CN113360538A

CN113360538A - Space-time convergence and query method of energy consumption data

Info

Publication number: CN113360538A
Application number: CN202110797362.8A
Authority: CN
Inventors: 胡新; 汤雪松; 吴维农; 李柯沂; 吴彬; 杨永刚; 胡洛娜; 陈柯; 钟淘淘; 任雪松; 甘玉芳
Original assignee: State Grid Corp of China SGCC; Information and Telecommunication Branch of State Grid Chongqing Electric Power Co Ltd
Current assignee: State Grid Corp of China SGCC; Information and Telecommunication Branch of State Grid Chongqing Electric Power Co Ltd
Priority date: 2021-07-14
Filing date: 2021-07-14
Publication date: 2021-09-07
Anticipated expiration: 2041-07-14
Also published as: CN113360538B

Abstract

The invention relates to a space-time convergence method of energy consumption data, which comprises the following steps: s1, constructing a hierarchical spatio-temporal data structure; s2, sequentially converging the energy consumption data at the minimum position granularity according to the sequence of time granularity from small to large to obtain converged data of the minimum position granularity at the maximum time granularity; s3, inserting the converged data of the minimum position granularity at the maximum time granularity into the energy consumption data of the position granularity at the minimum time granularity of the previous stage to form the converged data of the position granularity at the minimum time granularity of the previous stage; s4, sequentially converging the position granularity of the upper level from small to large according to the analogy of the steps S2-S3 to obtain converged data of the maximum position granularity at the maximum time granularity; therefore, the accuracy and the execution efficiency of data aggregation are ensured; the query method can realize the spatio-temporal query of the converged data by adopting a specially designed query language through the query engine, and has high efficiency and quick response.

Description

Space-time convergence and query method of energy consumption data

Technical Field

The invention relates to the field of intelligent power distribution networks, in particular to a space-time convergence and query method of energy consumption data.

Background

The intelligent power grid can not only improve efficiency and increase value, but also provide data-centered service for customers by analyzing mass energy data acquired by the distributed monitoring equipment, thereby realizing the goal of digital transformation. The amount of data generated by smart grids is enormous and growing at an explosive rate each year. In 2020, the holding capacity of the European intelligent electric meters reaches 2.4 hundred million, and China particularly reaches 4 hundred million. The massive energy data not only causes heavy burden on data storage, but also makes data scalability analysis very difficult. Data generated by the smart grid arrives in the form of continuous and unpredictable data streams, and a data structure is required to support quick updating and efficient query of the data streams. Taking the metering data as an example, although the data generated by the smart meters is generally smooth (terminal ID, installation location, metering reading), many decision-making functions require aggregation and analysis of data generated by a large number of smart meters at different spatiotemporal levels (e.g., energy usage per hour for all smart meters in a particular region).

The existing big data aggregation and query method cannot be directly applied to energy consumption data. Firstly, frequent item mining technology is mainly adopted in data stream processing methods such as Spark, Flink and Storm, important data only need to be kept during data aggregation, the rest data are discarded, and energy bills and power grid operation state estimation (for analyzing power characteristics of a power grid) are not allowed to discard any data. Secondly, the above method cannot capture the spatiotemporal characteristics of the data. For example, energy consumption data usually comes from smart meters located at the client side, in order to ensure smooth operation of the smart grid, aggregation and analysis are required for different periods (minutes, hours, days, weeks, years), different locations (smart meters, districts, power supply stations, branch companies, head offices) or both, and traditional database management systems, time-series database management systems, big data and data flow processing tools and the like cannot support special data hierarchy analysis and temporal analysis.

Disclosure of Invention

In view of the above, an object of the present invention is to overcome the defects in the prior art, and provide a method for performing spatio-temporal aggregation and query on energy-based data, which can ensure the accuracy and execution efficiency of data aggregation, reduce the storage space, and select an appropriate granularity according to the aggregation requirement. The query engine can realize the time-space query of the converged data, and has high efficiency and quick response.

The energy data space-time convergence method comprises the following steps:

s1, constructing a hierarchical spatio-temporal data structure; the hierarchical spatiotemporal data structure comprises a time dimension structure and a location dimension structure;

s2, sequentially converging the energy consumption data at the minimum position granularity of the position dimension structure according to the sequence of time granularity in the time dimension structure from small to large to obtain converged data of the minimum position granularity at the maximum time granularity;

s3, inserting the converged data of the minimum position granularity at the maximum time granularity into the energy consumption data of the position granularity at the first level above the minimum position granularity at the minimum time granularity to form converged data of the position granularity at the first level above the minimum position granularity at the minimum time granularity;

s4, sequentially converging the position granularity from the position granularity at the first level above the minimum position granularity from the position granularity at the first level in the step S2-S3 according to the sequence from small to large of the position granularity in the position dimension structure to obtain converged data of the maximum position granularity at the maximum time granularity.

Further, the hierarchical spatiotemporal data structure is composed of a node set arranged in a grid form; the nodes adopt cylindrical data structures.

Further, the cylindrical data structure includes N independent time queue data structures.

Further, the time queue data structure is a first-in first-out circular buffer, and the number of buffer units of the circular buffer is C.

Further, the number N of the independent time queue data structures is determined according to the number of the intelligent electric meters of the target position granularity.

Further, step S2 specifically includes:

s21, setting the number of time queue data structures according to the position of the target position granularity, and setting the number of cache units of each time queue data structure of the target position granularity according to the time granularity;

s22, acquiring energy consumption data collected by all intelligent electric meters, and calculating the variable quantity of the energy consumption data of all the intelligent electric meters;

s23, starting from the minimum time granularity, inserting the variable quantity into a target cache unit of a target time queue data structure;

s24, if the number of the cache units with variable data in the target time queue data structure in the step S23 reaches the set number, inserting all the variable data in the target time queue data structure into the target cache unit of the time queue data structure corresponding to the last level of the minimum time granularity;

s25, according to the analogy of the steps S23-S24, the energy consumption data aggregation from the small time granularity to the large time granularity is completed, and the aggregated data of the target position granularity at the maximum time granularity is obtained.

Further, in step S23, the target buffer unit of the target time queue data structure is determined according to the following steps:

a. setting data stream tau generated by intelligent electric meter to tau ═<e,t,△_e>(ii) a Wherein e is an identifier of the smart meter, t is a timestamp, and delta_eThe variation of the energy consumption data;

b. in a plurality of time queue data structures, taking a time queue data structure with the number size of e as a target time queue data structure;

c. selecting a corresponding numerical value from the timestamp t according to the target time granularity, and taking the numerical value as a target numerical value;

d. and in a plurality of cache units of the target time queue data structure, taking the cache unit numbered as the target numerical value as a target cache unit.

A method for querying energy consumption data comprises the following steps:

A1. constructing a query language model;

A2. adjusting each parameter value in the query language model to enable the query language model to generate a query language meeting the query requirement, and taking the query language as a target query language;

A3. target data is queried from the hierarchical spatio-temporal data structure using a target query language.

Further, the query language model is determined according to the following formula:

f(Q)[{+,-,*,/}]；

wherein f is an operation function, Q is a parameter of f, [ { +, -,/} ] is an optional query syntax for performing f (Q) operation; q is [ Time, tr ], [ Loc, lr ], Time is an attribute value of a Time hierarchy, Loc is an attribute value of a location hierarchy, tr is a specific Time, and lr is a specific location.

The invention has the beneficial effects that: the invention discloses a space-time convergence and query method of energy-using data, which adopts a grid-shaped hierarchical space-time data structure to ensure the accuracy and the execution efficiency of data convergence, realizes the gradual convergence of data at a high level of a hierarchical structure, and stores the latest data in a granularity form at a low level, thereby not only reducing the storage space, but also selecting the proper granularity according to the convergence requirement. The query engine adopts a specially designed query language, so that the spatiotemporal query of the aggregated data can be realized, and the method has high efficiency and quick response.

Drawings

The invention is further described below with reference to the following figures and examples:

FIG. 1 is a schematic flow chart of the spatiotemporal convergence method of the present invention;

FIG. 2 is a schematic diagram of a hierarchical spatiotemporal data structure according to the present invention.

Detailed Description

The invention is further described with reference to the drawings, as shown in fig. 1:

the invention relates to a space-time convergence method of energy utilization data, which comprises the following steps:

The time granularity can be divided into Minutes (MI), Hours (HR), Days (DY), Weeks (WE) and Years (YE) according to the order of the granularity from small to large, and a time hierarchical relationship is formed among the time granularities; the position granularity can be divided into a Smart Meter (SM), a station area (SD), a power supply station (PS), a branch company (BO) and a main company (HO) according to the order of the granularity from small to large, position hierarchical relation is formed among the position granularities, the station area is used as the upper stage of the smart meter, the power supply station is used as the upper stage of the station area, and the like. The intelligent electric meter belongs to the terminal.

A grid-shaped hierarchical Data Structure (STDS) is adopted to ensure the accuracy and the execution efficiency of Data aggregation, realize the gradual aggregation of Data at a high level of the hierarchical Structure, store the latest Data in a granularity form at a low level, reduce the storage space and select the proper granularity according to the aggregation requirement.

In this embodiment, the grid-like hierarchical spatiotemporal data structure is composed of a set of nodes arranged in a grid form; the nodes store and gather energy data from the smart meters in a Cylindrical Shape Data Structure (CSDS) mode.

In this embodiment, the cylindrical data structure includes N independent Time Queue Data Structures (TQDS).

In this embodiment, the time queue data structure is a first-in first-out circular buffer, and the number of buffer units of the circular buffer is set to C. By storing N data streams in each CSDS of the current hierarchy, each TQDS aggregates and sums the latest C data streams and sends the aggregated C data streams to the CSDS of the previous hierarchy, and deletes the C data streams at the same time. This way of aggregation not only stores data in aggregated form, but also ensures that no data is discarded or lost.

As shown in fig. 2, nodes [0,0], [1,0], …, [4,0] are distributed along the time granularity, but are all at the smart meter level of the location, which are referred to as level 0 of the STDS. Similarly, nodes [0,1], [1,1], [4,1] are referred to as level 1 of the STDS, and so on. It can be seen that the higher level nodes of the STDS contain aggregated data from the lower level nodes. For example, a level 1 node contains data for a level 0 node, a level 2 node contains data for a level 1 node, and so on.

If more hours of data need to be stored in the "HR-SD" node, such as storing the past 100 hours of data, C may be set to 100. Since the previous layer of the "HR-SD" node is the "DA-SD" node (time dimension indicated by the solid line), when the "HR-SD" node sets C to 24, the TQDS aggregates the 24 data and sends the aggregated data to a buffer unit of the "DA-SD" node TQDS. During the queuing operation, a non-zero sum is returned when C insertions occur, otherwise 0 is returned. The returned non-zero sum is the aggregated data sent to the nodes of the previous layer.

In this embodiment, the number N of the independent time queue data structures is determined according to the number of the smart meters of the target location granularity. If the number of the smart meters arranged in the terminal hierarchy is 10, the number N of the time queue data structures in each CSDS of the terminal location hierarchy is 10.

In this embodiment, the step S2 specifically includes:

s21, setting the number of time queue data structures according to the position of the target position granularity, and setting the number of cache units of each time queue data structure at the target position granularity according to the time granularity;

setting a position vector phi ═ {16,8,4,2} and a time vector Ψ ═ {60,24,7,52,10 }; the position vector is used to determine the number N of independent TQDS in the CSDS, which means that the node CSDS labeled SM has 16 TQDS; the node CSDS labeled SD has 8 TQDS, the node CSDS labeled PS has 4 TQDS, the node CSDS labeled BO has 2 TQDS, and the node CSDS labeled HO has 1 TQDS.

The time vector is used to determine the number of buffer units of different levels of TQDS. The time vector Ψ ═ {60,24,7,52,10} set above means that the number of buffer units of the TQDS in the nodes of each hierarchy is: the node labeled MI has 60 cache units per TQDS, the node labeled HR has 24 cache units per TQDS, the node labeled DA has 7 cache units per TQDS, the node labeled WE has 52 cache units per TQDS, and the node labeled YE has 10 cache units per TQDS.

S22, acquiring energy consumption data collected by all intelligent electric meters, and calculating the variable quantity of the energy consumption data of all the intelligent electric meters; wherein, the intelligent electric meter continuously sends data v, and the STDS stores the variation delta of the energy consumption data collected twice continuously_e＝v_t-1-v_tI.e. the energy used since the previous reading.

S23, starting from the minimum time granularity, inserting the variable quantity data into a target cache unit of a target time queue data structure;

s24, if the number of the cache units with variable data in the target time queue data structure in the step S23 reaches the set number, inserting all the variable data in the target time queue data structure into the target cache unit of the time queue data structure corresponding to the last level of the minimum time granularity; wherein, the node marked as MI has 60 buffer units per TQDS, and the number of buffer units set at this time is 60.

In this embodiment, in step S23, the target buffer unit of the target time queue data structure is determined according to the following steps:

a. setting data stream tau generated by intelligent electric meter to tau ═<e,t,Δ_e>(ii) a Wherein e is an identifier of the smart meter, t is a timestamp, and delta_eThe variation of the energy consumption data;

Wherein, each node of the STDS is associated with positions SM, DS, PS, BO and HO, i.e. each position is mapped to a positive integer j by using one-to-one mapping, and the data of the position is inserted into the TQDS located at the index j of the CSDS. For example, the smart meter collection data numbered 9 is inserted into the 9 th TQDS of the CSDS labeled SM node. Specifically, when the received data stream is τ ═ c<e＝5,t＝13/05/20200845,Δ_e＝152>The collected data indicating that the smart meter with the number (corresponding to the identifier) of 5 is 45 minutes at 8 am on 13 d.5/2020/e is 152, e is used for locating the TQDS, and t is used for determining the buffer unit in the TQDS. The data 152 is inserted into the 45 th buffer unit of the 5 th TQDS of the node marked MI (the lowest node of STDS), and when more than 15 arrays (up to 1 hour) are received, the 60 data are aggregated and inserted into the 8 th buffer unit of the 5 th TQDS of the node marked HR (the 60 data belong to the hour of 8 am).

By the space-time convergence method, the storage resource requirements and the time complexity of data updating are analyzed as follows:

storage resource requirements: when the time hierarchy vector and the position hierarchy vector set by the STDS are psi and phi respectively, C is psi, N is phi, and the storage resource required by the CSDS is C multiplied by N. Specifically, for a certain node n in the STDS_ijThe storage resource required for CSDS is C_i×N_jThe total memory resources required for the STDS are then

It is shown that the storage space required for layering the STDS is independent of the data size, and depends only on setting the temporal level vector Ψ and the location level vector Φ. For example, setting the time hierarchy vector Ψ ═ {60,24,7,52,10} and the location hierarchy vector Φ ═ 8239,2000,200,50,1}, respectively, then the storage resources required for the STDS are:

{60,24,7,52,10}×{8239,2000,200,50,1}＝1601370

meaning that the total number of buffer units required for the STDS is 1601370, if the data storage amount required for each buffer unit is 100 bytes, then the total storage space required for the STDS is 160137000 bytes 152.72 MB.

Time complexity of data update: for each energy use data generated by the smart meter, the data in the circular buffer needs to be updated. The circular buffer adopts a data structure based on Hash, one-time Hash operation is needed to update data, and after 60 times of data updating, the circular buffer stores 60 arrays v₀,…,v₅₉Calculating Δ ═ v₅₉-v₀It is inserted into MI-node [0,0] of the right diagonal line of the grid in FIG. 2],[0,1],[0,2],[03],[0,4]In (1). The calculation of Δ and the insertion of the node are performed once, respectively, the loop buffer receives 60 sets of the operations (60+5 × (1 (calculation Δ) +1 (insertion data))) -70 operations, and the time complexity of performing one data update is 70/60 ═ O (1).

An energy consumption data query method based on a space-time convergence method of energy consumption data comprises the following steps:

A1. constructing a query language model;

In this embodiment, the query language model is determined according to the following formula:

f(Q)[{+,-,*,/}]；

Wherein the query language supports a wide range of high-level user queries. For example, the query operation in the form of "how much energy is used per week recorded by each smart meter" is All ([ WE, ], [ SM, ]). Similarly, the query operation "which block has a higher energy demand in the last 5 weeks" is Max ([ WE,1-5], [ SD, ]); the query operation of "average daily energy for each smart meter in the last year" is All ([ YE,1], [ SM ]/365).

By the query method, the time complexity of data query is analyzed as follows:

1) single-point query: in the single point query process, the CSDS for locating the STDS node needs to perform one operation, and the TQDS for locating the CSDS needs to perform one operation. Therefore, querying the data in STSD requires two location operations with a time complexity of O (1).

2) And (3) range query: the number of positioning operations of the range query depends on the time and location range tr, lr. For the time range tr, tr single-point queries need to be initiated; for the location range lr, lr single-point queries need to be initiated, i.e., the time complexity of the range query is O (tr × lr).

Finally, the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all of them should be covered in the claims of the present invention.

Claims

1. A method for spatio-temporal convergence of energy-using data, comprising: the method comprises the following steps:

2. The method for spatio-temporal convergence of energy data according to claim 1, characterized in that: the hierarchical spatio-temporal data structure consists of a node set arranged in a grid form; the nodes adopt cylindrical data structures.

3. The method for spatio-temporal convergence of energy data according to claim 2, wherein: the cylindrical data structure includes N independent time queue data structures.

4. The method for spatio-temporal convergence of energy data according to claim 3, wherein: the time queue data structure is a first-in first-out circular buffer, and the number of buffer units of the circular buffer is C.

5. The method for spatio-temporal convergence of energy data according to claim 3, wherein: and the number N of the independent time queue data structures is determined according to the number of the intelligent electric meters of the target position granularity.

6. The method for spatio-temporal convergence of energy data according to claim 4, wherein: the step S2 specifically includes:

7. The method for spatio-temporal convergence of energy data according to claim 6, wherein: in step S23, the target buffer unit of the target time queue data structure is determined according to the following steps:

8. An energy consumption data query method based on any one of claims 1 to 7, characterized in that: the method comprises the following steps:

A1. constructing a query language model;

9. The energy consumption data query method of claim 8, wherein: the query language model is determined according to the following formula:

f(Q)[{+,-,*,/}]；