CN111049861A - Adaptive integer segmentation based data compression method for wide area electric energy metering system - Google Patents

Abstract

The invention provides a self-adaptive integer-based segmented data compression method for a wide area electric energy metering system, which is characterized in that various transmission texts are converted into binary strings, each byte of the binary strings is regarded as an unsigned integer, the optimal segmented number is calculated, all integers are compressed into a large integer by adopting a segmented integer compression method, the large integer is expressed by the binary strings, the integers are encoded, and the length of an output character string is remarkably reduced. The invention adopts the self-adaptive integer segmentation data compression method, can effectively reduce the communication data flow of the message, lightens the data transmission communication pressure of the wide area electric energy metering system, shortens the transmission time, improves the transmission efficiency, reduces the data transmission cost and saves the communication flow cost for the electric power enterprises.

Description

Adaptive integer segmentation based data compression method for wide area electric energy metering system

Technical Field

The invention relates to a self-adaptive integer segmentation based data compression method for data transmission messages in a wide area electric energy metering system for competitive electric power retail market and network load interaction, and belongs to the field of wide area electric energy metering systems.

Background

With the continuous release of the spot market construction and the electricity selling market in the electric power system in China, the number of market main bodies is continuously increased, the types of the main bodies tend to be diversified, the market competition is continuously enhanced, the price mechanism is more flexible, users can flexibly switch among electricity vendors, the rates of the users gradually show the difference of dates and time periods, and various personalized and innovative electricity selling packages are correspondingly presented on the electricity selling side. In the operation of the spot market, the electricity selling company faces the problem of checking and settling the deviation between the reported electric quantity (or physical contract electric quantity) and the actual electric quantity in the market at the day before, so that the settlement relation of the spot market is extremely complex, and a brand new challenge is provided for the conventional electric energy metering system.

In the current power system, a wide area electric energy metering system comprises a master station, a wide area electric energy metering terminal, an intelligent electric meter and the like, wherein the master station and the wide area electric energy metering terminal generally adopt 4G wireless public network communication, and application layer data exchange is realized mainly through framing transmission of an application layer data unit (APDU) in an interoperability data exchange communication protocol DL/T698.45-2017. In the message compression process, if the input is regarded as a character string corresponding to a hexadecimal numerical value to be compressed, the compressible space of the character string is small, because the existing lossless compression method for the character string mainly performs variable length coding according to the occurrence frequency of characters, so that the average coding length of each symbol is smaller than the ASCII code length (8 bits) of one character per se, however, the occurrence frequency of each character in messages such as actual current, voltage, power and the like in a wide area electric energy metering system is not obviously different, and the coding length of each character per se is only 4 bits, so that the average coding length of sixteen characters from 0 to 9 and from A to F is not obviously smaller than 4. The bandwidth condition and the communication mode in the existing power grid structure cannot effectively solve the problem of how to improve the granularity of the load curve and the electric energy curve of each retail user acquired by the wide-area electric energy metering system, and the high real-time performance of refreshing the load and electric energy data of each user in a minute-scale real-time manner cannot be achieved.

Disclosure of Invention

The invention provides a data compression method based on an adaptive integer segment for transmitting an application layer data unit (APDU) based on a DL/T698.45-2017 data exchange protocol, which aims to solve the problem that a wide area electric energy metering system collects the load curve of each retail user and the real-time transmission of the electric energy curve, and retransmits the application layer data unit (APDU) in an interoperability data exchange communication protocol DL/T698.45-2017 between a master station and a wide area electric energy metering terminal in a data lossless compression mode so as to improve the data transmission performance and adapt to high sampling frequency.

The technical solution of the invention is as follows: a wide area electric energy metering system is based on a self-adaptive integer segmentation data compression method, various transmission texts are converted into binary strings, each byte of the binary strings is regarded as an unsigned integer, the optimal segmentation quantity is calculated, all integers are compressed into a large integer by adopting a segmentation integer compression method, the large integer is expressed by the binary strings, the integers are encoded, and the length of an output character string is remarkably reduced; the method specifically comprises the following three steps:

① converts the input binary string into an integer sequence S ═ a₁,a₂,…,a_n}；

② divide S into m segments S₁,S₂,…,S_mI.e. S₁＝{a₁,a₂,…,a_k}，S_i＝{a_(i-1)*k+1,a_(i-1)*k+2,…,a_i*k}，S_m＝{a_(m-1)*k+1,a_(m-1)*k+2,…,a_n}；

③ for each segment S_i＝{a_(i-1)*k+1,a_(i-1)*k+2,…,a_i*k}, the following two variables are calculated:

X_i＝min{a_(i-1)*k+1,a_(i-1)*k+1,L,a_i*k}；

Y_i＝max{a_(i-1)*k+1,a_(i-1)*k+1,L,a_i*k}-X+2；

④ calculating the segment S_iCompression result R of_iThe calculation formula is as follows:

⑤ compressing the result R_iConversion into binary string BR_i；

⑥ repeat steps ③ through ⑤ until all segments are processed;

⑦ mixing BR₁,BR₂,…,BR_mArranging and outputting in sequence;

2) optimal segmentation step

① initializing m to 1, calculating Y value, and making the minimum average Y value Y' to Y;

② calculating Y values of m segments₁,…,Y_mAnd the average thereof

③ if m > n, terminating the optimal segmentation step;

④ if

Then order

Otherwise, the current m value is the optimal segmentation quantity, and the algorithm is terminated;

⑤ add segmentation, let m be m +1, go to step ② again;

3) data decompression step

① mixing BR_iConversion to integer R_iOf variable b₀＝R_i；

② calculating a according to the following formula_(i-1)*k+j＝(b_j-1modY_i)+X_iUp to

And b is_j>0, where j ≧ 1, the data decompression step is terminated.

The invention has the advantages that:

1) by adopting the self-adaptive integer segmentation data compression method, the communication data flow of the message can be effectively reduced, and the data transmission communication pressure of the wide-area electric energy metering system is reduced.

2) The transmission time is shortened, the transmission efficiency is improved, the data transmission cost is reduced, and the communication flow cost is saved for power enterprises.

Drawings

FIG. 1 is an optimal segmentation algorithm flow based on the adaptive integer segmentation data compression method of the present invention.

FIG. 2 is an application flow of framing interaction based on the adaptive integer segmented data compression method of the present invention.

FIG. 3 is an interoperability data exchange communication protocol DL/T698.45-2017 link layer communication protocol format between a master station and a wide area power metering terminal based on an adaptive integer segmented data compression method.

Detailed Description

The technical solution of the present invention is further illustrated by the following examples.

A wide area electric energy metering system is based on a self-adaptive integer segmentation data compression method, various transmission texts are converted into binary strings, each byte of the binary strings is regarded as an unsigned integer, the optimal segmentation quantity is calculated, all integers are compressed into a large integer by adopting a segmentation integer compression method, the large integer is expressed by the binary strings, the integers are encoded, and the length of an output character string is remarkably reduced.

The method mainly comprises the following three steps:

1) data compression step

Converting n bytes contained in an input binary string into n integers, i.e. S ═ a₁,a₂,…,a_nDividing S into m sections S₁、S₂、…、S_mEach segment comprising

An integer, i.e. S₁＝{a₁,a₂,…,a_k}，S_i＝{a_(i-1)*k+1,a_(i-1)*k+2,…,a_i*k}，S_m＝{a_(m-1)*k+1,a_(m-1)*k+2,…,a_n}; the main flow of the data compression step is as follows:

① converts the input binary string into an integer sequence S ═ a₁,a₂,…,a_n}；

② divide S into m segments S₁,S₂,…,S_mI.e. S₁＝{a₁,a₂,…,a_k}，S_i＝{a_(i-1)*k+1,a_(i-1)*k+2,…,a_i*k}，S_m＝{a_(m-1)*k+1,a_(m-1)*k+2,…,a_n}；

③ for each segment S_i＝{a_(i-1)*k+1,a_(i-1)*k+2,…,a_i*k}, the following two variables are calculated:

X_i＝min{a_(i-1)*k+1,a_(i-1)*k+1,L,a_i*k}；

Y_i＝max{a_(i-1)*k+1,a_(i-1)*k+1,L,a_i*k}-X+2；

④ calculating the segment S_iCompression result R of_iThe calculation formula is as follows:

⑤ compressing the result R_iConversion into binary string BR_i；

⑥ repeat steps ③ through ⑤ until all segments are processed;

⑦ mixing BR₁,BR₂,…,BR_mArranging and outputting in sequence;

2) optimal segmentation step

The number of segments directly affects the data compression efficiency, so the optimal number of segments needs to be found for each input string, as shown in fig. 1, the specific algorithm flow and execution steps are as follows:

① initializing m to 1, calculating Y value, making the minimum average Y value Y 'to Y, where the variable Y' represents the minimum Y average value and is a temporary variable, and since m is 1, the average value of Y is itself;

② calculating Y values of m segments₁,…,Y_mAnd the average thereof

③ if m > n, terminating the optimal segmentation step;

④ if

Then order

Otherwise, the current m value is the optimal segmentation quantity, the algorithm is terminated, and the value is the value of the variable m in the current iteration process;

⑤ add segment m-m +1, execute step ② again;

3) data decompression step

For each segment data, the decompression process is the inverse process of the compression process; for segment Si, if the compression result BR is known_iAnd middleVariable X_iAnd Y_iThe specific steps of decompression are as follows:

① mixing BR_iConversion to integer R_iOf variable b₀＝R_i；

② calculating a according to the following formula_(i-1)*k+j＝(b_j-1modY_i)+X_iUp to

And b is_j>0, where j ≧ 1, the data decompression step is terminated.

The APDU data compression algorithm of the communication message provided by the invention is mainly used for a wide area electric energy metering terminal, and the acquired data is compressed and then sent to a master station so as to improve the data transmission performance and adapt to high sampling frequency. Because the related data such as electric energy and the like are basically numerical types, the frequency of each number (0-9) has no obvious difference, and the efficiency is not high by simply adopting a compression algorithm based on statistics and a dictionary. Certain optimization strategies need to be designed according to the correlation among the same-class data, certain ideas of a lossy compression algorithm are combined with a lossless compression algorithm, and lossless data compression and efficient compression are guaranteed.

At present, apdu framing transmission is generally adopted for data transmission, and two conclusions are obtained through repeated experiments and test research: the longer the communication message is, the better the compression effect is; a decrease in the repetition rate of encrypted message data leads to a sudden increase in the compression ratio. The maximum processable APDU length exists in the practical use of the wide area electric energy metering system, and when the preprocessing length is larger than the maximum processable length, the APDU unit which needs to be processed currently is returned for the next processing. Because the message is limited in length due to encryption, if the message needs to be encrypted for transmission, the message is compressed, then encrypted and finally framed. If encryption is not required, the transmission should be compressed and then framed. An application flow of framing interaction based on the adaptive integer segmentation data compression method is proposed, and a specific flow is shown in fig. 2. The invention organically combines three processes of encryption, framing and compression before data transmission, reduces the communication data flow of the message and lightens the communication pressure of the system.

Examples

The invention refers to an interoperability data exchange communication protocol DL/T698.45-2017 link layer communication protocol format between a main station and a wide area electric energy metering terminal, and the format is shown in the following figure 3: the link user data comprises a complete byte sequence of an application layer data unit (APDU) or a framing fragment of the APDU, and the marking rule of the application layer data unit (APDU) follows the abstract syntax of ASN.1, which is described in GB/T16262.1-2006; the coding rules of application layer data units (APDUs) follow A-XDR, see DL/T790.6-2010 for details.

As an embodiment, a processing procedure of an adaptive integer segmented data compression method for a segment of a packet is specifically listed as follows:

given the raw data to be transmitted: 01011204000101120400010112040240000001011204000101112004100101120412010112041201, respectively;

1) converting the original data into an unsigned integer every two bytes, the following sequence of integers can be obtained:

257 4612 1 274 1024 257 4612 576 0 257 4612 1 273 8196 4097 274 1042257 4612 4609；

2) dividing the integer sequence to obtain the optimal number of segments:

(1) let m be 1: y' ═ 8198;

(2) let m be 2: y1 ═ 4614, Y2 ═ 8197,

(3) let m be 3: y1-4613, Y2-8198, Y3-4357,

(4) let m be 4: y1 ═ 4613, Y2 ═ 4614, Y3 ═ 8197, Y4 ═ 4357,

(5) let m be 5: y1 ═ 4613, Y2 ═ 4357, Y3 ═ 4614, Y4 ═ 7925, Y5 ═ 4357,

(6) let m be 6: y1-4613, Y2-4357, Y3-4614, Y4-8197, Y5-3825, Y6-4357,

(7) let m be 7: y1-4613, Y2-769, Y3-4614, Y4-4613, Y5-7925, Y6-787, Y7-5,

(8) let m be 8: y1-4613, Y2-769, Y3-4614, Y4-4613, Y5-7925, Y6-3825, Y7-787, Y8-5,

at this time

The value of more than m is 7, so that the optimal segmentation quantity is m is 7;

3) data compression for each sequence of integers

(1) The first segment of data sequence is: 25746121, respectively;

calculating to obtain R1-5468891407;

the encrypted sequence BR1 ═ 10100010,11111100,01010100,10000111, 1;

(2) the second segment of data sequence is: 2741024257, respectively;

calculating to obtain R2-10642960;

the encrypted sequence BR2 is 10100010,01100110,00010000;

(3) the third data sequence is: 46125760, respectively;

calculating to obtain R3-98187507216;

the encrypted sequence BR3 is 10110110,11100011,01110011,10110000,10000;

(4) the fourth segment of data sequence is: 25746121, respectively;

calculating to obtain R4-5468891407;

the encrypted sequence BR4 ═ 10100010,11111100,01010100,10000111, 1;

(5) the fifth data sequence is: 27381964097, respectively;

calculating to obtain R5-62793599;

the encrypted sequence BR5 ═ 11101111,10001001,11011111, 11;

(6) the sixth data sequence is: 2741042257, respectively;

calculating to obtain R6-11147068;

the encrypted sequence BR6 is 10101010,00010111,00111100;

(7) the seventh data sequence is: 46124609, respectively;

calculating to obtain R7-15;

1111 of the encrypted sequence BR 7;

in this example, the original data is 40 bytes (320 bits), the compressed data is 181 bits (about 23 bytes), and the compression ratio is 57.5%.

The present embodiments are described above with reference to the accompanying drawings, in which like reference numerals are used to designate like elements, and in which like reference numerals are used to designate like elements.

Unless otherwise defined, terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art; further, it will be understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense.

Claims (4)

1. The wide area electric energy metering system is based on a self-adaptive integer segmentation data compression method, and is characterized in that various transmission texts are converted into binary strings, each byte of the binary strings is regarded as an unsigned integer, the optimal segmentation quantity is calculated, all the integers are compressed into a large integer by adopting a segmentation integer compression method, the large integer is expressed by the binary strings, the integer is encoded, and the length of an output character string is remarkably reduced; the method specifically comprises the following three steps: 1) a data compression step; 2) optimal segmentation step; 3) and data decompression step.

2. The wide area electric energy metering system data compression method based on the adaptive integer segmentation as claimed in claim 1, wherein the specific flow and execution steps of the 1) data compression step are as follows:

① converts the input binary string into an integer sequence S ═ a₁,a₂,…,a_n}；

② divide S into m segments S₁,S₂,…,S_mI.e. S₁＝{a₁,a₂,…,a_k}，S_i＝{a_(i-1)*k+1,a_(i-1)*k+2,…,a_i*k}，S_m＝{a_(m-1)*k+1,a_(m-1)*k+2,…,a_n}；

③ for each segment S_i＝{a_(i-1)*k+1,a_(i-1)*k+2,…,a_i*k}, the following two variables are calculated:

X_i＝min{a_(i-1)*k+1,a_(i-1)*k+1,L,a_i*k}；

Y_i＝max{a_(i-1)*k+1,a_(i-1)*k+1,L,a_i*k}-X+2；

④ calculating the segment S_iCompression result R of_iThe calculation formula is as follows:

⑤ compressing the result R_iConversion into binary string BR_i；

⑥ repeat steps ③ through ⑤ until all segments are processed;

⑦ mixing BR₁,BR₂,…,BR_mOutput in sequence。

3. The method for compressing data of the wide area electric energy metering system based on the adaptive integer segmentation as claimed in claim 1, wherein the specific flow and execution steps of the 2) optimal segmentation step are as follows:

① initializing m to 1, calculating Y value, and making the minimum average Y value Y' to Y;

② calculating Y values of m segments₁,…,Y_mAnd the average thereof

③ if m > n, terminating the optimal segmentation step;

④ if

Then order

Otherwise, the current m value is the optimal segmentation quantity, and the algorithm is terminated;

⑤ increments segment by m +1 and step ② is performed again.

4. The method for compressing data of the wide area electric energy metering system based on the adaptive integer segmentation as claimed in claim 1, wherein the specific flow and execution steps of the 3) data decompression step are as follows:

① mixing BR_iConversion to integer R_iB, carrying out the following steps of; let variable b₀＝R_i；

② calculating a according to the following formula_(i-1)*k+j＝(b_j-1modY_i)+X_iUp to

And b is_j>0, wherein j is more than or equal to 1,

the data decompression step is terminated.

Images