CN117312257A - Data storage optimization method and system of consistency test platform of demand response equipment - Google Patents
Data storage optimization method and system of consistency test platform of demand response equipment Download PDFInfo
- Publication number
- CN117312257A CN117312257A CN202311027375.2A CN202311027375A CN117312257A CN 117312257 A CN117312257 A CN 117312257A CN 202311027375 A CN202311027375 A CN 202311027375A CN 117312257 A CN117312257 A CN 117312257A
- Authority
- CN
- China
- Prior art keywords
- data
- block
- demand response
- test platform
- consistency test
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 68
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000004044 response Effects 0.000 title claims abstract description 46
- 238000013500 data storage Methods 0.000 title claims abstract description 31
- 238000005457 optimization Methods 0.000 title claims abstract description 26
- 238000007906 compression Methods 0.000 claims abstract description 27
- 230000006835 compression Effects 0.000 claims abstract description 27
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 15
- 238000005096 rolling process Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 17
- 238000012795 verification Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013144 data compression Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 7
- 238000003860 storage Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 3
- 238000009826 distribution Methods 0.000 claims description 3
- 238000011084 recovery Methods 0.000 claims description 3
- 238000006467 substitution reaction Methods 0.000 claims description 3
- 230000001960 triggered effect Effects 0.000 claims description 3
- 230000008901 benefit Effects 0.000 abstract description 4
- 238000004378 air conditioning Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000006837 decompression Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 230000003252 repetitive effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011076 safety test Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
- G06F16/322—Trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
- G06F16/325—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
Abstract
The invention discloses a data storage optimization method and a system of a consistency test platform of demand response equipment, wherein the method comprises the following steps: acquiring consistency test platform data of the demand response equipment; rolling compression is carried out on the test data by utilizing a sliding window, and variable length coding is carried out on an output result; and encrypting the compressed data by adopting an encryption algorithm, and transmitting the encrypted data to obtain corresponding data of the consistency test platform of the demand response equipment. The invention provides a high-efficiency, intelligent, flexible and reliable data storage optimization method, and has wide application prospect and remarkable economic benefit.
Description
Technical Field
The invention relates to the technical field of demand response, in particular to a data storage optimization method and system of a demand response device consistency test platform.
Background
The central air conditioning system is used as an important demand response resource and can effectively participate in power grid peak shaving. With the rise of urban electrification level year by year, the number of central air conditioners participating in power grid peak shaving is increased. Central air conditioning equipment of different make, model and specification vary in function and performance. In terms of communication interoperation, different central air conditioning devices and systems may adopt different communication protocols or support multiple communication protocols, so consistency testing becomes a primary task for the central air conditioning devices to stably and efficiently participate in demand response of an electric power system, so as to ensure compatibility and consistency of the devices and other systems. In addition, the consistency test platform has large data volume, multiple types, quick growth, low data transmission efficiency, guaranteeing no data safety and influenced performance of the test platform.
Disclosure of Invention
The present invention has been made in view of the above-described problems.
Therefore, the technical problems solved by the invention are as follows: the existing data storage optimization method of the consistency test platform of the demand response equipment has the technical problem that the data safety cannot be guaranteed.
In order to solve the technical problems, the invention provides the following technical scheme: the data storage optimization method of the consistency test platform of the demand response equipment comprises the following steps:
acquiring consistency test platform data of the demand response equipment;
rolling compression is carried out on the test data by utilizing a sliding window, and variable length coding is carried out on an output result;
encrypting the compressed data by adopting an encryption algorithm, and transmitting the encrypted data;
and decrypting and decoding the received encrypted data to obtain corresponding demand response equipment consistency test platform data.
As a preferable scheme of the data storage optimization method of the consistency test platform of the demand response equipment, the invention comprises the following steps: the platform data comprises equipment basic information, equipment power consumption, equipment demand response capability, fault detection and recovery, test results, performance indexes and parameter settings.
As a preferable scheme of the data storage optimization method of the consistency test platform of the demand response equipment, the invention comprises the following steps: the test data comprises traversing the test data by utilizing a sliding window, compressing the data by utilizing a repeated structure of the data, and performing variable length coding on a compression result by taking a block as a unit.
As a preferable scheme of the data storage optimization method of the consistency test platform of the demand response equipment, the invention comprises the following steps: the test data also comprises sliding window traversal test data, the largest character string is matched, and the output tuple replaces repeated data;
processing the output tuple by adopting a self-adaptive block variable length coding data compression method, and calculating the entropy value of the data; determining the size of a data block based on an entropy value, constructing a binary tree by taking the block as a unit, traversing each block, counting the occurrence times of each character of each block, recording each character and the corresponding frequency thereof in each block by using an array, and constructing the binary tree of the current traversed block; generating a corresponding code for each character by traversing the Huffman tree, and updating the binary tree according to the code information of the current block after processing one block so as to adapt to the code of the next block;
and finally traversing all the blocks, summarizing the generated coding table, and replacing characters in the consistency test platform data of the demand response equipment with corresponding codes one by one.
As a preferable scheme of the data storage optimization method of the consistency test platform of the demand response equipment, the invention comprises the following steps: the compression may also include the steps of,
calculating weight distribution:
wherein W is i,j,k Weights representing the i, j, k-th block data; b (B) i,j,k Represented in three-dimensional data spaceThe i, j, k block data in (a); p (x) represents the probability of x in the data block; s (B) i,j,k ) Representing sparsity of the data blocks; l (B) i,j,k ) Representing locality of the data block; λ and μ represent weight parameters;
data compression and encoding:
C i,j,k ＝σ(B i,j,k， α·W i,j,k +β·S(B i,j,k )+γ·L(B i,j,k ))
wherein C is i,j,k Representing the compressed data block; sigma represents a compression function; alpha, beta and gamma represent adjustment parameters.
As a preferable scheme of the data storage optimization method of the consistency test platform of the demand response equipment, the invention comprises the following steps: verifying the original data block and the decompressed data block by using a plurality of hash functions;
the verification may include a verification of the identity of the user,
wherein,and->N-th hash values of the original data block and the compressed data block, respectively; n represents the index of the hash function; τ n Representing an nth hash function;
if all hash values are matched, verifying the data integrity; if a mismatch is found during the verification process, a report is generated and the process of recompression and adjustment of the compression parameters is triggered.
As a preferable scheme of the data storage optimization method of the consistency test platform of the demand response equipment, the invention comprises the following steps: the encryption comprises the steps of encrypting the coded data by adopting a GAN-AES encryption algorithm, taking a 128-bit block and a key optimized by GAN as inputs, and performing operation byte substitution, row shift, column confusion and round key addition on a byte array of 4*4; the first nine rounds are byte replacement, row shift, column confusion and round key addition operation, and the tenth round is byte replacement, row shift and round key addition operation.
The data storage optimization system of the demand response equipment consistency test platform adopting any one of the methods is characterized in that:
the data acquisition module is used for acquiring the data of the consistency test platform of the demand response equipment;
the data compression module is used for rolling and compressing the test data based on the sliding window and performing variable length coding on the output result so as to reduce the data storage space;
the encryption module is used for encrypting the compressed data by adopting an encryption algorithm and transmitting the encrypted data;
and the decryption module is used for decrypting and decoding the received encrypted data to obtain corresponding consistency test platform data of the call demand response equipment.
A computer device, comprising: a memory and a processor; the memory stores a computer program characterized in that: the processor, when executing the computer program, implements the steps of the method of any of the present invention.
A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program, when executed by a processor, implements the steps of the method of any of the present invention.
The invention has the beneficial effects that: the data storage optimization method of the consistency test platform of the demand response equipment can realize more efficient data storage by dividing the data into blocks and distributing different weights and compression levels for each block. This means that more data can be stored in a smaller memory space, thereby saving memory costs. The invention can more intelligently compress the data by considering the weight, sparsity and locality of the data blocks. This ensures that important data is properly processed, while unimportant or redundant data can be highly compressed. By using multiple hash functions for verification, the present invention provides a powerful mechanism to verify the integrity and security of data. This ensures that the data is not tampered with or damaged during storage and transmission. When needed, the data block can be decompressed by using a corresponding decompression algorithm, so that the data can be recovered quickly and accurately. By means of a feedback mechanism, the system can automatically adjust the compression parameters or recompress the data if any mismatch is found during the verification process, ensuring the best compression effect. By intelligently selecting the compression algorithm and parameters, the present invention can reduce unnecessary computations, thereby saving computing resources. Because the invention considers the three-dimensional structure of the data, the invention can be widely applied to various fields needing to process a large amount of data, such as medical imaging, meteorological data, geographic information systems and the like. The invention provides a high-efficiency, intelligent, flexible and reliable data storage optimization method, and has wide application prospect and remarkable economic benefit.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein:
FIG. 1 is a flowchart illustrating a data storage optimization method of a consistency test platform of a demand response device according to a first embodiment of the present invention.
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments, some of which are illustrated in the appended drawings. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below.
Further, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic can be included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
While the embodiments of the present invention have been illustrated and described in detail in the drawings, the cross-sectional view of the device structure is not to scale in the general sense for ease of illustration, and the drawings are merely exemplary and should not be construed as limiting the scope of the invention. In addition, the three-dimensional dimensions of length, width and depth should be included in actual fabrication.
Also in the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper, lower, inner and outer", etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the apparatus or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first, second, or third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
The terms "mounted, connected, and coupled" should be construed broadly in this disclosure unless otherwise specifically indicated and defined, such as: can be fixed connection, detachable connection or integral connection; it may also be a mechanical connection, an electrical connection, or a direct connection, or may be indirectly connected through an intermediate medium, or may be a communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
Example 1
Referring to fig. 1, for one embodiment of the present invention, a data storage optimization method of a consistency test platform of a demand response device is provided, including:
s1: and acquiring the consistency test platform data of the demand response equipment.
Further, the platform data includes device basic information, device power consumption, device demand response capability, fault detection and recovery, test results, performance indexes and parameter settings.
Still further, the test data includes traversing the test data using a sliding window, compressing the data using a repeating structure of the data, and variable length encoding the compressed result in units of blocks.
S2: and rolling and compressing the test data by utilizing the sliding window, and performing variable length coding on the output result.
It is to be noted that, based on the sliding window traversal test data, the data is scroll-compressed by using the repetitive structure of the data, the largest character string is matched, and the tuple (offset, string length, character to be matched) is output to replace the repetitive data. The compression result is variable length coded in units of blocks.
Specifically, an adaptive block variable length coding data compression method is adopted to process the output tuple, the entropy value of the data is calculated, and the size of the data block is determined based on the entropy value. Constructing a binary tree by taking blocks as units, traversing each block, counting the occurrence times of each character of each block, recording each character and the corresponding frequency of each character in each block by using an array, and constructing the binary tree of the current traversed block; and generating a corresponding code for each character by traversing the Huffman tree, and updating the binary tree according to the code information of the current block after processing one block so as to adapt to the code of the next block. And finally traversing all the blocks, summarizing the generated coding table, and replacing characters in the consistency test platform data of the demand response equipment with corresponding codes one by one.
The specific steps of rolling compression and variable length coding data based on the sliding window are as follows:
step 1, setting a sliding window and the size of a buffer zone, and traversing test data;
step 2, comparing the data in the searching buffer area, matching the longest repeated data, and outputting a tuple (offset, string length, character to be matched);
and step 3, updating the sliding window and searching the data in the buffer area, and repeating the steps until all the test data are scrolled.
Step 4, calculating the entropy value of the test data sample obtained in the step 3;
H(X)＝-Σ(p(x)*log2(p(x)))
where p (x) represents the probability of each data occurrence, 0 < = p (x) <=1, and Σ (p (x))=1;
step 5. Taking a block as an example, in the binary tree set { M ] 1 ,M 2 ,...M n Two trees with the minimum weight are selected to be respectively used as left and right subtrees, a new binary tree is constructed, and the weight of the root node of the new binary tree is the sum of the weights w of the left and right subtrees;
step 6, adding the binary tree newly constructed in the step 5 into a binary tree set, and deleting the binary tree in the binary tree set;
and 7, repeating the step 5, and repeating the step 6 until only one tree remains in the binary tree set, traversing the binary tree finally obtained by blocking, and generating a corresponding code for each character.
And 8, updating the binary tree, and encoding the next block data until all the data sample blocks are traversed. And replacing characters in the test data with corresponding codes one by using the generated code table.
S3: and encrypting the compressed data by adopting an encryption algorithm, and transmitting the encrypted data.
Further, the compression may further include,
calculating weight distribution:
wherein W is i,j,k Weights representing the i, j, k-th block data; b (B) i,j,k I, j, k-th block data represented in a three-dimensional data space; p (x) represents the probability of x in the data block; s (B) i,j,k ) Representing sparsity of a block of data, calculated as a proportion of zero or near zero data points in the block; l (B) i,j,k ) Representing the locality of a data block, and calculating the average difference between the data point and the neighbor of the data point in the block; λ and μ represent weight parameters that can be determined by cross-validation.
Data compression and encoding:
C i,j,k ＝σ(B i,j,k’ α·W i,j,k +β·S(B i,j,k )+γ·L(B i,j,k ))
wherein C is i,j,k Representing the compressed data block; sigma represents a compression function; α, β, and γ represent adjustment parameters that can be set as needed.
Further, verifying the original data block and the decompressed data block by using a plurality of hash functions; the verification may include a verification of the identity of the user,
wherein,and->N-th hash values of the original data block and the compressed data block, respectively; n represents the index of the hash function; τ n Representing an nth hash function;
if all hash values are matched, verifying the data integrity; if a mismatch is found during the verification process, a report is generated and the process of recompression and adjustment of the compression parameters is triggered.
It is to be appreciated that in many applications in the real world, data often has a three-dimensional structure, such as spatial data, time series data, and the like. By partitioning the data, we can better handle large amounts of data and assign different weights and compression levels to each data block. At the same time, not all data is equally important. Some data blocks may contain important information, while other data blocks may contain a large amount of redundant or unimportant information. By assigning weights to each data block, taking into account its sparsity and locality, we can more intelligently perform data compression. The integrity and security of the data is critical. By using multiple hash functions for verification, we can improve the accuracy and reliability of data verification.
First, the entire data set is divided into small blocks, each block having its own three-dimensional coordinates (i, j, k), for each data block, its weight, sparsity, and locality are calculated. The higher the weight the higher the compression level will be assigned. Each data block is compressed using an appropriate compression algorithm. The compression level will be determined based on the weight, sparsity, and locality of the data blocks. The data blocks are decompressed, if necessary, using a corresponding decompression algorithm. The original data block and the decompressed data block are verified using a plurality of hash functions. If all hash values match, the data integrity is verified. If any mismatch is found during the verification process, the system will generate a report and may trigger a process of recompression or adjustment of the compression parameters.
The encryption algorithm is GAN-AES encryption algorithm. Specifically, encrypting the data using the GAN-AES encryption algorithm includes: taking the 128-bit grouping block and the key optimized by GAN as input, and performing operation byte substitution, row shift, column confusion and round key addition on a byte array of 4*4; the first nine rounds are byte replacement, row shift, column confusion and round key addition operation, and the tenth round is byte replacement, row shift and round key addition operation.
Specifically, the AES encryption technique key that is optimized with GAN includes: firstly, defining a GAN structure, designing a generator and a discriminator structure, and taking a real AES key as training data. A set of random noise vectors is used as input to a generator to generate a candidate AES key. The arbiter takes the key generated by the receiving generator and the real key as inputs, and outputs the result as the probability that the input key is the real key. The method comprises the steps of alternately training the generator and the discriminator, generating a secret key through the generator and inputting the secret key into the discriminator for classification, and then adjusting parameters of the generator according to classification results of the discriminator, so that the quality of the secret key generated by the generator is gradually improved. After training is complete, the test set may be used to evaluate the quality and security of the keys generated by the generator.
And finally, decrypting and decoding the received encrypted data to obtain corresponding data of the consistency test platform of the demand response equipment.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile memory may include read only memory, magnetic tape, floppy disk, flash memory, optical memory, high density embedded nonvolatile memory, resistive memory, magnetic memory, ferroelectric memory, phase change memory, graphene memory, and the like. Volatile memory can include random access memory, external cache memory, or the like. By way of illustration, and not limitation, RAM can take many forms, such as static random access memory or dynamic random access memory. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like.
The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
Example 2
The following provides a data storage optimization method of a consistency test platform of demand response equipment, and in order to verify the beneficial effects of the invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.
Recording the size of the original data, respectively compressing the data by using a comparison group method and an experimental group method, recording the compressed data size and compression time, verifying the integrity of the data by using a hash function, and recording the success rate. And decompressing the data and recording the decompression time. As shown in table 1.
Table 1 experimental data table:
from the above experimental data, it can be seen that for the same data set, a higher compression rate can be obtained using the method of the present invention, saving more computing resources, and ensuring higher data integrity. This further verifies the beneficial effects of the present invention and demonstrates its superiority in practical applications.
Table 2 shows the security comparison of the present invention with the conventional method in data storage, and the verification is performed through multi-scenario experiments. Recording data is the proportion of content that the data is stolen.
Table 2 safety test data table:
| scene 1 | Scene 2 | Scene 3 | |
| The invention is that | 8％ | 15％ | 11％ |
| Conventional method | 34％ | 30％ | 44％ |
It can be seen that the information is not easy to steal and is relatively stable under different scenes, and the information stealing proportion of the traditional method is higher. The invention effectively protects the platform information through data encryption.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.
Claims (10)
1. The data storage optimization method of the consistency test platform of the demand response equipment is characterized by comprising the following steps of:
acquiring consistency test platform data of the demand response equipment;
rolling compression is carried out on the test data by utilizing a sliding window, and variable length coding is carried out on an output result;
and encrypting the compressed data by adopting an encryption algorithm, and transmitting the encrypted data to obtain corresponding data of the consistency test platform of the demand response equipment.
2. The data storage optimization method of a demand response device consistency test platform of claim 1, wherein: the platform data comprises equipment basic information, equipment power consumption, equipment demand response capability, fault detection and recovery, test results, performance indexes and parameter settings.
3. The data storage optimization method of the demand response device consistency test platform of claim 2, wherein: the test data comprises traversing the test data by utilizing a sliding window, compressing the data by utilizing a repeated structure of the data, and performing variable length coding on a compression result by taking a block as a unit.
4. A method of optimizing data storage for a demand response device consistency test platform as recited in claim 3, wherein: the test data also comprises sliding window traversal test data, the largest character string is matched, and the output tuple replaces repeated data;
processing the output tuple by adopting a self-adaptive block variable length coding data compression method, and calculating the entropy value of the data; determining the size of a data block based on an entropy value, constructing a binary tree by taking the block as a unit, traversing each block, counting the occurrence times of each character of each block, recording each character and the corresponding frequency thereof in each block by using an array, and constructing the binary tree of the current traversed block; generating a corresponding code for each character by traversing the Huffman tree, and updating the binary tree according to the code information of the current block after processing one block so as to adapt to the code of the next block;
and finally traversing all the blocks, summarizing the generated coding table, and replacing characters in the consistency test platform data of the demand response equipment with corresponding codes one by one.
5. The data storage optimization method of the demand response device consistency test platform of claim 4, wherein: the compression may also include the steps of,
calculating weight distribution:
wherein W is i,j,k Weights representing the i, j, k-th block data; b (B) i,j,k I, j, k-th block data represented in a three-dimensional data space; p) x_represents the probability of x in the data block; s (B) i,j,k ) Representing sparsity of the data blocks; l (B) i,j,k ) Representing locality of the data block; λ and μ represent weight parameters;
data compression and encoding:
C i,j,k ＝σ(B i,j,k ′α·W i,j,k +β·S(B i,j,k )+γ·L(B i,j,k ))
wherein C is i,j,k Representing the compressed data block; sigma represents a compression function; alpha, beta and gamma represent adjustment parameters.
6. The data storage optimization method of the demand response device consistency test platform of claim 5, wherein: verifying the original data block and the decompressed data block by using a plurality of hash functions;
the verification may include a verification of the identity of the user,
wherein,and->N-th hash values of the original data block and the compressed data block, respectively; n represents the index of the hash function; τ n Representing an nth hash function;
if all hash values are matched, verifying the data integrity; if a mismatch is found during the verification process, a report is generated and the process of recompression and adjustment of the compression parameters is triggered.
7. The data storage optimization method of the demand response device consistency test platform of claim 6, wherein: the encryption comprises the steps of encrypting the coded data by adopting a GAN-AES encryption algorithm, taking a 128-bit block and a key optimized by GAN as inputs, and performing operation byte substitution, row shift, column confusion and round key addition on a byte array of 4*4; the first nine rounds are byte replacement, row shift, column confusion and round key addition operation, and the tenth round is byte replacement, row shift and round key addition operation.
8. A data storage optimization system for a demand response device consistency test platform employing the method of any of claims 1-7, wherein:
the data acquisition module is used for acquiring the data of the consistency test platform of the demand response equipment;
the data compression module is used for rolling and compressing the test data based on the sliding window and performing variable length coding on the output result so as to reduce the data storage space;
the encryption module is used for encrypting the compressed data by adopting an encryption algorithm and transmitting the encrypted data;
and the decryption module is used for decrypting and decoding the received encrypted data to obtain corresponding data of the consistency test platform of the demand response equipment.
9. A computer device, comprising: a memory and a processor; the memory stores a computer program characterized in that: the processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program implementing the steps of the method of any of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311027375.2A CN117312257A (en) | 2023-08-15 | 2023-08-15 | Data storage optimization method and system of consistency test platform of demand response equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311027375.2A CN117312257A (en) | 2023-08-15 | 2023-08-15 | Data storage optimization method and system of consistency test platform of demand response equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117312257A true CN117312257A (en) | 2023-12-29 |
Family
ID=89261133
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311027375.2A Pending CN117312257A (en) | 2023-08-15 | 2023-08-15 | Data storage optimization method and system of consistency test platform of demand response equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117312257A (en) |
-
2023
- 2023-08-15 CN CN202311027375.2A patent/CN117312257A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230068246A1 (en) | Database utilizing spatial probability models for data compression | |
| Tao et al. | Significantly improving lossy compression for scientific data sets based on multidimensional prediction and error-controlled quantization | |
| US8983959B2 (en) | Optimized partitions for grouping and differentiating files of data | |
| Porwal et al. | Data compression methodologies for lossless data and comparison between algorithms | |
| WO2023040365A1 (en) | Image encryption method based on multi-scale compressed sensing and markov model | |
| CN107493405A (en) | Encrypted image reversible information hidden method based on coding compression | |
| US20160021396A1 (en) | Systems and methods for digital media compression and recompression | |
| CN112035695B (en) | Spatial data encryption method suitable for mobile terminal | |
| KR20170048398A (en) | Data adaptive compression and data encryption using kronecker products | |
| Di et al. | Optimization of error-bounded lossy compression for hard-to-compress HPC data | |
| Di et al. | Efficient lossy compression for scientific data based on pointwise relative error bound | |
| CN107291935B (en) | Spark and Huffman coding based CPIR-V nearest neighbor privacy protection query method | |
| Dolgorsuren et al. | StarZIP: Streaming graph compression technique for data archiving | |
| US20230222354A1 (en) | A method for a distributed learning | |
| Karthikeyan et al. | An efficient image compression method by using optimized discrete wavelet transform and Huffman encoder | |
| CN117312257A (en) | Data storage optimization method and system of consistency test platform of demand response equipment | |
| Sun et al. | A novel non-equilibrium hyperchaotic system and application on color image steganography with FPGA implementation | |
| Nandi et al. | Windowed Huffman Coding with limited distinct symbols | |
| CN113378186B (en) | Method and device for encrypting hidden information based on coded image | |
| GB2527099A (en) | Apparatus and method for data compression | |
| Žalik et al. | Can burrows-Wheeler transform be replaced in chain code compression? | |
| Ryabko | Compression-based methods for nonparametric density estimation, on-line prediction, regression and classification for time series | |
| CN107896143B (en) | Joint coding method based on high-dimensional chaotic system and arithmetic coding | |
| Kekre et al. | Vector quantized codebook optimization using modified genetic algorithm | |
| Štrbac et al. | Upgraded Petri net model and analysis of adaptive and static arithmetic coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |