CN117040542B - Intelligent comprehensive distribution box energy consumption data processing method - Google Patents
Intelligent comprehensive distribution box energy consumption data processing method Download PDFInfo
- Publication number
- CN117040542B CN117040542B CN202311289037.6A CN202311289037A CN117040542B CN 117040542 B CN117040542 B CN 117040542B CN 202311289037 A CN202311289037 A CN 202311289037A CN 117040542 B CN117040542 B CN 117040542B
- Authority
- CN
- China
- Prior art keywords
- data
- segment
- data segment
- item
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005265 energy consumption Methods 0.000 title claims abstract description 37
- 238000009826 distribution Methods 0.000 title claims abstract description 31
- 238000003672 processing method Methods 0.000 title claims abstract description 12
- 238000007906 compression Methods 0.000 claims abstract description 76
- 230000006835 compression Effects 0.000 claims abstract description 73
- 230000011218 segmentation Effects 0.000 claims abstract description 11
- 238000000034 method Methods 0.000 claims description 22
- 230000006978 adaptation Effects 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 9
- 230000005611 electricity Effects 0.000 description 17
- 238000003860 storage Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013144 data compression Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000004134 energy conservation Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R19/00—Arrangements for measuring currents or voltages or for indicating presence or sign thereof
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R21/00—Arrangements for measuring electric power or power factor
- G01R21/06—Arrangements for measuring electric power or power factor by measuring current and voltage
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R22/00—Arrangements for measuring time integral of electric power or current, e.g. electricity meters
- G01R22/06—Arrangements for measuring time integral of electric power or current, e.g. electricity meters by electronic methods
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R23/00—Arrangements for measuring frequencies; Arrangements for analysing frequency spectra
- G01R23/02—Arrangements for measuring frequency, e.g. pulse repetition rate; Arrangements for measuring period of current or voltage
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Power Engineering (AREA)
- Theoretical Computer Science (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention relates to the technical field of data processing, in particular to an intelligent comprehensive distribution box energy consumption data processing method, which comprises the following steps: collecting energy consumption data and differentiating, obtaining a data segment according to a coding section of the differential data, obtaining an initial compression rate according to a coding length of the data segment, a data quantity and an original bit number of data, obtaining a compression rate weight according to the initial compression rate and the compression rate of the traditional differential coding, obtaining a merging suitability according to the coding length of the data segment, the data quantity and the original bit number of the data, obtaining merging necessity according to the compression rate weight and the merging suitability, obtaining a data segment to be merged according to the merging necessity, merging the data segment to be merged to obtain a final data segment, and performing differential coding on the final data segment. Aiming at the problem that the conventional differential coding can not achieve the ideal compression rate of partial data, the invention enables all data to achieve better compression effect by the operation of initial segmentation and merging the data segments with poor compression effect.
Description
Technical Field
The invention relates to the technical field of data processing of data compression storage, in particular to an intelligent comprehensive distribution box energy consumption data processing method.
Background
Currently, intelligent comprehensive distribution boxes play an important role in energy management and energy conservation. However, with the increasing growth of energy consumption data, efficient processing and utilization of energy consumption data becomes a challenge. The energy consumption data of the intelligent comprehensive distribution box is usually a large amount of time sequence data, and comprises a large amount of time stamps and power consumption information. For large-scale energy consumption data sets, direct storage requires a large amount of storage space. The energy consumption data of the intelligent comprehensive distribution box is usually required to be transmitted to a data center or a cloud platform for further analysis and processing. Therefore, the intelligent comprehensive distribution box energy consumption data needs to be stored and managed.
The common intelligent comprehensive distribution box energy consumption data processing method is a differential compression algorithm, the traditional differential compression algorithm is a fixed-length compression algorithm, the coding length of data is determined based on the maximum value in the data differential value, and therefore a part of data which can be compressed to a smaller length in the data cannot reach an ideal compression rate.
Disclosure of Invention
The invention provides an intelligent comprehensive distribution box energy consumption data processing method, which aims to solve the existing problems: and (5) optimally storing the intelligent comprehensive power distribution box energy consumption data.
The invention discloses an intelligent comprehensive distribution box energy consumption data processing method which adopts the following technical scheme:
the embodiment of the invention provides an intelligent comprehensive distribution box energy consumption data processing method, which comprises the following steps:
collecting intelligent comprehensive distribution box energy consumption data, and respectively calculating differential values of all data in the energy consumption data to obtain differential data;
presetting a plurality of coding intervals and obtaining the coding length of each coding interval, segmenting the differential data according to the coding interval to which each data in the differential data belongs to obtain a plurality of data segments of each data and the coding length of each data segment, and calculating the initial compression rate of the data segments according to the coding length of the data segments, the data quantity contained in the data segments and the original bit number of one data;
acquiring the merging suitability of each data segment according to the coding length and the data quantity of each data segment and the adjacent data segment and the original bit quantity of one data, acquiring the compression rate weight of the data segment according to the initial compression rate of the data segment, the coding length of the data segment, the data quantity contained in the data segment and the original bit quantity of one data, weighting the merging suitability according to the compression rate weight of the data segment to acquire the merging necessity of each data segment, acquiring the data segment to be merged according to the merging necessity and the merging necessity threshold, and merging the data segments to be merged to acquire all final data segments;
and compressing and storing the final data segment by using a segmented differential encoding algorithm.
Preferably, the method for segmenting the differential data according to the coding section to which each data in the differential data belongs includes the following specific steps:
and recording a group of power consumption data which belong to the same coding section and are adjacent in time sequence in the differential data as a data segment.
Preferably, the coding length of each data segment is the coding length of the coding section where the differential data in the data segment is located.
Preferably, the calculating the initial compression rate of the data segment according to the coding length of the data segment, the data quantity contained in the data segment and the original bit number of one data includes the following specific steps:
wherein,is the firstItem data itemThe initial compression rate of the individual data segments,is the firstItem data itemThe number of data contained in the individual data segments,is the firstItem data itemThe length of the code of the individual data segments,is the original number of bits of a data.
Preferably, the method for obtaining the merging suitability of each data segment according to the coding length and the data quantity of each data segment and the adjacent data segment and the original bit number of one data includes the following specific steps:
wherein,is the firstItem data itemThe combined suitability of the individual data segments,represent the firstAnd (b)The number of data contained in the data segment with the smallest coding length in the data segments,respectively represent the firstItem data itemAnd (b)The encoded length of the individual data segments,is the original number of bits of a data.
Preferably, the method for obtaining the compression ratio weight of the data segment according to the initial compression ratio of the data segment, the encoding length of the data segment, the data quantity contained in the data segment and the original bit number of one data includes the following specific steps:
wherein,is the firstItem data itemThe compression ratio weights of the individual data segments,is the firstItem data itemThe initial compression rate of the individual data segments,is the firstThe number of data included in the item data,is the firstAll data segments in the item data correspond to the largest encoding length,is the original number of bits of a data.
Preferably, the method for obtaining the data segments to be combined according to the combining necessity and the combining necessity threshold includes the following specific steps:
comparing the merging necessity of each data segment with a preset merging necessity threshold value, and recording the data segments with the merging necessity larger than the merging necessity threshold value as data segments to be merged.
Preferably, the method for merging the data segments to be merged to obtain all final data segments includes the following specific steps:
and starting from the first data segment to be combined of each item of data, combining the data segment to be combined with the next data segment, calculating the combination necessity of the combined new data segment, if the combination necessity of the new data segment is still greater than the preset combination necessity, continuing to combine until the combination necessity of the data segment is less than the combination necessity threshold value, and recording the combined data segment as a final data segment, wherein the coding length of each data segment is equal to the maximum value of the coding lengths of all the data segments participating in combination, and recording the data segment which does not participate in combination as the final data segment.
Preferably, the coding interval isWherein n is any non-negative integer, and the coding length of the coding interval is n+1.
Preferably, the original number of bits of the one data represents the number of bits of the data when stored in the computer without being compressed.
The technical scheme of the invention has the beneficial effects that: aiming at the problem that a part of data which can be compressed to a smaller space in the data cannot reach an ideal compression rate by conventional differential coding, the scheme ensures that all data can reach the beneficial effect of the ideal compression rate as far as possible by carrying out initial segmentation on the data and merging data segments with poor compression effects after the initial segmentation.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of steps of an intelligent comprehensive distribution box energy consumption data processing method of the invention;
fig. 2 is a diagram of an example of segmenting data based on coding intervals.
Detailed Description
In order to further describe the technical means and effects adopted by the invention to achieve the preset aim, the following is a detailed description of specific implementation, structure, characteristics and effects of an intelligent comprehensive power distribution box energy consumption data processing method according to the invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The invention provides a specific scheme of an intelligent comprehensive distribution box energy consumption data processing method, which is specifically described below with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of a method for processing energy consumption data of an intelligent integrated distribution box according to an embodiment of the present invention is shown, where the method includes the following steps:
step S001: and collecting intelligent comprehensive distribution box energy consumption data, respectively calculating differential values of all data in the energy consumption data, and obtaining differential data.
It should be noted that, the electricity consumption data of the intelligent comprehensive distribution box mainly includes a time stamp, a device number and electricity consumption data, the electricity consumption data mainly includes voltage, current, frequency and the like, different data are recorded as different data, and differential encoding compression is mainly performed on each item of data in the electricity consumption data.
Specifically, first, the energy consumption data of the intelligent distribution box needs to be collected. The collection to intelligent comprehensive block terminal data can be accomplished to the mode of installation electric energy meter reading, in order to realize the real-time supervision to intelligent block terminal energy consumption data, can set up the collection frequency to intelligent block terminal energy consumption data for once every 15 minutes, does not inject data acquisition frequency in this scheme, in other embodiments, the implementation personnel can set up the collection frequency according to actual implementation condition. Classifying all the acquired data according to the equipment numbers, arranging the data of the same equipment according to the sequence from small to large of the acquired time stamps, and acquiring different items of data in the intelligent comprehensive distribution box energy consumption data.
In this scheme, the data is compressed by differential encoding, and the differential value of adjacent data is calculated, so that the differential value is not changed drastically, and the differential value is calculated and compressed for the same item of data.
Specifically, a device electricity consumption data table is obtained. The first of the electricity consumption dataThe item data is indexed according to time sequence, the item data is the first itemThe data minus the firstData, the difference is recorded as the firstItem data of itemAnd differential values. First, theAll differential values of item data constitute the firstAnd the item differential data, and all differential values of all item data form a power consumption data differential table.
For the electricity consumption data, each data is individually stored in the computer by assigning a fixed bit number to each data, and the bit number of the data stored in the computer without being compressed is recorded as the original bit number。
So far, the equipment electricity consumption data table and the electricity consumption data difference table of the intelligent comprehensive distribution box energy consumption data are obtained through the method.
Step S002: and calculating the initial compression rate of the data segment according to the length of the differential data segment and the interval to which the differential data belongs.
It should be noted that, in the conventional differential compression coding, the fixed-length coding corresponding to the coding length of the maximum differential value is directly set for all data according to the magnitude of the differential value, but the fluctuation degree of the energy consumption data of the intelligent comprehensive distribution box is larger, and the fixed-length coding is directly set for the data according to the maximum differential value, so that the data which can be compressed by using the shorter coding can be coded into a longer length, and therefore, the effect of reducing the coding length is achieved by segmenting the data and setting the differential coding length as small as possible for each data segment.
It should be further noted that the distribution box energy consumption data are all represented in the form of numbers, and the smaller the binary form of the differential value in the data, the fewer bits are required.
Specifically, for any non-negative integerWill beRecorded as code length ofIs a coding section of (a).
The coding length required for differential coding depends on the coding length of the coding section in which the differential value is located. For example, referring to fig. 2, for a series of differential data:. Wherein,in the coding sectionWhileIn the coding sectionThe differential data string is divided into three segments. The data may be segmented according to the differential value size.
Specifically, the method for initially segmenting all item data comprises the following steps:
according to each difference data in the electricity consumption data difference table of the equipment electricity consumption data table, according to the integer power of 2Dividing section. The first power consumption data difference tableThe term difference data belongs to the same coding interval and is adjacent in time sequence to a group of electricity consumption numbersRecorded as a data segment. The differential values of all the data in the same data segment are in the same coding section, the coding length of the coding section where the differential data in the data segment is located is recorded as the coding length of the data segment, and the first is recordedItem data itemThe section code length corresponding to each data section is recorded as the data section code length。
According to the data segment dividing modeItem data segment division, item dataAll data segments of item data constitute the firstA sequence of item data segments, obtain the firstThe code length of each data segment in the sequence of item data segments will beItem data itemThe number of data contained in each data segment is recorded as. And carrying out initial segmentation on all the item data to obtain a data segment sequence of each item of data.
It should be noted that, the initial segmentation of the data can shorten the average encoding length of the data, but when the data segments are differentially encoded with different encoding lengths, the recording of the reference data and the encoding length of each data segment also results in an increase in the encoding length. The initial compression rate of the data segment under the initial segment can be obtained according to the average coding length of the data segment and the data quantity contained in any one data segment.
Specifically, for the first of the electricity consumption dataItem data according to the firstThe data amount contained in the data segmentCalculating the initial compression rate obtained by initial segmentation according to the coding length of the data segment and the original bit number of one data:
wherein,is the firstItem data itemThe initial compression rate of the individual data segments,is the firstItem data itemThe number of data contained in the individual data segments,is the firstItem data itemThe length of the code of the individual data segments,is the original number of bits of a data.
In the formulaThe encoded length of the representative data segment is obtained by multiplying the encoded length of the data segment by the number of data contained in the data segment minus one, and the encoded length of the representative data segmentEach data is encoded to be of lengthThe code length of a data segment can be obtained by adding the code length of the reference value required by each data segment through differential coding, namely the original bit of one data. In the formulaThe method comprises the steps of multiplying the number of data contained in a data segment by the original bit of one data, obtaining the storage space required by the data segment which is not subjected to differential coding compression, multiplying the coding length of the coded data segment by the storage space required before coding, and subtracting the quotient value from 1 to obtain the initial compression rate of the data segment.
It should be further noted that, the initial compression rate represents the compression rate of the data segment by the initial segment, and the larger the initial compression rate, the better the compression effect of differential encoding compression after the initial segment.
Specifically, obtain the firstInitial compression ratio of all data segments in item data, initial compression ratio of all data segments forming the firstAnd the initial compression rate sequence of the item data is used for acquiring the initial compression rate of all data segments in all item data.
So far, the initial compression rate of all data segments in all items of data is obtained through the method.
Step S003: and acquiring the merging necessity of the adjacent data segments according to the initial compression rate and the merging suitability of the adjacent data segments, merging the data segments according to the merging necessity of the data segments, and acquiring the final segments of all data.
It should be noted that, after the initial segmentation is performed on the electricity consumption data table, each differential value is allocated with an optimal coding length, but because of a certain fluctuation of electricity consumption data of the intelligent comprehensive distribution box, the data is segmented into too many data segments according to the optimal coding length in a violent manner, so that a great amount of segmentation costs such as data segment initial data, coding length marks and the like are generated, and a part of data segments are reversely optimized in a violent segmentation manner. Therefore, the length of the data segments is increased in a data segment merging mode, and the excessive segmentation cost caused by excessive data segments is eliminated.
It should be further noted that, the two data segments are combined into one segment, so that the space occupation of the reference block required by each data segment is reduced, but the coding length of the data segment with the original coding length being short is also made longer, and the suitability of combining the two data segments is judged by judging the countermeasure relation between the coding length being longer and the space optimization of the reference block.
Specifically, for the firstItem data of itemAnd (b)And the data segments are combined and adaptive according to the coding length, the data quantity and the space size required by the stored data before compression of the two data segments:
wherein,is the firstItem data itemThe combined suitability of the individual data segments,represent the firstAnd (b)The number of data contained in the data segment of smaller coding length in the individual data segments,respectively represent the firstItem data itemAnd (b)The encoded length of the individual data segments,is the original number of bits of a data.
It should be noted that, after the two data segments are combined, the differential encoding length of the combined data segment takes the one with the larger encoding length of the two combined data segments, so the encoding length increase caused by the combination is mainly reflected in the fact that the encoding length of the data segment with the shorter encoding length before the combination is changed into the one with the larger encoding length of the two data segments. Therefore, the number and two of the data contained in the data segment with shorter encoding length can be combinedMultiplying the difference of the coding lengths of the data segments to obtain the data segments and combining the increment of the coding lengths, namely the molecular part in the formula:. Combining two data segments reduces the use of one reference data that cannot be compressed by differential encoding, so that the data segment combining shortens the encoding to the number of bits needed to compress the previous data, the denominator in the formula:. The partial manufacturers which make the codes grow and shorten through merging the data segments acquire the merging suitability of the two data segments, wherein the merging suitability is larger than that of an instruction two data segments, so that the shortening of the coding length of the data can be optimized, and the larger the merging suitability is, the larger the optimizing effect is.
Specifically, the merging necessity of all the data segments except the last data segment in all the item data is acquired.
It should be further noted that, whether two data segments are suitable for merging can be determined directly according to the merging suitability of the two data segments, however, if the merging suitability of one data segment and two adjacent data segments is relatively low, the compression effect of the data segment is far different from that of the traditional compression algorithm, and the data segment needs to be merged.
Specifically, according to the firstItem data itemThe compression rate of each data segment is obtained by the quotient of the compression rate of the original data and the compression rate of the original dataCompression ratio weight of each data segment:
wherein,is the firstItem data itemThe compression ratio weights of the individual data segments,is the firstItem data itemThe initial compression rate of the individual data segments,is the firstThe number of data included in the item data,is the firstAll data segments in the item data correspond to the largest encoding length,is the original number of bits of a data.
In the formulaRepresents the firstThe compression rate at which the entire item data is compressed by differential encoding is recorded as the reference encoded length, and when the entire item data is differentially compressed, it is necessary to store all differential values for the length of the compression encoding, and therefore all the differential values are selectedThe maximum coding length in the data segment is to ensure that all data can be compressed by this coding length. Obtaining the first through an initial compression rate calculation formula of the data segmentCompression ratio of the item data as a whole. Reuse of the firstItem data of itemCompression ratio and the first data segmentThe compression ratio of the whole item data is greater than a value indicating the first itemItem data itemThe compression rate of the individual data segments is better than directly compressing the whole data.
It should be noted that according to the first embodimentCompression ratio weight pair of item dataThe merging suitability of the data segments is weighted to obtain the merging necessity of the data segments.
Specifically, according to the firstItem data of itemThe compression ratio weight of each data segment weights the merging suitability of the data segment to obtain the first data segmentMerging necessity of individual data segments:
wherein,is the firstItem data itemThe necessity of merging the individual data segments,is the firstItem data itemThe compression ratio weights of the individual data segments,is the firstItem data itemThe combined suitability of the individual data segments.
The merging suitability of the data segments is weighted by the compression ratio weight, so that the merging necessity of the data segments is obtained, and the adjacent data segments can be merged according to the merging necessity.
Specifically, all data segments except the last data segment in all item data are weighted according to the formula, and the merging suitability of the data segments is obtained according to the compression ratio weight of the data segments. The calculation of the merging necessity of all the data segments is guaranteed to be carried out under the condition of the same dimension, and the merging necessity of the data segments is measured more accurately.
Presetting a combination requirementSex thresholdIn this exampleThe present embodiment is not limited to the merge necessity threshold, and in other embodiments, the practitioner may set the merge necessity threshold according to actual real-time conditions.
Specifically, for the firstAll data segments in the item data except the last data segment, combining necessity of the data segments with a combining necessity thresholdComparing, combining necessity is greater thanThe data segments of the data are marked as data segments to be combined, and the first data segment is obtainedAll the data segments to be merged in the item data. Merging the data segment to be merged with the next data segment from the first data segment to be merged, and calculating the merging necessity of the new data segment after merging if the merging necessity of the new data segment is still larger than that of the new data segmentMerging is continued until the merging necessity of the data segments is smaller thanThe data segments are then marked as a final data segment, and the code length of each data segment is marked as the largest of the code lengths of all the data segments to be combined, wherein the data segments not participating in the combination are also marked as final data segments.
So far, all the data segments are combined according to the method to obtain all the final data segments,
and acquiring all data segments except the last data segment in all item data, calculating the merging necessity, merging the data segments, acquiring the final data segment of all item data, and acquiring the coding length of the final data segment according to the maximum of the coding lengths of all data segments participating in merging.
So far, the final data segment division of all item data is obtained through the method.
Step S004: and carrying out sectional coding on the electricity consumption data according to the final data segment and storing the electricity consumption data.
The final data segment of all items of data is obtained, and differential encoding is performed on the final data segment, so that the overall differential encoding length of the data segment can be minimized.
Specifically, according to the final data segment of all the data, the first data of each data segment is set as a data segment reference value, the rest data are compressed into differential codes with the same coding length as the data segments through differential codes, the differential codes of the power consumption data are obtained, the codes are stored, and the optimal storage of the intelligent comprehensive distribution box energy consumption data is realized. The differential encoding compression is a well-known data encoding compression algorithm, and specific implementation procedures are not repeated in this embodiment.
This embodiment is completed.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the invention, but any modifications, equivalent substitutions, improvements, etc. within the principles of the present invention should be included in the scope of the present invention.
Claims (3)
1. The intelligent comprehensive distribution box energy consumption data processing method is characterized by comprising the following steps of:
collecting intelligent comprehensive distribution box energy consumption data, and respectively calculating differential values of all data in the energy consumption data to obtain differential data;
presetting a plurality of coding intervals and obtaining the coding length of each coding interval, segmenting the differential data according to the coding interval to which each data in the differential data belongs to obtain a plurality of data segments of each data and the coding length of each data segment, and calculating the initial compression rate of the data segments according to the coding length of the data segments, the data quantity contained in the data segments and the original bit number of one data;
acquiring the merging suitability of each data segment according to the coding length and the data quantity of each data segment and the adjacent data segment and the original bit quantity of one data, acquiring the compression rate weight of the data segment according to the initial compression rate of the data segment, the coding length of the data segment, the data quantity contained in the data segment and the original bit quantity of one data, weighting the merging suitability according to the compression rate weight of the data segment to acquire the merging necessity of each data segment, acquiring the data segment to be merged according to the merging necessity and the merging necessity threshold, and merging the data segments to be merged to acquire all final data segments;
compressing and storing the final data segment by using a segmentation differential coding algorithm;
the method for segmenting the differential data according to the coding section to which each data in the differential data belongs comprises the following specific steps:
recording the power consumption data which belong to the same coding section and are adjacent in time sequence in the differential data as a data segment;
the coding length of each data segment is the coding length of a coding section where differential data in the data segment are located;
the method for calculating the initial compression rate of the data segment according to the coding length of the data segment, the data quantity contained in the data segment and the original bit number of one data comprises the following specific steps:
；
wherein,is->Item data item->Initial compression rate of individual data segments, +.>Is->Item data item->Number of data contained in the data section, < >>Is->Item data item->Encoding length of individual data segment,/->The original number of bits for one data;
the method for obtaining the merging suitability of each data segment according to the coding length and the data quantity of each data segment and the adjacent data segment and the original bit quantity of one data comprises the following specific steps:
；
wherein,is->Item data item->Merging adaptation of individual data segmentsCompatibility (I) of (I)>Indicate->Person and->The number of data contained in the data section with the smallest coding length in the data sections,/for>Respectively represent +.>Item data item->Person and->Coding length of individual data segments,/->The original number of bits for one data;
the method for obtaining the compression ratio weight of the data segment according to the initial compression ratio of the data segment, the coding length of the data segment, the data quantity contained in the data segment and the original bit number of one data comprises the following specific steps:
；
wherein,is->Item data item->Compression ratio weight of individual data segment, +.>Is->Item data item->Initial compression rate of individual data segments, +.>Is->Number of data contained in item data, +.>Is->All data segments in the item data correspond to the maximum coding length,/->The original number of bits for one data;
the method for acquiring the data segments to be combined according to the combining necessity and the combining necessity threshold comprises the following specific steps:
comparing the merging necessity of each data segment with a preset merging necessity threshold value, and marking the data segments with the merging necessity larger than the merging necessity threshold value as data segments to be merged;
the method for merging the data segments to be merged to obtain all final data segments comprises the following specific steps:
and starting from the first data segment to be combined of each item of data, combining the data segment to be combined with the next data segment, calculating the combination necessity of the combined new data segment, if the combination necessity of the new data segment is still greater than the preset combination necessity, continuing to combine until the combination necessity of the data segment is less than the combination necessity threshold value, and recording the combined data segment as a final data segment, wherein the coding length of each data segment is equal to the maximum value of the coding lengths of all the data segments participating in combination, and recording the data segment which does not participate in combination as the final data segment.
2. The method for processing energy consumption data of intelligent integrated distribution box according to claim 1, wherein the coding interval isWherein n is any non-negative integer, and the coding length of the coding interval is n+1.
3. The method for processing intelligent integrated distribution box energy consumption data according to claim 1, wherein the original number of bits of the one data represents the number of bits of the data stored in the computer without being compressed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311289037.6A CN117040542B (en) | 2023-10-08 | 2023-10-08 | Intelligent comprehensive distribution box energy consumption data processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311289037.6A CN117040542B (en) | 2023-10-08 | 2023-10-08 | Intelligent comprehensive distribution box energy consumption data processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117040542A CN117040542A (en) | 2023-11-10 |
| CN117040542B true CN117040542B (en) | 2024-01-12 |
Family
ID=88630334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311289037.6A Active CN117040542B (en) | 2023-10-08 | 2023-10-08 | Intelligent comprehensive distribution box energy consumption data processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117040542B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117785818B (en) * | 2024-02-26 | 2024-05-10 | 山东惠分仪器有限公司 | Gas chromatograph data optimized storage method and system |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101075237A (en) * | 2006-06-28 | 2007-11-21 | 腾讯科技（深圳）有限公司 | Method for storing, fetching and indexing data |
| CN101933327A (en) * | 2007-10-26 | 2010-12-29 | 富士胶片株式会社 | Data compression apparatus, data compression program and image-taking apparatus |
| CN102938685A (en) * | 2012-11-15 | 2013-02-20 | 大连理工大学 | Wireless sensor network data compression method based on variable-length encoding |
| CN111510156A (en) * | 2020-06-03 | 2020-08-07 | 合肥师范学院 | Method for dynamically compressing and decompressing large file based on segmentation |
| CN113630125A (en) * | 2021-08-11 | 2021-11-09 | 深圳市联影高端医疗装备创新研究院 | Data compression method, data encoding method, data decompression method, data encoding device, data decompression device, electronic equipment and storage medium |
| CN115361027A (en) * | 2022-10-18 | 2022-11-18 | 江苏量超科技有限公司 | Sewage treatment effect identification method |
| CN115459782A (en) * | 2022-08-29 | 2022-12-09 | 北京思维实创科技有限公司 | Industrial Internet of things high-frequency data compression method based on time sequence segmentation and clustering |
| CN115567058A (en) * | 2022-08-28 | 2023-01-03 | 湖南大学 | Time sequence data lossy compression method combining prediction and coding |
| CN116016606A (en) * | 2023-01-09 | 2023-04-25 | 安徽皖欣环境科技有限公司 | Sewage treatment operation and maintenance data efficient management system based on intelligent cloud |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2008034213A1 (en) * | 2006-09-18 | 2008-03-27 | Infobright Inc. | A method and system for data compression in a relational database |
| TW202102010A (en) * | 2019-05-24 | 2021-01-01 | 瑞典商赫爾桑茲股份有限公司 | Methods, devices and computer program products for lossless data compression and decompression |
-
2023
- 2023-10-08 CN CN202311289037.6A patent/CN117040542B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101075237A (en) * | 2006-06-28 | 2007-11-21 | 腾讯科技（深圳）有限公司 | Method for storing, fetching and indexing data |
| CN101933327A (en) * | 2007-10-26 | 2010-12-29 | 富士胶片株式会社 | Data compression apparatus, data compression program and image-taking apparatus |
| CN102938685A (en) * | 2012-11-15 | 2013-02-20 | 大连理工大学 | Wireless sensor network data compression method based on variable-length encoding |
| CN111510156A (en) * | 2020-06-03 | 2020-08-07 | 合肥师范学院 | Method for dynamically compressing and decompressing large file based on segmentation |
| CN113630125A (en) * | 2021-08-11 | 2021-11-09 | 深圳市联影高端医疗装备创新研究院 | Data compression method, data encoding method, data decompression method, data encoding device, data decompression device, electronic equipment and storage medium |
| CN115567058A (en) * | 2022-08-28 | 2023-01-03 | 湖南大学 | Time sequence data lossy compression method combining prediction and coding |
| CN115459782A (en) * | 2022-08-29 | 2022-12-09 | 北京思维实创科技有限公司 | Industrial Internet of things high-frequency data compression method based on time sequence segmentation and clustering |
| CN115361027A (en) * | 2022-10-18 | 2022-11-18 | 江苏量超科技有限公司 | Sewage treatment effect identification method |
| CN116016606A (en) * | 2023-01-09 | 2023-04-25 | 安徽皖欣环境科技有限公司 | Sewage treatment operation and maintenance data efficient management system based on intelligent cloud |
Non-Patent Citations (1)
| Title |
|---|
| 相控阵天气雷达数据压缩技术;胡林宏等;《气象科技》;第51卷(第3期);第356-362页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117040542A (en) | 2023-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117040542B (en) | Intelligent comprehensive distribution box energy consumption data processing method | |
| CN115361027B (en) | Sewage treatment effect identification method | |
| CN115204754B (en) | Heating power supply and demand information management platform based on big data | |
| CN116702708B (en) | Road pavement construction data management system | |
| CN116208172B (en) | Data management system for building engineering project | |
| CN117278054B (en) | Intelligent power grid monitoring data storage method and system | |
| CN112633316A (en) | Load prediction method and device based on boundary estimation theory | |
| CN116016606B (en) | Sewage treatment operation and maintenance data efficient management system based on intelligent cloud | |
| CN116541828B (en) | Intelligent management method for service information data | |
| CN115543946B (en) | Financial big data optimized storage method | |
| CN116051156B (en) | New energy dynamic electricity price data management system based on digital twin | |
| CN116388245A (en) | Method for configuring energy storage capacity of optical storage and charging integrated power station and related equipment | |
| CN116700630A (en) | Organic-inorganic compound fertilizer production data optimized storage method based on Internet of things | |
| CN117097906B (en) | Method and system for efficiently utilizing regional medical resources | |
| CN116582133B (en) | Intelligent management system for data in transformer production process | |
| CN116933734B (en) | Intelligent diagnosis method for cutter faults of shield machine | |
| CN106790550B (en) | A kind of system suitable for the compression of power distribution network monitoring data | |
| CN115964347B (en) | Intelligent storage method for data of market supervision and monitoring center | |
| CN117278055A (en) | Vehicle sales information optimized storage method based on data analysis | |
| CN110995396A (en) | Compression method of communication messages of electricity consumption information acquisition system based on hierarchical structure | |
| CN116737085A (en) | Efficient elevator maintenance data storage method | |
| CN115913249A (en) | Battery data compression method, storage method and battery management system | |
| CN102811062B (en) | Curve sparse processing method for high-density time sequence data in power system wide-area measurement system (WAMS) | |
| CN114817831B (en) | Computing auxiliary method for building engineering economy | |
| CN116109009B (en) | Optical storage type park load curve dynamic identification method based on multi-feature extraction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |