CN117097810A - Data center transmission optimization method based on cloud computing - Google Patents
Data center transmission optimization method based on cloud computing Download PDFInfo
- Publication number
- CN117097810A CN117097810A CN202311344747.4A CN202311344747A CN117097810A CN 117097810 A CN117097810 A CN 117097810A CN 202311344747 A CN202311344747 A CN 202311344747A CN 117097810 A CN117097810 A CN 117097810A
- Authority
- CN
- China
- Prior art keywords
- current data
- buffer
- dictionary
- segment
- data segment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000005540 biological transmission Effects 0.000 title claims abstract description 37
- 238000005457 optimization Methods 0.000 title claims abstract description 22
- 238000007906 compression Methods 0.000 claims abstract description 71
- 230000006835 compression Effects 0.000 claims abstract description 70
- 230000008859 change Effects 0.000 claims abstract description 32
- 230000008569 process Effects 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 8
- 238000005265 energy consumption Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Computational Biology (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention relates to the technical field of data transmission, in particular to a data center transmission optimization method based on cloud computing, which comprises the following steps: collecting a current data sequence of the intelligent equipment; obtaining a range change trend value according to the current data sequence; obtaining a current data record according to the current data sequence; obtaining a buffer current data section according to the current data record; obtaining dictionary current data segments according to the buffer current data segments; obtaining a range change trend difference value according to the range change trend value, the buffer current data segment and the dictionary current data segment; obtaining the regional similarity according to the range change trend difference value; obtaining the data segment compression necessity according to the region similarity; obtaining a final buffer current data segment according to the data segment compression necessity; and carrying out compression transmission according to the final buffer current data segment. The invention improves the data quantity which can be compressed, and further improves the transmission efficiency of the current element data.
Description
Technical Field
The invention relates to the technical field of data transmission, in particular to a data center transmission optimization method based on cloud computing.
Background
With the continuous update and development of the internet of things technology, more and more intelligent devices are developed and are widely applied to daily life. The intelligent device can generate a large amount of energy consumption data during normal operation, so that the energy consumption performance of the intelligent device is better known, the intelligent device with better performance is developed, the energy consumption data is required to be transmitted to a data center, and the data center is used for data analysis. Because the intelligent equipment needs electric energy to support operation, the energy consumption data mainly consumed are current data, and the current data are transmitted to a data center; however, since the intelligent device generates a large amount of current data, in order to improve the transmission efficiency of the current data, it is necessary to compress the current data and transmit the compressed current data.
The traditional compression processing method utilizes the LZ77 compression algorithm to compress repeated data segments in the current data, but because the current data randomly generates fluctuation of different degrees, the repeated data segments are fewer, the data compressed by the LZ77 compression algorithm is less, the compression efficiency is low, the data volume of the compressed current data is still larger, and the transmission efficiency of the current data is lower.
Disclosure of Invention
The invention provides a data center transmission optimization method based on cloud computing, which aims to solve the existing problems: because the current data randomly generates fluctuation of different degrees, repeated data segments are fewer, so that the LZ77 compression algorithm compresses less data, the compression efficiency is low, the data volume of the compressed current data is still larger, and the transmission efficiency of the current data is lower.
The data center transmission optimization method based on cloud computing adopts the following technical scheme:
the method comprises the following steps:
collecting a current data sequence of the intelligent equipment, wherein the current data sequence comprises a plurality of current data;
obtaining a range variation trend value of each current data according to the current data sequence; obtaining a plurality of current data records according to the current data sequence, wherein the current data records comprise a dictionary window and a look-ahead buffer area;
obtaining a plurality of buffer current data segments of each preceding buffer zone according to the current data record; obtaining a plurality of dictionary current data segments of each buffer current data segment according to the buffer current data segments and the dictionary windows; obtaining a plurality of current data pairs and range variation trend difference values of each current data pair according to the range variation trend values, the buffer current data segments and the dictionary current data segments; obtaining the regional similarity of each current data pair according to the range change trend difference value; obtaining a plurality of dictionary current data segments of each buffer current data segment and data segment compression necessity of each buffer current data segment according to the region similarity;
obtaining a final buffer current data segment of each current data record according to the data segment compression necessity; and carrying out compression transmission according to the final buffer current data segment.
Preferably, the range variation trend value of each current data is obtained according to the current data sequence, which comprises the following specific steps:
in the method, in the process of the invention,indicate->Initial range trend value of individual current data, < >>；/>Indicate->-individual current data; />Indicate->-individual current data; />Indicate->-individual current data; />The representation takes absolute value; obtaining initial range change trend values of all current data in a current data sequence, carrying out linear normalization on all initial range change trend values, and recording each normalized initial range change trend value as a range change trend value.
Preferably, the method for obtaining a plurality of current data records according to the current data sequence includes the following specific steps:
respectively marking the preset text window length, dictionary window length and look-ahead buffer area length as T1, T2 and T3; wherein t1=t2+t3, obtaining a text window with a text window length of T1; the window area with the length of the front T2 in the text window is marked as a dictionary window, and the window area with the length of the rear T3 in the text window is marked as a look-ahead buffer area;
and for the first current data in the current data sequence as a starting point, the step length is 1, sequentially inputting the current data into text windows in the order from right to left, simultaneously recording the current data of the text windows once, and recording the current data as current data record until all the current data in the current data sequence are input.
Preferably, the method for obtaining the plurality of buffered current data segments of each preceding buffer according to the current data record includes the following specific steps:
for any one of the current data records, the current data in the preceding buffer areas are respectively denoted as w1, w2, w3 … …, wn; the buffer current data segments are w1, w1w2w3, … …, w1w2 … … wn, respectively.
Preferably, the obtaining a plurality of dictionary current data segments of each buffer current data segment according to the buffer current data segments and the dictionary window includes the following specific steps:
for any buffer current data segment, the length of the buffer current data segment is recorded as a reference length; and sequentially taking the current data points in the dictionary window as starting points, acquiring a data segment with the length of a reference length, and recording the data segment as a dictionary current data segment of the buffer current data segment.
Preferably, the obtaining a plurality of current data pairs and the range variation trend difference value of each current data pair according to the range variation trend value, the buffer current data section and the dictionary current data section includes the following specific methods:
for any dictionary current data segment of any buffer current data segment, recording any current data in the dictionary current data segment as dictionary current data; in the buffer current data section, recording current data with the same sequence number as the dictionary current data as buffer current data, and recording the dictionary current data and the buffer current data as a current data pair;
and recording the absolute value of the difference value between the range variation trend value of the dictionary current data in the current data pair and the range variation trend value of the buffer current data as the range variation trend difference value of the current data pair.
Preferably, the obtaining the area similarity of each current data pair according to the range variation trend difference value includes the following specific steps:
for any pair of current data pairs, recording the absolute value of the difference value between dictionary current data and buffer current data in the current data pairs as a current data difference value of the current data pairs;
in the method, in the process of the invention,representing the regional similarity of current data pairs; />A range variation trend difference value representing a current data pair; />A current data difference value representing a current data pair; />An exponential function based on a natural constant is represented.
Preferably, the obtaining the dictionary current data segments of each buffer current data segment and the data segment compression necessity of each buffer current data segment according to the region similarity includes the following specific methods:
for any one of the dictionary current data pieces of any one of the buffer current data pieces, in the formula,representing compression necessity of dictionary current data segments and buffer current data segments; />Representing the minimum value of the range variation trend difference values of all current data pairs in the dictionary current data section and the buffer current data section; />Representing the maximum value of the range variation trend difference values of all current data pairs in the dictionary current data section and the buffer current data section; />A variance representing the regional similarity of all current data pairs in the dictionary current data segment and the buffer current data segment; />A mean value representing the regional similarity of all current data pairs in the dictionary current data section and the buffer current data section; and acquiring compression necessity of all dictionary current data segments and buffer current data segments of the buffer current data segments, carrying out linear normalization on all compression necessity, and marking each normalized compression necessity as data segment compression necessity.
Preferably, the final buffered current data segment of each current data record is obtained according to the data segment compression necessity, which comprises the following specific steps:
marking a preset data segment compression necessity threshold as T4; for any dictionary current data segment of any buffer current data segment on any current data record, if the compression necessity of the dictionary current data segment and the data segment of the buffer current data segment is greater than T4, marking the dictionary current data segment as a matched current data segment; and acquiring all the matching current data segments of the buffer current data segments, acquiring all the matching current data segments of each buffer current data segment, and recording the buffer current data segment with the largest number of the matching current data segments as the final buffer current data segment.
Preferably, the compression transmission is performed according to the final buffered current data segment, which comprises the following specific methods:
for a final buffer current data segment of any current data record, in all the matching current data segments of the final buffer current data segment, the matching current data segment with the greatest compression necessity with the data segment of the final buffer current data segment is recorded as the final matching current data segment, the final matching current data segment is replaced by the final buffer current data segment to obtain a replaced final buffer current data segment, and the replaced final buffer current data segment is compressed by using an LZ77 compression algorithm to obtain a compressed data segment on the current data record; and acquiring compressed data segments on all current data records, and transmitting the compressed data segments to a data center.
The technical scheme of the invention has the beneficial effects that: obtaining a range change trend value of current data according to a current data sequence, obtaining a dictionary current data segment and a buffer current data segment, obtaining a range change trend difference value according to the range change trend value, the dictionary current data segment and the buffer current data segment, obtaining region similarity of current data pairs according to the range change trend difference value, obtaining data segment compression necessity according to the region similarity, obtaining a final buffer current data segment, and carrying out compression transmission according to the final buffer current data segment; compared with the prior art, the self-adaptive compression method can not carry out self-adaptive compression according to random fluctuation generated by current data; the range change trend value reflects the trend of slope change of current data around the current data, the regional similarity reflects the change degree of current data around dictionary current data and buffer current data, the data segment compression necessity reflects the degree of substitution of the buffer current data segment, the compressible data volume is improved, the compression efficiency is improved, and the transmission efficiency of current element data is further improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of steps of a data center transmission optimization method based on cloud computing.
Detailed Description
In order to further describe the technical means and effects adopted by the present invention to achieve the preset purpose, the following detailed description refers to specific implementation, structure, characteristics and effects of the cloud computing-based data center transmission optimization method according to the present invention with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following specifically describes a specific scheme of the data center transmission optimization method based on cloud computing provided by the invention with reference to the accompanying drawings.
Referring to fig. 1, a flowchart illustrating steps of a data center transmission optimization method based on cloud computing according to an embodiment of the present invention is shown, where the method includes the following steps:
step S001: and collecting a current data sequence of the intelligent device.
It should be noted that, in the conventional compression processing method, the data segments repeatedly appearing in the current data are compressed by using the LZ77 compression algorithm, but because the current data randomly generate fluctuations with different degrees, the repeated data segments are fewer, so that the data compressed by the LZ77 compression algorithm is fewer, the compression efficiency is low, the data volume of the compressed current data is still larger, and the transmission efficiency of the current data is lower. Therefore, the embodiment provides a data center transmission optimization method based on cloud computing.
Specifically, in order to implement the data center transmission optimization method based on cloud computing provided in this embodiment, a current data sequence needs to be collected first, and the specific process is as follows: acquiring current data of a week in an intelligent device database; the current data are arranged according to the sequence from the early to the late of the recording time, and the arranged sequence is recorded as a current data sequence. Wherein the current data sequence comprises a plurality of current data.
Thus, a current data sequence is obtained by the method.
Step S002: obtaining a range variation trend value of each current data according to the current data sequence; and obtaining a plurality of current data records according to the current data sequence.
It should be noted that, in the conventional LZ77 compression algorithm, a data segment with the same content as the maximum length in the buffer advance area is searched in the dictionary window, and the data segment is not compressed as long as the data segments are different; the current data always generates random numerical fluctuation, and the overall trend shows certain regularity although the degree of the numerical fluctuation can irregularly fluctuate within a certain range. The range change trend value can be obtained by analyzing the trend change between the current data for subsequent analysis processing.
Specifically, the first in the current data sequenceThe current data are taken as examples, according to +.>The difference between the individual current data and the surrounding current data is obtained +.>A range change trend value of the individual current data; wherein->The calculation method of the range change trend value of the individual current data comprises the following steps:
in the method, in the process of the invention,indicate->Initial range trend value of individual current data, < >>；/>Indicate->-individual current data; />Indicate->-individual current data; />Indicate->-individual current data; />The representation takes absolute value. Wherein if%>The larger the initial range trend value of the individual current data, the description +.>The larger the current data change around the individual current data, reflecting +.>The more the slope of the current data varies around the individual current data, the more pronounced the trend. Obtaining initial range change trend values of all current data in a current data sequence, carrying out linear normalization on all initial range change trend values, and recording each normalized initial range change trend value as a range change trend value.
Further, a text window length T1, a dictionary window length T2, and a look-ahead buffer length T3 are preset, where t1=20, t2=15, and t3=5 are taken as examples in this embodiment, and the present embodiment is not limited specifically, where T1, T2, and T3 may be determined according to specific implementation situations; acquiring a text window with a text window length of T1; the window area with the length of the front T2 in the text window is marked as a dictionary window, and the window area with the length of the rear T3 in the text window is marked as a look-ahead buffer. Wherein each text window contains a dictionary window and a look-ahead buffer. It should be noted that the preset T1, T2, and T3 need to satisfy the functional relationship of t1=t2+t3.
Further, the first current data in the current data sequence is taken as a starting point, the step length is 1, the current data are sequentially input into text windows from right to left, the current data of the text windows are recorded at the same time, the current data are recorded as current data records, and a plurality of current data records are obtained until all the current data in the current data sequence are input. Wherein each current data record has a text window containing a plurality of current data.
So far, a plurality of current data records are obtained through the method.
Step S003: obtaining a plurality of buffer current data segments of each preceding buffer zone according to the current data record; obtaining a plurality of dictionary current data segments of each buffer current data segment according to the buffer current data segments and the dictionary windows; obtaining a plurality of current data pairs and range variation trend difference values of each current data pair according to the range variation trend values, the buffer current data segments and the dictionary current data segments; obtaining the regional similarity of each current data pair according to the range change trend difference value; and obtaining a plurality of dictionary current data segments of each buffer current data segment and the data segment compression necessity of each buffer current data segment according to the region similarity.
It should be noted that, since the overall variation trend of the data segment is represented by the combined action of the plurality of current data in the data segment, for two data segments with similar variation trend, there is a certain similarity between the current data at corresponding positions in the two data segments, and there is a similarity between the current data at corresponding positions in different local data segments formed by the current data at different positions; for two local data segments with higher similarity, the data characteristics of the two data segments are not very different, so that the partial data lost by the compressed data segment obtained by the replacement compression process can be ignored.
Specifically, taking any one of current data records as an example, on the current data records, current data in the advance buffer areas are respectively denoted as w1, w2, w3 … …, wn; the buffer current data segments are w1, w1w2w3, … … and w1w2 … … wn respectively; taking any buffer current data segment as an example, and recording the length of the buffer current data segment as a reference length; and sequentially taking the current data points in the dictionary window as starting points, acquiring a data segment with the length of a reference length, and recording the data segment as a dictionary current data segment of the buffer current data segment. Each preceding buffer zone contains a plurality of buffer current data segments, and each buffer current data segment corresponds to a plurality of dictionary current data segments.
Further, taking any one dictionary current data segment of any one buffer current data segment as an example, and recording any one current data in the dictionary current data segment as dictionary current data; in the buffer current data section, recording current data with the same sequence number as the dictionary current data as buffer current data, and recording the dictionary current data and the buffer current data as a current data pair; recording the absolute value of the difference value between the range variation trend value of the dictionary current data in the current data pair and the range variation trend value of the buffer current data as the range variation trend difference value of the current data pair; recording the absolute value of the difference value between the dictionary current data and the buffer current data in the current data pair as a current data difference value of the current data pair; and obtaining the regional similarity of the current data pair according to the range variation trend difference value and the current data difference value of the current data pair. The dictionary current data section and the buffer current data section comprise a plurality of current data pairs, and each current data pair comprises dictionary current data and buffer current data; the method for calculating the regional similarity of the current data pair comprises the following steps:
in the method, in the process of the invention,representing the regional similarity of the current data pairs; />A range variation trend difference value representing the current data pair; />A current data difference value representing the current data pair; />An exponential function that is based on a natural constant; example use->The functions are used for representing inverse proportion relation and normalization processing, and an implementer can select inverse according to actual conditionsA scaling function and a normalization function. Wherein if the area similarity between the dictionary current data and the buffer current data is larger, the difference between the dictionary current data and the buffer current data is smaller, and the current data changes around the dictionary current data and the buffer current data are reflected to be similar. And obtaining the regional similarity of the dictionary current data section and all current data pairs in the buffer current data section.
Further, the compression necessity of the dictionary current data segment and the buffer current data segment is obtained according to the regional similarity of all current data pairs in the dictionary current data segment and the buffer current data segment. The method for calculating the compression necessity of the dictionary current data segment and the buffer current data segment comprises the following steps:
in the method, in the process of the invention,representing the compression necessity of the dictionary current data section and the buffer current data section; />Representing the minimum value of the range variation trend difference values of all current data pairs in the dictionary current data section and the buffer current data section; />Representing the maximum value of the range variation trend difference values of all current data pairs in the dictionary current data section and the buffer current data section; />A variance representing the regional similarity of all current data pairs in the dictionary current data segment and the buffer current data segment; />Represented in the dictionary current data section and the buffer current data section, allThe mean of the regional similarity of the current data pairs. The greater the compression necessity of the dictionary current data segment and the buffer current data segment, the greater the possibility that the buffer current data segment can be replaced by the dictionary current data segment, and the lower the difficulty of decompression after compression of the buffer current data segment is reflected. Acquiring all dictionary current data segments of the buffer current data segment and compression necessity of the buffer current data segment, carrying out linear normalization on all compression necessity, and marking each normalized compression necessity as data segment compression necessity; and acquiring all dictionary current data segments of each buffer current data segment and data segment compression necessity of each buffer current data segment on the current data record.
So far, all dictionary current data segments of each buffer current data segment and the data segment compression necessity of each buffer current data segment on the current data record are obtained through the method.
Step S004: obtaining a final buffer current data segment of each current data record according to the data segment compression necessity; and carrying out compression transmission according to the final buffer current data segment.
Specifically, a data segment compression necessity threshold T4 is preset, where the embodiment is described by taking t4=0.7 as an example, and the embodiment is not specifically limited, where T4 may be determined according to the specific implementation situation; taking any dictionary current data segment of any buffer current data segment on the current data record as an example, if the compression necessity of the dictionary current data segment and the data segment of the buffer current data segment is greater than T4, marking the dictionary current data segment as a matched current data segment; and acquiring all the matching current data segments of the buffer current data segments, acquiring all the matching current data segments of each buffer current data segment, and recording the buffer current data segment with the largest number of the matching current data segments as the final buffer current data segment.
Further, in all the matched current data segments of the final buffer current data segment, the matched current data segment with the greatest compression necessity of the data segment of the final buffer current data segment is recorded as the final matched current data segment, the final matched current data segment is replaced by the final buffer current data segment to obtain a replaced final buffer current data segment, and the LZ77 compression algorithm is utilized to compress the replaced final buffer current data segment to obtain a compressed data segment on the current data record; and acquiring compressed data segments on all current data records, transmitting the compressed data segments to a data center, and decompressing the compressed data segments to obtain data segments before compression when analysis is needed. The process of compressing the data segment is well known in the LZ77 compression algorithm, and this embodiment will not be described.
This embodiment is completed.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the invention, but any modifications, equivalent substitutions, improvements, etc. within the principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. The data center transmission optimization method based on cloud computing is characterized by comprising the following steps of:
collecting a current data sequence of the intelligent equipment, wherein the current data sequence comprises a plurality of current data;
obtaining a range variation trend value of each current data according to the current data sequence; obtaining a plurality of current data records according to the current data sequence, wherein the current data records comprise a dictionary window and a look-ahead buffer area;
obtaining a plurality of buffer current data segments of each preceding buffer zone according to the current data record; obtaining a plurality of dictionary current data segments of each buffer current data segment according to the buffer current data segments and the dictionary windows; obtaining a plurality of current data pairs and range variation trend difference values of each current data pair according to the range variation trend values, the buffer current data segments and the dictionary current data segments; obtaining the regional similarity of each current data pair according to the range change trend difference value; obtaining a plurality of dictionary current data segments of each buffer current data segment and data segment compression necessity of each buffer current data segment according to the region similarity;
obtaining a final buffer current data segment of each current data record according to the data segment compression necessity; and carrying out compression transmission according to the final buffer current data segment.
2. The cloud computing-based data center transmission optimization method according to claim 1, wherein the obtaining the range change trend value of each current data according to the current data sequence comprises the following specific steps:
in the method, in the process of the invention,indicate->Initial range trend value of individual current data, < >>；/>Indicate->-individual current data; />Indicate->-individual current data; />Indicate->-individual current data; />The representation takes absolute value; obtaining initial range change trend values of all current data in a current data sequence, carrying out linear normalization on all initial range change trend values, and recording each normalized initial range change trend value as a range change trend value.
3. The cloud computing-based data center transmission optimization method according to claim 1, wherein the obtaining a plurality of current data records according to a current data sequence comprises the following specific methods:
respectively marking the preset text window length, dictionary window length and look-ahead buffer area length as T1, T2 and T3; wherein t1=t2+t3, obtaining a text window with a text window length of T1; the window area with the length of the front T2 in the text window is marked as a dictionary window, and the window area with the length of the rear T3 in the text window is marked as a look-ahead buffer area;
and for the first current data in the current data sequence as a starting point, the step length is 1, sequentially inputting the current data into text windows in the order from right to left, simultaneously recording the current data of the text windows once, and recording the current data as current data record until all the current data in the current data sequence are input.
4. The cloud computing-based data center transmission optimization method according to claim 1, wherein the obtaining a plurality of buffered current data segments of each look-ahead buffer according to the current data record comprises the following specific steps:
for any one of the current data records, the current data in the preceding buffer areas are respectively denoted as w1, w2, w3 … …, wn; the buffer current data segments are w1, w1w2w3, … …, w1w2 … … wn, respectively.
5. The cloud computing-based data center transmission optimization method according to claim 1, wherein the obtaining a plurality of dictionary current data segments of each buffer current data segment according to the buffer current data segments and the dictionary window comprises the following specific steps:
for any buffer current data segment, the length of the buffer current data segment is recorded as a reference length; and sequentially taking the current data points in the dictionary window as starting points, acquiring a data segment with the length of a reference length, and recording the data segment as a dictionary current data segment of the buffer current data segment.
6. The cloud computing-based data center transmission optimization method according to claim 1, wherein the obtaining a plurality of current data pairs and range variation trend difference values of each current data pair according to the range variation trend values, the buffer current data segments and the dictionary current data segments comprises the following specific methods:
for any dictionary current data segment of any buffer current data segment, recording any current data in the dictionary current data segment as dictionary current data; in the buffer current data section, recording current data with the same sequence number as the dictionary current data as buffer current data, and recording the dictionary current data and the buffer current data as a current data pair;
and recording the absolute value of the difference value between the range variation trend value of the dictionary current data in the current data pair and the range variation trend value of the buffer current data as the range variation trend difference value of the current data pair.
7. The cloud computing-based data center transmission optimization method according to claim 1, wherein the obtaining the regional similarity of each current data pair according to the range variation trend difference value comprises the following specific steps:
for any pair of current data pairs, recording the absolute value of the difference value between dictionary current data and buffer current data in the current data pairs as a current data difference value of the current data pairs;
in the method, in the process of the invention,representing the regional similarity of current data pairs; />A range variation trend difference value representing a current data pair;a current data difference value representing a current data pair; />An exponential function based on a natural constant is represented.
8. The cloud computing-based data center transmission optimization method according to claim 1, wherein the obtaining the dictionary current data segments of each buffer current data segment and the data segment compression necessity of each buffer current data segment according to the regional similarity comprises the following specific steps:
for any one of the dictionary current data pieces of any one of the buffer current data pieces, in the formula,representing compression necessity of dictionary current data segments and buffer current data segments; />Representing the minimum value of the range variation trend difference values of all current data pairs in the dictionary current data section and the buffer current data section; />Expressed in dictionary electricityThe maximum value of range variation trend difference values of all current data pairs in the current data section and the buffer current data section; />A variance representing the regional similarity of all current data pairs in the dictionary current data segment and the buffer current data segment; />A mean value representing the regional similarity of all current data pairs in the dictionary current data section and the buffer current data section; and acquiring compression necessity of all dictionary current data segments and buffer current data segments of the buffer current data segments, carrying out linear normalization on all compression necessity, and marking each normalized compression necessity as data segment compression necessity.
9. The cloud computing-based data center transmission optimization method according to claim 1, wherein the obtaining the final buffered current data segment of each current data record according to the data segment compression necessity comprises the following specific steps:
marking a preset data segment compression necessity threshold as T4; for any dictionary current data segment of any buffer current data segment on any current data record, if the compression necessity of the dictionary current data segment and the data segment of the buffer current data segment is greater than T4, marking the dictionary current data segment as a matched current data segment; and acquiring all the matching current data segments of the buffer current data segments, acquiring all the matching current data segments of each buffer current data segment, and recording the buffer current data segment with the largest number of the matching current data segments as the final buffer current data segment.
10. The cloud computing-based data center transmission optimization method according to claim 9, wherein the compression transmission is performed according to the final buffer current data segment, comprising the following specific steps:
for a final buffer current data segment of any current data record, in all the matching current data segments of the final buffer current data segment, the matching current data segment with the greatest compression necessity with the data segment of the final buffer current data segment is recorded as the final matching current data segment, the final matching current data segment is replaced by the final buffer current data segment to obtain a replaced final buffer current data segment, and the replaced final buffer current data segment is compressed by using an LZ77 compression algorithm to obtain a compressed data segment on the current data record; and acquiring compressed data segments on all current data records, and transmitting the compressed data segments to a data center.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311344747.4A CN117097810B (en) | 2023-10-18 | 2023-10-18 | Data center transmission optimization method based on cloud computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311344747.4A CN117097810B (en) | 2023-10-18 | 2023-10-18 | Data center transmission optimization method based on cloud computing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117097810A true CN117097810A (en) | 2023-11-21 |
| CN117097810B CN117097810B (en) | 2024-01-02 |
Family
ID=88775360
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311344747.4A Active CN117097810B (en) | 2023-10-18 | 2023-10-18 | Data center transmission optimization method based on cloud computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117097810B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117316301A (en) * | 2023-11-22 | 2023-12-29 | 北华大学 | Intelligent compression processing method for gene detection data |
| CN117349778A (en) * | 2023-12-04 | 2024-01-05 | 湖南蓝绿光电科技有限公司 | Online real-time monitoring system of consumer based on thing networking |
| CN117473351A (en) * | 2023-12-26 | 2024-01-30 | 深圳市胜威南方科技有限公司 | Power supply information remote transmission system based on Internet of things |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2001063934A1 (en) * | 2000-02-22 | 2001-08-30 | Infineon Technologies Ag | Method for compressing data |
| WO2013136935A1 (en) * | 2012-03-13 | 2013-09-19 | インフォメティス株式会社 | Sensor, sensor signal processor, and power line signal encoder |
| CN106656200A (en) * | 2016-12-13 | 2017-05-10 | 合肥工业大学 | Program counter compression method and hardware circuit thereof |
| CN115359807A (en) * | 2022-10-21 | 2022-11-18 | 金叶仪器(山东)有限公司 | Noise online monitoring system for urban noise pollution |
| CN115834720A (en) * | 2022-09-09 | 2023-03-21 | 国网福建省电力有限公司营销服务中心 | Data compression method for photovoltaic communication data |
| CN116455941A (en) * | 2023-04-28 | 2023-07-18 | 中国通信建设集团设计院有限公司 | Indoor environment multi-source data transmission method and system based on Internet of things |
| CN116707534A (en) * | 2023-05-23 | 2023-09-05 | 河南正宇电气有限公司 | Electric power metering box data processing system based on internet of things technology |
-
2023
- 2023-10-18 CN CN202311344747.4A patent/CN117097810B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2001063934A1 (en) * | 2000-02-22 | 2001-08-30 | Infineon Technologies Ag | Method for compressing data |
| WO2013136935A1 (en) * | 2012-03-13 | 2013-09-19 | インフォメティス株式会社 | Sensor, sensor signal processor, and power line signal encoder |
| CN106656200A (en) * | 2016-12-13 | 2017-05-10 | 合肥工业大学 | Program counter compression method and hardware circuit thereof |
| CN115834720A (en) * | 2022-09-09 | 2023-03-21 | 国网福建省电力有限公司营销服务中心 | Data compression method for photovoltaic communication data |
| CN115359807A (en) * | 2022-10-21 | 2022-11-18 | 金叶仪器(山东)有限公司 | Noise online monitoring system for urban noise pollution |
| CN116455941A (en) * | 2023-04-28 | 2023-07-18 | 中国通信建设集团设计院有限公司 | Indoor environment multi-source data transmission method and system based on Internet of things |
| CN116707534A (en) * | 2023-05-23 | 2023-09-05 | 河南正宇电气有限公司 | Electric power metering box data processing system based on internet of things technology |
Non-Patent Citations (1)
| Title |
|---|
| 吴科永;陈东;辛宁;曹桂兴;: "压缩感知和相似性约束的图像超分辨率重构算法", 计算机应用研究, no. 05 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117316301A (en) * | 2023-11-22 | 2023-12-29 | 北华大学 | Intelligent compression processing method for gene detection data |
| CN117316301B (en) * | 2023-11-22 | 2024-03-05 | 北华大学 | Intelligent compression processing method for gene detection data |
| CN117349778A (en) * | 2023-12-04 | 2024-01-05 | 湖南蓝绿光电科技有限公司 | Online real-time monitoring system of consumer based on thing networking |
| CN117349778B (en) * | 2023-12-04 | 2024-02-20 | 湖南蓝绿光电科技有限公司 | Online real-time monitoring system of consumer based on thing networking |
| CN117473351A (en) * | 2023-12-26 | 2024-01-30 | 深圳市胜威南方科技有限公司 | Power supply information remote transmission system based on Internet of things |
| CN117473351B (en) * | 2023-12-26 | 2024-03-22 | 深圳市胜威南方科技有限公司 | Power supply information remote transmission system based on Internet of things |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117097810B (en) | 2024-01-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117097810B (en) | Data center transmission optimization method based on cloud computing | |
| CN102143039B (en) | Data segmentation method and equipment for data compression | |
| CN115204754B (en) | Heating power supply and demand information management platform based on big data | |
| US20200294629A1 (en) | Gene sequencing data compression method and decompression method, system and computer-readable medium | |
| CN115269526B (en) | Method and system for processing semiconductor production data | |
| CN115801902B (en) | Compression method of network access request data | |
| CN117290364B (en) | Intelligent market investigation data storage method | |
| CN115858476A (en) | Efficient storage method for user-defined form acquisition data in web development system | |
| CN115567589B (en) | Compression transmission method, device and equipment of JSON data and storage medium | |
| CN116610265B (en) | Data storage method of business information consultation system | |
| CN117014519B (en) | Data transmission method and intelligent hospital transmission system | |
| CN115543946A (en) | Financial big data optimized storage method | |
| CN116861041A (en) | Electronic document processing method and system | |
| CN117376430B (en) | Industrial data rapid transmission method and system based on DCS | |
| CN113590818B (en) | Government text data classification method based on integration of CNN (carbon fiber network), GRU (grid-like network) and KNN (K-nearest neighbor network) | |
| CN109687875A (en) | A kind of time series data processing method | |
| CN116318172A (en) | Design simulation software data self-adaptive compression method | |
| CN111479286A (en) | Data processing method for reducing communication flow of edge computing system | |
| US5708429A (en) | Method of compressing electroencephalographic signals | |
| CN118018033B (en) | Intelligent compression transmission method for motor performance data | |
| CN117221414B (en) | Intelligent data transmission method for modem | |
| CN117880767B (en) | Short message data transmission method with low delay | |
| Raju et al. | Compression of wearable body sensor network data | |
| CN116896769B (en) | Optimized transmission method for motorcycle Bluetooth sound data | |
| CN108449163A (en) | A kind of polarization code method for analyzing performance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |