CN117411947B - Cloud edge cooperation-based water service data rapid transmission method - Google Patents
Cloud edge cooperation-based water service data rapid transmission method Download PDFInfo
- Publication number
- CN117411947B CN117411947B CN202311724653.XA CN202311724653A CN117411947B CN 117411947 B CN117411947 B CN 117411947B CN 202311724653 A CN202311724653 A CN 202311724653A CN 117411947 B CN117411947 B CN 117411947B
- Authority
- CN
- China
- Prior art keywords
- data
- segment
- water flow
- section
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 196
- 238000000034 method Methods 0.000 title claims abstract description 37
- 230000005540 biological transmission Effects 0.000 title claims abstract description 21
- 230000003044 adaptive effect Effects 0.000 claims abstract description 24
- 238000013144 data compression Methods 0.000 claims abstract description 16
- 238000003064 k means clustering Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 description 7
- 238000007906 compression Methods 0.000 description 6
- 230000006835 compression Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 235000005607 chanvre indien Nutrition 0.000 description 3
- 244000261228 chanvre indien Species 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010224 classification analysis Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000007907 direct compression Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention relates to the technical field of digital information transmission, in particular to a cloud-edge cooperation-based water service data rapid transmission method. The method comprises the following steps: acquiring water flow data and determining the overall fluctuation degree of the water flow data; constructing a coordinate system, determining a water service coordinate, and dividing water flow data into segment data of at least two time segments based on the water service coordinate; determining flow rate increase coefficients of all segments of data in each time period; determining a segment fluctuation degree of each time segment; determining self-adaptive coding parameters by combining the mean value, the flow increase coefficient, the section fluctuation degree and the overall fluctuation degree of the section data; performing Columbus coding processing on the corresponding segment data according to the adaptive coding parameters to obtain coded data; and carrying out data compression on the encoded data to obtain compressed data, and transmitting the compressed data based on a cloud edge data pipeline. In summary, the invention can effectively reduce the storage space occupied by the water flow data and improve the efficiency of transmitting the water flow data.
Description
Technical Field
The invention relates to the technical field of digital information transmission, in particular to a cloud-edge cooperation-based water service data rapid transmission method.
Background
The water flow data is water service data information for estimating the river flow, is an important reference basis for decisions such as water service planning, equipment investment, resource allocation and the like, and is limited in transmission bandwidth, limited in storage resources, delayed in transmission and the like due to long duration and huge data volume of the water flow data to be counted.
In the related art, the Columbus code is used for carrying out data coding processing on the water flow data so as to improve the stability of transmission, under the mode, because the overall fluctuation of the water flow data is larger, for example, the water flow data of different rivers change greatly in different time periods, the Columbus code transmission mode can lead to the compression of smaller data and larger data by using the same coding parameter, and the smaller data is finally coded into longer codes, namely, under the mode, the storage space occupied by the water flow data is larger, and the efficiency of transmitting the water flow data is insufficient.
Disclosure of Invention
In order to solve the technical problems of large storage space occupied by water flow data and insufficient efficiency of transmitting the water flow data in the related art, the invention provides a cloud edge cooperation-based water service data rapid transmission method, which adopts the following technical scheme:
the invention provides a cloud edge cooperation-based water service data rapid transmission method, which comprises the following steps:
acquiring water flow data at different time points in time sequence, and determining the overall fluctuation degree of the water flow data according to the numerical values of all the water flow data;
constructing a coordinate system by taking time points as abscissa and the numerical value of water flow data as ordinate, wherein the water flow data of different time points correspond to water service coordinates, clustering the water flow data of all time points according to all water service coordinates, and dividing the water flow data of all time points into segment data of at least two time segments;
performing straight line fitting on water service coordinates of all the section data of each time period, and determining flow increase coefficients of all the section data of each time period; determining the section fluctuation degree of each time section according to the numerical value of all section data of each time section; according to the average value, the flow increase coefficient, the section fluctuation degree and the overall fluctuation degree of all section data of each time section, determining the self-adaptive coding parameters of all section data of each time section;
performing Columbus coding processing on the corresponding segment data according to the adaptive coding parameters to obtain coded data; and carrying out data compression on the encoded data to obtain compressed data, and transmitting the compressed data based on a cloud edge data pipeline.
Further, the determining the overall fluctuation degree of the water flow data according to the values of all the water flow data comprises the following steps:
and calculating a normalized value of the variance of the numerical values of all the water flow data as the overall fluctuation degree of the water flow data.
Further, the clustering the water flow data at all time points according to all water affair coordinates divides the water flow data at all time points into segment data of at least two time segments, including:
determining a preset K value based on an elbow method, and performing K-Means clustering on all the water service coordinates based on a K-Means clustering algorithm and the preset K value to obtain a cluster;
dividing the water flow data into segment data of at least two time segments according to the clustering clusters.
Further, the dividing the water flow data into segment data of at least two time segments according to the cluster includes:
and determining end point values of the abscissa values of all water service coordinates in each cluster, and dividing the water flow data into segment data of at least two time segments according to the end point values.
Further, the linear fitting is performed on the water affair coordinates of all the section data of each time period, and the determining of the flow rate increase coefficient of all the section data of each time period includes:
performing linear fitting on water affair coordinates of all the section data of each time period based on a linear fitting algorithm, and determining fitting lines of all the section data of each time period;
and carrying out normalization processing on the absolute value of the slope of the fitting straight line, and taking the absolute value as a flow rate increase coefficient of all the section data of the corresponding time period.
Further, the determining the segment fluctuation degree of each time segment according to the values of all segment data of each time segment includes:
the variance of the values of all the segment data for each time period is calculated as the degree of the segment fluctuation for each time period.
Further, the determining the adaptive coding parameters of all the segment data of each time segment according to the average value of all the segment data of each time segment, the flow rate increase coefficient, the segment fluctuation degree and the overall fluctuation degree comprises the following steps:
calculating the average value of all the segment data in each time period, and calculating the product of the average value and a preset adjustment coefficient as a segment parameter index;
calculating an inverse proportion normalization value of the product of the flow increase coefficient, the section fluctuation degree and the overall fluctuation degree as a parameter adjustment index;
and determining the self-adaptive coding parameters of all the segment data in the time period according to the segment parameter index and the parameter adjustment index.
Further, the determining the adaptive coding parameters of all the segment data in the time period according to the segment parameter index and the parameter adjustment index includes:
and calculating the product of the segment parameter index and the parameter adjustment index as the adaptive coding parameter of the corresponding segment data.
Further, the performing golomb coding processing on the corresponding segment data according to the adaptive coding parameters to obtain coded data includes:
based on a Columbus coding algorithm, the adaptive coding parameter is used as a Columbus coding parameter, the Columbus coding processing is carried out on the corresponding segment data, all the segment data are traversed, and the coded data are recombined according to the time sequence.
Further, the data compression of the encoded data to obtain compressed data includes:
and carrying out data compression on the coded data based on an LZ77 algorithm to obtain compressed data.
The invention has the following beneficial effects:
the method is applied to the technical field of digital information transmission, and by acquiring the water flow data at different time points in time sequence, the overall fluctuation degree of the water flow data is determined according to the numerical values of all the water flow data, and the overall fluctuation degree characterizes the overall fluctuation condition of the water flow data, so that the overall change of the water flow data is conveniently analyzed; then, because the overall change of the water flow data is often larger in reality, for the convenience of analysis, segmenting the water flow data, establishing a coordinate system and determining water service coordinates, and clustering divides the water flow data into segment data, so that each segment of data can be respectively subjected to data coding in the follow-up process, the self-adaptive coding can be realized according to the change of the water flow data, and the coding effect is improved; the method comprises the steps of obtaining a flow increase coefficient, a section fluctuation degree, a mean value and an overall fluctuation degree of section data, and obtaining self-adaptive coding parameters, wherein the self-adaptive coding parameters are coding parameters obtained by analyzing by combining the numerical condition of each section of data with the characteristics of the flow increase coefficient, the section fluctuation degree, the overall fluctuation degree and the like, and performing Columbus coding processing on the section data according to the self-adaptive coding parameters to obtain coded data; the method comprises the steps of carrying out data compression on encoded data to obtain compressed data, transmitting the compressed data based on cloud edge data pipelines, carrying out Columbus encoding through self-adaptive encoding parameters, then carrying out data compression, carrying out self-adaptive encoding compression according to the change degree, simultaneously effectively shortening the whole storage space, and comprehensively, effectively reducing the storage space occupied by water flow data and improving the efficiency of transmitting the water flow data.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for quickly transmitting water service data based on cloud-edge collaboration according to an embodiment of the present invention.
Detailed Description
In order to further describe the technical means and effects adopted by the invention to achieve the preset aim, the following is a detailed description of specific implementation, structure, characteristics and effects thereof based on cloud-edge cooperation of the water affair data rapid transmission method according to the invention with reference to the accompanying drawings and the preferred embodiment. In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The invention provides a cloud-edge cooperation-based water affair data rapid transmission method and a specific scheme thereof, which are specifically described below with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of a method for quickly transmitting water service data based on cloud-edge collaboration according to an embodiment of the present invention is shown, where the method includes:
s101: acquiring water flow data at different time points in time sequence, and determining the overall fluctuation degree of the water flow data according to the numerical values of all the water flow data.
The specific application scenario of the embodiment of the invention is that the water flow data is obtained, and then, the water flow data is subjected to data coding and compression so as to improve the transmission efficiency of the water flow data, and it can be understood that in the conventional water flow data statistics process, the water flow data is usually represented by a positive integer due to the fact that the water flow is larger and the sensitivity to the water flow is lower, and the water flow is greatly changed due to the fact that the water flow is influenced by weather, time, human factors and other aspects, so that the compression effect of the traditional Columbus coding mode is poor.
In the embodiment of the invention, in the process of acquiring water flow data, water flow can be acquired by using flow metering equipment, corresponding water flow is represented by using an integer form, and common flow metering equipment comprises vortex shedding flowmeter, electromagnetic flowmeter, instantaneous flowmeter and the like. That is, the present invention periodically obtains the water flow data at different time points by using the corresponding flowmeter, for example, obtain the water flow data every 1 hour, which is not limited.
In the embodiment of the invention, the overall fluctuation degree can be determined by combining the acquired numerical values of all water flow data, wherein the overall fluctuation degree is the fluctuation condition of the water flow in the corresponding time period, namely the speed condition of water flow change.
Further, in some embodiments of the present invention, determining the overall extent of fluctuation of the water flow data from the values of all the water flow data includes: and calculating a normalized value of the variance of the numerical values of all the water flow data as the overall fluctuation degree of the water flow data. According to the embodiment of the invention, the overall fluctuation degree of the water flow data is determined in a variance form, when the variance value is larger, the corresponding water flow change is more intense, namely, the overall fluctuation degree is higher, and when the variance value is smaller, the corresponding water flow change is flatter, namely, the overall fluctuation degree is lower.
After the overall analysis of the water flow data is carried out, the invention can carry out local analysis in stages according to the change condition of the water flow in different stages, and the specific process of the local analysis is referred to the subsequent embodiment.
S102: and constructing a coordinate system by taking the time points as the abscissa and the numerical value of the water flow data as the ordinate, wherein the water flow data of different time points correspond to the water service coordinates, clustering the water flow data of all time points according to all the water service coordinates, and dividing the water flow data of all the time points into segment data of at least two time segments.
In the embodiment of the invention, the water flow data at different time points can be divided according to the corresponding water flow distribution condition to obtain segment data corresponding to a plurality of time segments, and the segment data can be processed in a clustering mode in the dividing process.
Further, in some embodiments of the present invention, the water flow data at all time points are clustered according to all water affair coordinates, and the water flow data at all time points are divided into segment data of at least two time segments, including: determining a preset K value based on an elbow method, and performing K-Means clustering on all water coordinates based on a K-Means clustering algorithm and the preset K value to obtain a cluster; dividing the water flow data into segment data of at least two time segments according to the cluster.
The elbow method is an existing method for confirming clustering k values, and because the water flow changes in different time periods, namely the segmentation conditions of the water flow in different water flow data are different, the fixed preset k values cannot be simply used for direct analysis, so that the accurate and more objective preset k values are confirmed through the elbow method, and then clustering processing is carried out based on the preset k values.
The K-Means clustering algorithm is a clustering algorithm well known to those skilled in the art, based on a preset K value, K-Means clustering processing can be performed on all water service coordinates by using the K-Means clustering algorithm to obtain clusters, that is, water service coordinates are combined into one cluster in a similar distance, and water flow data corresponding to each cluster can be characterized as being in the same water flow stage at adjacent moments.
In the embodiment of the invention, the water service data is clustered based on the K-Means clustering algorithm to obtain the cluster, that is, the corresponding cluster can divide the water flow data into data.
Further, in some embodiments of the present invention, dividing the water flow data into segment data of at least two time segments according to the cluster includes: and determining end point values of the abscissa values of all the water service coordinates in each cluster, and dividing the water flow data into segment data of at least two time segments according to the end point values.
It can be understood that, since the water flow changes linearly, that is, the probability of abrupt change of the data value is low in the normal water flow changing process, all water coordinates corresponding to each cluster obtained in the embodiment of the present invention should be continuous in time sequence, and the embodiment of the present invention uses the end points of the abscissa values of all water coordinates in each cluster as the end points of the segments, and divides the water flow data into segment data of at least two time segments according to the end point values. Wherein, because the abscissa is time, namely the segment data is the water flow data corresponding to a continuous time segment. The time period is a sub-segment of the overall water flow time period.
For example, when the time point corresponding to the water flow data is 1-10 and the end point values of the cluster are 3 and 7, respectively, the water flow data can be divided into three time periods { 1-3 }, { 4-7 }, and { 8-10 }, and the water flow data corresponding to each time period is used as the segment data of the same time point.
The embodiment of the invention uses a clustering mode to segment, so that the water flow data can be effectively segmented according to the characteristic of smooth linearity of the water flow change, different water flow periods can be accurately divided, the water flow data analysis can be ensured to carry out classification analysis according to the data value of the water flow data, and the direct compression mode is avoided.
S103: performing straight line fitting on water service coordinates of all the section data of each time period, and determining flow increase coefficients of all the section data of each time period; determining the section fluctuation degree of each time section according to the numerical value of all section data of each time section; and determining the self-adaptive coding parameters of all the section data of each time period according to the average value, the flow increase coefficient, the section fluctuation degree and the overall fluctuation degree of all the section data of each time period.
In the embodiment of the invention, in order to analyze the distribution characteristics of the segment data, indexes such as a flow increase coefficient, a segment fluctuation degree, a mean value of the segment data and the like are quoted for self-adaptive coding determination.
Further, in some embodiments of the present invention, performing straight line fitting on water coordinates of all segments of data of each time period to determine a flow rate increase coefficient of all segments of data of each time period, including: performing linear fitting on water affair coordinates of all the section data of each time period based on a linear fitting algorithm, and determining fitting lines of all the section data of each time period; and carrying out normalization processing on the absolute value of the slope of the fitting straight line to serve as a flow rate increase coefficient of all the section data of the corresponding time period.
It can be understood that the segment data of each time segment can represent water flow data under the condition that water flow is similar, and the situation can be specifically such as various situations of water flow increase, water flow decrease, water flow maintenance, and the like. And the slope of the fitting straight line can represent the change trend of the corresponding segment data.
In the embodiment of the invention, when the absolute value of the slope of the fitting straight line is larger, the larger the change of the water flow data is indicated, namely the larger the corresponding water flow is influenced by external force, at the moment, the higher the change degree of the data is, namely the data numerical value difference among the data is larger, so that the slope of the fitting straight line is used as the flow increase coefficient by normalization in the embodiment of the invention.
Further, in some embodiments of the present invention, determining the degree of segment fluctuation for each time segment based on the values of all segment data for each time segment includes: the variance of the values of all the segment data for each time period is calculated as the degree of the segment fluctuation for each time period.
The invention also takes the numerical variance of all the section data of each time section as the section fluctuation degree of each time section, similar to the calculation of the integral fluctuation degree, and it can be understood that the section fluctuation degree represents the fluctuation degree of the data of each time section, namely the water flow change condition of the water flow at different time points.
Thus, after determining the flow rate increase coefficient, the section fluctuation degree, and the overall fluctuation degree, the coding condition of the section data in each time section can be adjusted in combination with the obtained analysis result.
Further, in some embodiments of the present invention, determining adaptive coding parameters for all segment data of each time segment according to the average value, the flow rate increase coefficient, the segment fluctuation degree, and the overall fluctuation degree of all segment data of each time segment includes: calculating the average value of all the segment data in each time period, and calculating the product of the average value and a preset adjustment coefficient as a segment parameter index; calculating an inverse proportion normalization value of the product of the flow increase coefficient, the section fluctuation degree and the integral fluctuation degree as a parameter adjustment index; and determining the self-adaptive coding parameters of all the segment data in the time period according to the segment parameter index and the parameter adjustment index.
The preset adjustment coefficient may be, for example, specifically 0.5, or may be adjusted according to the actual detection situation, which is not limited to this.
In the embodiment of the invention, the average value of all the segment data in each time period is calculated, the average value can specifically represent the overall condition of the segment data, the product of the average value and the preset adjustment coefficient is calculated as the segment parameter index, and then the segment parameter index can be adjusted.
The inverse proportion normalized value of the product of the flow rate increase coefficient, the section fluctuation degree and the integral fluctuation degree is calculated as a parameter adjustment index, and it can be understood that the larger the flow rate increase coefficient and the section fluctuation degree is, the higher the change degree of all section data of the corresponding time section can be represented, and the integral fluctuation degree table is used for sorting the change degree of the corresponding water flow data, the larger the parameter adjustment index is, and the lower the change degree of the section data of the corresponding time section is represented.
It can be understood that in the embodiment of the present invention, a golomb coding algorithm is required to be used for data coding, and the water flow data is usually data with a larger value, and the value is usually between 100 and 3000, and specific data is cited for explanation, in the water flow statistics process, at 9 am of a certain day, the water flow data of Ganja in the Jiang state is 1220 cubic meters per second, and the water flow data of Ganja in the Ganja is 184 cubic meters per second, that is, when the encoding statistics is performed by using the golomb coding, the fixed coding parameters can cause the coding gap of different water flow degrees to be too large, and the corresponding transmission efficiency is poor.
Further, in some embodiments of the present invention, determining adaptive coding parameters of all segment data in a time period according to a segment parameter index and a parameter adjustment index includes: and calculating the product of the segment parameter index and the parameter adjustment index as the adaptive coding parameter of the corresponding segment data.
In the embodiment of the invention, the segment parameter index and the parameter adjustment index are in positive correlation with the adaptive coding parameter, that is, when the variation degree of all the segment data in the time period is higher, the corresponding adaptive coding parameter may be smaller, and when the variation degree of all the segment data in the time period is lower, the corresponding adaptive coding parameter may be larger, so that the adaptive coding parameter is adjusted according to the variation degree and the segment parameter index.
S104: performing Columbus coding processing on the corresponding segment data according to the adaptive coding parameters to obtain coded data; and carrying out data compression on the encoded data to obtain compressed data, and transmitting the compressed data based on a cloud edge data pipeline.
In the embodiment of the present invention, the data is encoded by using a golomb coding algorithm, and further, in some embodiments of the present invention, the corresponding segment data is respectively encoded according to adaptive coding parameters to obtain encoded data, which includes: based on a Columbus coding algorithm, taking the adaptive coding parameter as the Columbus coding parameter, performing Columbus coding processing on the corresponding segment data, traversing all the segment data, and recombining according to a time sequence order to obtain the coded data.
It should be noted that, when the value of the coding parameter is selected properly, the golomb coding algorithm can effectively shorten the final coding length, but codes the water flow parameter with larger data by using the coding parameter with smaller value, and the final coding is still longer, so that the invention characterizes the corresponding data with smaller change when codes the data with smaller change degree by using the golomb coding algorithm, and the similarity of the codes is higher when codes the data with smaller difference are coded by using the golomb coding algorithm, and the adaptive coding parameter with lower change degree can be set relatively lower when codes the data with larger change degree by using the adaptive coding parameter with lower value, so that the code with simpler distribution can be obtained.
Further, in some embodiments of the present invention, data compression is performed on the encoded data to obtain compressed data, including: and carrying out data compression on the coded data based on an LZ77 algorithm to obtain compressed data.
The LZ77 algorithm is a lossless data compression algorithm, and the corresponding compression effect of the LZ77 algorithm is better when the LZ77 algorithm faces repeated data with larger regularity, so that the embodiment of the invention analyzes the data through the importance degree, when the LZ77 algorithm is used for data compression, the complexity degree of water flow data with larger importance can be lower, the water flow data is easier to compress by the LZ77 algorithm, the complexity degree of the water flow data with smaller importance is higher, but the byte number of the water flow data is less in the encoded data, and the overall transmission efficiency in the transmission process can be improved.
After compressed data is obtained through compression, the embodiment of the invention uses the cloud edge data pipeline to transmit the compressed data, wherein the cloud edge data pipeline is a pipeline for transmitting data in a cloud edge system.
The method is applied to the technical field of digital information transmission, and by acquiring the water flow data at different time points in time sequence, the overall fluctuation degree of the water flow data is determined according to the numerical values of all the water flow data, and the overall fluctuation degree characterizes the overall fluctuation condition of the water flow data, so that the overall change of the water flow data is conveniently analyzed; then, because the overall change of the water flow data is often larger in reality, for the convenience of analysis, segmenting the water flow data, establishing a coordinate system and determining water service coordinates, and clustering divides the water flow data into segment data, so that each segment of data can be respectively subjected to data coding in the follow-up process, the self-adaptive coding can be realized according to the change of the water flow data, and the coding effect is improved; the method comprises the steps of obtaining a flow increase coefficient, a section fluctuation degree, a mean value and an overall fluctuation degree of section data, and obtaining self-adaptive coding parameters, wherein the self-adaptive coding parameters are coding parameters obtained by analyzing by combining the numerical condition of each section of data with the characteristics of the flow increase coefficient, the section fluctuation degree, the overall fluctuation degree and the like, and performing Columbus coding processing on the section data according to the self-adaptive coding parameters to obtain coded data; the method comprises the steps of carrying out data compression on encoded data to obtain compressed data, transmitting the compressed data based on cloud edge data pipelines, carrying out Columbus encoding through self-adaptive encoding parameters, and then carrying out data compression, so that the occupied storage space of the compressed data with higher importance is smaller, meanwhile, the whole storage space is effectively shortened, and in combination, the occupied storage space of water flow data can be effectively reduced, and the efficiency of transmitting the water flow data is improved.
It should be noted that: the sequence of the embodiments of the present invention is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments.
Claims (4)
1. A cloud edge cooperation-based water affair data rapid transmission method is characterized by comprising the following steps:
acquiring water flow data at different time points in time sequence, and determining the overall fluctuation degree of the water flow data according to the numerical values of all the water flow data;
constructing a coordinate system by taking time points as abscissa and the numerical value of water flow data as ordinate, wherein the water flow data of different time points correspond to water service coordinates, clustering the water flow data of all time points according to all water service coordinates, and dividing the water flow data of all time points into segment data of at least two time segments;
performing straight line fitting on water service coordinates of all the section data of each time period, and determining flow increase coefficients of all the section data of each time period; determining the section fluctuation degree of each time section according to the numerical value of all section data of each time section; according to the average value, the flow increase coefficient, the section fluctuation degree and the overall fluctuation degree of all section data of each time section, determining the self-adaptive coding parameters of all section data of each time section;
performing Columbus coding processing on the corresponding segment data according to the adaptive coding parameters to obtain coded data; carrying out data compression on the encoded data to obtain compressed data, and transmitting the compressed data based on a cloud edge data pipeline;
the step of determining the overall fluctuation degree of the water flow data according to the values of all the water flow data comprises the following steps:
calculating normalized values of variances of the numerical values of all the water flow data as the overall fluctuation degree of the water flow data;
clustering the water flow data of all time points according to all water affair coordinates, dividing the water flow data of all time points into segment data of at least two time segments, wherein the clustering comprises the following steps:
determining a preset K value based on an elbow method, and performing K-Means clustering on all the water service coordinates based on a K-Means clustering algorithm and the preset K value to obtain a cluster;
dividing the water flow data into segment data of at least two time segments according to a cluster;
the dividing the water flow data into segment data of at least two time segments according to a cluster comprises:
determining end point values of abscissa values of all water service coordinates in each cluster, and dividing the water flow data into segment data of at least two time segments according to the end point values;
the linear fitting is performed on the water affair coordinates of all the section data of each time period, and the flow rate increase coefficients of all the section data of each time period are determined, including:
performing linear fitting on water affair coordinates of all the section data of each time period based on a linear fitting algorithm, and determining fitting lines of all the section data of each time period;
normalizing the absolute value of the slope of the fitting straight line to be used as a flow increase coefficient of all the data segments in the corresponding time period;
the determining the segment fluctuation degree of each time segment according to the values of all segment data of each time segment comprises the following steps:
calculating the variance of the numerical values of all the segment data of each time segment as the segment fluctuation degree of each time segment;
the self-adaptive coding parameters of all the section data of each time section are determined according to the average value of all the section data of each time section, the flow increase coefficient, the section fluctuation degree and the overall fluctuation degree, and the self-adaptive coding parameters comprise the following steps:
calculating the average value of all the segment data in each time period, and calculating the product of the average value and a preset adjustment coefficient as a segment parameter index;
calculating an inverse proportion normalization value of the product of the flow increase coefficient, the section fluctuation degree and the overall fluctuation degree as a parameter adjustment index;
and determining the self-adaptive coding parameters of all the segment data in the time period according to the segment parameter index and the parameter adjustment index.
2. The method for quickly transmitting the water service data based on cloud edge cooperation as claimed in claim 1, wherein said determining the adaptive coding parameters of all the segment data in the time period according to the segment parameter index and the parameter adjustment index comprises:
and calculating the product of the segment parameter index and the parameter adjustment index as the adaptive coding parameter of the corresponding segment data.
3. The method for rapidly transmitting water service data based on cloud edge cooperation as claimed in claim 1, wherein the performing golomb coding processing on the corresponding segment data according to the adaptive coding parameters to obtain coded data includes:
based on a Columbus coding algorithm, the adaptive coding parameter is used as a Columbus coding parameter, the Columbus coding processing is carried out on the corresponding segment data, all the segment data are traversed, and the coded data are recombined according to the time sequence.
4. The method for rapidly transmitting water service data based on cloud edge cooperation as claimed in claim 1, wherein the data compression of the encoded data to obtain compressed data comprises:
and carrying out data compression on the coded data based on an LZ77 algorithm to obtain compressed data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311724653.XA CN117411947B (en) | 2023-12-15 | 2023-12-15 | Cloud edge cooperation-based water service data rapid transmission method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311724653.XA CN117411947B (en) | 2023-12-15 | 2023-12-15 | Cloud edge cooperation-based water service data rapid transmission method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117411947A CN117411947A (en) | 2024-01-16 |
| CN117411947B true CN117411947B (en) | 2024-02-23 |
Family
ID=89496624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311724653.XA Active CN117411947B (en) | 2023-12-15 | 2023-12-15 | Cloud edge cooperation-based water service data rapid transmission method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117411947B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2451604A1 (en) * | 2001-06-29 | 2003-01-09 | Qualcomm Incorporated | Dct compression using golomb-rice coding |
| US6850649B1 (en) * | 1999-03-26 | 2005-02-01 | Microsoft Corporation | Image encoding using reordering and blocking of wavelet coefficients combined with adaptive encoding |
| EP1653628A1 (en) * | 2004-10-29 | 2006-05-03 | Microsoft Corporation | Lossless adaptive golomb/rice encoding and decoding of integer data using backward-adaptive rules |
| WO2009044346A1 (en) * | 2007-10-05 | 2009-04-09 | Nokia Corporation | System and method for combining adaptive golomb coding with fixed rate quantization |
| GB201303660D0 (en) * | 2013-03-01 | 2013-04-17 | Gurulogic Microsystems Oy | Data encoder, data decoder and method |
| CN112332853A (en) * | 2020-11-02 | 2021-02-05 | 重庆邮电大学 | Time sequence data compression and recovery method based on power system |
| DE102021110443A1 (en) * | 2020-07-31 | 2022-02-03 | Samsung Electronics Co., Ltd. | FRAME BUFFER COMPRESSION CIRCUIT AND IMAGE PROCESSING DEVICE |
| CN114726926A (en) * | 2022-03-30 | 2022-07-08 | 电子科技大学 | Self-adaptive variable length coding method for Laplace information source |
| CN115459782A (en) * | 2022-08-29 | 2022-12-09 | 北京思维实创科技有限公司 | Industrial Internet of things high-frequency data compression method based on time sequence segmentation and clustering |
| CN116452686A (en) * | 2023-03-03 | 2023-07-18 | 湖北工业大学 | Point painting generation method based on adaptive clustering and linear programming optimization point selection |
-
2023
- 2023-12-15 CN CN202311724653.XA patent/CN117411947B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6850649B1 (en) * | 1999-03-26 | 2005-02-01 | Microsoft Corporation | Image encoding using reordering and blocking of wavelet coefficients combined with adaptive encoding |
| CA2451604A1 (en) * | 2001-06-29 | 2003-01-09 | Qualcomm Incorporated | Dct compression using golomb-rice coding |
| EP1653628A1 (en) * | 2004-10-29 | 2006-05-03 | Microsoft Corporation | Lossless adaptive golomb/rice encoding and decoding of integer data using backward-adaptive rules |
| WO2009044346A1 (en) * | 2007-10-05 | 2009-04-09 | Nokia Corporation | System and method for combining adaptive golomb coding with fixed rate quantization |
| GB201303660D0 (en) * | 2013-03-01 | 2013-04-17 | Gurulogic Microsystems Oy | Data encoder, data decoder and method |
| DE102021110443A1 (en) * | 2020-07-31 | 2022-02-03 | Samsung Electronics Co., Ltd. | FRAME BUFFER COMPRESSION CIRCUIT AND IMAGE PROCESSING DEVICE |
| CN112332853A (en) * | 2020-11-02 | 2021-02-05 | 重庆邮电大学 | Time sequence data compression and recovery method based on power system |
| CN114726926A (en) * | 2022-03-30 | 2022-07-08 | 电子科技大学 | Self-adaptive variable length coding method for Laplace information source |
| CN115459782A (en) * | 2022-08-29 | 2022-12-09 | 北京思维实创科技有限公司 | Industrial Internet of things high-frequency data compression method based on time sequence segmentation and clustering |
| CN116452686A (en) * | 2023-03-03 | 2023-07-18 | 湖北工业大学 | Point painting generation method based on adaptive clustering and linear programming optimization point selection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117411947A (en) | 2024-01-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106649026B (en) | Monitoring data compression method suitable for operation and maintenance automation system | |
| CN115659070A (en) | Water flow data transmission method based on NB-IOT intelligent water meter | |
| CN115359807B (en) | Noise online monitoring system for urban noise pollution | |
| CN116827350B (en) | Flexible work platform intelligent supervision method and system based on cloud edge cooperation | |
| CN116996673B (en) | Intelligent cloud management system based on passing in and out management and equipment running state | |
| CN115987294A (en) | Multidimensional data processing method of Internet of things | |
| CN117411947B (en) | Cloud edge cooperation-based water service data rapid transmission method | |
| CN114461594A (en) | Data compression method, edge device and computer storage medium | |
| US10211851B2 (en) | Method and system for compressing data from smart meter | |
| CN116582133B (en) | Intelligent management system for data in transformer production process | |
| WO2023246700A1 (en) | Point cloud attribute encoding method, point cloud attribute decoding method, and storage medium | |
| CN117082153A (en) | Carrier communication data acquisition and data optimization method | |
| CN109286817B (en) | Method for processing quantization distortion information of DCT (discrete cosine transformation) coefficient in video coding | |
| CN115292393B (en) | Data management system for intelligent gateway | |
| CN103746701A (en) | Rapid encoding option selecting method applied to Rice lossless data compression | |
| Nascimento et al. | Improved transient data compression algorithm based on wavelet spectral quantization models | |
| CN115866251A (en) | Semantic segmentation based image information rapid transmission method | |
| CN111339155A (en) | Correlation analysis system | |
| US20150078674A1 (en) | Component sorting based encoding for 3d mesh compression | |
| CN115618229A (en) | Line loss allocation algorithm based on big data | |
| CN104320661A (en) | Image coding quality predicting method based on difference entropy and structural similarity | |
| CN117459187B (en) | High-speed data transmission method based on optical fiber network | |
| CN115270895B (en) | Fault detection method for diesel engine | |
| Beknazarova et al. | Experimental Evaluation of the Efficiency of Compression of Files by Fractal-Spectral Codec | |
| CN111316326A (en) | Image encoding method, apparatus and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |