CN117376429A - Intelligent data compression method for wireless sensor network - Google Patents
Intelligent data compression method for wireless sensor network Download PDFInfo
- Publication number
- CN117376429A CN117376429A CN202311303843.4A CN202311303843A CN117376429A CN 117376429 A CN117376429 A CN 117376429A CN 202311303843 A CN202311303843 A CN 202311303843A CN 117376429 A CN117376429 A CN 117376429A
- Authority
- CN
- China
- Prior art keywords
- added
- dictionary
- character string
- data
- character
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000013144 data compression Methods 0.000 title claims abstract description 16
- 230000005540 biological transmission Effects 0.000 claims abstract description 97
- 238000007906 compression Methods 0.000 claims abstract description 31
- 230000006835 compression Effects 0.000 claims abstract description 31
- 238000010606 normalization Methods 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 8
- 238000012544 monitoring process Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000005265 energy consumption Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000013068 supply chain management Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/18—Self-organising networks, e.g. ad-hoc networks or sensor networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention relates to the technical field of data coding compression, in particular to an intelligent data compression method for a wireless sensor network. The method comprises the steps of obtaining an initial dictionary according to the type number of transmission data, and obtaining the assignability of each type of character according to the residual space of the initial dictionary and the occurrence frequency of each type of character; carrying out LZW algorithm coding on the transmission data to obtain a character string to be added; according to the occurrence frequency of the first character in the character string to be added, the residual data quantity of the transmission data and the relative frequency of the character string to be added, the condition frequency is obtained, and the optimal degree of the character string to be added is obtained by combining the distributability of the first character in the character string to be added; and updating the initial dictionary according to the preference degree, and continuing to perform LZW algorithm coding to continuously update the initial dictionary until all transmission data are coded, so as to obtain compressed data. The method optimizes the dictionary updating process in the LZW algorithm, improves the data compression efficiency, and ensures that the compression effect is better.
Description
Technical Field
The invention relates to the technical field of data coding compression, in particular to an intelligent data compression method for a wireless sensor network.
Background
Through the data transmission of the wireless sensor network, the real-time monitoring, remote control, intelligent decision and the like of the data can be realized, and higher efficiency, reliability and sustainable development are brought to various industries. The wireless sensor network is widely applied to the aspects of industrial automation, agriculture and environment monitoring, construction and intelligent home, medical care, logistics and supply chain management and the like. However, the wireless sensor network data often has a larger data volume, which causes that the data consumes a large amount of transmission bandwidth when the data is transmitted through the network, and reduces the transmission speed, so the data compression process is very important in the data transmission of the wireless sensor network.
In wireless sensor network data, because the data to be monitored often has larger concentration, most of the data has lower and even no changes along with the change of time, so that a large amount of repeated data exists in the data, the LZW algorithm is often used for lossless coding compression in the existing coding compression technology, and in the existing coding process adopting the LZW algorithm for coding, due to the characteristic of diversity of the wireless sensor network data, when the wireless sensor network data is added into a dictionary for coding compression, some character strings with lower frequency not only have low coding efficiency of the data, but also have lower compression rate of the LZW algorithm on the data, and the final compression effect is poor.
Disclosure of Invention
In order to solve the technical problems of low data coding efficiency, low compression rate of the LZW algorithm on data and poor final compression effect in the prior art, the invention aims to provide an intelligent data compression method for a wireless sensor network, and the adopted technical scheme is as follows:
the invention provides an intelligent data compression method for a wireless sensor network, which comprises the following steps:
acquiring transmission data; according to the variety number of characters in the transmission data, obtaining an initial dictionary corresponding to the transmission data; obtaining the distribution degree of each type of characters according to the residual space size of the initial dictionary and the occurrence frequency of each type of characters in the transmission data;
in the process of carrying out LZW algorithm coding on transmission data through an initial dictionary, obtaining a dictionary updating position and a corresponding character string to be added; acquiring the condition frequency of the character string to be added according to the occurrence frequency of the first character in the character string to be added, the relative frequency of the character string to be added and the data quantity of the transmission data after the corresponding dictionary updating position; obtaining the preference degree of the character string to be added according to the condition frequency of the character to be added and the distribution degree of the first character in the character string to be added;
according to the preference degree of the character string to be added, updating the initial dictionary of the dictionary updating position; and carrying out LZW algorithm coding on the transmission data after the dictionary updating position through the updated initial dictionary to obtain a new dictionary updating position, updating the initial dictionary until all the transmission data are coded, and obtaining compressed data.
Further, the method for acquiring the initial dictionary comprises the following steps:
counting the number of types of characters in the transmission data, and obtaining the size of an initial dictionary according to the number of types;
the frequency of each type of character in the transmission data is used as the occurrence frequency of each type of character, the corresponding characters are sequentially added into the initial dictionary according to the sequence from big to small of the occurrence frequency, codes corresponding to each type of character are generated, and the initial dictionary is obtained.
Further, the specific expression for obtaining the size of the initial dictionary is:
where S is expressed as the size of the initial dictionary, n is expressed as the number of categories, log is expressed as a logarithmic function,represented as a round-up function.
Further, the method for acquiring the distributable degree comprises the following steps:
taking the difference value between the size and the category number of the initial dictionary as the residual space size of the initial dictionary; and (3) rounding down the product of the occurrence frequency of each type of characters and the size of the residual space to obtain the allocability of each type of characters.
Further, the method for acquiring the conditional frequency comprises the following steps:
counting the data quantity of the transmission data after updating the position of the dictionary, obtaining the data quantity to be encoded, calculating the ratio of the data quantity to be encoded to the total data quantity of the transmission data, and taking the ratio of the negative correlation mapping and normalization processing as an encoding adjustment value; taking the frequency of the character string to be added in the transmission data after the dictionary updating position as the relative frequency of the character string to be added;
obtaining the adjustment frequency of the character string to be added according to the code adjustment value and the relative frequency of the character string to be added;
and calculating the ratio of the adjustment frequency of the character string to be added to the occurrence frequency of the first character in the character string to be added, and obtaining the conditional frequency of the character string to be added.
Further, the specific expression of the adjustment frequency of the character string to be added is:
wherein Pw represents an adjustment frequency of a character string to be added, ps represents a relative frequency of the character string to be added, D represents a total data amount of transmission data, du represents a data amount to be encoded, norm () represents a normalization function, a represents a preset adjustment parameter, and the preset adjustment parameter is greater than 0 and less than 1.
Further, the initial dictionary updating of the dictionary updating position according to the preference degree of the character string to be added includes:
taking the ratio of the residual space size of the initial dictionary to the variety number of the transmission data as a preferred threshold;
and when the preference degree of the character strings to be added is larger than the preference threshold value, adding the character strings to be added into the initial dictionary, and generating codes corresponding to the character strings to be added to obtain the updated initial dictionary.
Further, the step of performing LZW algorithm encoding on the transmission data after the dictionary updating position through the updated initial dictionary to obtain a new dictionary updating position, updating the initial dictionary until all the transmission data are encoded, and obtaining compressed data includes:
in the LZW algorithm coding process, when the character string is not in the updated initial dictionary for the first time, the corresponding coding position is used as a new dictionary updating position; initial dictionary updating for the new dictionary updating location;
carrying out LZW algorithm coding on the transmission data after the last new dictionary updating position through the initial dictionary updated last time to obtain compression coding data corresponding to the transmission data;
forming a character string sequence from all character strings in the initial dictionary updated last time; the compressed encoded data, the character string sequence and the initial dictionary which are not updated are taken as compressed data of the transmission data.
Further, the method for obtaining the preference degree comprises the following steps:
and calculating the product of the conditional frequency of the character to be added and the assignability of the first character in the character string to be added, and obtaining the preference degree of the character string to be added.
Further, in the process of performing LZW algorithm encoding on the transmission data through the initial dictionary, obtaining a dictionary update position and a corresponding character string to be added, including:
in the LZW algorithm coding process, when the character string is not in the initial dictionary after the first retrieval, the corresponding coding position is used as a dictionary updating position; and taking the character strings which are not in the initial dictionary and correspond to the dictionary updating positions as character strings to be added.
The invention has the following beneficial effects:
according to the invention, an initial dictionary is obtained according to the number of types of transmission data, the allocable degree of each type of character is obtained according to the residual space of the initial dictionary and the occurrence frequency of each type of character, and the allocable degree is used for reflecting the number degree of character strings of different types which can be added in the dictionary of the LZW algorithm, so that the importance degree of the character strings to be added in the subsequent process is conveniently analyzed. And further carrying out LZW algorithm coding on the transmission data through the initial dictionary, and analyzing the dictionary updating position before the character strings need to be added into the initial dictionary to obtain the dictionary updating position and the corresponding character strings to be added in consideration of different use frequencies of the character strings needing to be added into the initial dictionary in the transmission data. The importance of the character string to be added is analyzed from two aspects by considering the data quantity of transmission data which is not coded after the dictionary is updated and the duty ratio degree of all the character strings corresponding to the first character of the character string to be added, the condition frequency of the character string to be added is obtained by corresponding to the data quantity of the transmission data and the relative frequency of the character string to be added after the dictionary is updated according to the occurrence frequency of the first character of the character string to be added, the use frequency of the character string to be added is analyzed more comprehensively and accurately, and the space size of the character string to be added which can be allocated, namely the allocability degree of the first character of the character string to be added is further considered, so that the preference degree of the character string to be added is obtained, and the importance of the character string to be added is reflected. Finally updating the initial dictionary according to the preferred degree to obtain a better dictionary, so that the retrieval rate of the dictionary in coding is higher, the coding effect is better, the LZW algorithm coding is continuously carried out on the rest of the transmission data, the initial dictionary is continuously updated until all the transmission data are coded, and the compressed data are obtained.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for intelligently compressing data in a wireless sensor network according to an embodiment of the present invention.
Detailed Description
In order to further describe the technical means and effects adopted by the invention to achieve the preset aim, the following is a detailed description of specific implementation, structure, characteristics and effects of the intelligent compression method for wireless sensor network data according to the invention with reference to the accompanying drawings and the preferred embodiment. In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following specifically describes a specific scheme of the wireless sensor network data intelligent compression method provided by the invention with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of a method for intelligently compressing data in a wireless sensor network according to an embodiment of the present invention is shown, where the method includes the following steps:
s1: acquiring transmission data; according to the variety number of characters in the transmission data, obtaining an initial dictionary corresponding to the transmission data; and obtaining the allocability of each type of character according to the residual space size of the initial dictionary and the occurrence frequency of each type of character in the transmission data.
The wireless sensor network data mainly comprises sensor data, position data, energy data, network topology data, event data and the like, wherein the sensor data is usually the environmental parameters of a place, such as temperature, humidity, illumination intensity, pressure, sound and the like; sensor location data, such as location coordinates of sensor nodes; the energy consumption of the sensor, such as the energy consumption of the data sensor node; network topology data of the sensor, such as connection relation, routing information and the like between sensor network nodes; event data for the sensor, such as a specific time detectable by the wireless sensor, such as a fire, an earthquake, etc. The primary purpose of collecting such data is to obtain real-time information and status of the environment, device or system for monitoring, analysis and decision making. And analyzing data characteristics through the collected sensor network data, and predicting the data so as to optimize the actual scenes such as energy sources, resource environments and the like.
In the wireless sensor network data, the acquired data volume is large and contains more character types, but the acquired data belongs to monitoring data, the concentration of the monitoring data is strong, and the repeatability of the data is also high, so that the LZW algorithm is generally adopted for encoding compression, and a good compression effect can be realized. Firstly, the collected data needing to be compressed is taken as transmission data, the LZW algorithm realizes compression by establishing a character string table, namely a dictionary and replacing long character strings by shorter codes, so that the initial dictionary is firstly constructed for coding and compressing the transmission data. In the embodiment of the invention, the dictionary space prepared in advance by the computer is a fixed array space obtained according to the number of types, so that dictionary optimization can be conveniently carried out according to the use condition of the space.
Preferably, the number of kinds of characters in the transmission data is counted, the size of the initial dictionary is obtained according to the number of kinds, the size of the initial dictionary is the maximum number of character strings which can be stored in the dictionary, and the specific expression for obtaining the size of the initial dictionary is as follows:
where S is expressed as the size of the initial dictionary, n is expressed as the number of categories, log is expressed as a logarithmic function,represented as a round-up function.
Wherein,expressed as a maximum coding length obtained from the number of categories, for example, when the number of categories is 5, the coding length is 4, and then the initial dictionary size is 16 according to the power of 2 operation. The initial dictionary is of a size of a pre-prepared empty spaceThe space size is not changed even if the initial dictionary is not filled with data.
The occurrence frequency of each type of character in the transmission data is obtained, the occurrence frequency is the occurrence probability of each type of character in the transmission data, the characters are sequentially added into the initial dictionary according to the sequence from the big to the small of the occurrence frequency, codes corresponding to each type of character are generated, and the initial dictionary is obtained. It should be noted that, the arrangement sequence from large to small ensures that the higher frequency characters can be quickly searched when the dictionary is searched, the coding compression efficiency is improved, the codes corresponding to each type of characters are generated and filled in according to the sequence from small to large of natural numbers, and the coding process for constructing the dictionary as the LZW algorithm is a technical means well known to those skilled in the art, and will not be repeated here.
In the transmission data, the higher the appearance frequency of the character, the higher the frequency of the character string with the character of the type as the first position, and the more the space position occupied by the character string with the higher frequency as the first position can be when the initial dictionary is updated due to the addition of the character string. Therefore, the allocatable degree of each type of character is obtained according to the residual space size of the initial dictionary and the occurrence frequency of each type of character in the transmission data, and the allocatable degree reflects the allocatable degree of the character strings with the characters of each type in the residual space.
Preferably, the difference between the size of the initial dictionary and the number of types is taken as the residual space size of the initial dictionary, the product of the occurrence frequency of each type of characters and the residual space size is rounded down, and the allocable degree of each type of characters is obtained, and in the embodiment of the present invention, the specific expression of the allocable degree of the characters is:
in Al i The assignability of the i-th character, S is the size of the initial dictionary, n is the number of kinds and P i Expressed as the frequency of occurrence of the i-th character,expressed as a round-down function, by means of the round-down, it is ensured that the remaining space can be fully allocated for each type of character.
Thus, the primary analysis of the transmission data is completed, and an initial dictionary which can be encoded and the assignability of each type of character used for analyzing the use frequency of the character string are obtained.
S2: in the process of carrying out LZW algorithm coding on transmission data through an initial dictionary, obtaining a dictionary updating position and a corresponding character string to be added; obtaining the condition frequency of the character string to be added according to the character string to be added, the frequency of the first character in the character string to be added and the number of the transmission data which is not coded at the updating position of the corresponding dictionary; and obtaining the preference degree of the character string to be added according to the condition frequency of the character to be added and the assignable degree of the first character in the character string to be added.
The transmission data may be coded by the LZW algorithm according to the initial dictionary, and the LZW algorithm coding process is a well-known technique known to those skilled in the art, and will not be described herein in detail. In the coding process of the LZW algorithm, character strings are directly added into a dictionary according to the appearance sequence of the character strings, the use frequency of the character strings is not considered, a great number of positions are occupied in the dictionary by some character strings with extremely low use frequency, the retrieval time of a subsequent dictionary is increased, the dictionary is easily filled up faster, the compression time of the algorithm is long, the compression efficiency is low, and the compression degree is also low. Therefore, the embodiment of the invention mainly analyzes the importance degree of the character string when the new character string is added into the dictionary, and if the character string does not meet the selection condition, the character string is not added into the dictionary.
First, in the process of performing LZW algorithm coding on transmission data through an initial dictionary, a dictionary updating position and a corresponding character string to be added are obtained, and in one embodiment of the invention, when the character string is not in the initial dictionary in the LZW algorithm coding process, the corresponding coding position is taken as the dictionary updating position, and the character string which is not in the initial dictionary in the dictionary updating position is taken as the character string to be added. For example, for a group of data abcabaa, the data is not encoded, the character string that is not in the dictionary is detected as ab for the first time, that is, the character string to be added is ab, the first character in the character string to be added is a, and the corresponding encoding position, that is, the dictionary updating position, is the position before the first character a in the character string ab.
As encoding proceeds, since the data to be encoded is reduced, the data to be encoded is transmission data which is not encoded, that is, transmission data after the dictionary updating position, the influence on the relative frequency of the character string to be added is increased, for example, in two cases where the remaining data amount of the data to be encoded is 10 and the remaining data amount is 5, the importance degree when the number of occurrences of the character string to be added is 2 is different, and the character string to be added is more important in the case where the remaining data amount is 5 because the proportion is higher. The less data to be encoded remains, the more the frequency of character strings to be added is affected. Meanwhile, as coding proceeds, the residual allocation space of the character string to be added in the initial dictionary is reduced, so that the conditional frequency of the character string to be added is obtained according to the relative frequency of the character string to be added, the occurrence frequency of the first character in the character string to be added and the number of the transmission data which are not coded at the updating position of the corresponding dictionary.
Preferably, the data quantity of the transmission data after the dictionary is updated is counted, the ratio of the data quantity to be encoded to the total data quantity of the transmission data is calculated as the data quantity to be encoded, the code adjustment value is obtained by mapping and normalizing the ratio of the negative correlation, and the importance degree of the relative frequency of the character string to be added is reflected by the duty ratio degree of the uncoded data quantity. The frequency of the character string to be added in the transmission data after the dictionary updating position is used as the relative frequency of the character string to be added, and the frequency of the character string to be added after the dictionary updating position is calculated can show the use frequency of the character string to be added more because the character string to be added is a new character string which is not added into the dictionary.
According to the code adjustment value and the relative frequency of the character string to be added, obtaining the adjustment frequency of the character string to be added, wherein the specific expression of the adjustment frequency is as follows:
wherein Pw represents an adjustment frequency of a character string to be added, ps represents a relative frequency of the character string to be added, D represents a total data amount of transmission data, du represents a data amount to be encoded, a represents a preset adjustment parameter, and the preset adjustment parameter is greater than 0 and less than 1. The Norm () is represented as a normalization function, and the normalization is a technical means well known to those skilled in the art, and the normalization function may be selected by linear normalization or standard normalization, and the specific normalization method is not limited herein.
Wherein,the code adjustment value is indicated as the code adjustment value, and the smaller the data quantity to be coded, that is, the less transmission data which is not coded, the more important the character string to be added is, and the larger the code adjustment value is. In the embodiment of the present invention, the preset adjustment parameter a is set to 0.8, so as to prevent the adjustment degree of the relative frequency from being too large, which affects the subsequent analysis of the important degree.
After the relative frequency of the character strings to be added is adjusted according to the data volume which is not coded, the frequency of occurrence of the first character in the character strings to be added is further adjusted again, and the greater the proportion of the character strings to be added in the same type of character strings, namely the greater the proportion of the character strings to be added in the character strings with the same first character, the higher the coding compression efficiency of the character strings to be added after the character strings to be added are added into the dictionary at the moment is indicated.
Therefore, preferably, a ratio of an adjustment frequency of a character string to be added to an occurrence frequency of a first character in the character string to be added is calculated to obtain a condition frequency of the character string to be added, and in the embodiment of the present invention, a specific expression of the condition frequency is as follows:
wherein Cf is represented as the conditional frequency of the character string to be added, pw is represented as the adjustment frequency of the character string to be added, and P i The first character in the character string to be added is represented as the occurrence frequency of the ith character.
When the condition frequency is larger, the use frequency of the character string to be added in the subsequent data is larger, the coding compression rate is improved by adding the dictionary corresponding to the character string to be added, meanwhile, the higher the distributability of the first character in the character string to be added is, the higher the possibility that the character string to be added can be added is, and therefore the preference degree of the character string to be added is obtained according to the condition frequency of the character string to be added and the distributability of the first character in the character string to be added.
Preferably, the product of the conditional frequency of the character to be added and the assignable degree of the first character in the character string to be added is calculated to obtain the preference degree of the character string to be added, and the importance degree of the character string to be added is reflected through the preference degree, and in the embodiment of the invention, the specific expression of the preference degree is as follows:
Pd＝Cf×Al i
wherein Pd is expressed as the preference degree of the character string to be added, cf is expressed as the conditional frequency of the character string to be added, al i The first character in the character string to be added is the assignability of the ith character.
Because the degree of assignability reflects the size of the allocation space of each type of character, the larger the allocation space, the greater the likelihood that a character string to be added can be added to the dictionary, and the smaller the allocation space, the less the likelihood that a character string to be added considers to be added to the dictionary. The condition frequency reflects the use frequency of the character string to be added, and when the use frequency is larger, the possibility that the character string to be added can be added into the dictionary is larger, and the use frequency is smaller, the possibility that the character string to be added is added into the dictionary is smaller.
So far, the analysis of the importance degree of the character string to be added is completed, and the preference degree of the character string to be added in the encoding process is obtained.
S3: according to the preference degree of the character string to be added, updating the initial dictionary of the dictionary updating position; and carrying out LZW algorithm coding on the transmission data after the dictionary updating position through the updated initial dictionary to obtain a new dictionary updating position, updating the initial dictionary until all the transmission data are coded, and obtaining compressed data.
In the embodiment of the invention, whether the character string to be added can be added into the dictionary is judged by setting the preference threshold, namely, the initial dictionary of the dictionary updating position is updated according to the preference degree of the character string to be added. The selection of the optimal threshold value influences the filling condition of the dictionary, when the optimal threshold value is too large, the added character strings are fewer, so that a large amount of space in the dictionary cannot be filled, encoded data obtained by encoding are more, the ideal compression rate cannot be achieved, when the optimal threshold value is too small, the space in the dictionary is filled too early, the space of the dictionary needs to be updated, and the encoding compression calculation amount is increased.
Therefore, in order to obtain a more reasonable preference threshold, in one embodiment of the present invention, the ratio of the remaining space size of the initial dictionary to the number of types of transmission data is set by the remaining space size of the initial dictionary and the number of types of transmission data, as the preference threshold.
Further, when the preference degree of the character string to be added is larger than the preference threshold value, the character string to be added is important, the subsequent coding compression efficiency is higher after the character string to be added is added into the initial dictionary, codes corresponding to the character string to be added are generated, and the updated initial dictionary is obtained. It should be noted that, in the embodiment of the present invention, the code corresponding to the character string to be added is generated by adding one to the natural number of the code corresponding to the previous character string data in the dictionary, and the specific code generating process in the LZW algorithm is a well-known technology known to those skilled in the art, and will not be described herein.
In the embodiment of the invention, if the preference degree of the character string to be added is smaller than or equal to the preference threshold value, the initial dictionary is not updated, so that the character string to be added is directly encoded according to the character string already in the initial dictionary, for example, for a group of data abcacaa, if the character string ab to be added cannot update the initial dictionary, the character a is directly encoded, the character b needs to be further judged to judge the retrieval condition of bc in the dictionary, if the character string to be added is abc and cannot update the initial dictionary, the character b is indicated to be the character string already in the dictionary, the character string ab is directly encoded, and the character c needs to be further judged according to the retrieval condition of ca in the dictionary. The encoding process refers to an LZW algorithm encoding process, and the encoding process is a public technical means and will not be further described herein.
And (3) carrying out LZW algorithm coding on the transmission data after the dictionary updating position through the updated initial dictionary, repeating the process of step S2, and in the LZW algorithm coding process, when the character string is not in the updated initial dictionary for the first time, taking the corresponding coding position as a new dictionary updating position, after the new dictionary updating position is obtained, updating the initial dictionary of the new dictionary updating position until all the transmission data are coded, stopping updating, and carrying out LZW algorithm coding on the transmission data after the last new dictionary updating position through the last updated initial dictionary to obtain compression coding data corresponding to the transmission data, wherein the compression coding data is the compression result of lossless compression of the transmission data.
In the decoding process of the LZW algorithm, a dictionary needs to be reconstructed according to the compressed coding data, in order to ensure that the compressed coding data can be decoded, an initial dictionary which is not updated needs to be used as corresponding compressed data, and the initial dictionary which is not updated is the initial dictionary obtained in the step S1, wherein the initial dictionary only comprises various characters and corresponding codes. And meanwhile, all character strings in the initial dictionary updated for the last time are formed into character string sequences, wherein the character string sequences are character strings which can be added into the dictionary, and whether the character strings are added into the dictionary can be judged in the process of decoding and constructing the dictionary, so that the dictionary constructed by decoding can be consistent with the encoding process. It should be noted that, the decoding process of the LZW algorithm is a public technical means well known to those skilled in the art, and will not be described herein.
And finally, taking the compressed and encoded data, the character string sequence and the initial dictionary which is not updated as compressed data of transmission data.
In summary, the invention obtains the initial dictionary through the category number, and obtains the assignable degree of each category of character according to the residual space of the initial dictionary and the occurrence frequency of each category of character, wherein the assignable degree is used for reflecting the joinable degree of different categories of character strings in the dictionary of the LZW algorithm. The method comprises the steps of carrying out LZW algorithm coding on transmission data through an initial dictionary, analyzing dictionary updating positions of the character strings before the initial dictionary is needed to be added in consideration of different use frequencies of the character strings needing to be added in the transmission data, obtaining the character strings to be added in the dictionary updating positions, and obtaining the preferred degree of the character strings to be added by considering the importance influence of the relative frequency of the character strings to be added in consideration of the transmission data quantity which is not coded after the dictionary updating positions and the influence of the duty ratio degree of all the character strings which are possibly corresponding to the first characters in the character strings to be added, obtaining the condition frequency of the character strings to be added by the occurrence frequency of the first characters in the character strings to be added, the data quantity of the transmission data after the dictionary updating positions and the relative frequency of the character strings to be added. Finally updating the initial dictionary according to the preferred degree, so that the searching speed of the initial dictionary is higher, the coding effect is better, the LZW algorithm coding is continuously carried out on the residual transmission data, the initial dictionary is continuously updated until all the transmission data are coded, and compressed data are obtained.
It should be noted that: the sequence of the embodiments of the present invention is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments.
Claims (10)
1. An intelligent compression method for wireless sensor network data is characterized by comprising the following steps:
acquiring transmission data; according to the variety number of characters in the transmission data, obtaining an initial dictionary corresponding to the transmission data; obtaining the distribution degree of each type of characters according to the residual space size of the initial dictionary and the occurrence frequency of each type of characters in the transmission data;
in the process of carrying out LZW algorithm coding on transmission data through an initial dictionary, obtaining a dictionary updating position and a corresponding character string to be added; acquiring the condition frequency of the character string to be added according to the occurrence frequency of the first character in the character string to be added, the relative frequency of the character string to be added and the data quantity of the transmission data after the corresponding dictionary updating position; obtaining the preference degree of the character string to be added according to the condition frequency of the character to be added and the distribution degree of the first character in the character string to be added;
according to the preference degree of the character string to be added, updating the initial dictionary of the dictionary updating position; and carrying out LZW algorithm coding on the transmission data after the dictionary updating position through the updated initial dictionary to obtain a new dictionary updating position, updating the initial dictionary until all the transmission data are coded, and obtaining compressed data.
2. The intelligent data compression method of the wireless sensor network according to claim 1, wherein the initial dictionary acquisition method comprises the following steps:
counting the number of types of characters in the transmission data, and obtaining the size of an initial dictionary according to the number of types;
the frequency of each type of character in the transmission data is used as the occurrence frequency of each type of character, the corresponding characters are sequentially added into the initial dictionary according to the sequence from big to small of the occurrence frequency, codes corresponding to each type of character are generated, and the initial dictionary is obtained.
3. The intelligent data compression method of the wireless sensor network according to claim 2, wherein the specific expression for obtaining the size of the initial dictionary is:
where S is expressed as the size of the initial dictionary, n is expressed as the number of categories, log is expressed as a logarithmic function,represented as a round-up function.
4. The intelligent data compression method of the wireless sensor network according to claim 2, wherein the method for acquiring the distributable degree comprises the following steps:
taking the difference value between the size and the category number of the initial dictionary as the residual space size of the initial dictionary; and (3) rounding down the product of the occurrence frequency of each type of characters and the size of the residual space to obtain the allocability of each type of characters.
5. The intelligent data compression method of the wireless sensor network according to claim 1, wherein the method for acquiring the conditional frequency comprises the following steps:
counting the data quantity of the transmission data after updating the position of the dictionary, obtaining the data quantity to be encoded, calculating the ratio of the data quantity to be encoded to the total data quantity of the transmission data, and taking the ratio of the negative correlation mapping and normalization processing as an encoding adjustment value; taking the frequency of the character string to be added in the transmission data after the dictionary updating position as the relative frequency of the character string to be added;
obtaining the adjustment frequency of the character string to be added according to the code adjustment value and the relative frequency of the character string to be added;
and calculating the ratio of the adjustment frequency of the character string to be added to the occurrence frequency of the first character in the character string to be added, and obtaining the conditional frequency of the character string to be added.
6. The intelligent data compression method of the wireless sensor network according to claim 5, wherein the specific expression of the adjustment frequency of the character string to be added is:
wherein Pw represents an adjustment frequency of a character string to be added, ps represents a relative frequency of the character string to be added, D represents a total data amount of transmission data, du represents a data amount to be encoded, norm () represents a normalization function, a represents a preset adjustment parameter, and the preset adjustment parameter is greater than 0 and less than 1.
7. The intelligent data compression method of the wireless sensor network according to claim 4, wherein the initial dictionary update of the dictionary update location according to the preference degree of the character string to be added comprises:
taking the ratio of the residual space size of the initial dictionary to the variety number of the transmission data as a preferred threshold;
and when the preference degree of the character strings to be added is larger than the preference threshold value, adding the character strings to be added into the initial dictionary, and generating codes corresponding to the character strings to be added to obtain the updated initial dictionary.
8. The intelligent data compression method of the wireless sensor network according to claim 1, wherein the step of performing LZW algorithm encoding on the transmission data after the dictionary updating position through the updated initial dictionary to obtain a new dictionary updating position, updating the initial dictionary until all the transmission data are encoded, and obtaining the compressed data includes:
in the LZW algorithm coding process, when the character string is not in the updated initial dictionary for the first time, the corresponding coding position is used as a new dictionary updating position; initial dictionary updating for the new dictionary updating location;
carrying out LZW algorithm coding on the transmission data after the last new dictionary updating position through the initial dictionary updated last time to obtain compression coding data corresponding to the transmission data;
forming a character string sequence from all character strings in the initial dictionary updated last time; the compressed encoded data, the character string sequence and the initial dictionary which are not updated are taken as compressed data of the transmission data.
9. The intelligent data compression method of the wireless sensor network according to claim 1, wherein the method for obtaining the preference degree comprises the following steps:
and calculating the product of the conditional frequency of the character to be added and the assignability of the first character in the character string to be added, and obtaining the preference degree of the character string to be added.
10. The method for intelligently compressing data in a wireless sensor network according to claim 1, wherein the obtaining the dictionary update location and the corresponding character string to be added in the process of LZW algorithm encoding the transmission data through the initial dictionary includes:
in the LZW algorithm coding process, when the character string is not in the initial dictionary after the first retrieval, the corresponding coding position is used as a dictionary updating position; and taking the character strings which are not in the initial dictionary and correspond to the dictionary updating positions as character strings to be added.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311303843.4A CN117376429A (en) | 2023-10-10 | 2023-10-10 | Intelligent data compression method for wireless sensor network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311303843.4A CN117376429A (en) | 2023-10-10 | 2023-10-10 | Intelligent data compression method for wireless sensor network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117376429A true CN117376429A (en) | 2024-01-09 |
Family
ID=89403385
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311303843.4A Pending CN117376429A (en) | 2023-10-10 | 2023-10-10 | Intelligent data compression method for wireless sensor network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117376429A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117708513A (en) * | 2024-02-05 | 2024-03-15 | 贵州省畜牧兽医研究所 | Planting data management method for valerian characteristic research |
| CN117874049A (en) * | 2024-03-08 | 2024-04-12 | 北京龙创悦动网络科技有限公司 | Method and system for updating hand-tour data |
-
2023
- 2023-10-10 CN CN202311303843.4A patent/CN117376429A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117708513A (en) * | 2024-02-05 | 2024-03-15 | 贵州省畜牧兽医研究所 | Planting data management method for valerian characteristic research |
| CN117708513B (en) * | 2024-02-05 | 2024-04-19 | 贵州省畜牧兽医研究所 | Planting data management method for valerian characteristic research |
| CN117874049A (en) * | 2024-03-08 | 2024-04-12 | 北京龙创悦动网络科技有限公司 | Method and system for updating hand-tour data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117376429A (en) | Intelligent data compression method for wireless sensor network | |
| CN116192971B (en) | Intelligent cloud energy operation and maintenance service platform data management method | |
| CN109542944B (en) | Intelligent home user control behavior recommendation method based on time sequence causality analysis | |
| Fernández et al. | Efficient RDF interchange (ERI) format for RDF data streams | |
| Chiarot et al. | Time series compression survey | |
| CN113094346A (en) | Big data coding and decoding method and device based on time sequence | |
| CN115882868A (en) | Intelligent storage method for gas monitoring data | |
| CN116681036A (en) | Industrial data storage method based on digital twinning | |
| Al-Qurabat et al. | Two-level energy-efficient data reduction strategies based on SAX-LZW and hierarchical clustering for minimizing the huge data conveyed on the internet of things networks | |
| WO2023202149A1 (en) | State selection method and system for finite state entropy encoding, and storage medium and device | |
| Kennel et al. | Context-tree modeling of observed symbolic dynamics | |
| CN115913247A (en) | Deep lossless compression method and system for high-frequency power data | |
| CN1426629A (en) | Method and apparatus for optimized lossless compression using plurality of coders | |
| CN116680269A (en) | Time sequence data coding and compressing method, system, equipment and medium | |
| CN115695564B (en) | Efficient transmission method of Internet of things data | |
| CN115567609B (en) | Communication method of Internet of things for boiler | |
| CN112783865A (en) | Semi-supervised man-machine combined operation and maintenance fault library generation method and system | |
| CN116841973A (en) | Data intelligent compression method and system for embedded database | |
| CN108829930B (en) | Lightweight method for designing MBD model by three-dimensional digital process | |
| CN109510627B (en) | Integer coding method, device and storage medium | |
| Siemens et al. | Internet of things data transfer method using neural network autoencoder | |
| CN114301468A (en) | FSE encoding method, device, equipment and storage medium | |
| CN117273764B (en) | Anti-counterfeiting management method and system for electronic atomizer | |
| CN117171399B (en) | New energy data optimized storage method based on cloud platform | |
| CN116738354B (en) | Method and system for detecting abnormal behavior of electric power Internet of things terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |