CN117767960B - Sensor data optimization acquisition and storage method - Google Patents
Sensor data optimization acquisition and storage method Download PDFInfo
- Publication number
- CN117767960B CN117767960B CN202410195039.7A CN202410195039A CN117767960B CN 117767960 B CN117767960 B CN 117767960B CN 202410195039 A CN202410195039 A CN 202410195039A CN 117767960 B CN117767960 B CN 117767960B
- Authority
- CN
- China
- Prior art keywords
- temperature data
- real
- character
- data
- historical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000005457 optimization Methods 0.000 title abstract description 9
- 238000007906 compression Methods 0.000 claims abstract description 60
- 230000006835 compression Effects 0.000 claims abstract description 59
- 230000000694 effects Effects 0.000 claims abstract description 18
- 238000005070 sampling Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Landscapes
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention relates to the technical field of data processing, in particular to a sensor data optimization acquisition and storage method, which comprises the following steps: collecting historical temperature data and real-time temperature data; obtaining the influence of the frequency difference on the compression rate of Fei Nuo coding algorithm according to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data, and obtaining new real-time temperature data; obtaining a reconstructed Fischer-Tropsch coding tree, and obtaining first coding data and second coding data according to the reconstructed Fischer-Tropsch coding tree and Fei Nuo coding trees; the first encoded data or the second encoded data is stored. The invention self-adaptively reconstructs the Fischer code tree, better adapts to the real-time change condition of the data, further improves the overall compression effect of the data, and reduces the size of the storage space occupied by the sensor data, thereby reducing the resources required by storage.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a sensor data optimization acquisition and storage method.
Background
Sensor data is widely used in various fields such as the internet of things, smart cities, industrial automation, and the like. The acquisition and storage of sensor data is a key element in these applications. However, sensor data is typically generated at a high frequency and has a large amount of redundant information, resulting in a huge amount of data, which presents challenges for data acquisition and storage. Therefore, it is necessary to compress the data collected by the sensor to reduce the size of the storage space occupied by the sensor data, thereby reducing the resources required for storage.
In conventional fisher coding, fei Nuo code trees are pre-constructed based on the probability of occurrence of characters in the data samples. Since the sensor data is continuously input with time, the data frequency distribution characteristics input in real time deviate from the expected distribution in Fei Nuo coding trees, and the coding efficiency is reduced.
Disclosure of Invention
In order to solve the problems, the invention provides a sensor data optimization acquisition and storage method.
The invention relates to a sensor data optimization acquisition and storage method which adopts the following technical scheme:
One embodiment of the invention provides a sensor data optimized acquisition and storage method, which comprises the following steps:
Collecting historical temperature data and real-time temperature data by using a temperature sensor;
Obtaining the types of all characters in the historical temperature data and the frequency of each character in the historical temperature data according to the historical temperature data; obtaining the types of all characters in the real-time temperature data and the frequency of each character in the real-time temperature data according to the real-time temperature data;
Presetting a character type range, and obtaining the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of a Fei Nuo coding algorithm according to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data;
Obtaining the total number of temperature values contained in the adjusted real-time temperature data according to the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm and the total number of temperature values contained in the real-time temperature data, and obtaining new real-time temperature data according to the total number of temperature values contained in the adjusted real-time temperature data;
Obtaining Fei Nuo coding trees corresponding to the historical temperature data according to the frequency numbers of all characters in the historical temperature data, obtaining merging temperature data and a reconstructed Fischer-Tropsch coding tree according to the influence of the frequency difference of the corresponding characters in the new real-time temperature data and the historical temperature data on the compression rate of a Fei Nuo coding algorithm, obtaining first coding data according to the reconstructed Fischer-Tropsch coding tree and the merging temperature data, and obtaining second coding data according to Fei Nuo coding trees corresponding to the historical temperature data and the merging temperature data;
The first encoded data or the second encoded data is stored.
Further, the method for acquiring the historical temperature data and the real-time temperature data by using the temperature sensor comprises the following specific steps:
The method comprises the steps of utilizing a temperature sensor to collect indoor temperature data, wherein the sampling time of the temperature sensor is to output a temperature value every TQ seconds, TQ is preset first time, time sequence data formed by all temperature values collected in the last TW hours is recorded as indoor temperature data, TW is preset second time, time sequence data formed by the last th temperature values in the indoor temperature data is used as real-time temperature data, data except the real-time temperature data in the indoor temperature data is used as historical temperature data, th is the number of preset temperature values, the indoor temperature data comprises a plurality of temperature values, and each temperature value can be regarded as a character.
Further, the category of all characters in the historical temperature data and the frequency of each character in the historical temperature data are obtained according to the historical temperature data; obtaining the types of all characters in the real-time temperature data and the frequency of each character in the real-time temperature data according to the real-time temperature data, wherein the method comprises the following specific steps:
acquiring the types of all characters in the historical temperature data according to the time sequence from left to right, and counting the frequency of each character in the historical temperature data to obtain the frequency of each character in the historical temperature data;
and acquiring the types of all characters in the real-time temperature data according to the time sequence from left to right, and counting the frequency of each character in the real-time temperature data to obtain the frequency of each character in the real-time temperature data.
Further, according to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data, the influence of the frequency difference of the real-time temperature data and the corresponding character in the historical temperature data on the compression rate of the Fei Nuo coding algorithm is obtained, and the method comprises the following specific steps:
Obtaining the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data according to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data, and obtaining the influence of the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm according to the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data.
Further, according to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data, the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data is obtained, and the method comprises the following specific steps:
In the method, in the process of the invention, Is the frequency of the occurrence of the ith character in the character type range in real-time temperature data,/>For the frequency of occurrence of the ith character in the range of character types in the historical temperature data,/>Frequency of j-th character in character type range in real-time temperature dataFrequency of occurrence of jth character in the character type range in historical temperature data,/>Is the total number of all characters in the character category range,/>The frequency difference of the ith character in the character type range in the real-time temperature data and the historical temperature data is obtained.
Further, according to the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data, the influence of the frequency difference of the corresponding character in the real-time temperature data and the historical temperature data on the compression rate of the Fei Nuo coding algorithm is obtained, and the method comprises the following specific steps:
In the method, in the process of the invention, Is the frequency difference of the ith character in the character type range in the real-time temperature data and the historical temperature data,/>Is the frequency of the occurrence of the ith character in the character type range in real-time temperature data,/>Is the total number of all characters in the character category range,/>As a logarithmic function with 2 as the base,/>As an exponential function with a base of natural constant,For the influence of the frequency difference of corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm, the method comprises the following steps ofTo take absolute value.
Further, the method for obtaining the total number of the temperature values contained in the adjusted real-time temperature data according to the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of the Fei Nuo coding algorithm and the total number of the temperature values contained in the real-time temperature data comprises the following specific steps:
In the method, in the process of the invention, For the total number of temperature values contained in the real-time temperature data,/>For preset parameters,/>For the influence of the frequency difference of corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm, the method comprises the following steps ofAs an exponential function based on natural constants,/>The total number of temperature values contained in the adjusted real-time temperature data.
Further, the step of obtaining new real-time temperature data according to the total number of temperature values contained in the adjusted real-time temperature data includes the following specific steps:
taking time sequence data formed by the last th1 temperature values in the indoor temperature data as new real-time temperature data, wherein th1 is the total number of temperature values contained in the adjusted real-time temperature data.
Further, the Fei Nuo code tree corresponding to the historical temperature data is obtained according to the frequency of all the characters in the historical temperature data, which comprises the following specific steps:
And coding according to the frequency of all characters in the historical temperature data by using a Fisher coding algorithm to obtain Fei Nuo coding trees corresponding to the historical temperature data.
Further, the method for obtaining the combined temperature data and the reconstructed fisher code tree according to the influence of the frequency difference of the corresponding characters in the new real-time temperature data and the historical temperature data on the compression rate of the Fei Nuo coding algorithm, obtaining the first coding data according to the reconstructed fisher code tree and the combined temperature data, and obtaining the second coding data according to the Fei Nuo coding tree and the combined temperature data corresponding to the historical temperature data comprises the following specific steps:
Presetting an empirical threshold, denoted as T, if ，/>Combining the historical temperature data and the new real-time temperature data according to a time sequence order to obtain combined temperature data according to the influence of frequency difference of corresponding characters in the new real-time temperature data and the historical temperature data on compression rate of Fei Nuo coding algorithm, obtaining a reconstructed Fisher coding tree according to the frequency of all characters in the combined temperature data by utilizing the Fisher coding algorithm, and coding the combined temperature data according to the reconstructed Fisher coding tree by utilizing the Fisher coding algorithm to obtain first coding data;
If it is And combining the historical temperature data with the new real-time temperature data according to a time sequence order to obtain combined temperature data, and encoding the combined temperature data by utilizing a Fisher encoding algorithm according to Fei Nuo encoding trees corresponding to the historical temperature data to obtain second encoded data.
The technical scheme of the invention has the beneficial effects that: the conventional fixed fisher code tree is built in advance through the character frequency in the statistical data, and the frequency of the real-time data may be changed due to the continuous input of the real-time data in the sensor data, which may be affected by external factors, and the compression effect of compressing the real-time data by using the original code tree may be deteriorated.
According to the invention, the frequency characteristic of the real-time temperature data is analyzed, the difference between the real-time temperature data and the frequency distribution characteristic of the historical temperature data is calculated according to the frequency distribution characteristic of the real-time temperature data, and the Fei Nuo coding tree is reconstructed according to the self-adaptive dynamic of the difference between the real-time temperature data and the frequency distribution characteristic of the real-time temperature data, so that the self-adaptive reconstructed coding tree can be better adapted to the real-time change condition of the data, the overall compression effect of the data is improved, the size of the storage space occupied by the sensor data is reduced, and the resources required by storage are reduced.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of steps of a method for optimizing acquisition and storage of sensor data according to an embodiment of the present invention.
Detailed Description
In order to further describe the technical means and effects adopted by the invention to achieve the preset aim, the following is a detailed description of specific implementation, structure, characteristics and effects of a sensor data optimization acquisition and storage method according to the invention with reference to the attached drawings and the preferred embodiment. In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following specifically describes a specific scheme of the sensor data optimization acquisition and storage method provided by the invention with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of steps of a method for optimizing acquisition and storage of sensor data according to an embodiment of the present invention is shown, the method includes the following steps:
Step S001, acquiring historical temperature data and real-time temperature data by using a temperature sensor.
In this embodiment, the sensor in the smart home system is used to monitor indoor temperature, illumination, air quality, etc. so as to realize automatic control and energy-saving management of the smart home, and in this embodiment, only the temperature is taken as an example, and data needs to be collected first.
Specifically, the temperature sensor is used to collect indoor temperature data, wherein the sampling time of the temperature sensor is that one temperature value is output every TQ seconds, TQ is preset first time, the embodiment describes with tq=10 that time sequence data formed by all temperature values collected in the last TW hours is recorded as indoor temperature data, TW is preset second time, the embodiment describes with tw=1 that time sequence data formed by the last th temperature values in the indoor temperature data is used as real-time temperature data, and data except the real-time temperature data in the indoor temperature data is used as historical temperature data. th is the number of preset temperature values, and in this embodiment, th=100 is described, and it should be noted that, in the temperature data of this embodiment, the minimum difference between different temperature values is 0.1, the indoor temperature data includes a plurality of temperature values, and each temperature value can be regarded as a character, that is, the historical temperature data and the real-time temperature data both include a plurality of characters.
So far, historical temperature data and real-time temperature data are obtained.
And step S002, obtaining the types of all characters in the historical temperature data and the frequency of each character in the historical temperature data according to the historical temperature data.
It should be noted that, the indoor temperature data collected by the temperature sensor may be affected by various factors, for example, in summer, the temperature difference between day and night is large, the indoor temperature data collected by the temperature sensor is not necessarily exactly equal to the set temperature of the air conditioner, the indoor temperature may be affected by the outdoor temperature, and the interference of external factors may cause the change of the indoor temperature data. That is, the interference of external factors may cause the frequency distribution of the indoor temperature data collected by the temperature sensor to change. And Fei Nuo is a coding tree constructed based on the probability distribution of the historical data, if the temperature character frequency collected by the temperature sensor changes, the character frequency distribution in the real-time temperature data will have a difference with the character frequency distribution in the historical temperature data, and the coding effect is worse when the difference is larger. Therefore, the frequency characteristic of the real-time temperature data needs to be analyzed, the frequency change characteristic of the characters in the real-time temperature data is captured in time, the Fei Nuo coding tree is updated by self-adaptation through analyzing the frequency distribution difference of the real-time temperature data and the historical temperature data, and the compression effect of compressing the real-time temperature data by using the updated coding tree is better.
In the indoor temperature data collected by the temperature sensor, the frequency of various characters in the indoor temperature data collected by the temperature sensor also changes due to the influence of external environment factors. For example, when the external temperature increases or decreases, even if the current air conditioner is in an operating state, the indoor temperature is still affected, the distribution of various characters of the indoor temperature data is changed, and the compression effect of the data by using the coding tree is affected by the change of the character distribution. Therefore, the frequency of various characters in the historical temperature data acquired by the temperature sensor needs to be counted.
Specifically, the types of all characters in the historical temperature data and the frequency of occurrence of each character in the historical temperature data are obtained according to the historical temperature data, and the method specifically comprises the following steps:
and acquiring the types of all the characters in the historical temperature data according to the time sequence from left to right, and counting the frequency of each character in the historical temperature data to obtain the frequency of each character in the historical temperature data.
Thus, the frequency of occurrence of each character in the historical temperature data is obtained.
And step S003, obtaining the types of all characters in the real-time temperature data and the frequency of each character in the real-time temperature data according to the real-time temperature data.
It should be noted that, due to the interference of the external environment, there is a certain difference between the real-time temperature data and the historical temperature data. When compressing real-time temperature data using the fisher code, it is important to keep the frequency distribution of the real-time temperature data synchronized with the code tree constructed using the historical temperature data. The closer the frequency of each character in the real-time temperature data and the corresponding character in the coding tree is, the better the compression effect is. When the character frequency in the real-time temperature data is different from the character frequency in the historical temperature data, the compression effect is affected, and the larger the difference is, the worse the compression effect is. Therefore, the frequency of various characters in the real-time temperature data needs to be counted, and the difference between the frequency and the historical temperature data is calculated.
Specifically, the types of all characters in the real-time temperature data and the frequency of occurrence of each character in the real-time temperature data are obtained according to the real-time temperature data, and the method specifically comprises the following steps:
and acquiring the types of all characters in the real-time temperature data according to the time sequence from left to right, and counting the frequency of each character in the real-time temperature data to obtain the frequency of each character in the real-time temperature data.
Thus, the frequency of each character in the real-time temperature data is obtained.
Step S004, presetting a character type range, obtaining the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data according to the frequency of any character in the character type range in the historical temperature data and the historical temperature data, and obtaining the influence of the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm according to the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data.
It should be noted that, when the frequency difference of the characters in the real-time temperature data and the frequency difference of the characters in the historical temperature data are too large, the compression effect of compressing the real-time temperature data by using the Fei Nuo coding tree corresponding to the historical temperature data is poor, so that the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data needs to be calculated according to the frequency of the corresponding characters in the real-time temperature data and the historical temperature data.
Specifically, a character type range is preset, and according to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data, the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data is obtained, wherein the frequency difference is specifically as follows:
Presetting a character type range, which is recorded as This example shows/>Is thatFor example, the character type range includes a plurality of different characters, and the minimum difference between the different characters is 0.1, and it should be specifically noted that the character type range may include all characters appearing in the acquired historical temperature data and real-time temperature data.
In the method, in the process of the invention,Is the frequency of the occurrence of the ith character in the character type range in real-time temperature data,/>For the frequency of occurrence of the ith character in the range of character types in the historical temperature data,/>Frequency of j-th character in character type range in real-time temperature dataFrequency of occurrence of jth character in the character type range in historical temperature data,/>Is the total number of all characters in the character category range,/>The frequency difference of the ith character in the character type range in the real-time temperature data and the historical temperature data is obtained.
If it isThe larger the absolute value of (2) is, the larger the frequency difference between the real-time temperature data and the historical temperature data of the ith character in the character type range is; if/>The smaller the value of (2) is, the smaller the frequency difference between the real-time temperature data and the historical temperature data of the ith character in the character type range is.
It should be noted that, the principle of Fei Nuo coding is that the higher the frequency of a character is, the fewer bits the character occupies, so if the frequency of a character is changed greatly, the influence on the compression rate is also great, and the greater the influence on the compression rate is, the worse the compression effect is when the Fei Nuo coding tree corresponding to the historical temperature data is used for compressing the real-time temperature data. Therefore, the influence of the difference between the real-time temperature data and the character frequency difference in the historical temperature data on the compression rate needs to be calculated.
Specifically, according to the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data, the influence of the frequency difference of the corresponding character in the real-time temperature data and the historical temperature data on the compression rate of the Fei Nuo coding algorithm is obtained, and the method specifically comprises the following steps:
In the method, in the process of the invention, For the frequency difference of the ith character in the character type range in the real-time temperature data and the historical temperature data, it should be noted that/>, in the above formulaNot equal to 0, if/>Equal to 0, then directly set the/>Corresponding toIs 0,/>Is the frequency of the occurrence of the ith character in the character type range in real-time temperature data,/>Is the total number of all characters in the character category range,/>As a logarithmic function with 2 as the base,/>As an exponential function based on natural constants,/>For the influence of the frequency difference of corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm, the method comprises the following steps ofTo take absolute value.
It should be noted that the number of the substrates,A symbol representing the frequency difference between the real-time temperature data and the historical temperature data of the ith character in the character type range, if the frequency of the ith character in the character type range in the real-time temperature data is larger than the frequency of the ith character in the historical temperature data, namely/>Then/>; If the frequency of occurrence of the ith character in the character type range in the real-time temperature data is smaller than the frequency of occurrence in the historical temperature data, namely/>Then/>，Representing the/>, in real-time temperature dataThe larger the difference of the compression effect when the class characters are encoded by using Fei Nuo encoding trees corresponding to the historical temperature data, the larger the difference of the frequencies of the corresponding characters in the real-time temperature data and the historical temperature data, the larger the influence of the real-time temperature data on the compression effect is compressed by using Fei Nuo encoding tables corresponding to the historical temperature data, and the smaller the difference of the frequencies of the corresponding characters in the real-time temperature data and the historical temperature data, the smaller the influence of the real-time temperature data on the compression effect is compressed by using Fei Nuo encoding tables corresponding to the historical temperature data.
So far, the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm is obtained.
And S005, obtaining the total number of the temperature values contained in the adjusted real-time temperature data according to the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of the Fei Nuo coding algorithm and the total number of the temperature values contained in the real-time temperature data, and obtaining new real-time temperature data according to the total number of the temperature values contained in the adjusted real-time temperature data.
When calculating the frequency difference between the real-time temperature data and the historical temperature data, if the length of the real-time temperature data is too long, that is, th is too large, the compression effect when encoding and compressing the real-time temperature data by using the Fei Nuo encoding tree corresponding to the historical temperature data becomes poor when the character frequency in the real-time temperature data is greatly changed, and if the length of the real-time temperature data is too short, that is, th is too small, the calculation amount in the compression process is too large. Therefore, the preset length range for acquiring the real-time temperature data needs to be adjusted according to the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate.
Specifically, according to the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm and the total number of the temperature values contained in the real-time temperature data, the total number of the temperature values contained in the adjusted real-time temperature data is obtained, specifically as follows:
In the method, in the process of the invention, For the total number of temperature values contained in the real-time temperature data,/>To preset parameters, the embodiment usesTo describe,/>For the influence of the frequency difference of corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm, the method comprises the following steps ofAs an exponential function based on natural constants,/>The total number of temperature values contained in the adjusted real-time temperature data.
It should be noted that, if the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data has a greater influence on the compression rate, the total number of temperature values contained in the adjusted real-time temperature data is smaller, and the frequency change condition of various characters in the real-time temperature data is reflected; if the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data has smaller influence on the compression rate, the total number of the temperature values contained in the adjusted real-time temperature data is larger.
Further, new real-time temperature data is obtained according to the total number of temperature values contained in the adjusted real-time temperature data, and the method specifically comprises the following steps:
taking time sequence data formed by the last th1 temperature values in the indoor temperature data as new real-time temperature data, wherein th1 is the total number of temperature values contained in the adjusted real-time temperature data.
So far, new real-time temperature data are obtained.
Step S006, obtaining Fei Nuo coding trees corresponding to the historical temperature data according to the frequency of all characters in the historical temperature data, obtaining merging temperature data and a reconstructed Fischer-Tropsch coding tree according to the influence of the frequency difference of the corresponding characters in the new real-time temperature data and the historical temperature data on the compression rate of the Fei Nuo coding algorithm, obtaining first coding data according to the reconstructed Fischer-Tropsch coding tree and the merging temperature data, obtaining second coding data according to Fei Nuo coding trees corresponding to the historical temperature data and the merging temperature data, and storing the first coding data or the second coding data.
It should be noted that, the above obtained frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data affects the compression rate of the Fei Nuo coding algorithm, and similarly, the new frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data affects the compression rate of the Fei Nuo coding algorithm, and the reconstructed fisher code tree and the combined temperature data are obtained by judging the impact of the compression rate, and then the coding compression can be performed according to the reconstructed fisher code tree.
Specifically, a Fei Nuo coding tree corresponding to the historical temperature data is obtained according to the frequency of all characters in the historical temperature data, the combined temperature data and the reconstructed Fischer-Tropsch coding tree are obtained according to the influence of the frequency difference of the corresponding characters in the new real-time temperature data and the historical temperature data on the compression rate of the Fei Nuo coding algorithm, first coding data are obtained according to the reconstructed Fischer-Tropsch coding tree and the combined temperature data, second coding data are obtained according to a Fei Nuo coding tree corresponding to the historical temperature data and the combined temperature data, and the first coding data or the second coding data are stored, wherein the steps are as follows:
and coding according to the frequency of all characters in the historical temperature data by using a Fisher coding algorithm to obtain Fei Nuo coding trees corresponding to the historical temperature data. It should be noted that, according to the existing method of obtaining the Fei Nuo code tree corresponding to the historical temperature data according to the frequency of all the characters in the historical temperature data, the code tree is Fei Nuo code algorithm, which is not described in detail in this embodiment.
A threshold of empirical value is preset, denoted as T, in this embodiment described by t=0.5, if，/>The influence of the frequency difference of the corresponding characters in the new real-time temperature data and the historical temperature data on the compression rate of the Fei Nuo coding algorithm is that the frequency difference of the corresponding characters in the new real-time temperature data and the historical temperature data is overlarge, the influence on the compression rate is overlarge, and the Fei Nuo coding tree corresponding to the historical temperature data needs to be reconstructed.
The weight of the tool body is as follows: combining the historical temperature data and the new real-time temperature data according to a time sequence order to obtain combined temperature data, utilizing a Fisher encoding algorithm according to the frequency of all characters in the combined temperature data to obtain a reconstructed Fisher encoding tree, and encoding the combined temperature data according to the reconstructed Fisher encoding tree by utilizing the Fisher encoding algorithm to obtain first encoded data; it should be noted that, according to the frequency utilization fischer encoding algorithm of all characters in the merging temperature data, the existing method of reconstructing the fischer encoding tree into Fei Nuo encoding algorithm is obtained, and according to the existing method of reconstructing the fischer encoding tree into Fei Nuo encoding algorithm, the merging temperature data is encoded by utilizing the fischer encoding algorithm, which is not described in detail in this embodiment.
If it isAnd the fact that the frequency difference of characters corresponding to the new real-time temperature data and the historical temperature data is small is indicated, the influence on the compression rate is within an acceptable range, a Fei Nuo coding tree corresponding to the historical temperature data does not need to be reconstructed, the historical temperature data and the new real-time temperature data are combined according to a time sequence to obtain combined temperature data, and the combined temperature data is coded by utilizing a Fisher coding algorithm according to a Fei Nuo coding tree corresponding to the historical temperature data to obtain second coding data. It should be noted that, the merging temperature data is encoded by using the fisher coding algorithm according to the Fei Nuo coding tree corresponding to the historical temperature data, which is the existing method of the Fei Nuo coding algorithm, and this embodiment is not described again.
The first encoded data or the second encoded data is stored.
It should be noted that, the above is only to take one historical temperature data and one new real-time temperature data as an example for analysis, and if another new real-time temperature data exists, the combined temperature data may be regarded as one historical temperature data for analysis.
So far, through the temperature data that gathers temperature sensor, encode according to the frequency change of character, realize gathering the storage to the optimization of sensor data.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the invention, but any modifications, equivalent substitutions, improvements, etc. within the principles of the present invention should be included in the scope of the present invention.
Claims (6)
1. The sensor data optimized acquisition and storage method is characterized by comprising the following steps of:
Collecting historical temperature data and real-time temperature data by using a temperature sensor;
Obtaining the types of all characters in the historical temperature data and the frequency of each character in the historical temperature data according to the historical temperature data; obtaining the types of all characters in the real-time temperature data and the frequency of each character in the real-time temperature data according to the real-time temperature data; each temperature value in the temperature data is used as a character; the character type range comprises a plurality of different characters, the minimum difference value of the different characters is 0.1, and the character type range can fully comprise the acquired characters appearing in the historical temperature data and the real-time temperature data;
Presetting a character type range, and obtaining the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of a Fei Nuo coding algorithm according to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data;
Obtaining the total number of temperature values contained in the adjusted real-time temperature data according to the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm and the total number of temperature values contained in the real-time temperature data, and obtaining new real-time temperature data according to the total number of temperature values contained in the adjusted real-time temperature data;
Obtaining Fei Nuo coding trees corresponding to the historical temperature data according to the frequency numbers of all characters in the historical temperature data, obtaining merging temperature data and a reconstructed Fischer-Tropsch coding tree according to the influence of the frequency difference of the corresponding characters in the new real-time temperature data and the historical temperature data on the compression rate of a Fei Nuo coding algorithm, obtaining first coding data according to the reconstructed Fischer-Tropsch coding tree and the merging temperature data, and obtaining second coding data according to Fei Nuo coding trees corresponding to the historical temperature data and the merging temperature data;
storing the first encoded data or the second encoded data;
According to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data, the influence of the frequency difference of the real-time temperature data and the corresponding character in the historical temperature data on the compression rate of Fei Nuo coding algorithm is obtained, and the method comprises the following specific steps:
Obtaining the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data according to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data, and obtaining the influence of the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm according to the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data;
According to the frequency difference of any character in the character type range in the real-time temperature data and the historical temperature data, the influence of the frequency difference of the corresponding character in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm is obtained, and the method comprises the following specific steps:
In the method, in the process of the invention, For the frequency difference of the ith character in the character type range in the real-time temperature data and the historical temperature data,Is the frequency of the occurrence of the ith character in the character type range in real-time temperature data,/>Is the total number of all characters in the character category range,/>As a logarithmic function with 2 as the base,/>As an exponential function based on natural constants,/>For the magnitude of the effect of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm,Taking an absolute value;
The method comprises the specific steps of obtaining the total number of the temperature values contained in the adjusted real-time temperature data according to the influence of the frequency difference of the corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm and the total number of the temperature values contained in the real-time temperature data, wherein the specific steps are as follows:
In the method, in the process of the invention, For the total number of temperature values contained in the real-time temperature data,/>For preset parameters,/>For the influence of the frequency difference of corresponding characters in the real-time temperature data and the historical temperature data on the compression rate of Fei Nuo coding algorithm, the method comprises the following steps ofAs an exponential function based on natural constants,/>The total number of the temperature values contained in the adjusted real-time temperature data is calculated;
Obtaining merging temperature data and a reconstructed Fischer-Tropsch coding tree according to the influence of the frequency difference of corresponding characters in the new real-time temperature data and the historical temperature data on the compression rate of a Fei Nuo coding algorithm, obtaining first coding data according to the reconstructed Fischer-Tropsch coding tree and the merging temperature data, and obtaining second coding data according to Fei Nuo coding tree and the merging temperature data corresponding to the historical temperature data, wherein the method comprises the following specific steps of:
Presetting an empirical threshold, denoted as T, if ，/>Combining the historical temperature data and the new real-time temperature data according to a time sequence order to obtain combined temperature data according to the influence of frequency difference of corresponding characters in the new real-time temperature data and the historical temperature data on compression rate of Fei Nuo coding algorithm, obtaining a reconstructed Fisher coding tree according to the frequency of all characters in the combined temperature data by utilizing the Fisher coding algorithm, and coding the combined temperature data according to the reconstructed Fisher coding tree by utilizing the Fisher coding algorithm to obtain first coding data;
If it is And combining the historical temperature data with the new real-time temperature data according to a time sequence order to obtain combined temperature data, and encoding the combined temperature data by utilizing a Fisher encoding algorithm according to Fei Nuo encoding trees corresponding to the historical temperature data to obtain second encoded data.
2. The method for optimally acquiring and storing sensor data according to claim 1, wherein the step of acquiring the historical temperature data and the real-time temperature data by using the temperature sensor comprises the following specific steps:
The method comprises the steps of utilizing a temperature sensor to collect indoor temperature data, wherein the sampling time of the temperature sensor is that one temperature value is output every TQ seconds, TQ is preset first time, time sequence data formed by all temperature values collected in the last TW hours is recorded as indoor temperature data, TW is preset second time, time sequence data formed by the last th temperature values in the indoor temperature data is used as real-time temperature data, data except the real-time temperature data in the indoor temperature data is used as historical temperature data, th is the number of preset temperature values, the indoor temperature data comprises a plurality of temperature values, and each temperature value is regarded as a character.
3. The method for optimizing, collecting and storing sensor data according to claim 1, wherein the category of all characters in the historical temperature data and the frequency of occurrence of each character in the historical temperature data are obtained according to the historical temperature data; obtaining the types of all characters in the real-time temperature data and the frequency of each character in the real-time temperature data according to the real-time temperature data, wherein the method comprises the following specific steps:
acquiring the types of all characters in the historical temperature data according to the time sequence from left to right, and counting the frequency of each character in the historical temperature data to obtain the frequency of each character in the historical temperature data;
and acquiring the types of all characters in the real-time temperature data according to the time sequence from left to right, and counting the frequency of each character in the real-time temperature data to obtain the frequency of each character in the real-time temperature data.
4. The method for optimizing, collecting and storing sensor data according to claim 1, wherein the step of obtaining the frequency difference between the real-time temperature data and the historical temperature data of any character in the character type range according to the frequency of any character in the character type range in the historical temperature data and the real-time temperature data comprises the following specific steps:
In the method, in the process of the invention, Is the frequency of the occurrence of the ith character in the character type range in real-time temperature data,/>For the frequency of occurrence of the ith character in the range of character types in the historical temperature data,/>Frequency of j-th character in character type range in real-time temperature dataFrequency of occurrence of jth character in the character type range in historical temperature data,/>Is the total number of all characters in the character category range,/>The frequency difference of the ith character in the character type range in the real-time temperature data and the historical temperature data is obtained.
5. The method for optimizing acquisition and storage of sensor data according to claim 1, wherein the step of acquiring new real-time temperature data according to the total number of temperature values contained in the adjusted real-time temperature data comprises the following specific steps:
taking time sequence data formed by the last th1 temperature values in the indoor temperature data as new real-time temperature data, wherein th1 is the total number of temperature values contained in the adjusted real-time temperature data.
6. The method for optimizing, collecting and storing sensor data according to claim 1, wherein the Fei Nuo code tree corresponding to the historical temperature data is obtained according to the frequency of all characters in the historical temperature data, comprising the following specific steps:
And coding according to the frequency of all characters in the historical temperature data by using a Fisher coding algorithm to obtain Fei Nuo coding trees corresponding to the historical temperature data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410195039.7A CN117767960B (en) | 2024-02-22 | 2024-02-22 | Sensor data optimization acquisition and storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410195039.7A CN117767960B (en) | 2024-02-22 | 2024-02-22 | Sensor data optimization acquisition and storage method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117767960A CN117767960A (en) | 2024-03-26 |
| CN117767960B true CN117767960B (en) | 2024-06-18 |
Family
ID=90326068
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410195039.7A Active CN117767960B (en) | 2024-02-22 | 2024-02-22 | Sensor data optimization acquisition and storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117767960B (en) |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5945933A (en) * | 1998-01-27 | 1999-08-31 | Infit Ltd. | Adaptive packet compression apparatus and method |
| JP4374712B2 (en) * | 2000-04-04 | 2009-12-02 | ソニー株式会社 | Data processing apparatus, data processing method, and recording medium |
| CN112667712B (en) * | 2020-12-31 | 2023-03-17 | 安徽工业大学 | Grouped accurate histogram data publishing method based on differential privacy |
| CN116886104B (en) * | 2023-09-08 | 2023-11-21 | 西安小草植物科技有限责任公司 | Intelligent medical data analysis method based on artificial intelligence |
| CN117216023B (en) * | 2023-11-07 | 2024-01-26 | 陕西长瑞安驰信息技术集团有限公司 | Large-scale network data storage method and system |
-
2024
- 2024-02-22 CN CN202410195039.7A patent/CN117767960B/en active Active
Non-Patent Citations (2)
| Title |
|---|
| 一种通用在线自适应压缩编码;张立清，解林;计算机工程;19940620(03);全文 * |
| 实时数据库在企业能源管理系统(EMS)中的应用研究;茅心怡;牟磊;;中小企业管理与科技(下旬刊);20120725(07);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117767960A (en) | 2024-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116153452B (en) | Medical electronic medical record storage system based on artificial intelligence | |
| CN116153453B (en) | Intelligent ring information management method and system based on cloud edge cooperation | |
| CN116155298B (en) | Ship lock remote management system based on internet data | |
| CN117540238B (en) | Data security management method for industrial digital information acquisition device | |
| CN116910285B (en) | Intelligent traffic data optimized storage method based on Internet of things | |
| CN117235557B (en) | Electrical equipment fault rapid diagnosis method based on big data analysis | |
| CN117278054B (en) | Intelligent power grid monitoring data storage method and system | |
| CN115987294A (en) | Multidimensional data processing method of Internet of things | |
| CN117376429A (en) | Intelligent data compression method for wireless sensor network | |
| CN117608499A (en) | Intelligent traffic data optimal storage method based on Internet of things | |
| CN116827350A (en) | Flexible work platform intelligent supervision method and system based on cloud edge cooperation | |
| CN116095347A (en) | Construction engineering safety construction method and system based on video analysis | |
| CN117767960B (en) | Sensor data optimization acquisition and storage method | |
| CN116975503B (en) | Soil erosion information management method and system | |
| CN117040542B (en) | Intelligent comprehensive distribution box energy consumption data processing method | |
| CN117076408A (en) | Temperature monitoring big data transmission method | |
| CN116232935B (en) | Big data transmission method for monitoring Internet of things | |
| CN115695564B (en) | Efficient transmission method of Internet of things data | |
| CN116567269A (en) | Spectrum monitoring data compression method based on signal-to-noise separation | |
| CN117150224B (en) | User behavior data storage analysis method based on Internet of things | |
| CN117171399B (en) | New energy data optimized storage method based on cloud platform | |
| CN117294397B (en) | Wireless communication data storage method and system | |
| CN116450592B (en) | Working condition self-adaptive compression method and system based on industrial big data characteristics | |
| CN117792403B (en) | Distributed agricultural data storage management method based on stream big data technology | |
| CN112329923A (en) | Model compression method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |