CN117119022A - Energy consumption data processing method, system, equipment and medium based on MQTT protocol - Google Patents
Energy consumption data processing method, system, equipment and medium based on MQTT protocol Download PDFInfo
- Publication number
- CN117119022A CN117119022A CN202311387296.2A CN202311387296A CN117119022A CN 117119022 A CN117119022 A CN 117119022A CN 202311387296 A CN202311387296 A CN 202311387296A CN 117119022 A CN117119022 A CN 117119022A
- Authority
- CN
- China
- Prior art keywords
- sensor
- energy consumption
- consumption data
- flow
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000005265 energy consumption Methods 0.000 title claims abstract description 93
- 238000003672 processing method Methods 0.000 title claims abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 36
- 238000007405 data analysis Methods 0.000 claims abstract description 35
- 238000004891 communication Methods 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims abstract description 18
- 230000005540 biological transmission Effects 0.000 claims abstract description 8
- 238000009826 distribution Methods 0.000 claims description 57
- 230000006870 function Effects 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 15
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 9
- 238000007476 Maximum Likelihood Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 description 8
- 230000004044 response Effects 0.000 description 8
- 230000000875 corresponding effect Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000013459 approach Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 241000255925 Diptera Species 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013477 bayesian statistics method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/27—Regression, e.g. linear or logistic regression
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
- H04L69/162—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/26—Special purpose or proprietary protocols or architectures
Abstract
The application relates to the technical field of data processing, in particular to an energy consumption data processing method, system, equipment and medium based on an MQTT protocol, wherein the method specifically comprises the following steps: deploying a sensor network in a building, and collecting energy consumption data of the building through the sensor network; establishing real-time and efficient communication between the sensor network and a back-end server according to an MQTT protocol; according to the back-end server, calculating and analyzing the energy consumption data, generating an energy consumption data analysis result and transmitting the energy consumption data analysis result to a client; and acquiring the energy consumption data analysis result in real time on the client side based on WebSockets connection, and converting the energy consumption data analysis result into an energy consumption chart according to a graph function library. According to the application, real-time and efficient communication is established between the sensor network and the back-end server through the MQTT protocol, so that timely transmission and processing of energy consumption data are ensured.
Description
Technical Field
The application relates to the technical field of data processing, in particular to an energy consumption data processing method, system, equipment and medium based on an MQTT protocol.
Background
In the technology in the field of the internet of things at present, aiming at the data acquisition and analysis requirements, the technology is generally adopted that the data acquisition and the data analysis are integrated in the same module, so that the equipment can perform data acquisition and preliminary analysis in the same module and can perform instant decision and response according to analysis results, however, with the development of the internet of things, as the acquisition and analysis modules are integrated in the same module, the computing resources of terminal equipment are limited, and when the requirements of large-scale data acquisition and analysis are met, the acquisition and analysis simultaneously occupy the limited resources, so that the equipment cannot meet the data processing requirements, and the system performance and response speed are affected.
However, after the data acquisition and the data analysis are separately and independently set in the field of the internet of things, the data in the sensor network generally needs to be transmitted to a back-end server in time for processing, and under the condition of large-scale data, the traditional communication method can face the problems of communication delay and data loss. Second, because data within the sensor network typically needs to be transmitted to a backend server for processing, different sensors may generate different data traffic, and conventional approaches fail to effectively address traffic distribution and control issues, resulting in some sensors having data that is ignored or other sensors that are excessively accessed, thereby reducing the efficiency of the system. At the same time, conventional methods often fail to take into account parameter uncertainties, which may lead to model instability or unreliability in real-time systems, especially in rapidly changing environments.
Disclosure of Invention
The application aims to provide an energy consumption data processing method, system, equipment and medium based on an MQTT protocol, which are used for establishing real-time and efficient communication between a sensor network and a back-end server through the MQTT protocol, so as to ensure the timely transmission and processing of energy consumption data and solve at least one of the problems in the prior art.
The application provides an energy consumption data processing method based on an MQTT protocol, which specifically comprises the following steps:
deploying a sensor network in a building, and collecting energy consumption data of the building through the sensor network;
establishing real-time and efficient communication between the sensor network and a back-end server according to an MQTT protocol;
according to the back-end server, calculating and analyzing the energy consumption data, generating an energy consumption data analysis result and transmitting the energy consumption data analysis result to a client;
and acquiring the energy consumption data analysis result in real time on the client side based on WebSockets connection, and converting the energy consumption data analysis result into an energy consumption chart according to a graph function library.
Further, the establishing real-time and efficient communication between the sensor network and the backend server according to the MQTT protocol specifically includes:
setting an MQTT Broker at the back-end server, and configuring an MQTT client library for each sensor in the sensor network;
constructing a plurality of themes topic in an MQTT Broker according to the building area and the sensor type;
establishing a flow prediction model for each sensor based on a Bayesian regression algorithm, and dynamically adjusting the current weight of each sensor according to the future predicted flow of each sensor and the current actual flow of each sensor according to the flow prediction model;
and forwarding the energy consumption data packet of each sensor to the corresponding topic according to the current weight of each sensor.
Furthermore, the establishing a flow prediction model for each sensor based on the Bayesian regression algorithm specifically comprises the following steps:
acquiring a historical flow data set and a first feature set of each sensor, wherein the first feature set comprises message frequency, message size, message delay, topic distribution, sensor type, sensor position and sensor equipment energy consumption;
determining a second feature set from the first feature set based on pearson correlation coefficients;
based on the Bayesian regression model, a flow prediction model for each sensor is determined from the historical flow data set for each sensor and the second feature set.
Still further, the determining a second feature set from the first feature set based on the pearson correlation coefficient specifically includes:
setting a correlation coefficient threshold value, and calculating a pearson correlation coefficient of a historical flow data set of each sensor and each feature in the first feature set;
and eliminating the features with the pearson correlation coefficient lower than the correlation coefficient threshold value in the first feature set to obtain a second feature set.
Further, the pearson correlation coefficient satisfiesR represents the pearson correlation coefficient of the historical flow data set for each sensor with each feature in the first feature set, n represents the number of data points for each feature in the first feature set, i represents the serial number of data points for each feature in the historical flow data set or first feature set,values representing each data point of each feature in the first feature set, +.>A value representing each data point of the historical flow data set,/->Mean value representing each feature in the first set of features,/->Representing the average value of the historical traffic dataset.
Further, the determining, based on the bayesian regression model, a flow prediction model of each sensor according to the historical flow data set of each sensor and the second feature set specifically includes:
based on Bayesian regression formulaTaking the historical flow data set of each sensor as output y, and taking the second feature set as input feature matrix X to obtain a parameter vector +.>Data point set and error term->Is a set of data points;
using Gaussian distribution as parameter vectorError term->According to the a priori distribution of the parameter vector +.>Is used for determining the parameter vector +.>Is based on the a priori parameters of the a priori distribution of the error term +.>Is used for determining the error term +.>Prior parameters of the prior distribution of (a);
determining the parameter vector by maximum likelihood estimationLikelihood function value of (2) and said error term +.>Likelihood function values of (2);
based on the MCMC method, according to the parameter vectorIterative sampling is carried out on the data set, the prior distribution, the likelihood function value and the historical flow data set, and the parameter vector is determined through the samples on the converged MCMC chain>Posterior distribution of (2);
based on the MCMC method, according to the error termIterative sampling is carried out on the data set, the prior distribution, the likelihood function value and the historical flow data set, and the error item is determined through the samples on the converged MCMC chain>Posterior distribution of (2);
according to the parameter vectorPosterior distribution of (2) and the error term +.>Establishing a flow prediction model.
Further, the dynamically adjusting the current weight of each sensor according to the future predicted flow rate of each sensor and the current actual flow rate of each sensor according to the flow rate prediction model specifically includes:
setting an initial current weight and an adjustment coefficient for each sensor, wherein the adjustment coefficient is used for controlling the adjustment amplitude of the current weight of each sensor;
determining a future predicted flow for a next time step for each sensor according to the flow prediction model;
based on the weight adjustment formula, according to the initial timeDetermining a new current weight from the pre-weight, the adjustment coefficient, the future predicted flow of the next time step of each sensor, and the current actual flow of each sensor, the weight adjustment formula satisfyingWherein->Representing the new current weight, +.>Representing the initial current weight, +.>Representing adjustment coefficients->Future predicted flow representing the next time step of each,/i>Representing the current actual flow of each sensor;
taking the new current weight asSubstituting the weight adjustment formula to repeat iterative calculation and dynamically adjusting the current weight of each sensor.
The application also provides an energy consumption data processing system based on the MQTT protocol, which specifically comprises:
the data acquisition layer is used for deploying a sensor network in a building and acquiring energy consumption data of the building through the sensor network;
the data transmission layer is used for establishing real-time and efficient communication between the sensor network and the back-end server according to the MQTT protocol;
the data analysis layer is used for calculating and analyzing the energy consumption data according to the back-end server, generating an energy consumption data analysis result and transmitting the energy consumption data analysis result to the client;
and the client application layer is used for acquiring the energy consumption data analysis result in real time based on the WebSockets connection and converting the energy consumption data analysis result into an energy consumption chart according to the graphic function library.
The present application also provides a computer device comprising: memory and processor and computer program stored on the memory, which when executed on the processor, implements an MQTT protocol-based energy consumption data processing method according to any one of the above methods.
The application also provides a computer readable storage medium having stored thereon a computer program which, when run by a processor, implements an energy consumption data processing method based on the MQTT protocol as set forth in any one of the above methods.
Compared with the prior art, the application has at least one of the following technical effects:
1. and real-time and efficient communication is established between the sensor network and the back-end server through the MQTT protocol, so that timely transmission and processing of energy consumption data can be ensured.
2. And a Bayesian regression algorithm is used for establishing a flow prediction model for each sensor, dynamically adjusting the current weight of the sensor, helping to optimize distribution of data flow, ensuring that the high-flow sensor obtains more bandwidth, and improving the efficiency of the system.
3. The pearson correlation coefficient is used for determining the feature set, so that the most relevant features for energy consumption data flow prediction can be identified, and the prediction performance of the model is improved.
4. The Bayesian regression model is adopted, the prior distribution and the posterior distribution are combined, the uncertainty of parameters is better considered, and the robustness of the model is improved.
5. The client terminal is connected based on WebSockets, can acquire the analysis result of the energy consumption data in real time, and converts the result into an energy consumption chart through a graphic function library, so that a user can conveniently monitor and analyze the energy consumption condition.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of an energy consumption data processing method based on an MQTT protocol according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an energy consumption data processing system based on the MQTT protocol according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
Referring to fig. 1, an embodiment of the present application provides an energy consumption data processing method based on MQTT protocol, where the method specifically includes:
s101: deploying a sensor network in a building, and collecting energy consumption data of the building through the sensor network;
and establishing real-time and efficient communication between the sensor network and the back-end server according to the MQTT protocol.
In this embodiment, the MQTT refers to Message Queue Telemetry Transport (message queue telemetry transmission protocol), the communication model of the MQTT protocol includes publicher (Publisher), subscriber (Subscriber) and MQTT Broker (proxy server), the MQTT Broker is a message proxy server based on a publish/subscribe model, a theme needs to be created for publishing and subscribing sensor data, the MQTT Broker is responsible for receiving a message of the publicher and sending the message to the corresponding Subscriber, and is a core of the whole MQTT subscription/publishing, for example, the mos quito open source software can be selected as the MQTT Broker, and message forwarding and storage are realized by installing the mos quito open source software, monitoring ports and configuring an authentication mode.
The sensor network is responsible for sensing data and publishing the data as a message to a specific topic on an MQTT Broker through an MQTT protocol, the data is stored in a message queue of the Broker, a back-end server can selectively subscribe to a topic of interest through the subscriber function of the MQTT, and after subscribing to a certain topic, the back-end server can receive all published messages under the topic and can send notifications, alarms or other instructions through set conditions of overhigh response temperature, overhigh humidity and the like.
In some embodiments, the establishing real-time efficient communication between the sensor network and the backend server according to MQTT protocol specifically includes:
setting an MQTT Broker at the back-end server, and configuring an MQTT client library for each sensor in the sensor network;
constructing a plurality of themes topic in an MQTT Broker according to the building area and the sensor type;
establishing a flow prediction model for each sensor based on a Bayesian regression algorithm, and dynamically adjusting the current weight of each sensor according to the future predicted flow of each sensor and the current actual flow of each sensor according to the flow prediction model;
and forwarding the energy consumption data packet of each sensor to the corresponding topic according to the current weight of each sensor.
In this embodiment, an MQTT Broker is provided on the back-end server, e.g., using a molquitto, to communicate with each sensor in the sensor network, the MQTT client library on each sensor is configured so that they can connect to the MQTT Broker. A series of topics topic is built within MQTT Broker for the publishing and subscription of sensor data, depending on building area and sensor type, for example the following topics may be created: "building/area1/temperature" for temperature data of area1, "building/area2/lighting" for illumination data of area2, "building/area3/humidity" for humidity data of area3, and so on.
In order to achieve flow equalization and dynamic weight adjustment, a Bayesian regression algorithm is used to build a flow prediction model for each sensor, taking features as input, flow as output, and taking into account the uncertainty between features and flow. The future flow of each sensor is dynamically predicted through a flow prediction model, and the obtained predicted value is used for carrying out weight adjustment with the current actual flow of each sensor, so that according to the current weight of each sensor, the energy consumption data packet of the sensor can be dynamically forwarded to the corresponding topic, and the sensor with higher weight forwards more data packets, so that flow balance is realized.
In some embodiments, the establishing a flow prediction model for each sensor based on a bayesian regression algorithm specifically includes:
acquiring a historical flow data set and a first feature set of each sensor, wherein the first feature set comprises message frequency, message size, message delay, topic distribution, sensor type, sensor position and sensor equipment energy consumption;
determining a second feature set from the first feature set based on pearson correlation coefficients;
based on the Bayesian regression model, a flow prediction model for each sensor is determined from the historical flow data set for each sensor and the second feature set.
In this embodiment, a historical flow data set for each sensor is collected and a first feature set is determined, the feature set comprising the following features: message frequency, frequency at which each sensor sends messages; message size, average size of message sent by each sensor; message delay, transmission delay of each sensor message; topic distribution, each sensor issues a message to the distribution of different topics; sensor types, each sensor type, such as a temperature sensor, a humidity sensor, etc.; sensor locations, the location of each sensor in the building; sensor device energy consumption, device energy consumption condition of each sensor.
On the basis of the first feature set, pearson correlation coefficients are used to determine which features are highly correlated with the flow, and a correlation coefficient threshold is set to determine whether features are included in the second feature set.
In some embodiments, the determining the second feature set from the first feature set based on the pearson correlation coefficient specifically includes:
setting a correlation coefficient threshold value, and calculating a pearson correlation coefficient of a historical flow data set of each sensor and each feature in the first feature set;
and eliminating the features with the pearson correlation coefficient lower than the correlation coefficient threshold value in the first feature set to obtain a second feature set.
Specifically, the pearson correlation coefficient satisfiesR represents the pearson correlation coefficient of the historical flow data set of each sensor with each feature in the first feature set, n represents the number of data points for each feature in the first feature set, i represents the number of data points for each feature in the historical flow data set or the first feature set,/-a number of data points for each feature in the first feature set, and/-a number of data points for each feature in the first feature set>Values representing each data point of each feature in the first feature set, +.>A value representing each data point of the historical flow data set,/->Mean value representing each feature in the first set of features,/->Representing the average value of the historical traffic dataset.
In this embodiment, since the Bayesian regression algorithm is directed to features and outputs that have a linear relationship, the pearson correlation coefficient helps to understand the degree of linear correlation between the features and the outputs, with the pearson correlation coefficient ranging in value from-1 to 1. If r is close to 1, the strong positive correlation exists between the characteristic and the output; if r is close to-1, the strong negative correlation exists between the characteristic and the output; if r is close to 0, it means that there is substantially no linear relationship between the feature and the output. Therefore, a value in the interval of [0,1] can be selected as the correlation coefficient threshold.
In some embodiments, the determining, based on the bayesian regression model, a flow prediction model for each sensor based on the historical flow data set and the second feature set for each sensor specifically includes:
based on Bayesian regression formulaTaking the historical flow data set of each sensor as output y, and taking the second feature set as input feature matrix X to obtain a parameter vector +.>Data point set and error term->Is a set of data points;
using Gaussian distribution as parameter vectorError term->According to the a priori distribution of the parameter vector +.>Is used for determining the parameter vector +.>Is based on the a priori parameters of the a priori distribution of the error term +.>Is used for determining the error term +.>Prior parameters of the prior distribution of (a);
determining the parameter vector by maximum likelihood estimationLikelihood function value of (2) and said error term +.>Likelihood function values of (2);
based on the MCMC method, according to the parameter vectorIterative sampling is carried out on the data set, the prior distribution, the likelihood function value and the historical flow data set, and the parameter vector is determined through the samples on the converged MCMC chain>Posterior distribution of (2);
based on the MCMC method, according to the error termIterative sampling is carried out on the data set, the prior distribution, the likelihood function value and the historical flow data set, and the error item is determined through the samples on the converged MCMC chain>Posterior distribution of (2);
according to the parameter vectorPosterior distribution of (2) and the error term +.>Establishing a flow prediction model.
In this embodiment, in Bayesian regression, a priori distribution is used to express uncertainty of model parameters, and initial assumptions in the absence of observed data, assuming a parameter vectorIs a random variable with a priori distribution (e.g. gaussian distribution) while also assuming the error term +.>Gaussian distribution is obeyed because it has important properties in statistics and is suitable for modeling many natural phenomena.
For the case of gaussian distribution, two parameters need to be estimated: mean value ofSum of variances->This can be done by maximum likelihood estimation (Maximum Likelihood Estimation, MLE), i.e. finding the parameter value that maximizes the probability of the data under a given model. The objective of the maximum likelihood estimation is to maximize the likelihood function, i.e. the conditional probability of the observed data under a given parameter. In normal distribution, the estimates of the mean and variance can be calculated from the sample mean and sample variance.
In Bayesian statistics, the mean value can be calculatedSum of variances->Considered as random variables rather than fixed constants. The initial uncertainty for these parameters is represented by introducing a priori distributions, which are then updated using bayesian theorem, resulting in posterior distributions, i.e. by means of known a priori distributions and likelihood functions, the posterior distribution of the parameters can be calculated, giving a set of possible values of the parameters, each with a certain probability. Under the Bayesian framework, bayesian inferences can be made, such as calculating expected values of parameters, confidence intervals, etc.
However, since in the case of flow prediction, it is difficult to directly calculate the analytical expression of the posterior distribution, particularly for complex models. In this case, a numerical method, such as a markov chain monte carlo (Markov chain Monte Carlo, MCMC) method, may be used to sample from the posterior distribution to obtain the distribution information of the parameters. The MCMC method is a commonly used bayesian inference method, and is used for sampling from a posterior distribution, and in each iteration of MCMC, according to a certain sampling rule, a new parameter value is calculated through the value of the current parameter and the prior distribution, and this process generates a series of samples of the parameter value, which is called MCMC chain. After a certain number of iterative samplings, it is necessary to check whether the MCMC chain has converged to a stable posterior distribution, after which the samples on the chain can be used to approximate the posterior distribution of the calculated parameters. Specifically, an initial parameter value is selected as a starting point, a candidate parameter value is generated based on the current parameter value, and then an acceptance probability is calculated, which is a comparison of the ratio of the likelihood value under the current parameter value to the product of the prior distribution and the ratio of the likelihood value under the candidate parameter value to the product of the prior distribution. Whether to accept the candidate parameter value is determined based on the acceptance probability and a randomly generated value, and if the random number is smaller than the acceptance probability, the candidate parameter value is accepted as a new parameter value, otherwise the current parameter value is retained. After repeating the steps of employing and calculating the probability of reception a number of times, a series of parameter values are generated which gradually approach the target posterior distribution.
In some embodiments, the dynamically adjusting the current weight of each sensor according to the future predicted flow rate of each sensor and the current actual flow rate of each sensor according to the flow rate prediction model specifically includes:
setting an initial current weight and an adjustment coefficient for each sensor, wherein the adjustment coefficient is used for controlling the adjustment amplitude of the current weight of each sensor;
determining a future predicted flow for a next time step for each sensor according to the flow prediction model;
based on a weight adjustment formula, according to the initial current weight, the adjustment coefficient, the future predicted flow of the next time step of each sensor and each transmissionDetermining a new current weight of the current actual flow of the sensor, wherein the weight adjustment formula satisfies the following conditionsWherein->Representing the new current weight, +.>Representing the initial current weight, +.>Representing adjustment coefficients->Future predicted flow representing the next time step of each,/i>Representing the current actual flow of each sensor;
taking the new current weight asSubstituting the weight adjustment formula to repeat iterative calculation and dynamically adjusting the current weight of each sensor.
In this embodiment, the adjustment factor controls the adjustment amplitude of the weight, i.e. the update frequency of the current weight, which is a value between 0 and 1, a larger adjustment factor results in a faster weight adjustment, a smaller adjustment factor results in a slower weight adjustment, and the selection adjustment factor needs to be balanced according to the response speed and stability of the system. If it is desired that the system be able to respond quickly to changes in the load of the sensor, a larger adjustment factor may be selected, which may result in a faster approach of the weight to the predicted target weight. If a smoother adjustment of the weight of the sensor by the system is desired, a smaller adjustment coefficient may be selected, which may reduce the oscillations and instability of the weight.
In addition, the sensor weights may be periodically reset under certain conditions to ensure that the dispensing process does not over-bias a sensor, e.g., after each sensor selection, the selected sensor weights may be reduced appropriately, allowing more opportunities for other sensors to be selected.
S102: according to the back-end server, calculating and analyzing the energy consumption data, generating an energy consumption data analysis result and transmitting the energy consumption data analysis result to a client;
and acquiring the energy consumption data analysis result in real time on the client side based on WebSockets connection, and converting the energy consumption data analysis result into an energy consumption chart according to a graph function library.
In this embodiment, a WebSocket library is deployed on a backend server, and is used for processing connection requests and message delivery of clients, in which JavaScript can be used to create WebSocket clients in client applications, and after receiving sensor data, the backend server performs calculation and analysis, such as real-time usage amount, total energy consumption, energy saving advice, etc. of each resource, and after calculating an energy consumption data analysis result, the energy consumption data analysis result is sent to the connected clients in the form of WebSocket messages. The client listens for messages received from WebSocket and when receiving real-time data update messages, uses JavaScript graphics library (e.g. Chart. Js) to convert the data into an energy consumption graph. The client may dynamically update the chart whenever new data arrives to enable real-time monitoring.
Referring to fig. 2, the embodiment of the present application further provides an energy consumption data processing system 2 based on the MQTT protocol, where the system 2 specifically includes:
the data acquisition layer 201 is used for deploying a sensor network in a building and acquiring energy consumption data of the building through the sensor network;
a data transmission layer 202, configured to establish real-time and efficient communication between the backend servers of the sensor network according to MQTT protocol;
the data analysis layer 203 is configured to calculate and analyze the energy consumption data according to the back-end server, generate an energy consumption data analysis result, and transmit the energy consumption data analysis result to a client;
the client application layer 204 is configured to obtain the energy consumption data analysis result in real time based on WebSockets connection, and convert the energy consumption data analysis result into an energy consumption graph according to a graph function library.
It can be understood that the content in the embodiment of the energy consumption data processing method based on the MQTT protocol shown in fig. 1 is applicable to the embodiment of the energy consumption data processing system based on the MQTT protocol, and the functions specifically implemented by the embodiment of the energy consumption data processing system based on the MQTT protocol are the same as those in the embodiment of the energy consumption data processing method based on the MQTT protocol shown in fig. 1, and the achieved beneficial effects are the same as those in the embodiment of the energy consumption data processing method based on the MQTT protocol shown in fig. 1.
It should be noted that, because the content of information interaction and execution process between the above systems is based on the same concept as the method embodiment of the present application, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the system is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
Referring to fig. 3, an embodiment of the present application further provides a computer device 3, including: memory 302 and processor 301 and a computer program 303 stored on the memory 302, which computer program 303, when executed on the processor 301, implements an energy consumption data processing method based on the MQTT protocol as set forth in any of the above methods.
The computer device 3 may be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The computer device 3 may include, but is not limited to, a processor 301, a memory 302. It will be appreciated by those skilled in the art that fig. 3 is merely an example of the computer device 3 and is not meant to be limiting as the computer device 3, and may include more or fewer components than shown, or may combine certain components, or different components, such as may also include input-output devices, network access devices, etc.
The processor 301 may be a central processing unit (Central Processing Unit, CPU), the processor 301 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 302 may in some embodiments be an internal storage unit of the computer device 3, such as a hard disk or a memory of the computer device 3. The memory 302 may in other embodiments also be an external storage device of the computer device 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 3. Further, the memory 302 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 302 is used to store an operating system, application programs, boot loader (BootLoader), data, and other programs, such as program code for the computer program. The memory 302 may also be used to temporarily store data that has been output or is to be output.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when being run by a processor, implements the energy consumption data processing method based on the MQTT protocol as set forth in any one of the above methods.
In this embodiment, the integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/terminal apparatus, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (RAM, random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the disclosed embodiments of the application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the apparatus/terminal device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Claims (10)
1. The energy consumption data processing method based on the MQTT protocol is characterized by comprising the following steps of:
deploying a sensor network in a building, and collecting energy consumption data of the building through the sensor network;
establishing real-time and efficient communication between the sensor network and a back-end server according to an MQTT protocol;
according to the back-end server, calculating and analyzing the energy consumption data, generating an energy consumption data analysis result and transmitting the energy consumption data analysis result to a client;
and acquiring the energy consumption data analysis result in real time on the client side based on WebSockets connection, and converting the energy consumption data analysis result into an energy consumption chart according to a graph function library.
2. The method according to claim 1, wherein the establishing real-time efficient communication between the sensor network and the backend server according to MQTT protocol, in particular comprises:
setting an MQTT Broker at the back-end server, and configuring an MQTT client library for each sensor in the sensor network;
constructing a plurality of themes topic in an MQTT Broker according to the building area and the sensor type;
establishing a flow prediction model for each sensor based on a Bayesian regression algorithm, and dynamically adjusting the current weight of each sensor according to the future predicted flow of each sensor and the current actual flow of each sensor according to the flow prediction model;
and forwarding the energy consumption data packet of each sensor to the corresponding topic according to the current weight of each sensor.
3. The method according to claim 2, wherein the establishing a flow prediction model for each sensor based on a bayesian regression algorithm specifically comprises:
acquiring a historical flow data set and a first feature set of each sensor, wherein the first feature set comprises message frequency, message size, message delay, topic distribution, sensor type, sensor position and sensor equipment energy consumption;
determining a second feature set from the first feature set based on pearson correlation coefficients;
based on the Bayesian regression model, a flow prediction model for each sensor is determined from the historical flow data set for each sensor and the second feature set.
4. A method according to claim 3, characterized in that said determining a second feature set from said first feature set based on pearson correlation coefficients, in particular comprises:
setting a correlation coefficient threshold value, and calculating a pearson correlation coefficient of a historical flow data set of each sensor and each feature in the first feature set;
and eliminating the features with the pearson correlation coefficient lower than the correlation coefficient threshold value in the first feature set to obtain a second feature set.
5. The method of claim 4, wherein the pearson correlation coefficient satisfiesR represents the pearson correlation coefficient of the historical flow data set of each sensor with each feature in the first feature set, n represents the number of data points for each feature in the first feature set, i represents the number of data points for each feature in the historical flow data set or the first feature set,/-a number of data points for each feature in the first feature set, and/-a number of data points for each feature in the first feature set>Values representing each data point of each feature in the first feature set, +.>A value representing each data point of the historical flow data set,/->Mean value representing each feature in the first set of features,/->Representing the average value of the historical traffic dataset.
6. A method according to claim 3, wherein the determining a flow prediction model for each sensor based on the bayesian regression model from the historical flow data set for each sensor and the second feature set, in particular comprises:
based on Bayesian regression formulaHistory of each sensorTaking the flow data set as output y, taking the second feature set as input feature matrix X, and obtaining a parameter vector +.>Data point set and error term->Is a set of data points;
using Gaussian distribution as parameter vectorError term->According to the a priori distribution of the parameter vector +.>Is used for determining the parameter vector +.>Is based on the a priori parameters of the a priori distribution of the error term +.>Is used for determining the error term +.>Prior parameters of the prior distribution of (a);
determining the parameter vector by maximum likelihood estimationLikelihood function value of (2) and said error term +.>Likelihood function values of (2);
based on the MCMC method, according to the parameter vectorIterative sampling is carried out on the data set, the prior distribution, the likelihood function value and the historical flow data set, and the parameter vector is determined through the samples on the converged MCMC chain>Posterior distribution of (2);
based on the MCMC method, according to the error termIterative sampling is carried out on the data set, the prior distribution, the likelihood function value and the historical flow data set, and the error item is determined through the samples on the converged MCMC chain>Posterior distribution of (2);
according to the parameter vectorPosterior distribution of (2) and the error term +.>Establishing a flow prediction model.
7. The method according to claim 2, wherein the dynamically adjusting the current weight of each sensor for the future predicted flow of each sensor and the current actual flow of each sensor according to the flow prediction model comprises:
setting an initial current weight and an adjustment coefficient for each sensor, wherein the adjustment coefficient is used for controlling the adjustment amplitude of the current weight of each sensor;
determining a future predicted flow for a next time step for each sensor according to the flow prediction model;
based on a weight adjustment formula, according to the initial current weight, the adjustment coefficient and each sensorDetermining a new current weight for the future predicted flow and the current actual flow for each sensor for the next time step, the weight adjustment formula satisfyingWherein->Representing the new current weight, +.>Representing the initial current weight, +.>Representing adjustment coefficients->Future predicted flow representing the next time step of each,/i>Representing the current actual flow of each sensor;
taking the new current weight asSubstituting the weight adjustment formula to repeat iterative calculation and dynamically adjusting the current weight of each sensor.
8. An energy consumption data processing system based on an MQTT protocol, comprising:
the data acquisition layer is used for deploying a sensor network in a building and acquiring energy consumption data of the building through the sensor network;
the data transmission layer is used for establishing real-time and efficient communication between the sensor network and the back-end server according to the MQTT protocol;
the data analysis layer is used for calculating and analyzing the energy consumption data according to the back-end server, generating an energy consumption data analysis result and transmitting the energy consumption data analysis result to the client;
and the client application layer is used for acquiring the energy consumption data analysis result in real time based on the WebSockets connection and converting the energy consumption data analysis result into an energy consumption chart according to the graphic function library.
9. A computer device, comprising: memory and processor and computer program stored on the memory, which, when executed on the processor, implements the MQTT protocol-based energy consumption data processing method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, implements the MQTT-protocol-based energy consumption data processing method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311387296.2A CN117119022B (en) | 2023-10-25 | 2023-10-25 | Energy consumption data processing method, system, equipment and medium based on MQTT protocol |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311387296.2A CN117119022B (en) | 2023-10-25 | 2023-10-25 | Energy consumption data processing method, system, equipment and medium based on MQTT protocol |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117119022A true CN117119022A (en) | 2023-11-24 |
| CN117119022B CN117119022B (en) | 2024-01-30 |
Family
ID=88813252
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311387296.2A Active CN117119022B (en) | 2023-10-25 | 2023-10-25 | Energy consumption data processing method, system, equipment and medium based on MQTT protocol |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117119022B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104639625A (en) * | 2015-01-27 | 2015-05-20 | 华南理工大学 | Data concentrator acquisition control method based on MQTT (Message Queuing Telemetry Transport), data concentrator acquisition control device based on MQTT and data concentrator acquisition control system based on MQTT |
| CN106998358A (en) * | 2017-03-24 | 2017-08-01 | 武汉光庭信息技术股份有限公司 | Map sensor network method for establishing model and system based on MQTT and LCM |
| CN108366053A (en) * | 2018-01-30 | 2018-08-03 | 电子科技大学 | A kind of MQTT anomalous traffic detection methods based on naive Bayesian |
| CN109218416A (en) * | 2018-08-30 | 2019-01-15 | 北京赛佰特科技有限公司 | Internet of things system based on cloud platform |
| CN109451352A (en) * | 2018-12-11 | 2019-03-08 | 北京奇艺世纪科技有限公司 | A kind of video playing method for predicting and device |
| CN110839184A (en) * | 2019-10-15 | 2020-02-25 | 北京邮电大学 | Method and device for adjusting bandwidth of mobile fronthaul optical network based on flow prediction |
| CN113852785A (en) * | 2021-07-29 | 2021-12-28 | 浪潮软件科技有限公司 | Intelligent monitoring and scheduling method, device and system based on 5G |
| WO2022253454A2 (en) * | 2021-06-04 | 2022-12-08 | Telefonaktiebolaget Lm Ericsson (Publ) | Dimensioning of telecommunication infrastructure |
| CN115988028A (en) * | 2022-12-21 | 2023-04-18 | 上海应用技术大学 | LNG gas station real-time safety monitoring system based on internet of things |
| CN116758765A (en) * | 2023-06-09 | 2023-09-15 | 大连理工大学 | Multi-target signal control optimization method suitable for multi-mode traffic |
-
2023
- 2023-10-25 CN CN202311387296.2A patent/CN117119022B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104639625A (en) * | 2015-01-27 | 2015-05-20 | 华南理工大学 | Data concentrator acquisition control method based on MQTT (Message Queuing Telemetry Transport), data concentrator acquisition control device based on MQTT and data concentrator acquisition control system based on MQTT |
| CN106998358A (en) * | 2017-03-24 | 2017-08-01 | 武汉光庭信息技术股份有限公司 | Map sensor network method for establishing model and system based on MQTT and LCM |
| CN108366053A (en) * | 2018-01-30 | 2018-08-03 | 电子科技大学 | A kind of MQTT anomalous traffic detection methods based on naive Bayesian |
| CN109218416A (en) * | 2018-08-30 | 2019-01-15 | 北京赛佰特科技有限公司 | Internet of things system based on cloud platform |
| CN109451352A (en) * | 2018-12-11 | 2019-03-08 | 北京奇艺世纪科技有限公司 | A kind of video playing method for predicting and device |
| CN110839184A (en) * | 2019-10-15 | 2020-02-25 | 北京邮电大学 | Method and device for adjusting bandwidth of mobile fronthaul optical network based on flow prediction |
| WO2022253454A2 (en) * | 2021-06-04 | 2022-12-08 | Telefonaktiebolaget Lm Ericsson (Publ) | Dimensioning of telecommunication infrastructure |
| CN113852785A (en) * | 2021-07-29 | 2021-12-28 | 浪潮软件科技有限公司 | Intelligent monitoring and scheduling method, device and system based on 5G |
| CN115988028A (en) * | 2022-12-21 | 2023-04-18 | 上海应用技术大学 | LNG gas station real-time safety monitoring system based on internet of things |
| CN116758765A (en) * | 2023-06-09 | 2023-09-15 | 大连理工大学 | Multi-target signal control optimization method suitable for multi-mode traffic |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117119022B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kapsalis et al. | A cooperative fog approach for effective workload balancing | |
| US8892720B2 (en) | System and method for network optimization through predictive downloading | |
| CN107360032B (en) | Network flow identification method and electronic equipment | |
| CN114039918B (en) | Information age optimization method and device, computer equipment and storage medium | |
| CN105917632A (en) | A method for scalable distributed network traffic analytics in telco | |
| CN111124819A (en) | Method and device for monitoring full link | |
| CN111142942B (en) | Window data processing method and device, server and storage medium | |
| CN116032663A (en) | Privacy data processing system, method, equipment and medium based on edge equipment | |
| CN114567650B (en) | Data processing method and Internet of things platform system | |
| CN114065863A (en) | Method, device and system for federal learning, electronic equipment and storage medium | |
| CN115460153B (en) | Dynamic adjustment method and device for token bucket capacity, storage medium and electronic device | |
| CN108376099B (en) | Mobile terminal calculation migration method for optimizing time delay and energy efficiency | |
| CN112202595A (en) | Abstract model construction method based on time sensitive network system | |
| CN117119022B (en) | Energy consumption data processing method, system, equipment and medium based on MQTT protocol | |
| Kim et al. | P2P computing for trusted networking of personalized IoT services | |
| Song et al. | Dram: A drl-based resource allocation scheme for mar in mec | |
| Hou et al. | GP-NFSP: Decentralized task offloading for mobile edge computing with independent reinforcement learning | |
| CN112202596A (en) | Abstract model construction device based on time sensitive network system | |
| CN114900817B (en) | Distributed wireless sensor network information estimation method based on binary quantization | |
| CN112948114B (en) | Edge computing method and edge computing platform | |
| US20180366949A1 (en) | Systems Using Data Streams to Feed Optimization Engines in Power System Operations | |
| Cleland et al. | FedComm: Understanding communication protocols for edge-based federated learning | |
| Chang et al. | Towards an Efficient and Dynamic Allocation of Radio Access Network Slicing Resources for 5G Era | |
| Liu | Multimedia intelligent fog computing scheme based on robust perception for complex networks | |
| Guo et al. | Mathematical modeling of transaction latency on Ethereum |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |