CN117061449B - Data batch transmission method, device, equipment and medium of edge gateway - Google Patents

Data batch transmission method, device, equipment and medium of edge gateway Download PDF

Info

Publication number
CN117061449B
CN117061449B CN202311312525.4A CN202311312525A CN117061449B CN 117061449 B CN117061449 B CN 117061449B CN 202311312525 A CN202311312525 A CN 202311312525A CN 117061449 B CN117061449 B CN 117061449B
Authority
CN
China
Prior art keywords
energy consumption
consumption data
edge gateway
sensor node
dimensional element
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311312525.4A
Other languages
Chinese (zh)
Other versions
CN117061449A (en
Inventor
王晓明
李孔政
黄嘉荣
冯成斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Baxtrand Technology Co ltd
Original Assignee
Guangdong Baxtrand Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Baxtrand Technology Co ltd filed Critical Guangdong Baxtrand Technology Co ltd
Priority to CN202311312525.4A priority Critical patent/CN117061449B/en
Publication of CN117061449A publication Critical patent/CN117061449A/en
Application granted granted Critical
Publication of CN117061449B publication Critical patent/CN117061449B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling
    • H04L47/62Queue scheduling characterised by scheduling criteria
    • H04L47/625Queue scheduling characterised by scheduling criteria for service slots or service orders
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling
    • H04L47/62Queue scheduling characterised by scheduling criteria
    • H04L47/625Queue scheduling characterised by scheduling criteria for service slots or service orders
    • H04L47/6275Queue scheduling characterised by scheduling criteria for service slots or service orders based on priority
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling
    • H04L47/62Queue scheduling characterised by scheduling criteria
    • H04L47/625Queue scheduling characterised by scheduling criteria for service slots or service orders
    • H04L47/628Queue scheduling characterised by scheduling criteria for service slots or service orders based on packet size, e.g. shortest packet first
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/12Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00Reducing energy consumption in communication networks
    • Y02D30/70Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention relates to the technical field of data transmission, in particular to a data batch transmission method, a device, equipment and a medium of an edge gateway, wherein the method specifically comprises the following steps: based on communication connection between an edge gateway and a sensor node set, acquiring energy consumption data of the sensor node set through the edge gateway; according to a time sequence similarity algorithm and a clustering algorithm, data integration is carried out on the respective energy consumption data of each sensor node in the sensor node set, and an energy consumption data cluster set is obtained; and monitoring the network load condition of the edge gateway in real time, determining the energy consumption data cluster with the highest priority from the energy consumption data cluster set according to the current network load condition of the edge gateway, and sending the energy consumption data cluster with the highest priority to a server. According to the invention, the energy consumption data cluster with the highest priority is determined in real time according to the network load condition of the edge gateway, so that the real-time performance and the efficiency of transmission are ensured.

Description

Data batch transmission method, device, equipment and medium of edge gateway
Technical Field
The present invention relates to the field of data transmission technologies, and in particular, to a method, an apparatus, a device, and a medium for batch data transmission of an edge gateway.
Background
In an application scenario where large-scale terminal devices report data to a server, the data volume is huge and centralized reporting often results in excessive processing pressure of the server. To solve this problem, the current technology generally uses a method of directly transmitting all data to a server at one time. However, this conventional data reporting method has some drawbacks: a large number of terminal devices simultaneously transmit data to a server at one time, so that the server needs to process a large number of data requests simultaneously, and the load and response time of the server are increased; a large amount of data is simultaneously sent to a server, which may cause network congestion and delay or loss of data transmission; the terminal device needs to maintain a continuous network connection to send data, which consumes a large amount of power, reducing the battery life of the device, especially for mobile terminals.
Disclosure of Invention
The invention aims to provide a data batch transmission method, device, equipment and medium of an edge gateway, which integrate energy consumption data in a sensor node set by utilizing a time sequence similarity algorithm and a clustering algorithm to obtain an energy consumption data cluster set, thereby reducing data transmission quantity and optimizing network transmission efficiency so as to solve at least one of the existing problems.
The invention provides a data batch transmission method of an edge gateway, which specifically comprises the following steps:
based on communication connection between an edge gateway and a sensor node set, acquiring energy consumption data of the sensor node set through the edge gateway;
according to a time sequence similarity algorithm and a clustering algorithm, data integration is carried out on the respective energy consumption data of each sensor node in the sensor node set, and an energy consumption data cluster set is obtained;
and monitoring the network load condition of the edge gateway in real time, determining the energy consumption data cluster with the highest priority from the energy consumption data cluster set according to the current network load condition of the edge gateway, and sending the energy consumption data cluster with the highest priority to a server.
Further, according to the time sequence similarity algorithm and the clustering algorithm, data integration is performed on the respective energy consumption data of each sensor node in the sensor node set to obtain an energy consumption data cluster set, which specifically includes:
calculating Euclidean distance between time sequence points of energy consumption data of every two sensor nodes in the sensor node set, and obtaining a distance matrix between the energy consumption data of every two sensor nodes;
Calculating an accumulated distance matrix by using a dynamic time warping algorithm according to the distance matrix, and obtaining similarity scores between the energy consumption data of every two sensor nodes through the accumulated distance matrix;
constructing a similarity matrix according to similarity scores between energy consumption data of every two sensor nodes in the sensor node set;
and clustering the energy consumption data of each sensor node in the sensor node set according to a clustering algorithm based on the similarity matrix to obtain an energy consumption data cluster set.
Further, the calculating the euclidean distance between time sequence points of the energy consumption data of every two sensor nodes in the sensor node set, to obtain a distance matrix between the energy consumption data of every two sensor nodes, specifically includes:
obtaining energy consumption data of a first sensor node, and constructing a first time sequence set according to the energy consumption data of the first sensor node
Obtaining energy consumption data of a second sensor node, and constructing a second time sequence set according to the energy consumption data of the second sensor node
Wherein the first sensor node and the second sensor node are any two sensor nodes in the sensor node set;
Based on a Euclidean distance formula, calculating the Euclidean distance between any time sequence point of the first time sequence set and any time sequence point of the second time sequence set to obtain a Euclidean distance set, wherein the Euclidean distance formula meets the following requirementsWherein C is Euclidean distance between any time sequence point of the first time sequence set and any time sequence point of the second time sequence set, < ->For a first set of time sequencesIs any time sequence point,/->For any time sequence point in the second time sequence set, i is the sequence number of the time sequence point of the first time sequence set or the second time sequence set, and n is the Euclidean distance calculation times between any time sequence point of the first time sequence set and any time sequence point of the second time sequence set;
and constructing a distance matrix according to the Euclidean distance set.
Further, according to the distance matrix, a dynamic time warping algorithm is used to calculate a cumulative distance matrix, and a similarity score between the energy consumption data of every two sensor nodes is obtained through the cumulative distance matrix, which specifically includes:
taking a first two-dimensional element of the distance matrix as a starting position, and determining a path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element;
When the path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element is a unique path, determining the unique path as a target path;
when the path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element is not the only path, calculating the accumulated Euclidean distance between each path, and determining the path with the minimum accumulated Euclidean distance as the target path;
obtaining an accumulated distance matrix according to the accumulated Euclidean distance of each two-dimensional element except the first two-dimensional element and the target path between the first two-dimensional element;
and determining the cumulative Euclidean distance of the last two-dimensional element on the cumulative distance matrix as a similarity score between the energy consumption data of every two sensor nodes.
Further, the clustering of the energy consumption data of each sensor node in the sensor node set according to the clustering algorithm based on the similarity matrix to obtain an energy consumption data cluster set specifically includes:
randomly determining the energy consumption data of a plurality of sensor nodes as an initial cluster center point;
setting a similarity score threshold value, and clustering energy consumption data of other sensor nodes which are not initial cluster center points with corresponding initial cluster center points according to comparison of the similarity score of each two-dimensional element of each initial cluster center point on the similarity matrix and the similarity score threshold value to obtain an initial energy consumption data cluster set;
And obtaining a new cluster center point from each initial energy consumption data cluster in the initial energy consumption data cluster set by calculating an average value, clustering according to the new cluster center point, and repeating iteration for a preset number of times to obtain a target energy consumption data cluster set.
Further, the determining, according to the current network load condition of the edge gateway, the energy consumption data cluster with the highest priority from the energy consumption data cluster set specifically includes:
constructing a queuing model between the sensor node set and the edge gateway, and determining the arrival rate and the transmission rate of the edge gateway according to the queuing model, wherein the arrival rate is the average request times of the sensor nodes reaching the edge gateway in unit time, and the transmission rate is the average request times of the sensor nodes processed by the edge gateway in unit time;
determining the average queuing length of the queuing model according to the arrival rate and the transmission rate;
establishing a first linear relation model between the data size and the data transmission time of the edge gateway, and determining the data transmission time of the average queuing length according to the first linear relation model;
determining network delay of the edge gateway according to the average queuing length and the data transmission time;
Establishing a second linear relation model between the data size of the edge gateway and the network delay, and determining the priority of each energy consumption data cluster in the energy consumption data cluster set under the current network delay of the edge gateway according to the second linear relation model;
and determining the energy consumption data cluster with the highest priority according to the priority of each energy consumption data cluster.
Further, the average queuing length satisfies,/>,/>Wherein L is the average queuing length, lambda is the arrival rate of the edge gateway, mu is the transmission rate of the edge gateway, K is the total request times of the sensor nodes reaching the edge gateway in unit time, K is the total request times of the sensor nodes processed by the edge gateway in unit time, and t is unit time.
The invention also provides a data batch transmission device of the edge gateway, which specifically comprises:
the data acquisition module is used for acquiring energy consumption data of the sensor node set through the edge gateway based on communication connection between the edge gateway and the sensor node set;
the data integration module is used for integrating the respective energy consumption data of each sensor node in the sensor node set according to a time sequence similarity algorithm and a clustering algorithm to obtain an energy consumption data cluster set;
The data transmission module is used for monitoring the network load condition of the edge gateway in real time, determining the energy consumption data cluster with the highest priority from the energy consumption data cluster set according to the current network load condition of the edge gateway, and sending the energy consumption data cluster with the highest priority to the server.
The present invention also provides a computer device comprising: memory and processor and computer program stored on the memory, which when executed on the processor, implements a data batch transmission method of an edge gateway as claimed in any one of the above methods.
The invention also provides a computer readable storage medium, characterized in that it has stored thereon a computer program which, when run by a processor, implements a data batch transmission method of an edge gateway as described in any of the above methods.
Compared with the prior art, the invention has at least one of the following technical effects:
1. and integrating the energy consumption data in the sensor node set by using a time sequence similarity algorithm and a clustering algorithm to obtain an energy consumption data cluster set, so that the data transmission quantity is reduced, and the network transmission efficiency is optimized.
2. And the energy consumption data cluster with the highest priority is determined in real time according to the network load condition of the edge gateway, so that the real-time performance and the efficiency of transmission are ensured.
3. By establishing a queuing model, the transmission priority is dynamically adjusted in consideration of the relation between the network load and the data size, so that better network load control and resource utilization are realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a data batch transmission method of an edge gateway according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a data batch transmission device of an edge gateway according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
In addition, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
In an application scenario where large-scale terminal devices report data to a server, the data volume is huge and centralized reporting often results in excessive processing pressure of the server. To solve this problem, the current technology generally uses a method of directly transmitting all data to a server at one time. However, this conventional data reporting method has some drawbacks: a large number of terminal devices simultaneously transmit data to a server at one time, so that the server needs to process a large number of data requests simultaneously, and the load and response time of the server are increased; a large amount of data is simultaneously sent to a server, which may cause network congestion and delay or loss of data transmission; the terminal device needs to maintain a continuous network connection to send data, which consumes a large amount of power, reducing the battery life of the device, especially for mobile terminals.
Referring to fig. 1, an embodiment of the present invention provides a data batch transmission method for an edge gateway, where the method specifically includes:
s101: and acquiring the energy consumption data of the sensor node set through the edge gateway based on the communication connection between the edge gateway and the sensor node set.
And according to a time sequence similarity algorithm and a clustering algorithm, carrying out data integration on the respective energy consumption data of each sensor node in the sensor node set to obtain an energy consumption data cluster set.
In some embodiments, the data integration is performed on the energy consumption data of each sensor node in the sensor node set according to a time sequence similarity algorithm and a clustering algorithm to obtain an energy consumption data cluster set, which specifically includes:
calculating Euclidean distance between time sequence points of energy consumption data of every two sensor nodes in the sensor node set, and obtaining a distance matrix between the energy consumption data of every two sensor nodes;
calculating an accumulated distance matrix by using a dynamic time warping algorithm according to the distance matrix, and obtaining similarity scores between the energy consumption data of every two sensor nodes through the accumulated distance matrix;
Constructing a similarity matrix according to similarity scores between energy consumption data of every two sensor nodes in the sensor node set;
and clustering the energy consumption data of each sensor node in the sensor node set according to a clustering algorithm based on the similarity matrix to obtain an energy consumption data cluster set.
In this embodiment, the edge gateway may compress and sort the data by a time sequence similarity algorithm and a clustering algorithm after receiving and integrating the cluster data from the plurality of sensor edge nodes, thereby reducing the amount of data transmitted to the server, retaining key energy consumption information, and being helpful for improving the data transmission efficiency and the processing performance of the server.
Among the time series similarity algorithms, a dynamic time warping (DTW, dynamic Time Warping) algorithm may be used to calculate the best alignment path between time series, resulting in a similarity score between the data.
First, for each pair of time series, distance measurement is used to calculate the distance between them, and the euclidean distance is used to measure because the energy consumption value has obvious difference between different nodes in consideration of the characteristic of the continuity of the energy consumption data.
Next, a two-dimensional matrix is constructed, the rows and columns of which correspond to two time-series data points, respectively, and the distance between each pair of data points is calculated based on the selected distance metric, and the distance matrix is populated.
Then, starting from the upper left corner of the distance matrix, a cumulative distance matrix is calculated, in each cell, the cumulative distance of the path from the start point to the cell is stored, and the cumulative distance matrix is calculated using a recursive or dynamic programming method.
Then, starting from the bottom right corner of the cumulative distance matrix, the best path, i.e., the best time series alignment, is found back, and the path may be moved up, left, or up to the left in the grid until the top left corner is reached.
Finally, calculating the similarity of the time series according to the optimal path, wherein the similarity can be the sum of the distances of all grids on the path or the average distance on the path.
In some embodiments, the calculating the euclidean distance between time series points of the energy consumption data of each two sensor nodes in the sensor node set, to obtain a distance matrix between the energy consumption data of each two sensor nodes, specifically includes:
obtaining energy consumption data of a first sensor node, and constructing a first time sequence set according to the energy consumption data of the first sensor node
Obtaining energy consumption data of a second sensor node, and constructing a second time sequence set according to the energy consumption data of the second sensor node
Wherein the first sensor node and the second sensor node are any two sensor nodes in the sensor node set;
based on a Euclidean distance formula, calculating the Euclidean distance between any time sequence point of the first time sequence set and any time sequence point of the second time sequence set to obtain a Euclidean distance set, wherein the Euclidean distance formula meets the following requirementsWherein C is Euclidean distance between any time sequence point of the first time sequence set and any time sequence point of the second time sequence set, < ->For any time-series point in the first set of time-series, < > for>For any time sequence point in the second time sequence set, i is the sequence number of the time sequence point of the first time sequence set or the second time sequence set, and n is the Euclidean distance calculation times between any time sequence point of the first time sequence set and any time sequence point of the second time sequence set;
and constructing a distance matrix according to the Euclidean distance set.
In this embodiment, a blank two-dimensional matrix is first created, the first time series set is used as a row of the two-dimensional matrix, the second time series set is used as a column of the two-dimensional matrix, or the first time series set is used as a column of the two-dimensional matrix, and the second time series set is used as a row of the two-dimensional matrix. For the energy consumption data of every two sensor nodes, the euclidean distance between them is calculated, assuming a first time series set a= [3, 4, 7, 9, 2 ]A second set of time sequences b= [2, 5, 6, 8, 10]The first two-dimensional element of the distance matrix [,/>]Is->Until the euclidean distance between any time series point in the first time series set and any time series point in the second time series set is calculated, the distance matrix is gradually filled, and specific examples are shown in the following table.
A1(3) A2(4) A3(7) A4(9) A5(2)
B1(2) 1 2 5 7 0
B2(5) 2 1 2 4 3
B3(6) 3 2 1 3 4
B4(8) 5 4 1 1 6
B5(10) 7 6 3 1 8
In some embodiments, the calculating a cumulative distance matrix according to the distance matrix by using a dynamic time warping algorithm, and obtaining a similarity score between the energy consumption data of each two sensor nodes through the cumulative distance matrix specifically includes:
taking a first two-dimensional element of the distance matrix as a starting position, and determining a path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element;
when the path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element is a unique path, determining the unique path as a target path;
when the path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element is not the only path, calculating the accumulated Euclidean distance between each path, and determining the path with the minimum accumulated Euclidean distance as the target path;
Obtaining an accumulated distance matrix according to the accumulated Euclidean distance of each two-dimensional element except the first two-dimensional element and the target path between the first two-dimensional element;
and determining the cumulative Euclidean distance of the last two-dimensional element on the cumulative distance matrix as a similarity score between the energy consumption data of every two sensor nodes.
In this embodiment, in a Dynamic Time Warping (DTW) algorithm, a cumulative distance matrix is used to record the minimum cumulative distance from a certain data point in time series a to each cell in the matrix. The cumulative distance matrix is calculated starting from the upper left corner of the matrix, with the value of each cell representing the cumulative distance of the path from the starting point to the current cell. This process can be analogically like finding a path between two time series that aligns the two series in the time dimension while minimizing the distance between the corresponding data points.
Starting from the upper left corner of the distance matrix, the cumulative distance for each cell can be calculated by: in a first step, the cumulative distance of the initial position (upper left corner) is equal to the distance of the starting point, i.e. the value from the first cell of the matrix. Second, moving to the right along the first row of the matrix, the accumulated distance is equal to the accumulated distance of the last cell plus the distance of the current cell. Third, moving down the first column of the matrix, the cumulative distance is equal to the cumulative distance of the last cell plus the distance of the current cell. Fourth, for other locations, the cumulative distance is equal to the minimum cumulative distance of the path to the current cell, i.e., the minimum cumulative distance is selected from the left, top or top left cell plus the distance of the current cell.
By continually calculating each cell in the cumulative distance matrix, we can find a path that aligns time series A onto time series B so that the cumulative distance of this path is minimized. This path is the best alignment path of the time series and is also the core of the DTW algorithm. Finally, the value of the cell at the lower right corner of the cumulative distance matrix is the DTW distance between time series A and B. In addition, the DTW distance may be normalized, for example, by dividing the DTW distance by the length of the time series or the maximum distance, so as to fall within a specific range, for example, between 0 and 1.
In some embodiments, the clustering, based on the similarity matrix, the energy consumption data of each sensor node in the sensor node set according to a clustering algorithm, to obtain an energy consumption data cluster set specifically includes:
randomly determining the energy consumption data of a plurality of sensor nodes as an initial cluster center point;
setting a similarity score threshold value, and clustering energy consumption data of other sensor nodes which are not initial cluster center points with corresponding initial cluster center points according to comparison of the similarity score of each two-dimensional element of each initial cluster center point on the similarity matrix and the similarity score threshold value to obtain an initial energy consumption data cluster set;
And obtaining a new cluster center point from each initial energy consumption data cluster in the initial energy consumption data cluster set by calculating an average value, clustering according to the new cluster center point, and repeating iteration for a preset number of times to obtain a target energy consumption data cluster set.
In this embodiment, a certain number of energy consumption data of sensor nodes are randomly selected from the sensor node set as an initial cluster center point, and a similarity score threshold is set to determine whether the energy consumption data of other sensor nodes are similar to the initial cluster center point. And comparing each element in the similarity score matrix with a similarity score threshold value, and if the similarity score of the element exceeds the threshold value, clustering the energy consumption data of the corresponding sensor node with the central point of the initial cluster to construct an initial energy consumption data cluster set. And calculating the average value of the energy consumption data of the sensor nodes in the clusters for each cluster in the initial energy consumption data cluster set to obtain a new cluster center point. Repeating the steps, using the new cluster center point to re-cluster the energy consumption data of the sensor nodes, and repeating the iteration for a certain number of times or until the cluster center point does not change significantly. After iteration, a final energy consumption data cluster set is obtained, wherein each cluster has a relatively stable cluster center point.
The selection of the cluster center points can be gradually optimized by calculating new cluster center points and iterative clustering, so that a more accurate energy consumption data cluster set is obtained.
S102: and monitoring the network load condition of the edge gateway in real time, determining the energy consumption data cluster with the highest priority from the energy consumption data cluster set according to the current network load condition of the edge gateway, and sending the energy consumption data cluster with the highest priority to a server.
In some embodiments, the determining, according to the current network load condition of the edge gateway, the energy consumption data cluster with the highest priority from the energy consumption data cluster set specifically includes:
constructing a queuing model between the sensor node set and the edge gateway, and determining the arrival rate and the transmission rate of the edge gateway according to the queuing model, wherein the arrival rate is the average request times of the sensor nodes reaching the edge gateway in unit time, and the transmission rate is the average request times of the sensor nodes processed by the edge gateway in unit time;
determining the average queuing length of the queuing model according to the arrival rate and the transmission rate;
establishing a first linear relation model between the data size and the data transmission time of the edge gateway, and determining the data transmission time of the average queuing length according to the first linear relation model;
Determining network delay of the edge gateway according to the average queuing length and the data transmission time;
establishing a second linear relation model between the data size of the edge gateway and the network delay, and determining the priority of each energy consumption data cluster in the energy consumption data cluster set under the current network delay of the edge gateway according to the second linear relation model;
and determining the energy consumption data cluster with the highest priority according to the priority of each energy consumption data cluster.
Specifically, the average queuing length satisfies,/>,/>Wherein L is the average queuing length, lambda is the arrival rate of the edge gateway, mu is the transmission rate of the edge gateway, K is the total request times of the sensor nodes reaching the edge gateway in unit time, K is the total request times of the sensor nodes processed by the edge gateway in unit time, and t is unit time.
In this embodiment, a queuing model is built for the data transmission process of each sensor node, and considering the use of an M/M/1/K queuing model, M represents that the arrival process is a Poisson process, 1 represents that there is only one service device, K represents the capacity of the queue, and thenWhereas the arrival rate (lambda) and the transmission rate (mu) are defined. The first linear relation model and the second linear relation model may be found by a linear function y=mx+b using a least squares method to find the most suitable slope m and intercept b, i.e. ,/>In the first linear relationship model, +.>Is the data size, +.>Is the data transmission time, < >>Is the average of the data size, +.>Is the average value of the data transmission time, in a second linear relation model, < >>Is the data size, +.>Is network delay, +.>Is the average of the data size, +.>Is the average of the network delays.
Based on the queuing model, the first linear relation model and the second linear relation model, the priority of the energy consumption data cluster is calculated and determined, so that the edge gateway can select a proper energy consumption data cluster for transmission according to the current network delay condition, thereby optimizing a data transmission strategy, reducing network delay and improving data transmission efficiency.
Referring to fig. 2, the embodiment of the present invention further provides a data batch transmission device 2 of an edge gateway, where the device 2 specifically includes:
the data acquisition module 201 is configured to obtain, based on communication connection between an edge gateway and a sensor node set, energy consumption data of the sensor node set through the edge gateway;
the data integration module 202 is configured to integrate the respective energy consumption data of each sensor node in the sensor node set according to a time sequence similarity algorithm and a clustering algorithm to obtain an energy consumption data cluster set;
And the data transmission module 203 is configured to monitor the network load condition of the edge gateway in real time, determine the energy consumption data cluster with the highest priority from the energy consumption data cluster set according to the current network load condition of the edge gateway, and send the energy consumption data cluster with the highest priority to a server.
It can be understood that the contents of the embodiment of the data batch transmission method of the edge gateway shown in fig. 1 are applicable to the embodiment of the data batch transmission device of the edge gateway, and the functions of the embodiment of the data batch transmission device of the edge gateway are the same as those of the embodiment of the data batch transmission method of the edge gateway shown in fig. 1, and the advantages achieved are the same as those achieved by the embodiment of the data batch transmission method of the edge gateway shown in fig. 1.
It should be noted that, because the content of information interaction and execution process between the above devices is based on the same concept as the method embodiment of the present invention, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
Referring to fig. 3, an embodiment of the present invention further provides a computer device 3, including: memory 302 and processor 301 and a computer program 303 stored on memory 302, which computer program 303, when executed on processor 301, implements a data batch transmission method of an edge gateway according to any of the above methods.
The computer device 3 may be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The computer device 3 may include, but is not limited to, a processor 301, a memory 302. It will be appreciated by those skilled in the art that fig. 3 is merely an example of the computer device 3 and is not meant to be limiting as the computer device 3, and may include more or fewer components than shown, or may combine certain components, or different components, such as may also include input-output devices, network access devices, etc.
The processor 301 may be a central processing unit (Central Processing Unit, CPU), the processor 301 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 302 may in some embodiments be an internal storage unit of the computer device 3, such as a hard disk or a memory of the computer device 3. The memory 302 may in other embodiments also be an external storage device of the computer device 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 3. Further, the memory 302 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 302 is used to store an operating system, application programs, boot loader (BootLoader), data, and other programs, such as program code for the computer program. The memory 302 may also be used to temporarily store data that has been output or is to be output.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the data batch transmission method of the edge gateway according to any one of the above methods.
In this embodiment, the integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application implements all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/terminal apparatus, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (RAM, random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments disclosed in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other manners. For example, the apparatus/terminal device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Claims (8)

1. The data batch transmission method of the edge gateway is characterized by comprising the following steps:
based on communication connection between an edge gateway and a sensor node set, acquiring energy consumption data of the sensor node set through the edge gateway;
according to a time sequence similarity algorithm and a clustering algorithm, data integration is carried out on the respective energy consumption data of each sensor node in the sensor node set, and an energy consumption data cluster set is obtained;
the data integration is performed on the respective energy consumption data of each sensor node in the sensor node set according to a time sequence similarity algorithm and a clustering algorithm to obtain an energy consumption data cluster set, and the method specifically comprises the following steps:
calculating Euclidean distance between time sequence points of energy consumption data of every two sensor nodes in the sensor node set, and obtaining a distance matrix between the energy consumption data of every two sensor nodes;
Calculating an accumulated distance matrix by using a dynamic time warping algorithm according to the distance matrix, and obtaining similarity scores between the energy consumption data of every two sensor nodes through the accumulated distance matrix;
constructing a similarity matrix according to similarity scores between energy consumption data of every two sensor nodes in the sensor node set;
based on the similarity matrix, clustering the energy consumption data of each sensor node in the sensor node set according to a clustering algorithm to obtain an energy consumption data cluster set;
the calculating a cumulative distance matrix by using a dynamic time warping algorithm according to the distance matrix, and obtaining a similarity score between the energy consumption data of every two sensor nodes through the cumulative distance matrix specifically comprises the following steps:
taking a first two-dimensional element of the distance matrix as a starting position, and determining a path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element;
when the path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element is a unique path, determining the unique path as a target path;
when the path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element is not the only path, calculating the accumulated Euclidean distance between each path, and determining the path with the smallest accumulated Euclidean distance as the target path;
Obtaining an accumulated distance matrix according to the accumulated Euclidean distance of each two-dimensional element except the first two-dimensional element and the target path between the first two-dimensional element;
determining the cumulative Euclidean distance of the last two-dimensional element on the cumulative distance matrix as a similarity score between the energy consumption data of every two sensor nodes;
and monitoring the network load condition of the edge gateway in real time, determining the energy consumption data cluster with the highest priority from the energy consumption data cluster set according to the current network load condition of the edge gateway, and sending the energy consumption data cluster with the highest priority to a server.
2. The method according to claim 1, wherein the calculating the euclidean distance between time series points of the energy consumption data of each two sensor nodes in the set of sensor nodes, and obtaining the distance matrix between the energy consumption data of each two sensor nodes, specifically comprises:
obtaining energy consumption data of a first sensor node, and constructing a first time sequence set according to the energy consumption data of the first sensor node
Obtaining energy consumption data of a second sensor node, and constructing a second time sequence set according to the energy consumption data of the second sensor node
Wherein the first sensor node and the second sensor node are any two sensor nodes in the sensor node set;
based on a Euclidean distance formula, calculating the Euclidean distance between any time sequence point of the first time sequence set and any time sequence point of the second time sequence set to obtain a Euclidean distance set, wherein the Euclidean distance formula meets the following requirementsWherein C is Euclidean distance between any time sequence point of the first time sequence set and any time sequence point of the second time sequence set, < ->For any time-series point in the first set of time-series, < > for>For any time sequence point in the second time sequence set, i is the sequence number of the time sequence point of the first time sequence set or the second time sequence set, and n is the Euclidean distance calculation times between any time sequence point of the first time sequence set and any time sequence point of the second time sequence set;
and constructing a distance matrix according to the Euclidean distance set.
3. The method according to claim 1, wherein the clustering the energy consumption data of each sensor node in the set of sensor nodes according to a clustering algorithm based on the similarity matrix to obtain a set of energy consumption data clusters, specifically includes:
Randomly determining the energy consumption data of a plurality of sensor nodes as an initial cluster center point;
setting a similarity score threshold value, and clustering energy consumption data of other sensor nodes which are not initial cluster center points with corresponding initial cluster center points according to comparison of the similarity score of each two-dimensional element of each initial cluster center point on the similarity matrix and the similarity score threshold value to obtain an initial energy consumption data cluster set;
and obtaining a new cluster center point from each initial energy consumption data cluster in the initial energy consumption data cluster set by calculating an average value, clustering according to the new cluster center point, and repeating iteration for a preset number of times to obtain a target energy consumption data cluster set.
4. The method according to claim 1, wherein the determining the highest priority energy consumption data cluster from the set of energy consumption data clusters according to the current network load condition of the edge gateway specifically comprises:
constructing a queuing model between the sensor node set and the edge gateway, and determining the arrival rate and the transmission rate of the edge gateway according to the queuing model, wherein the arrival rate is the average request times of the sensor nodes reaching the edge gateway in unit time, and the transmission rate is the average request times of the sensor nodes processed by the edge gateway in unit time;
Determining the average queuing length of the queuing model according to the arrival rate and the transmission rate;
establishing a first linear relation model between the data size and the data transmission time of the edge gateway, and determining the data transmission time of the average queuing length according to the first linear relation model;
determining network delay of the edge gateway according to the average queuing length and the data transmission time;
establishing a second linear relation model between the data size of the edge gateway and the network delay, and determining the priority of each energy consumption data cluster in the energy consumption data cluster set under the current network delay of the edge gateway according to the second linear relation model;
and determining the energy consumption data cluster with the highest priority according to the priority of each energy consumption data cluster.
5. The method of claim 4, wherein the average queuing length satisfies,/>Wherein L is the average queuing length, lambda is the arrival rate of the edge gateway, mu is the transmission rate of the edge gateway, K is the total request times of the sensor nodes reaching the edge gateway in unit time, K is the total request times of the sensor nodes processed by the edge gateway in unit time, and t is unit time.
6. An edge gateway data batch transmission device, which is characterized in that the device specifically comprises:
the data acquisition module is used for acquiring energy consumption data of the sensor node set through the edge gateway based on communication connection between the edge gateway and the sensor node set;
the data integration module is used for integrating the respective energy consumption data of each sensor node in the sensor node set according to a time sequence similarity algorithm and a clustering algorithm to obtain an energy consumption data cluster set;
the data integration is performed on the respective energy consumption data of each sensor node in the sensor node set according to a time sequence similarity algorithm and a clustering algorithm to obtain an energy consumption data cluster set, and the method specifically comprises the following steps:
calculating Euclidean distance between time sequence points of energy consumption data of every two sensor nodes in the sensor node set, and obtaining a distance matrix between the energy consumption data of every two sensor nodes;
calculating an accumulated distance matrix by using a dynamic time warping algorithm according to the distance matrix, and obtaining similarity scores between the energy consumption data of every two sensor nodes through the accumulated distance matrix;
Constructing a similarity matrix according to similarity scores between energy consumption data of every two sensor nodes in the sensor node set;
based on the similarity matrix, clustering the energy consumption data of each sensor node in the sensor node set according to a clustering algorithm to obtain an energy consumption data cluster set;
the calculating a cumulative distance matrix by using a dynamic time warping algorithm according to the distance matrix, and obtaining a similarity score between the energy consumption data of every two sensor nodes through the cumulative distance matrix specifically comprises the following steps:
taking a first two-dimensional element of the distance matrix as a starting position, and determining a path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element;
when the path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element is a unique path, determining the unique path as a target path;
when the path between each two-dimensional element except the first two-dimensional element and the first two-dimensional element is not the only path, calculating the accumulated Euclidean distance between each path, and determining the path with the smallest accumulated Euclidean distance as the target path;
Obtaining an accumulated distance matrix according to the accumulated Euclidean distance of each two-dimensional element except the first two-dimensional element and the target path between the first two-dimensional element;
determining the cumulative Euclidean distance of the last two-dimensional element on the cumulative distance matrix as a similarity score between the energy consumption data of every two sensor nodes;
the data transmission module is used for monitoring the network load condition of the edge gateway in real time, determining the energy consumption data cluster with the highest priority from the energy consumption data cluster set according to the current network load condition of the edge gateway, and sending the energy consumption data cluster with the highest priority to the server.
7. A computer device, comprising: memory and processor and computer program stored on the memory, which when executed on the processor, implements the data batch transmission method of an edge gateway as claimed in any one of claims 1 to 5.
8. A computer readable storage medium, having stored thereon a computer program which, when executed by a processor, implements the data batch transmission method of an edge gateway as claimed in any one of claims 1 to 5.
CN202311312525.4A 2023-10-11 2023-10-11 Data batch transmission method, device, equipment and medium of edge gateway Active CN117061449B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311312525.4A CN117061449B (en) 2023-10-11 2023-10-11 Data batch transmission method, device, equipment and medium of edge gateway

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311312525.4A CN117061449B (en) 2023-10-11 2023-10-11 Data batch transmission method, device, equipment and medium of edge gateway

Publications (2)

Publication Number Publication Date
CN117061449A CN117061449A (en) 2023-11-14
CN117061449B true CN117061449B (en) 2024-01-30

Family

ID=88659471

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311312525.4A Active CN117061449B (en) 2023-10-11 2023-10-11 Data batch transmission method, device, equipment and medium of edge gateway

Country Status (1)

Country Link
CN (1) CN117061449B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111556457A (en) * 2020-04-27 2020-08-18 山东省科学院海洋仪器仪表研究所 Task migration method and system of underwater self-organizing network based on edge gateway
CN111726768A (en) * 2020-06-16 2020-09-29 天津理工大学 Edge-oriented computation reliable data collection method based on compressed sensing
WO2020233320A1 (en) * 2019-05-20 2020-11-26 深圳壹账通智能科技有限公司 Reminding task allocation method and apparatus, computer device, and storage medium
CN113282409A (en) * 2021-05-13 2021-08-20 广东电网有限责任公司广州供电局 Edge calculation task processing method and device and computer equipment
CN113365323A (en) * 2021-05-10 2021-09-07 华东桐柏抽水蓄能发电有限责任公司 Low-energy-consumption multipath transmission method for wireless sensor network
CN114710499A (en) * 2022-03-30 2022-07-05 重庆川仪自动化股份有限公司 Load balancing method, device and medium for edge computing gateway based on computational power routing

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9639598B2 (en) * 2014-07-31 2017-05-02 International Business Machines Corporation Large-scale data clustering with dynamic social context
US11048727B2 (en) * 2018-09-10 2021-06-29 Ciena Corporation Systems and methods for automated feature selection and pattern discovery of multi-variate time-series

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020233320A1 (en) * 2019-05-20 2020-11-26 深圳壹账通智能科技有限公司 Reminding task allocation method and apparatus, computer device, and storage medium
CN111556457A (en) * 2020-04-27 2020-08-18 山东省科学院海洋仪器仪表研究所 Task migration method and system of underwater self-organizing network based on edge gateway
CN111726768A (en) * 2020-06-16 2020-09-29 天津理工大学 Edge-oriented computation reliable data collection method based on compressed sensing
CN113365323A (en) * 2021-05-10 2021-09-07 华东桐柏抽水蓄能发电有限责任公司 Low-energy-consumption multipath transmission method for wireless sensor network
CN113282409A (en) * 2021-05-13 2021-08-20 广东电网有限责任公司广州供电局 Edge calculation task processing method and device and computer equipment
CN114710499A (en) * 2022-03-30 2022-07-05 重庆川仪自动化股份有限公司 Load balancing method, device and medium for edge computing gateway based on computational power routing

Also Published As

Publication number Publication date
CN117061449A (en) 2023-11-14

Similar Documents

Publication Publication Date Title
CN111125089B (en) Time sequence data storage method, device, server and storage medium
CN108874535B (en) Task adjusting method, computer readable storage medium and terminal device
US20110119270A1 (en) Apparatus and method for processing a data stream
CN111694839B (en) Time sequence index construction method and device based on big data and computer equipment
CN112328688B (en) Data storage method, device, computer equipment and storage medium
CN117193503B (en) Heat dissipation host power supply operation monitoring method based on Internet of things
CN112289379B (en) Method and device for determining cell type, storage medium and electronic device
CN117061449B (en) Data batch transmission method, device, equipment and medium of edge gateway
CN112596985B (en) IT asset detection method, device, equipment and medium
CN111160614B (en) Training method and device of resource transfer prediction model and computing equipment
CN115795368B (en) Enterprise internal training data processing method and system based on artificial intelligence
CN115035017A (en) Cell density grouping method, device, electronic apparatus and storage medium
CN107665291B (en) Mutation detection method based on cloud computing platform Spark
CN111490889B (en) Method and device for estimating wireless service growth
CN112468546A (en) Account position determining method, account position determining device, server and storage medium
CN113590322A (en) Data processing method and device
CN112241767A (en) Pipe network water pressure monitoring node arrangement method, device, equipment and storage medium
CN112596903A (en) Intelligent information processing method and device based on big data
CN111124681A (en) Cluster load distribution method and device
CN113822768A (en) Community network processing method, device, equipment and storage medium
CN105468603A (en) Data selection method and apparatus
CN115134333B (en) Rapid networking method and system based on photovoltaic power generation and cloud platform
CN117235297B (en) Image selection method and computer equipment
CN116405500B (en) System resource management method based on data analysis and cloud computing data analysis
CN117350052A (en) Method and system for constructing power computing service capability model based on power computing network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant