CN116527525A - Equipment data acquisition method and system based on edge calculation - Google Patents

Equipment data acquisition method and system based on edge calculation Download PDF

Info

Publication number
CN116527525A
CN116527525A CN202310494119.8A CN202310494119A CN116527525A CN 116527525 A CN116527525 A CN 116527525A CN 202310494119 A CN202310494119 A CN 202310494119A CN 116527525 A CN116527525 A CN 116527525A
Authority
CN
China
Prior art keywords
data
module
data acquisition
cluster
clustering
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310494119.8A
Other languages
Chinese (zh)
Inventor
方彬
祝视
杨芳僚
田建伟
余琦
薛静远
廖铭鼎
李浩志
李轶佳
徐宁
向行
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Corp of China SGCC
State Grid Hunan Electric Power Co Ltd
Information and Telecommunication Branch of State Grid Hunan Electric Power Co Ltd
Original Assignee
State Grid Corp of China SGCC
State Grid Hunan Electric Power Co Ltd
Information and Telecommunication Branch of State Grid Hunan Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Corp of China SGCC, State Grid Hunan Electric Power Co Ltd, Information and Telecommunication Branch of State Grid Hunan Electric Power Co Ltd filed Critical State Grid Corp of China SGCC
Priority to CN202310494119.8A priority Critical patent/CN116527525A/en
Publication of CN116527525A publication Critical patent/CN116527525A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/04—Processing captured monitoring data, e.g. for logfile generation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y10/00—Economic sectors
    • G16Y10/35—Utilities, e.g. electricity, gas or water
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/10—Detection; Monitoring
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/46—Cluster building
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/04—Protocols for data compression, e.g. ROHC
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Security & Cryptography (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a device data acquisition method based on edge calculation, which comprises the steps of acquiring working data information of data acquisition devices; initializing and classifying the data acquisition equipment; clustering the data acquisition equipment; determining data items to be compressed for each cluster; data is compressed and then uploaded; updating the clustering result; and repeating the steps to complete the equipment data acquisition based on the edge calculation. The invention also discloses a system for realizing the equipment data acquisition method based on the edge calculation. According to the invention, the compressible data items are calculated, and clustering and compressing items are not required to be manually specified; meanwhile, the invention can effectively utilize the calculation capability of the data acquisition equipment, reduce the amount of uploading data, and improve the reliability of data acquisition by reducing the amount of uploading data; therefore, the invention has high reliability, good accuracy and higher efficiency.

Description

Equipment data acquisition method and system based on edge calculation
Technical Field
The invention belongs to the field of Internet of things, and particularly relates to a device data acquisition method and system based on edge calculation.
Background
Along with the development of economic technology and the improvement of living standard of people, electric energy becomes an indispensable secondary energy source in the production and living of people, and brings endless convenience to the production and living of people. Therefore, ensuring stable and reliable supply of electric energy becomes one of the most important tasks of the electric power system.
At present, with the development and perfection of the power internet, more and more data acquisition devices with computing capability, storage capability and communication capability are accessed into a power system, and the power grid operation related data acquired and transmitted by the devices plays an extremely important role for the power system. Therefore, the stable and reliable acquisition of the data of the equipment is particularly important for the power system.
At present, in an actual power system, management of data acquisition equipment and collection processing of data are mainly realized through an internet of things management platform, and the data acquisition equipment is generally connected to the internet of things management platform in a wireless manner through an APN network and performs data interaction with the platform. However, in the current data acquisition scheme, the data acquisition devices are isolated and lack of unified connection; although the redundancy of the data uploaded by the device is smaller at the individual device level, the data uploaded by the massive data acquisition device has great redundancy at the whole platform side. The transmission of the redundant data needs to consume a large amount of extra bandwidth and communication resources, so that the burden of an Internet of things management platform is greatly increased; meanwhile, redundant data are transmitted in a large quantity, and the efficiency of the data acquisition process is greatly reduced.
Disclosure of Invention
The invention aims to provide an edge calculation-based equipment data acquisition method which is high in reliability, accuracy and efficiency.
The second object of the present invention is to provide a system for implementing the method for collecting device data based on edge calculation.
The invention provides a device data acquisition method based on edge calculation, which comprises the following steps:
s1, acquiring working data information of data acquisition equipment;
s2, initializing and classifying the data acquisition equipment according to the working data information acquired in the step S1;
s3, clustering the data acquisition equipment based on the classification algorithm and the classification result of the step S2;
s4, determining data items to be compressed of each cluster according to the currently obtained clustering result;
s5, each data acquisition device compresses acquired data according to the data item confirmed in the step S4 and then uploads the compressed data;
s6, updating the clustering result obtained in the step S3;
s7, repeating the steps S4 to S6 to finish equipment data acquisition based on edge calculation.
The step S2 of initializing and classifying the data acquisition device according to the working data information acquired in the step S1 specifically includes the following steps:
classifying the data acquisition devices adopting the same data item into one type, and setting the data uploading formats of the data acquisition devices of the one type to be the same.
The step S3 of clustering the data acquisition equipment based on the classification algorithm and the classification result of the step S2 specifically comprises the following steps:
A. acquiring historical data of all N data acquisition devices, wherein each data acquisition device acquires Z pieces of data, and the data node set is expressed as X= { X 1 ,x 2 ,...,x N X, where x n Representing an nth data acquisition device D n And x is the feature matrix of (2) n Is M x Z, M being the nth data acquisition device D n The number of data items collected; variable(s)Representing a feature matrix x n Element values with medium coordinates (m, z);
B. initializing a cluster head set:
randomly selecting a node from the set X as an initial cluster head C 1 The method comprises the steps of carrying out a first treatment on the surface of the Each node calculates the shortest distance d between itself and the current existing cluster head nk ；
Calculating that each node is selected as the next cluster head nodeProbability p n Is thatd n The distance between the nth node and the existing cluster head is the distance;
according to the calculated probability p of each node n Randomly selecting one node as the next cluster head node;
repeating the steps until the cluster head nodes with the set number K are obtained, so that the cluster head nodes combined with C to be C= { C is obtained 1 ,C 2 ,...,C K }；
C. Clustering each node: calculating the distance between each node and all cluster head nodes obtained currently; each node adds the node into the cluster where the cluster head closest to the node is located, so that the clustering process of each node is completed;
D. updating the cluster head:
for each cluster, the reference cluster head of the current cluster is calculated by adopting the following formula
In |G k The i indicates the number of nodes in the current cluster;
selecting a node closest to the reference cluster head from the current cluster as a new cluster head of the current cluster, which is expressed asWherein->Represents x n And->A distance therebetween;
after this step is performed on each cluster, the obtained product is obtainedIs set of cluster heads C' = { C 1 ',C' 2 ,...,C' K And clustering results;
E. and C-D, repeating the steps until the cluster head set is not updated any more or the preset maximum iteration times are reached, and obtaining a final clustering result and the cluster head set.
The distance in the step B and the step C is a Hamming distance; in specific implementation, the following formula is adopted to calculate the Hamming distance d nk ：
In the middle ofIs an exclusive or operation; />Is cluster head C' k Element values corresponding to the intermediate coordinates (m, z);
according to the clustering result obtained at present, the step S4 of determining the data item to be compressed for each cluster specifically includes the following steps:
for each cluster G k Acquiring the latest data of each node, wherein the composition dimension is |G k Data Y of |xm k ，|G k The I is the number of nodes in the current cluster;
for data Y k Performing line-by-line differential operation to obtainWherein->The first row data in (2) is calculated by the following formula:
wherein the method comprises the steps ofFor data Y k Row 1 data in (a), and the dimension is 1×m; the value of l is 1,2 k -1; f (x, y) is a processing function for normalizing the differential result, and f (x, y) is calculated as +.> Is a set decimal;
based on dataDetermining the data item to be compressed:
if the data isOne of the columns is 0, and the data item corresponding to the column is added into the compression list L k In which the data items corresponding to columns which are not all 0 are added to the normal list +.>
Will compress list L k The values of the data items of (2) are combined and calculated to obtain the corresponding hash value h k 。
Each data acquisition device in step S5 compresses the acquired data according to the data item confirmed in step S4 and then uploads the compressed data, and specifically includes the following steps:
each data acquisition device acquires the compressed list L obtained in the step S4 k List of normalhash value h k And corresponding parameters;
each data acquisition device selects the compressed column after the data acquisition is completedTable L k The data items in the hash table are combined and calculated to obtain a corresponding hash value H k And (3) judging:
if H k ＝h k The data acquisition device only uploads the normal listData of the corresponding data item;
if H k ≠h k Uploading the data of all the data items by the data acquisition equipment;
after receiving the data, judging the length of the received data:
if the received data length is equal to the normal listThe corresponding data length is consistent, the received data is matched with the compressed list L k After the corresponding data are summarized, complete data are obtained;
if the received data length is equal to the normal listIf the corresponding data length is inconsistent, the received data is used as complete data.
The step S6 of updating the clustering result obtained in the step S3 specifically comprises the following steps:
the compression rate p of each cluster is calculated as followsWherein B is the number of compressed data, A is the total data volume uploaded by the current cluster;
judging the compression ratio:
if p 2 ＜p≤p 1 S4, processing the corresponding cluster, and re-determining the data item to be compressed of the corresponding cluster; p is p 1 For a first threshold value set, p 2 Is a set second threshold;
if p is less than or equal to p 2 And then adopting the step S3 to carry out cluster division again, and then adopting the step S4 to determine the data item needing to be compressed for the re-divided clusters.
The step S6 of updating the clustering result obtained in the step S3 specifically comprises the following steps:
setting clustering time T;
and (3) re-dividing all the data acquisition equipment into clusters in the step S3 at intervals of set clustering time T, and determining data items needing to be compressed in the re-divided clusters in the step S4.
The invention also discloses a system for realizing the equipment data acquisition method based on the edge calculation, which comprises a data acquisition module, an initial classification module, a clustering module, a compression module, an uploading module, an updating module and a circulation module; the data acquisition module, the initial classification module, the clustering module, the compression module, the uploading module and the updating module are sequentially connected in series; the input end of the circulation module is connected with the output end of the updating module, and the output end of the circulation module is connected with the compression module; the data acquisition module is used for acquiring working data information of the data acquisition equipment and uploading the data to the initial classification module; the initial classification module is used for carrying out initial classification on the data acquisition equipment according to the received data and uploading the data to the clustering module; the clustering module is used for clustering the data acquisition equipment according to the received data and uploading the data to the compression module; the compression module is used for determining data items to be compressed of each cluster according to received data and the currently obtained clustering result, and uploading the data to the uploading module; the uploading module is used for compressing the acquired data according to the received data items by each data acquisition device and uploading the acquired data according to the confirmed data items, and the data uploading and updating module; the updating module is used for updating the clustering result according to the received data and uploading the data to the circulating module; the circulation module is used for repeating the operation of the compression module, the uploading module and the updating module to finish the equipment data acquisition based on the edge calculation.
According to the equipment data acquisition method and system based on edge calculation, the compressible data items are calculated, and clustering and compression items do not need to be manually specified; meanwhile, the invention can effectively utilize the calculation capability of the data acquisition equipment, reduce the amount of uploading data, and improve the reliability of data acquisition by reducing the amount of uploading data; therefore, the invention has high reliability, good accuracy and higher efficiency.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
FIG. 2 is a schematic diagram of functional modules of the system of the present invention.
Detailed Description
A schematic process flow diagram of the method of the present invention is shown in fig. 1: the invention provides a device data acquisition method based on edge calculation, which comprises the following steps:
s1, acquiring working data information of data acquisition equipment;
s2, initializing and classifying the data acquisition equipment according to the working data information acquired in the step S1; the method specifically comprises the following steps:
classifying data acquisition equipment adopting the same data item into one type, and setting the same format of uploading data by the data acquisition equipment;
s3, clustering the data acquisition equipment based on the classification algorithm and the classification result of the step S2; the method specifically comprises the following steps:
A. acquiring historical data of all N data acquisition devices, wherein each data acquisition device acquires Z pieces of data, and the data node set is expressed as X= { X 1 ,x 2 ,...,x N X, where x n Representing an nth data acquisition device D n And x is the feature matrix of (2) n Is M x Z, M being the nth data acquisition device D n The number of data items collected; variable(s)Representing a feature matrix x n Element values with medium coordinates (m, z);
B. initializing a cluster head set:
randomly select from the set XA node is used as an initial cluster head C 1 The method comprises the steps of carrying out a first treatment on the surface of the Each node calculates the shortest distance d between itself and the current existing cluster head nk The method comprises the steps of carrying out a first treatment on the surface of the In the specific implementation, if the selected distance is the Hamming distance, the Hamming distance d is calculated by the following formula nk ：
In the middle ofIs an exclusive or operation; />Is cluster head C' k Element values corresponding to the intermediate coordinates (m, z);
calculating probability p that each node is selected as the next cluster head node n Is thatd n The distance between the nth node and the existing cluster head is the distance;
according to the calculated probability p of each node n Randomly selecting one node as the next cluster head node;
repeating the steps until the cluster head nodes with the set number K are obtained, so that the cluster head nodes combined with C to be C= { C is obtained 1 ,C 2 ,...,C K }；
C. Clustering each node: calculating the distance between each node and all cluster head nodes obtained currently; each node adds the node into the cluster where the cluster head closest to the node (preferably the Hamming distance) is located, so that the clustering process of each node is completed;
D. updating the cluster head:
for each cluster, the reference cluster head of the current cluster is calculated by adopting the following formula
In |G k The i indicates the number of nodes in the current cluster;
selecting a node closest to the reference cluster head from the current cluster as a new cluster head of the current cluster, which is expressed asWherein->Represents x n And->A distance therebetween;
after this step is performed on each cluster, a new cluster head set C' = { C is obtained 1 ',C' 2 ,...,C' K And clustering results;
E. repeating the steps C-D until the cluster head set is not updated any more or the preset maximum iteration times are reached, and obtaining a final clustering result and the cluster head set;
the reason for the clustering is to divide the devices with similar collected data into one cluster based on the historical data, so that the probability of compression is higher;
s4, determining data items to be compressed of each cluster according to the currently obtained clustering result; the method specifically comprises the following steps:
for each cluster G k Acquiring the latest data of each node, wherein the composition dimension is |G k Data Y of |xm k ，|G k The I is the number of nodes in the current cluster;
for data Y k Performing line-by-line differential operation to obtainWherein->The first row data in (2) is calculated by the following formula:
wherein the method comprises the steps ofFor data Y k Row 1 data in (a), and the dimension is 1×m; the value of l is 1,2 k -1; f (x, y) is a processing function for normalizing the differential result, and f (x, y) is calculated as +.> Is a set decimal;
based on dataDetermining the data item to be compressed:
if the data isOne of the columns is 0, and the data item corresponding to the column is added into the compression list L k In which the data items corresponding to columns which are not all 0 are added to the normal list +.>
Will compress list L k The values of the data items of (2) are combined and calculated to obtain the corresponding hash value h k ；
S5, each data acquisition device compresses acquired data according to the data item confirmed in the step S4 and then uploads the compressed data; the method specifically comprises the following steps:
each data acquisitionThe device acquires the compressed list L obtained in the step S4 k List of normalhash value h k And corresponding parameters;
after the data acquisition is completed, each data acquisition device selects the compressed list L k The data items in the hash table are combined and calculated to obtain a corresponding hash value H k And (3) judging:
if H k ＝h k The data acquisition device only uploads the normal listData of the corresponding data item;
if H k ≠h k Uploading the data of all the data items by the data acquisition equipment;
after receiving the data, judging the length of the received data:
if the received data length is equal to the normal listThe corresponding data length is consistent, the received data is matched with the compressed list L k After the corresponding data are summarized, complete data are obtained;
if the received data length is equal to the normal listIf the corresponding data length is inconsistent, the received data is used as complete data;
s6, updating the clustering result obtained in the step S3;
when the specific updating is carried out, an active updating mode or a passive updating mode can be adopted;
if active updating is adopted, the method specifically comprises the following steps:
the compression rate p of each cluster is calculated as followsWherein B is the number of compressed data, A is the total data volume uploaded by the current cluster;
judging the compression ratio:
if p 2 ＜p≤p 1 S4, processing the corresponding cluster, and re-determining the data item to be compressed of the corresponding cluster; p is p 1 For a first threshold value set, p 2 Is a set second threshold;
if p is less than or equal to p 2 Then adopting the step S3 to carry out cluster division again, and then adopting the step S4 to determine the data item needing to be compressed for the re-divided clusters;
if the passive update is adopted, the method specifically comprises the following steps:
setting clustering time T;
every set clustering time T, adopting a step S3 to re-divide clusters of all data acquisition equipment, and adopting a step S4 to determine data items needing to be compressed for the re-divided clusters;
s7, repeating the steps S4 to S6 to finish equipment data acquisition based on edge calculation.
FIG. 2 is a schematic diagram of functional modules of the system of the present invention: the system for realizing the equipment data acquisition method based on edge calculation comprises a data acquisition module, an initial classification module, a clustering module, a compression module, an uploading module, an updating module and a circulation module; the data acquisition module, the initial classification module, the clustering module, the compression module, the uploading module and the updating module are sequentially connected in series; the input end of the circulation module is connected with the output end of the updating module, and the output end of the circulation module is connected with the compression module; the data acquisition module is used for acquiring working data information of the data acquisition equipment and uploading the data to the initial classification module; the initial classification module is used for carrying out initial classification on the data acquisition equipment according to the received data and uploading the data to the clustering module; the clustering module is used for clustering the data acquisition equipment according to the received data and uploading the data to the compression module; the compression module is used for determining data items to be compressed of each cluster according to received data and the currently obtained clustering result, and uploading the data to the uploading module; the uploading module is used for compressing the acquired data according to the received data items by each data acquisition device and uploading the acquired data according to the confirmed data items, and the data uploading and updating module; the updating module is used for updating the clustering result according to the received data and uploading the data to the circulating module; the circulation module is used for repeating the operation of the compression module, the uploading module and the updating module to finish the equipment data acquisition based on the edge calculation.

Claims (9)

1. An equipment data acquisition method based on edge calculation comprises the following steps:
s1, acquiring working data information of data acquisition equipment;
s2, initializing and classifying the data acquisition equipment according to the working data information acquired in the step S1;
s3, clustering the data acquisition equipment based on the classification algorithm and the classification result of the step S2;
s4, determining data items to be compressed of each cluster according to the currently obtained clustering result;
s5, each data acquisition device compresses acquired data according to the data item confirmed in the step S4 and then uploads the compressed data;
s6, updating the clustering result obtained in the step S3;
s7, repeating the steps S4 to S6 to finish equipment data acquisition based on edge calculation.
2. The method for collecting device data based on edge calculation according to claim 1, wherein the step S2 is characterized in that the step S1 is performed according to the working data information, and the method comprises the following steps:
classifying the data acquisition devices adopting the same data item into one type, and setting the data uploading formats of the data acquisition devices of the one type to be the same.
3. The method for collecting data of equipment based on edge calculation according to claim 2, wherein the step S3 is based on the classification algorithm and the classification result of step S2, and the method specifically comprises the following steps:
A. acquiring historical data of all N data acquisition devices, wherein each data acquisition device acquires Z pieces of data, and the data node set is expressed as X= { X 1 ,x 2 ,...,x N X, where x n Representing an nth data acquisition device D n And x is the feature matrix of (2) n Is M x Z, M being the nth data acquisition device D n The number of data items collected; variable(s)Representing a feature matrix x n Element values with medium coordinates (m, z);
B. initializing a cluster head set:
randomly selecting a node from the set X as an initial cluster head C 1 The method comprises the steps of carrying out a first treatment on the surface of the Each node calculates the shortest distance d between itself and the current existing cluster head nk ；
Calculating probability p that each node is selected as the next cluster head node n Is thatd n The distance between the nth node and the existing cluster head is the distance;
according to the calculated probability p of each node n Randomly selecting one node as the next cluster head node;
repeating the steps until the cluster head nodes with the set number K are obtained, so that the cluster head nodes combined with C to be C= { C is obtained 1 ,C 2 ,...,C K }；
C. Clustering each node: calculating the distance between each node and all cluster head nodes obtained currently; each node adds the node into the cluster where the cluster head closest to the node is located, so that the clustering process of each node is completed;
D. updating the cluster head:
for each cluster, the reference cluster head of the current cluster is calculated by adopting the following formula
In |G k The i indicates the number of nodes in the current cluster;
selecting a node closest to the reference cluster head from the current cluster as a new cluster head of the current cluster, which is expressed asWherein->Represents x n And->A distance therebetween;
after the step is carried out on each cluster, a new cluster head set C ' = { C ' is obtained ' 1 ,C' 2 ,...,C' K And clustering results;
E. and C-D, repeating the steps until the cluster head set is not updated any more or the preset maximum iteration times are reached, and obtaining a final clustering result and the cluster head set.
4. The method for collecting device data based on edge calculation according to claim 3, wherein the distance in step B and step C is a hamming distance; in specific implementation, the following formula is adopted to calculate the Hamming distance d nk ：
In the middle ofIs an exclusive or operation; />Is cluster head C' k Element values corresponding to the middle coordinates (m, z).
5. The method for collecting device data based on edge computing as recited in claim 3, wherein the determining, in step S4, the data item to be compressed for each cluster according to the currently obtained clustering result specifically includes the following steps:
for each cluster G k Acquiring the latest data of each node, wherein the composition dimension is |G k Data Y of |xm k ，|G k The I is the number of nodes in the current cluster;
for data Y k Performing line-by-line differential operation to obtainWherein->The first row data in (2) is calculated by the following formula:
wherein the method comprises the steps ofFor data Y k Row 1 data in (a), and the dimension is 1×m; the value of l is 1,2 k -1; f (x, y) is a processing function for normalizing the differential result, and f (x, y) is calculated as +.> Is a set decimal;
based on dataDetermining the data item to be compressed:
if the data isOne of the columns is 0, and the data item corresponding to the column is added into the compression list L k In which the data items corresponding to columns which are not all 0 are added to the normal list +.>
Will compress list L k The values of the data items of (2) are combined and calculated to obtain the corresponding hash value h k 。
6. The method for collecting data of edge-based computing device according to claim 4, wherein each data collecting device in step S5 compresses the collected data according to the data item confirmed in step S4 and then uploads the compressed data, and specifically comprises the following steps:
each data acquisition device acquires the compressed list L obtained in the step S4 k List of normalhash value h k And corresponding parameters;
after the data acquisition is completed, each data acquisition device selects the compressed list L k The data items in the hash table are combined and calculated to obtain a corresponding hash value H k And (3) judging:
if H k ＝h k The data acquisition device only uploads the normal columnWatch (watch)Data of the corresponding data item;
if H k ≠h k Uploading the data of all the data items by the data acquisition equipment;
after receiving the data, judging the length of the received data:
if the received data length is equal to the normal listThe corresponding data length is consistent, the received data is matched with the compressed list L k After the corresponding data are summarized, complete data are obtained;
if the received data length is equal to the normal listIf the corresponding data length is inconsistent, the received data is used as complete data.
7. The method for collecting device data based on edge calculation according to claim 6, wherein the step S6 of updating the clustering result obtained in the step S3 specifically includes the following steps:
the compression rate p of each cluster is calculated as followsWherein B is the number of compressed data, A is the total data volume uploaded by the current cluster;
judging the compression ratio:
if p 2 ＜p≤p 1 S4, processing the corresponding cluster, and re-determining the data item to be compressed of the corresponding cluster; p is p 1 For a first threshold value set, p 2 Is a set second threshold;
if p is less than or equal to p 2 Then adopting step S3 to reenterAnd (3) line clustering division is performed, and then a data item needing to be compressed is determined by adopting a step S4 for the re-divided clusters.
8. The method for collecting device data based on edge calculation according to claim 6, wherein the step S6 of updating the clustering result obtained in the step S3 specifically includes the following steps:
setting clustering time T;
and (3) re-dividing all the data acquisition equipment into clusters in the step S3 at intervals of set clustering time T, and determining data items needing to be compressed in the re-divided clusters in the step S4.
9. A system for implementing the edge computing-based device data acquisition method according to any one of claims 1 to 8, comprising a data acquisition module, an initial classification module, a clustering module, a compression module, an uploading module, an updating module and a circulation module; the data acquisition module, the initial classification module, the clustering module, the compression module, the uploading module and the updating module are sequentially connected in series; the input end of the circulation module is connected with the output end of the updating module, and the output end of the circulation module is connected with the compression module; the data acquisition module is used for acquiring working data information of the data acquisition equipment and uploading the data to the initial classification module; the initial classification module is used for carrying out initial classification on the data acquisition equipment according to the received data and uploading the data to the clustering module; the clustering module is used for clustering the data acquisition equipment according to the received data and uploading the data to the compression module; the compression module is used for determining data items to be compressed of each cluster according to received data and the currently obtained clustering result, and uploading the data to the uploading module; the uploading module is used for compressing the acquired data according to the received data items by each data acquisition device and uploading the acquired data according to the confirmed data items, and the data uploading and updating module; the updating module is used for updating the clustering result according to the received data and uploading the data to the circulating module; the circulation module is used for repeating the operation of the compression module, the uploading module and the updating module to finish the equipment data acquisition based on the edge calculation.
CN202310494119.8A 2023-05-05 2023-05-05 Equipment data acquisition method and system based on edge calculation Pending CN116527525A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310494119.8A CN116527525A (en) 2023-05-05 2023-05-05 Equipment data acquisition method and system based on edge calculation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310494119.8A CN116527525A (en) 2023-05-05 2023-05-05 Equipment data acquisition method and system based on edge calculation

Publications (1)

Publication Number Publication Date
CN116527525A true CN116527525A (en) 2023-08-01

Family

ID=87389860

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310494119.8A Pending CN116527525A (en) 2023-05-05 2023-05-05 Equipment data acquisition method and system based on edge calculation

Country Status (1)

Country Link
CN (1) CN116527525A (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101841932A (en) * 2010-05-10 2010-09-22 南京邮电大学 Distributed compression sensing method based on dynamic clustering in wireless sensor network
KR101072448B1 (en) * 2010-04-27 2011-10-11 강원대학교산학협력단 Wireless sensor network system and clustering method thereof
CN104703262A (en) * 2015-03-20 2015-06-10 湘潭大学 Compressed sensing-based clustered data collecting method
CN105025498A (en) * 2015-06-08 2015-11-04 南京邮电大学 A sensing network clustering type space time compression method based on network coding and compression sensing
CN107124693A (en) * 2017-06-07 2017-09-01 北京邮电大学 The MTC device data transmission method and system of a kind of cluster compression
CN109525956A (en) * 2019-01-02 2019-03-26 吉林大学 The energy-efficient method of data capture of sub-clustering in wireless sense network based on data-driven
CN110996371A (en) * 2019-12-16 2020-04-10 南京邮电大学 Clustering algorithm for prolonging life cycle of wireless sensor network
CN111093166A (en) * 2019-12-06 2020-05-01 北京京航计算通讯研究所 Compressed data collection system using sparse measurement matrix in internet of things

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101072448B1 (en) * 2010-04-27 2011-10-11 강원대학교산학협력단 Wireless sensor network system and clustering method thereof
CN101841932A (en) * 2010-05-10 2010-09-22 南京邮电大学 Distributed compression sensing method based on dynamic clustering in wireless sensor network
CN104703262A (en) * 2015-03-20 2015-06-10 湘潭大学 Compressed sensing-based clustered data collecting method
CN105025498A (en) * 2015-06-08 2015-11-04 南京邮电大学 A sensing network clustering type space time compression method based on network coding and compression sensing
CN107124693A (en) * 2017-06-07 2017-09-01 北京邮电大学 The MTC device data transmission method and system of a kind of cluster compression
CN109525956A (en) * 2019-01-02 2019-03-26 吉林大学 The energy-efficient method of data capture of sub-clustering in wireless sense network based on data-driven
CN111093166A (en) * 2019-12-06 2020-05-01 北京京航计算通讯研究所 Compressed data collection system using sparse measurement matrix in internet of things
CN110996371A (en) * 2019-12-16 2020-04-10 南京邮电大学 Clustering algorithm for prolonging life cycle of wireless sensor network

Similar Documents

Publication Publication Date Title
CN102687404B (en) Data value occurrence information for data compression
CN112488070A (en) Neural network compression method for remote sensing image target detection
CN113610227B (en) Deep convolutional neural network pruning method for image classification
CN112817940B (en) Gradient compression-based federated learning data processing system
CN108509592A (en) Date storage method, read method based on Redis and device
CN114640356A (en) Big data compression method, system and storage medium based on neural network
CN109165006B (en) Design optimization and hardware implementation method and system of Softmax function
CN110198171B (en) Data compression method and device, computer readable medium and electronic equipment
CN114943342A (en) Optimization method of federated learning system
CN115858476A (en) Efficient storage method for user-defined form acquisition data in web development system
CN117743870B (en) Water conservancy data management system based on big data
CN116527525A (en) Equipment data acquisition method and system based on edge calculation
CN102394718A (en) Sensing network data compression coding/decoding method
CN107944045B (en) Image search method and system based on t distribution Hash
CN102724508A (en) Distinguishability self-adapting node tree encoding method of JPEG (joint photographic experts group) 2000
US20230325374A1 (en) Generation method and index condensation method of embedding table
CN114329663B (en) Slope unit dividing method based on scale of historical geological disasters
CN116050579A (en) Building energy consumption prediction method and system based on depth feature fusion network
CN112016004A (en) Multi-granularity information fusion-based job crime screening system and method
CN116341689B (en) Training method and device for machine learning model, electronic equipment and storage medium
CN117648063B (en) Intelligent operation management system and method based on big data analysis
CN114189825B (en) Data processing method and system based on industrial Internet and intelligent manufacturing
CN114679184B (en) Data compression method and system for time sequence database
CN117294314B (en) Fruit and vegetable can production information data record management method
CN116743179B (en) Ammeter data optimization acquisition processing method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination