CN117592556B - Semi-federal learning system based on GNN and operation method thereof - Google Patents
Semi-federal learning system based on GNN and operation method thereof Download PDFInfo
- Publication number
- CN117592556B CN117592556B CN202410069642.0A CN202410069642A CN117592556B CN 117592556 B CN117592556 B CN 117592556B CN 202410069642 A CN202410069642 A CN 202410069642A CN 117592556 B CN117592556 B CN 117592556B
- Authority
- CN
- China
- Prior art keywords
- data
- training
- local
- model
- terminal equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000012549 training Methods 0.000 claims abstract description 224
- 230000005540 biological transmission Effects 0.000 claims abstract description 90
- 230000002776 aggregation Effects 0.000 claims abstract description 41
- 238000004220 aggregation Methods 0.000 claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 36
- 239000011159 matrix material Substances 0.000 claims description 55
- 238000004422 calculation algorithm Methods 0.000 claims description 40
- 238000011156 evaluation Methods 0.000 claims description 39
- 238000007781 pre-processing Methods 0.000 claims description 36
- 238000005457 optimization Methods 0.000 claims description 28
- 238000003860 storage Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 14
- 230000008569 process Effects 0.000 claims description 13
- 238000004458 analytical method Methods 0.000 claims description 12
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 230000010365 information processing Effects 0.000 claims description 12
- 239000013598 vector Substances 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 11
- 230000002159 abnormal effect Effects 0.000 claims description 7
- 230000004931 aggregating effect Effects 0.000 claims description 6
- 230000003190 augmentative effect Effects 0.000 claims description 6
- 230000006835 compression Effects 0.000 claims description 6
- 238000007906 compression Methods 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 5
- 230000003416 augmentation Effects 0.000 claims description 4
- 238000004519 manufacturing process Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000013500 data storage Methods 0.000 claims description 3
- 230000004069 differentiation Effects 0.000 claims description 3
- 230000006837 decompression Effects 0.000 claims 1
- 230000002457 bidirectional effect Effects 0.000 abstract description 3
- 238000012360 testing method Methods 0.000 description 8
- 238000012854 evaluation process Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 230000001537 neural effect Effects 0.000 description 4
- 238000012512 characterization method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007621 cluster analysis Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/098—Distributed learning, e.g. federated learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention belongs to the technical field of federal learning and edge computing, and discloses a semi-federal learning system based on GNN and an operation method thereof, wherein the system comprises a cloud service center, a plurality of edge servers and terminal equipment, wherein bidirectional connection exists between the edge servers and the cloud service center, bidirectional links exist between the edge servers and the terminal equipment, the edge servers are used for analyzing the terminal equipment needing to upload local data, issuing a transmission decision, and receiving model parameters uploaded by the terminal equipment for carrying out local training for aggregation after determining the terminal equipment for local training; and simultaneously receiving local data sent by the terminal equipment which cannot perform local training, performing model training after performing deduplication processing, using the obtained model parameters for aggregation, and uploading the aggregated model parameters to the cloud service center. The method improves the inclusion of the system and the performance of the model, and solves the problem of model overfitting caused by repeated data training.
Description
Technical Field
The invention belongs to the technical field of federal learning and edge calculation, and particularly relates to a semi-federal learning system based on GNN and an operation method thereof.
Background
Federal learning is a machine learning method that aims to solve the problems of data privacy and data dispersion by training a model on a local device and aggregating model parameters on a central server to achieve co-training of a global model in a distributed environment. In the existing federal learning system, after each user device performs model training by using local data, local gradients obtained by training are uploaded to a base station, and then model parameters are uploaded to a central server for aggregation, so that a global model is obtained. Traditional federal learning assumes that the data between devices is distributed, each device has its own local data set, and model training is performed on the device.
However, in federal learning in an edge scenario of the internet of things, many devices do not have the ability to perform local training, such as sensors, cameras, etc., which can only collect data, transmit the data out but do not have enough computing power to participate in model training; devices with computing power, such as smart bracelets, have limited storage space and need to process other tasks, and cannot use all device resources for federal learning; in addition, some devices such as industrial control devices and intelligent home controllers are limited by network communication modules and environmental factors, and may face the problem of unstable network connection, and parameter updating of each round cannot be performed stably. An edge server is therefore required to assist these devices in model training for federal learning.
In addition, in a system considering the task of assisting with an edge server, the necessity and rationality of uploading data by a terminal device are not considered, meanwhile, the uploading data is lack of necessary processing, and repeated redundant data wastes computational resources, and meanwhile, the problem of model overfitting is caused.
Disclosure of Invention
In order to solve the technical problems, the invention provides a semi-federal learning system based on GNN and an operation method thereof, wherein the GNN (graph neural network) is used for analyzing and processing the real-time state and performance of each terminal device, and the characteristics of each terminal device are used for executing a device selection task and giving a transmission decision so that a device which cannot be trained can participate in model training through an optimal transmission link, thereby improving the inclusion of the system and the performance of the model; meanwhile, before training the original data uploaded by part of equipment, the edge server can remove the data uploaded repeatedly through matching of a similarity algorithm for two times, so that the problem of model fitting caused by repeated data training is solved.
In order to achieve the above purpose, the invention is realized by the following technical scheme:
the invention relates to a half federal learning system based on GNN, which comprises a cloud service center, a plurality of edge servers and terminal equipment connected with the edge servers and participating in training,
The cloud service center is used for issuing training tasks to the edge servers at the beginning of training, then carrying out aggregation optimization on global models uploaded by the edge servers, updating the global models and issuing the global models to the edge servers;
the terminal equipment is heterogeneous, comprises terminal equipment for carrying out local training and terminal equipment incapable of carrying out local training, has high-performance computing resources, high-capacity storage resources and high-speed stable network connection, is suitable for processing complex computing tasks and storing large-scale data, can meet real-time parameter updating requirements, can train a model issued by the edge server to obtain model parameters, and uploads the model parameters to the edge server; some or some of computing, storing or communication resources of the terminal equipment which cannot perform local training are limited, the acquired local data need to be sent to the edge server, and the terminal equipment cooperates with the edge server to complete model training of federal learning;
the edge server is used for analyzing the terminal equipment needing to upload local data, issuing a transmission decision, determining the terminal equipment carrying out local training, and then receiving the model parameters uploaded by the terminal equipment carrying out the local training for aggregation; meanwhile, local data sent by terminal equipment incapable of performing local training is received, the local data is subjected to de-duplication processing and then subjected to model training, the obtained model parameters are used for aggregation, and the aggregated model parameters are uploaded to the cloud service center, wherein the edge server comprises a data receiving and transmitting module, a data preprocessing module, a transmission decision module and a model training aggregator, the model training aggregator comprises a model parameter preprocessing module, a data similarity relation processing module, a model training module and a model aggregation module, and the data receiving and transmitting module is used for receiving local resource information uploaded by the terminal equipment capable of performing local training, model parameters obtained after the local data training and the local data uploaded by the terminal equipment incapable of performing local training; transmitting training tasks and updated model parameters issued by a cloud service center; the data preprocessing module is used for judging received local data, if the local resource information uploaded by the terminal equipment for carrying out local training is obtained, the local resource information is uploaded to the transmission decision module, if the local data transmitted by the terminal equipment for carrying out local training is received, the local data is uploaded to the data similarity relation processing module, and if part of model parameters transmitted by the terminal equipment for carrying out local training are received, the local resource information is uploaded to the model parameter preprocessing module; the transmission decision module performs offline learning on the local resource information through a GNN algorithm based on the local resource information of the equipment, judges which equipment has no computing capacity and data storage space according to a training result, and needs to transmit local data to an edge server to perform model training and make corresponding data transmission link decisions online; the model training aggregator is used for carrying out de-duplication and federal learning training on the received data, carrying out aggregation optimization on the received data and the model parameters uploaded by the terminal equipment for carrying out local training, carrying out encryption and compression processing on the aggregated model parameters, transmitting the model parameters to the data transceiving module to be sent to the cloud server for global aggregation update, and the data similarity relation processing module is used for matching out and deleting the local data sequences uploaded repeatedly by using a similarity algorithm and transmitting the processed local data to the model training module; the model training module is used for carrying out model training by using the local data transmitted by the data similarity relation processing module, updating model parameters, and aggregating the updated model parameters and the model parameters uploaded by the terminal equipment for carrying out the local training; the model parameter preprocessing module is used for preprocessing the model parameters sent by the terminal equipment for carrying out local training, and then transmitting the model parameters to the model aggregation module for updating the model parameters; the model aggregation module is used for carrying out aggregation optimization on the model parameters which are uploaded by the terminal equipment and processed by the model parameter preprocessing module and the model parameters obtained by the edge server through the local data training uploading by the terminal equipment which cannot carry out local training, and transmitting the aggregated model parameters to the data receiving and transmitting module for global aggregation updating after encryption and compression processing.
The transmission decision module comprises a characteristic information processing component, a GNN decision component, a decision evaluation component and an evaluation optimization component; the feature information processing component is used for receiving the local resource information sent by the data preprocessing module and extracting a plurality of feature vectors from the received data to generateGenerating a feature matrix X, arranging the extracted feature vector generation feature matrices X into training samples of the graph neural network, sending the training samples to a sample data set, and updating the sample data set in real time; the GNN decision component carries out offline learning on training samples in the sample data set based on a simulated learning algorithm, and generates a transmission decision according to a training result; the decision evaluation component is used for evaluating the rationality and accuracy of the transmission decision, and outputting the index of the target receiving device and each terminal device on line if the evaluation is qualifiedIf the evaluation is unqualified, sending an evaluation result to an evaluation optimization component, so that the evaluation optimization component optimizes training samples in the sample data set according to the evaluation result of the decision evaluation component; the evaluation optimization component optimizes training samples of the sample dataset. The evaluation process of the decision evaluation component comprises: randomly extracting 25% of data from the sample data set as a test data set for transmission test, and judging that the transmission decision is qualified if the classification accuracy reaches 95% or more; the strategy for evaluating the training samples of the sample dataset for optimization by the optimization component includes: and periodically removing the training samples and abnormal values which are farthest after the training samples enter the time index sequence of the sample data set, cross-verifying the training samples, amplifying the original data, and generating more samples so as to increase the size and the diversity of the data set.
The invention discloses an operation method of a half federal learning system based on GNN, which comprises the following steps:
step 1, broadcasting a training task issued by a cloud service center by an edge server, and requesting all terminal devices participating in the training task to report local information of the terminal devices;
step 2, the data preprocessing module of the edge server judges the received data, and if the received data is the local resource information uploaded by the terminal equipment, the received data is uploaded to the transmission decision module; if the local data sent by the terminal equipment is received, uploading the local data to a data similarity relation processing module; if part of model parameters sent by the terminal equipment are received, uploading the model parameters to a model parameter preprocessing module;
step 3, constructing a transmission decision module based on the GNN, judging terminal equipment needing to upload local data, and issuing a transmission decision of the proper terminal equipment;
step 4, the data similarity relation processing module processes the repeatedly uploaded local data, and the repeatedly uploaded local data is deleted and then uploaded to the model training module;
step 5, the model training module gathers the local data and then carries out model training, and the updated model parameters are uploaded to the model aggregation module;
Step 6, the model aggregation module aggregates the model parameters uploaded after the terminal equipment is trained by the local training and the model parameters obtained by the edge server by utilizing the terminal equipment which cannot be trained locally to upload the local data training, and uploads the updated global model to the cloud service center;
step 7, the cloud service center aggregates and updates the global model uploaded by the edge server and then transmits the global model to the edge server participating in training;
and 8, continuing to perform model iterative training of federal learning in the mode of step 1-step 7 until model parameter convergence is achieved or total iteration number J of federal learning is achieved.
The invention further improves that: the step 3 builds a transmission decision module based on the GNN network, judges the terminal equipment needing to upload the local data, and issues a transmission decision of the proper terminal equipment, and specifically comprises the following steps:
step 3.1, the characteristic information processing component obtains the local information of all terminal devices participating in federal learning, and generates data volume and distribution of the terminal devicesComputing resource->Device storage space->Calculating a plurality of eigenvectors including a data average arrival time interval and a data arrival time interval variance; extraction of the graph Structure to generate the graph- >Wherein->Is the terminal device in the figure->Set of->Is the side +.>Is a collection of (3); generating a weighting-free adjacency matrix>For adjacency matrix->The result is obtained after addition of self-loop->Using an augmented adjacency matrix->Generating an augmentation matrix of the device>：
,
Step 3.2, constructing a feature matrix by taking the plurality of feature vectors output by the feature information processing component as the multidimensional feature vector of each terminal device：
,
Wherein,representing the number of participating devices in the federal study of the present round, < >>A dimension representing a feature;
step 3.3, constructing an equipment analysis model based on the GNN algorithm, wherein the input of the equipment analysis model is an augmented adjacency matrix,
Breadth-increasing matrixFeature matrix->The output of the equipment analysis model is a transmission demand feature matrix, and the transmission demand feature matrix comprises full-image equipment information, link connection state information and data volume information:
,
wherein,representing the neural network->Characterization of layer node,/->Representing the neural network->Output node representation of layer, ">，/>To the (th) of the graph neural network>Weight matrix of layer,/>For adjacency matrix->Plus the result from the loop, < >>Is->Degree matrix of->Is a nonlinear activation function;
step 3.4, constructing four types of transmission information feature matrixes according to the parameter value range of the actual transmission requirements: a first transmission information feature matrix T limiting the computing resource, a second transmission information feature matrix D limiting the storage space of the device, a third transmission information feature matrix B limiting the communication time delay and a fourth transmission information feature matrix limiting the link state;
Step 3.5, calculating the similarity between the transmission demand feature matrix and the four transmission information feature matrices output by the equipment analysis model respectively, wherein equipment with insufficient computing resources can obtain lower similarity in the similarity calculation with the first transmission information feature matrix T of limited computing resources, equipment with insufficient storage space can obtain lower similarity in the similarity calculation with the second transmission information feature matrix D of limited equipment storage space, equipment with insufficient communication resources can obtain lower similarity in the similarity calculation with the third transmission information feature matrix B of limited communication time delay, equipment with unstable network connection can obtain lower similarity in the similarity calculation with the fourth transmission information feature matrix of limited link state, and only equipment with output result higher than a decision threshold is selected;
step 3.6, evaluating the result of the GNN network decision, wherein the evaluation process comprises the following steps: randomly extracting 25% of data from the sample data set as a test data set for transmission test, judging the transmission decision as qualified if the classification accuracy reaches 95% or more, optimizing the GNN-based transmission decision according to the decision evaluation result, and optimizing the component, wherein the optimizing strategy comprises the following steps: optimizing training samples of the sample dataset: removing the training samples and abnormal values which are farthest after the training samples enter the time index sequence of the sample data set, cross-verifying the training samples, amplifying local data, and generating more samples so as to increase the size and diversity of the data set; and (3) adjusting the decision threshold in the step 3.5.
The invention further improves that: the step 4 data similarity relation processing module processes the local data repeatedly uploaded and deletes the local data repeatedly uploaded, and specifically comprises the following steps:
step 4.1, data receiving: receiving data sent by a data preprocessing module, extracting a characteristic sequence of each data item in the data sent by the data preprocessing module, and converting the characteristic sequence into sequence data with the same length;
step 4.2, matching by a first similarity algorithm: calculating a distance matrix between the feature sequences in the sequence data obtained in the step 4.1 by using a DTW algorithm to obtain the distance between each feature sequence and other feature sequences, judging whether each feature sequence is similar to the other feature sequences according to a set similarity threshold, and finding out the approximate sequence interval of the repeated data;
step 4.3, matching by a second similarity algorithm: according to the rough sequence interval obtained by the first matching in the step 4.2, a DTW algorithm is used for dynamic adjustment, start and stop points of the sequence interval of the repeated data are precisely positioned, and the positions of the repeated data are accurately found by adjusting the start and stop points of a matching path;
step 4.4, deleting the duplicate redundant data: and deleting the repeated redundant data according to the sequence interval of the repeated data obtained by the second matching, and only reserving one part of non-repeated data.
The invention further improves that: the step 5 specifically comprises the following steps: the model training module collects local data and then carries out model training, namely, a target model for federal learning is trained by utilizing the data, and when the edge server carries out federal learning model training based on a local data set uploaded by the terminal equipment and federal learning model parameters broadcasted by the cloud server, the federal learning model parameters are updated through the following formula to obtain local federal learning model parameters:
,
in the method, in the process of the invention,representing edge server +.>In->Local federal learning model parameters obtained by training of local data sets uploaded by terminal equipment in round of iteration, </i >>Indicate->Global federal learning model parameters broadcasted by cloud server in round of iteration, < ->Representing a preset learning rate->Representing differentiation +.>Representing global federal learning model parameters asLower edge server->Local federal learning model loss function of (c):
,
in the method, in the process of the invention,is a model parameter->In data sample->Is a sample loss function of (1).
The invention further improves that: in step 6, the model aggregation module aggregates two types of model parameters, namely the model parameters uploaded after the training of the terminal equipment for carrying out the local training and the model parameters obtained by the edge server by uploading the local data training by the terminal equipment which cannot carry out the local training, and the model parameters are expressed as follows:
,
Wherein,for edge server->First->Model parameters updated in a second iteration, +.>Which means that the terminal device is provided with a communication interface,indicate->Sub-iteration terminal device->Locally trained raw federal learning model parameters, < ->Representing terminal device +.>Is>Representation->The total number of local data set samples for the individual terminal device,representing edge server +.>In->And training the obtained local federal learning model parameters by using the local data uploaded by the terminal equipment in the round of iteration.
The invention further improves that: in step 7, the cloud server aggregates and updates the model parameters uploaded by the edge server, and the update description expression is as follows:
,
wherein,express cloud service center->Model parameters updated iteratively in rounds, +.>Representing the total number of edge servers in semi-federal learning, < >>Representing edge server->Representing edge server +.>Is a local data sample number; />Representation->The total number of local data samples for each edge server.
The beneficial effects of the invention are as follows:
according to the semi-federation learning system and the learning method based on the GNN, which are deployed in the heterogeneous edge computing scene, the inclusion characteristics of semi-federation learning and the intelligence of the GNN in the aspect of extracting spatial characteristics and node information are combined, training equipment selection can be performed on the basis of heterogeneous edge federation learning, and the graph neural network can process the relation among the equipment in parallel, so that the system has good real-time performance; the image neural network can carry out robust processing on abnormal conditions and noise in the equipment network, can make transmission decisions faster and is suitable for the equipment network with dynamic change;
According to the invention, under the transmission decision made based on the graph neural network, the edge server selects the terminal equipment which cannot be subjected to local training due to the lack of computing capability, lack of storage capacity or unstable network connection based on the local resource characteristics of the equipment, and receives the local data of the equipment to perform model training at the edge end, so that the valuable data resources of the equipment are fully utilized, the model is more fair and accurate, different data distribution and characteristics are better adapted, the problem caused by single equipment data deviation is solved, and the performance of the model in various tasks and scenes is improved; the training tasks of part of the equipment are unloaded to the edge side, the resources of the edge side are fully utilized, the training efficiency of the federal learning system is improved, and the problem that the federal learning converges slowly due to the isomerism is solved;
according to the invention, before the edge server uses the local data uploaded by the terminal equipment to perform model training, the two-time similarity algorithm matching is used, the approximate sequence interval of the repeated data is found out by the first-time similarity algorithm matching, the starting point of the sequence interval is dynamically adjusted by the second-time similarity algorithm matching, the sequence interval of the repeated data is accurately positioned, the repeated redundant data is deleted, the data after the repeated removal is reused for model training, the repeated training of the same data in the training process is avoided, and the efficiency of model training is improved; meanwhile, by deleting the repeated training data, the deviation of the model can be avoided, the generalization capability of the model is improved, and the problem of over-fitting of the model is solved.
Drawings
FIG. 1 is a flow chart of a method of operation of the present invention.
Fig. 2 is a schematic diagram of a semi-federal learning system according to the present invention.
Fig. 3 is a schematic diagram of an edge server according to the present invention.
Detailed Description
Embodiments of the invention are disclosed in the drawings, and for purposes of explanation, numerous practical details are set forth in the following description. However, it should be understood that these practical details are not to be taken as limiting the invention. That is, in some embodiments of the invention, these practical details are unnecessary.
The invention provides a method for carrying out transmission decision on whether local data is required to be uploaded by equipment participating in training by the local data characteristics sent by the edge server mobile equipment, carrying out data receiving on equipment needing to assist training according to the local data of the terminal equipment connected with the edge server, carrying out de-duplication processing on the received local data, carrying out model training and model aggregation at an edge end, and uploading to a cloud service center for carrying out aggregation optimization of a global model.
As shown in fig. 2, the present invention provides a GNN-based semi-federal learning system, which includes a cloud service center, a plurality of edge servers, terminal devices connected to the edge servers and participating in training, the terminal devices being heterogeneous,
The cloud service center is used for transmitting training tasks to the edge servers at the beginning of training, then carrying out aggregation optimization on global models uploaded by the edge servers, updating the global models and transmitting the global models to the edge servers, so that the parameters of the models are uploaded to the cloud service center by the edge servers and the global models are transmitted to the edge servers by the cloud service center;
the terminal equipment is heterogeneous, comprises terminal equipment for carrying out local training and terminal equipment incapable of carrying out local training, has high-performance computing resources, high-capacity storage resources and high-speed stable network connection, is suitable for processing complex computing tasks and storing large-scale data, can meet real-time parameter updating requirements, can train a model issued by the edge server to obtain model parameters, and uploads the model parameters to the edge server; some or some of the computing, storage or communication resources of the terminal equipment which cannot perform local training are limited, and the terminal equipment needs to send the collected local data to the edge server to cooperate with the edge server to complete model training of federal learning.
The method comprises the steps that a bidirectional link exists between an edge server and terminal equipment to communicate with each other, the edge server is used for analyzing the terminal equipment needing to upload local data, issuing a transmission decision, determining the terminal equipment carrying out local training, and then receiving model parameters uploaded by the terminal equipment carrying out the local training for aggregation; and simultaneously receiving local data sent by the terminal equipment which cannot perform local training, performing model training after performing deduplication processing, using the obtained model parameters for aggregation, and uploading the aggregated model parameters to the cloud service center.
As shown in fig. 3, the edge server of the present invention includes a data transceiver module, a data preprocessing module, a transmission decision module and a model training aggregator, wherein the transmission decision module includes a feature information processing component, a GNN decision component, a decision evaluation component and an evaluation optimization component; the model training aggregator comprises a model parameter preprocessing module, a data similarity relation processing module, a model training module and a model aggregating module. Each module is described in detail below:
the data receiving and transmitting module is used for receiving local resource information uploaded by the terminal equipment for carrying out local training, model parameters obtained after the local data training is used and local data uploaded by the terminal equipment which cannot carry out the local training; and sending the training task and the updated model parameters issued by the cloud service center.
The data preprocessing module is used for judging received local data, and firstly needs to analyze the received data and convert the received data into a processable format. Depending on the format and structure of the data, a suitable parsing method, such as JSON parser, XML parser, etc., may be used to extract different data separately based on the specific fields or tags of the data. If the local resource information uploaded by the terminal equipment for carrying out the local training is obtained, the local resource information is uploaded to a transmission decision module, if the local data transmitted by the terminal equipment which cannot carry out the local training is received, the local resource information is uploaded to a data similarity relation processing module, and if the partial model parameters transmitted by the terminal equipment for carrying out the local training are received, the local resource information is uploaded to a model parameter preprocessing module.
The transmission decision module performs offline learning on the local resource information through the GNN algorithm based on the local resource information of the equipment, judges which equipment has no computing capacity and no data storage space according to the training result, and needs to transmit local data to an edge server for model training, and makes corresponding data transmission link decisions online.
The characteristic information processing component is used for receiving the local resource information sent by the data preprocessing module, extracting a plurality of characteristic vector production characteristic matrixes X from the received data, sorting the extracted characteristic vector production characteristic matrixes X into training samples of the graph neural network, sending the training samples to the sample data set, and updating the sample data set in real time;
The GNN decision component carries out offline learning on training samples in the sample data set based on a simulated learning algorithm, and generates a transmission decision according to a training result; the core advantage of mimicking learning is its off-line training and on-line decision-making approach. Thus, the trained GNN model can be effectively applied to real-time decisions;
the decision evaluation component is used for evaluating the rationality and accuracy of the transmission decision, and outputting the index of the target receiving device and each terminal device on line if the evaluation is qualifiedIf the evaluation is unqualified, sending an evaluation result to an evaluation optimization component, so that the evaluation optimization component optimizes training samples in the sample data set according to the evaluation result of the decision evaluation component;
the evaluation optimization component optimizes training samples of a sample data set, and the evaluation process of the decision evaluation component comprises: randomly extracting 25% of data from the sample data set as a test data set for transmission test, and judging that the transmission decision is qualified if the classification accuracy reaches 95% or more;
the strategy for optimizing training samples of a sample dataset by the evaluation optimization component includes: and periodically removing the training samples and abnormal values which are farthest after the training samples enter the time index sequence of the sample data set, cross-verifying the training samples, amplifying the original data to generate more samples so as to increase the size and diversity of the data set, and adjusting the decision threshold.
The model training aggregator is used for carrying out de-duplication and federal learning training on the received data, carrying out aggregation optimization on the received data and the model parameters uploaded by the terminal equipment for carrying out local training, carrying out encryption and compression processing on the aggregated model parameters, and transmitting the aggregated model parameters to the data receiving and sending module to the cloud server for global aggregation updating.
The data similarity relation processing module is used for matching out and deleting the local data sequences which are repeatedly uploaded by using a similarity algorithm, and then transmitting the processed local data to the model training module;
the model training module is used for carrying out model training by using the local data transmitted by the data similarity relation processing module, updating model parameters, and aggregating the updated model parameters and the model parameters uploaded by the terminal equipment for carrying out the local training;
the model parameter preprocessing module is used for preprocessing the model parameters sent by the terminal equipment for carrying out local training, and then transmitting the model parameters to the model aggregation module for updating the model parameters;
the model aggregation module is used for carrying out aggregation optimization on the model parameters which are uploaded by the terminal equipment and processed by the model parameter preprocessing module and the model parameters obtained by the edge server through the local data training uploading by the terminal equipment which cannot carry out local training, and transmitting the aggregated model parameters to the data receiving and transmitting module for global aggregation updating after encryption and compression processing.
Based on the semi-federation learning system, the invention also provides a semi-federation learning operation method based on GNN in a heterogeneous edge scene, as shown in figure 1, the operation method specifically comprises the following steps:
step 1, broadcasting a training task issued by a cloud service center by an edge server, and requesting all terminal devices participating in the training task to report local information of the terminal devices, wherein the training task issued by the cloud service center is to train a federal learning target model by using data, namely minimizing a global loss function, and the target model depends on an optimization problem:
,
wherein,representing model parameters of the object model,/->Represents the theoretical optimal model parameters of federal learning training,representing global loss functions on the data sets of all devices, in which step the local resource information uploaded by the terminal device comprises the terminal device data volume and distribution +.>Computing resource->Device storage space->Information contained therein.
Step 2, the data preprocessing module of the edge server judges the received data, and if the received data is the local resource information uploaded by the terminal equipment, the received data is uploaded to the transmission decision module; if the local data sent by the terminal equipment is received, uploading the local data to a data similarity relation processing module; if part of model parameters sent by the terminal equipment are received, uploading the model parameters to a model parameter preprocessing module;
Step 3, constructing a transmission decision module based on the GNN, judging terminal equipment needing to upload local data, and issuing a transmission decision of the proper terminal equipment, wherein the method specifically comprises the following steps:
step 3.1, the characteristic information processing component obtains the local information of all terminal devices participating in federal learning, and generates data volume and distribution of the terminal devicesComputing resource->Device storage space->Calculating a plurality of eigenvectors including a data average arrival time interval and a data arrival time interval variance; extraction of the graph Structure to generate the graph->Wherein->Is the terminal device in the figure->Set of->Is the side +.>Is a collection of (3); generating a weighting-free adjacency matrix>For adjacency matrix->The result is obtained after addition of self-loop->Using an augmented adjacency matrix->Generating an augmentation matrix of the device>：
；
Step 3.2, constructing a feature matrix by taking the plurality of feature vectors output by the feature information processing component as the multidimensional feature vector of each terminal device：
，
Wherein,representing the number of participating devices in the federal study of the present round, < >>A dimension representing a feature;
step 3.3, constructing an equipment analysis model based on the GNN algorithm, wherein the input of the equipment analysis model is an augmented adjacency matrix An augmentation matrix->Feature matrix->The output of the equipment analysis model is a transmission demand feature matrix, and the transmission demand feature matrix comprises full-image equipment information, link connection state information and data volume information:
，
wherein,representing the neural network->Characterization of layer node,/->Representing the neural network->Output node representation of layer, ">，/>To the (th) of the graph neural network>Weight matrix of layer,/>For adjacency matrix->Plus the result from the loop, < >>Is->Degree matrix of->Is a nonlinear activation function;
step 3.4, constructing four types of transmission information feature matrixes according to the parameter value range of the actual transmission requirements: a first transmission information feature matrix T limiting the computing resource, a second transmission information feature matrix D limiting the storage space of the device, a third transmission information feature matrix B limiting the communication time delay and a fourth transmission information feature matrix limiting the link state;
step 3.5, calculating the similarity between the transmission demand feature matrix and the four transmission information feature matrices output by the equipment analysis model respectively, wherein equipment with insufficient computing resources can obtain lower similarity in the similarity calculation with the first transmission information feature matrix T of limited computing resources, equipment with insufficient storage space can obtain lower similarity in the similarity calculation with the second transmission information feature matrix D of limited equipment storage space, equipment with insufficient communication resources can obtain lower similarity in the similarity calculation with the third transmission information feature matrix B of limited communication time delay, equipment with unstable network connection can obtain lower similarity in the similarity calculation with the fourth transmission information feature matrix of limited link state, and only equipment with output result higher than a decision threshold is selected; the calculation method of the similarity and the setting of the threshold are determined according to specific situations, and different data characteristics and application scenes may need to use different similarity measurement methods and threshold settings.
Step 3.6, evaluating the result of the GNN network decision, wherein the evaluation process comprises the following steps: randomly extracting 25% of data from the sample data set as a test data set for transmission test, judging the transmission decision as qualified if the classification accuracy reaches 95% or more, optimizing the GNN-based transmission decision according to the decision evaluation result, and optimizing the component, wherein the optimizing strategy comprises the following steps: optimizing training samples of the sample dataset: removing the training samples and abnormal values which are farthest after the training samples enter the time index sequence of the sample data set, cross-verifying the training samples, amplifying local data, and generating more samples so as to increase the size and diversity of the data set; and (3) adjusting the decision threshold in the step 3.5.
And 4, the data similarity relation processing module processes the repeatedly uploaded local data, deletes the repeatedly uploaded local data and uploads the repeatedly uploaded local data to the model training module. Deleting the local data repeatedly uploaded, which specifically comprises the following steps:
step 4.1, data receiving: receiving data sent by a data preprocessing module, extracting a characteristic sequence of each data item in the data sent by the data preprocessing module, and converting the characteristic sequence into sequence data with the same length;
Step 4.2, matching by a first similarity algorithm: using DTW (Dynamic Time Warping) algorithm to calculate the distance matrix between the feature sequences in the sequence data obtained in the step 4.1, obtaining the distance between each feature sequence and other feature sequences, judging whether each feature sequence is similar to other feature sequences according to the set similarity threshold, and finding out the approximate sequence interval of the repeated data;
step 4.3, matching by a second similarity algorithm: according to the rough sequence interval obtained by the first matching in the step 4.2, a DTW algorithm is used for dynamic adjustment, start and stop points of the sequence interval of the repeated data are precisely positioned, and the positions of the repeated data are accurately found by adjusting the start and stop points of a matching path;
step 4.4, deleting the duplicate redundant data: and deleting the repeated redundant data according to the sequence interval of the repeated data obtained by the second matching, and only reserving one part of non-repeated data.
When matching different types of data using a similarity algorithm, the similarity algorithm selected may vary from one type of data to another. Such as:
for text data, common similarity algorithms include edit distance (Levenshtein distance), jaccard similarity, cosine similarity, and the like. These algorithms may be used to compare the degree of similarity between text, e.g., for text deduplication, discovery of similar documents, etc.
For numerical data, the similarity between the numerical values may be calculated using an algorithm such as euclidean distance, manhattan distance, cosine similarity, or the like. The algorithms can be used in the fields of cluster analysis, anomaly detection and the like.
For image data, similarity algorithms for image data typically involve extraction and comparison of image features, such as using perceptual hash algorithms (Perceptual Hashing), structural similarity indices (Structural Similarity Index, SSIM), and the like to compare similarity between images.
For time series data, a dynamic time warping (Dynamic Time Warping, DTW) algorithm may be used to measure similarity between sequences, or a frequency domain similarity algorithm based on fourier transforms.
For audio data, a sound feature extraction algorithm, such as MFCC (Mel-frequency cepstral coefficients), may be used to calculate the similarity between the audio.
And 5, the model training module gathers the local data and then carries out model training, and the updated model parameters are uploaded to the model aggregation module. The method comprises the following steps: the model training module collects local data and then carries out model training, namely, a target model for federal learning is trained by utilizing the data, and when the edge server carries out federal learning model training based on a local data set uploaded by the terminal equipment and federal learning model parameters broadcasted by the cloud server, the federal learning model parameters are updated through the following formula to obtain local federal learning model parameters:
，
In the method, in the process of the invention,representing edge server +.>In->Local federal learning model parameters obtained by training of local data sets uploaded by terminal equipment in round of iteration, </i >>Indicate->Global federal learning model parameters broadcasted by cloud server in round of iteration, < ->Representing a preset learning rate->Representing differentiation +.>Representing global federal learning model parameters asLower edge server->Local federal learning model of (a)Loss function:
,
in the method, in the process of the invention,is a model parameter->In data sample->Is a sample loss function of (1).
The loss function may be different for different learning tasks. For example, for a simple linear regression federal learning algorithm,。
step 6, the model aggregation module aggregates two types of model parameters, namely the model parameters uploaded after the training of the terminal equipment for carrying out the local training and the model parameters obtained by the edge server by utilizing the local data uploading training of the terminal equipment which cannot carry out the local training, and the update description expression is as follows:
，
wherein,for edge server->First->Model parameters updated in a second iteration, +.>Representing the terminal device->Indicate->Sub-iteration terminal device->Locally trained raw federal learning model parameters, < ->Representing terminal device +. >Is>Representation->The total number of local data set samples for the individual terminal device, and (2)>Representing edge server +.>In->And training the obtained local federal learning model parameters by using the local data uploaded by the terminal equipment in the round of iteration.
Then the updated global model is obtained and then uploaded to a cloud service center;
and 7, the cloud service center carries out aggregation update on the global model uploaded by the edge server, wherein the update description expression is as follows:
，
wherein,express cloud service center->Model parameters updated iteratively in rounds, +.>Representing the total number of edge servers in semi-federal learning, < >>Representing edge server->Representing edge server +.>Is a local data sample number; />Representation->The total number of local data samples for each edge server.
Issuing the updated and aggregated model parameters to an edge server participating in training;
and 8, continuing to perform model iterative training of federal learning in the mode of step 1-step 7 until model parameter convergence is achieved or total iteration number J of federal learning is achieved.
The semi-federal learning system and the running method based on the GNN in the heterogeneous edge scene solve the problem of slow convergence of federal learning caused by isomerism, improve the generalization capability of the model and solve the problem of overfitting of the model.
The foregoing description is only illustrative of the invention and is not to be construed as limiting the invention. Various modifications and variations of the present invention will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, or the like, which is within the spirit and principles of the present invention, should be included in the scope of the claims of the present invention.
Claims (9)
1. An operation method of a half federal learning system based on GNN is characterized in that: the operation method specifically comprises the following steps:
step 1, broadcasting a training task issued by a cloud service center by an edge server, and requesting all terminal devices participating in the training task to report local information of the terminal devices;
step 2, the data preprocessing module of the edge server judges the received data, and if the received data is the local resource information uploaded by the terminal equipment, the received data is uploaded to the transmission decision module; if the local data sent by the terminal equipment is received, uploading the local data to a data similarity relation processing module; if part of model parameters sent by the terminal equipment are received, uploading the model parameters to a model parameter preprocessing module;
step 3, constructing a transmission decision module based on the GNN, judging terminal equipment needing to upload local data, and issuing a transmission decision of the proper terminal equipment;
Step 4, the data similarity relation processing module processes the repeatedly uploaded local data, and the repeatedly uploaded local data is deleted and then uploaded to the model training module;
step 5, the model training module gathers the local data and then carries out model training, and the updated model parameters are uploaded to the model aggregation module;
step 6, the model aggregation module aggregates the model parameters uploaded after the terminal equipment is trained by the local training and the model parameters obtained by the edge server by utilizing the terminal equipment which cannot be trained locally to upload the local data training, and uploads the updated global model to the cloud service center;
step 7, the cloud service center aggregates and updates the global model uploaded by the edge server and then transmits the global model to the edge server participating in training;
step 8, continuing to perform model iterative training of federal learning in the mode of step 1-step 7 until model parameter convergence is achieved or total iteration number J of federal learning is achieved;
the step 3 builds a transmission decision module based on the GNN network, judges terminal equipment needing to upload local data, and issues a transmission decision of the proper terminal equipment, and specifically comprises the following steps:
Step 3.1, the characteristic information processing component obtains the local information of all the terminal devices participating in federal learning, and generates data volume and distribution D of the terminal devices n Computing resource f n Device storage space S n Calculating a plurality of eigenvectors including a data average arrival time interval and a data arrival time interval variance; extracting the graph structure to generate a graph G (I, R), wherein I is a set of terminal equipment I in the graph, and R is a set of edges R of the graph; generating an adjacent matrix A without weight, and adding self-loop to the adjacent matrix A to obtain a resultUsing an augmented adjacency matrix->Generating an augmentation matrix of the device>
Step 3.2, taking a plurality of feature vectors output by the feature information processing component as multidimensional feature vectors of each terminal device, and constructing a feature matrix X:
wherein n represents the number of participating devices in the federal learning of the present round, and m represents the dimension of the feature;
step 3.3, constructing an equipment analysis model based on the GNN algorithm, wherein the input of the equipment analysis model is an augmented adjacency matrixIncrease matrix +.>The characteristic matrix X is output by the equipment analysis model, and the transmission demand characteristic matrix comprises full-image equipment information, link connection state information and data quantity information:
Wherein H is (l) Representation of layer I nodes of a graph neural network, H (l+1) Output node representation representing layer l+1 of the graph neural network, H (0) ＝X，W (l) For the weight matrix of the first layer of the graph neural network,add the result from the loop to adjacency matrix A, < >>Is thatIs a nonlinear activation function;
step 3.4, constructing four types of transmission information feature matrixes according to the parameter value range of the actual transmission requirements: a first transmission information feature matrix T limiting the computing resource, a second transmission information feature matrix D limiting the storage space of the device, a third transmission information feature matrix B limiting the communication time delay and a fourth transmission information feature matrix limiting the link state;
step 3.5, calculating the similarity between the transmission demand feature matrix output by the equipment analysis model and the four types of transmission information feature matrix constructed in the step 3.4, setting a decision threshold, and only selecting equipment with an output result higher than the decision threshold;
step 3.6, evaluating the result of the GNN network decision, optimizing the component, wherein the optimized strategy comprises the following steps: optimizing training samples of the sample dataset: removing the training samples and abnormal values which are farthest after the training samples enter the time index sequence of the sample data set, cross-verifying the training samples, amplifying local data, and generating more samples so as to increase the size and diversity of the data set; and (3) adjusting the decision threshold in the step 3.5.
2. A method of operating a GNN-based semi-federal learning system according to claim 1, wherein: the step 4 data similarity relation processing module processes the local data repeatedly uploaded and deletes the local data repeatedly uploaded, and specifically comprises the following steps:
step 4.1, data receiving: receiving data sent by a data preprocessing module, extracting a characteristic sequence of each data item in the data sent by the data preprocessing module, and converting the characteristic sequence into sequence data with the same length;
step 4.2, matching by a first similarity algorithm: calculating a distance matrix between the feature sequences in the sequence data obtained in the step 4.1 by using a DTW algorithm to obtain the distance between each feature sequence and other feature sequences, judging whether each feature sequence is similar to the other feature sequences according to a set similarity threshold, and finding out a sequence interval of repeated data;
step 4.3, matching by a second similarity algorithm: according to the sequence interval obtained by the first matching in the step 4.2, a DTW algorithm is used for dynamic adjustment, start and stop points of the sequence interval of the repeated data are accurately positioned, and the positions of the repeated data are accurately found by adjusting the start and stop points of a matching path;
Step 4.4, deleting the duplicate redundant data: and deleting the repeated redundant data according to the sequence interval of the repeated data obtained by the second matching, and only reserving one part of non-repeated data.
3. A method of operating a GNN-based semi-federal learning system according to claim 1, wherein: the step 5 specifically comprises the following steps: the model training module collects local data and then carries out model training, namely, a target model for federal learning is trained by utilizing the data, and when the edge server carries out federal learning model training based on a local data set uploaded by the terminal equipment and federal learning model parameters broadcasted by the cloud server, the federal learning model parameters are updated through the following formula to obtain local federal learning model parameters:
in the method, in the process of the invention,representing local federal learning model parameters, ω, obtained by edge server n training with local data set uploaded by terminal device in the t-th iteration t The global federal learning model parameters broadcasted by the cloud end server in the t-th iteration are represented, ρ represents a preset learning rate, and +.>Representing differentiation, L n (omega) represents the global federal learning model parameter omega t Local federal learning model loss function for lower edge server n:
Wherein l (omega; x) n ，y n ) Is a model parameter omega is obtained in the data sample (x n ，y n ) Is a sample loss function of (1).
4. A method of operating a GNN-based semi-federal learning system according to claim 1, wherein: in step 1, the training task issued by the cloud service center is to train a federal learning target model, i.e. minimize a global loss function, by using data, where the target model depends on an optimization problem:
wherein ω represents the target model parameters, ω * Representing theoretical optimal model parameters for federal learning training, L (ω) represents the global loss function over the dataset for all devices.
5. A method of operating a GNN-based semi-federal learning system according to claim 1, wherein: the semi-federation learning system based on the GNN is applied to heterogeneous edge computing scenes, and comprises a cloud service center, a plurality of edge servers and terminal devices connected with the edge servers and participating in training,
the cloud service center is used for issuing training tasks to the edge servers at the beginning of training, then carrying out aggregation optimization on global models uploaded by the edge servers, updating the global models and issuing the global models to the edge servers;
The terminal equipment is heterogeneous, the terminal equipment comprises terminal equipment for carrying out local training and terminal equipment incapable of carrying out local training, the terminal equipment for carrying out local training trains a model issued by the edge server to obtain model parameters, the model parameters are uploaded to the edge server, the terminal equipment incapable of carrying out local training sends collected local data to the edge server, and the terminal equipment cooperates with the edge server to complete model training of federal learning;
the edge server is used for analyzing the terminal equipment needing to upload local data, issuing a transmission decision, determining the terminal equipment for local training, and then receiving the model parameters uploaded by the terminal equipment for local training for aggregation; and simultaneously receiving local data sent by the terminal equipment which cannot perform local training, performing model training after performing deduplication processing, using the obtained model parameters for aggregation, and uploading the aggregated model parameters to the cloud service center.
6. A method of operating a GNN-based semi-federal learning system according to claim 5, wherein: the edge server comprises a data receiving and transmitting module, a data preprocessing module, a transmission decision module and a model training aggregator, wherein the model training aggregator comprises a model parameter preprocessing module, a data similarity processing module, a model training module and a model aggregating module,
The data receiving and transmitting module is used for receiving local resource information uploaded by the terminal equipment for carrying out local training, model parameters obtained after the local data training is used and local data uploaded by the terminal equipment which cannot carry out the local training; transmitting training tasks and updated model parameters issued by a cloud service center;
the data preprocessing module is used for judging received local data, if the local resource information uploaded by the terminal equipment for carrying out local training is obtained, the local resource information is uploaded to the transmission decision module, if the local data which cannot be sent by the terminal equipment for carrying out local training is received, the local resource information is uploaded to the data similarity relation processing module, and if part of model parameters which are sent by the terminal equipment for carrying out local training are received, the local resource information is uploaded to the model parameter preprocessing module;
the transmission decision module performs offline learning on the local resource information through a GNN algorithm based on the local resource information of the equipment, judges which equipment has no computing capacity and data storage space according to a training result, and needs to transmit local data to an edge server for model training, and makes corresponding data transmission link decisions online;
the model training aggregator is used for carrying out de-duplication and federal learning training on the received data, carrying out aggregation optimization on the received data and the model parameters uploaded by the terminal equipment for carrying out local training, carrying out encryption and compression processing on the aggregated model parameters, and transmitting the aggregated model parameters to the data receiving and sending module to the cloud server for global aggregation updating.
7. A method of operating a GNN-based semi-federal learning system according to claim 6, wherein: the transmission decision module comprises a characteristic information processing component, a GNN decision component, a decision evaluation component and an evaluation optimization component;
the characteristic information processing component is used for receiving the local resource information sent by the data preprocessing module, extracting a plurality of characteristic vector production characteristic matrixes X from the received data, sorting the extracted characteristic vector production characteristic matrixes X into training samples of the graph neural network, sending the training samples to the sample data set, and updating the sample data set in real time;
the GNN decision component carries out offline learning on training samples in the sample data set based on a simulated learning algorithm, and generates a transmission decision according to a training result;
the decision evaluation component is used for evaluating the rationality and accuracy of the transmission decision, if the evaluation is qualified, the index of the target receiving equipment and the calculation speed of each terminal equipment i are output on line, and if the evaluation is unqualified, the evaluation result is sent to the evaluation optimization component, so that the evaluation optimization component optimizes training samples in the sample data set according to the evaluation result of the decision evaluation component;
The evaluation optimization component optimizes training samples of a sample dataset.
8. A method of operating a GNN-based semi-federal learning system according to claim 7, wherein: the strategy for optimizing training samples of a sample dataset by the evaluation optimization component includes: and periodically removing the training samples and abnormal values which are farthest after the training samples enter the time index sequence of the sample data set, cross-verifying the training samples, amplifying the original data, and generating more samples so as to increase the size and the diversity of the data set.
9. A method of operating a GNN-based semi-federal learning system according to claim 6, wherein:
the data similarity relation processing module is used for matching out and deleting the local data sequences which are repeatedly uploaded by using a similarity algorithm, and then transmitting the processed local data to the model training module;
the model training module is used for carrying out model training by using the local data transmitted by the data similarity relation processing module, updating model parameters, and aggregating the updated model parameters with the model parameters uploaded by the terminal equipment for carrying out the local training;
the model parameter preprocessing module is used for preprocessing decryption and decompression of model parameters sent by the terminal equipment for carrying out local training and then transmitting the model parameters to the model aggregation module for updating the model parameters;
The model aggregation module is used for carrying out aggregation optimization on the model parameters which are uploaded by the terminal equipment and processed by the model parameter preprocessing module and the model parameters obtained by the edge server through the local data training uploaded by the terminal equipment which cannot be subjected to local training, and transmitting the aggregated model parameters to the data receiving and transmitting module for global aggregation updating after encryption and compression processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410069642.0A CN117592556B (en) | 2024-01-18 | 2024-01-18 | Semi-federal learning system based on GNN and operation method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410069642.0A CN117592556B (en) | 2024-01-18 | 2024-01-18 | Semi-federal learning system based on GNN and operation method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117592556A CN117592556A (en) | 2024-02-23 |
| CN117592556B true CN117592556B (en) | 2024-03-26 |
Family
ID=89918643
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410069642.0A Active CN117592556B (en) | 2024-01-18 | 2024-01-18 | Semi-federal learning system based on GNN and operation method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117592556B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113806735A (en) * | 2021-08-20 | 2021-12-17 | 北京工业大学 | Execution and evaluation dual-network personalized federal learning intrusion detection method and system |
| WO2023284511A1 (en) * | 2021-07-15 | 2023-01-19 | 支付宝(杭州)信息技术有限公司 | Privacy protection-based graphical model training method and apparatus, and device |
| CN117035059A (en) * | 2023-08-04 | 2023-11-10 | 青岛大学 | Efficient privacy protection recommendation system and method for communication |
-
2024
- 2024-01-18 CN CN202410069642.0A patent/CN117592556B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023284511A1 (en) * | 2021-07-15 | 2023-01-19 | 支付宝(杭州)信息技术有限公司 | Privacy protection-based graphical model training method and apparatus, and device |
| CN113806735A (en) * | 2021-08-20 | 2021-12-17 | 北京工业大学 | Execution and evaluation dual-network personalized federal learning intrusion detection method and system |
| CN117035059A (en) * | 2023-08-04 | 2023-11-10 | 青岛大学 | Efficient privacy protection recommendation system and method for communication |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117592556A (en) | 2024-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109347668B (en) | Training method and device for service quality assessment model | |
| CN111563275B (en) | Data desensitization method based on generation countermeasure network | |
| CN111506773B (en) | Video duplicate removal method based on unsupervised depth twin network | |
| CN109743356B (en) | Industrial internet data acquisition method and device, readable storage medium and terminal | |
| CN114580662A (en) | Federal learning method and system based on anchor point aggregation | |
| WO2023093177A1 (en) | Device fault diagnosis method and apparatus, and electronic device and storage medium | |
| CN114462623B (en) | Data analysis method, system and platform based on edge calculation | |
| CN112906859B (en) | Federal learning method for bearing fault diagnosis | |
| CN113037783B (en) | Abnormal behavior detection method and system | |
| CN113255895A (en) | Graph neural network representation learning-based structure graph alignment method and multi-graph joint data mining method | |
| CN115391576A (en) | Cluster-based federal graph learning method and system | |
| Yi et al. | Fedlora: Model-heterogeneous personalized federated learning with lora tuning | |
| Li et al. | CBFL: a communication-efficient federated learning framework from data redundancy perspective | |
| CN117592556B (en) | Semi-federal learning system based on GNN and operation method thereof | |
| Fellus et al. | Decentralized k-means using randomized gossip protocols for clustering large datasets | |
| CN104090885A (en) | Multi-view video object retrieval system based on local copy detection and method | |
| Zhao et al. | Communication-efficient federated learning for digital twin systems of industrial Internet of Things | |
| CN114640669A (en) | Edge calculation method and device | |
| CN117171686A (en) | Method and system for detecting abnormal data of intelligent power grid based on federal learning | |
| CN116362328A (en) | Federal learning heterogeneous model aggregation method based on fairness characteristic representation | |
| CN113887806B (en) | Long-tail cascade popularity prediction model, training method and prediction method | |
| CN114595816A (en) | Neural network model training method based on edge calculation | |
| Yang et al. | A novel adaptive gradient compression approach for communication-efficient Federated Learning | |
| CN116796850A (en) | Model training method, device, equipment and storage medium | |
| Zeyang | Research on intelligent acceleration algorithm for big data mining in communication network based on support vector machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |