CN114897837A - Power inspection image defect detection method based on federal learning and self-adaptive difference - Google Patents

Abstract

The invention discloses a power inspection image defect detection method based on federal learning and self-adaptive difference, which comprises the following steps: the method comprises the steps of building a federal learning scene, preprocessing data, issuing global model parameters by a central server, training model parameters by a client, uploading local model parameters, weighting and aggregating the client model parameters collected by the central server, calculating the total model loss of the server, and updating the client model parameters. On the premise of privacy protection and data safety sharing, the method ensures that all participants perform collaborative training, and ensures the high precision of the power inspection image defect detection; on the basis of federal learning, a differential privacy technology is introduced, the data storage and model training stage of deep learning is transferred to the local, a local user only interacts model parameters with a central server, and meanwhile, a client side adaptively controls noise addition according to data characteristics.

Description

Power inspection image defect detection method based on federal learning and self-adaptive difference

Technical Field

The invention relates to a power inspection image defect detection method, in particular to a power inspection image defect detection method based on federal learning and self-adaptive difference, and belongs to the technical field of power detection.

Background

With the vigorous development of the construction of the intelligent power grid and the artificial intelligence technology in China, the detection of the defects of the power inspection image becomes one of the important works for guaranteeing the safety of the power grid. The goal of power patrol image defect detection is to accurately discern the category and location information contained in each image. Mainstream power inspection image defect detection algorithms are classified into the following 2 types, one is a traditional defect detection algorithm based on machine learning, and the other is a defect detection algorithm based on deep learning. The former has quite low efficiency of determining and analyzing the state of the power inspection image, and the determination of the parameter range and the selection of the characteristics mainly depend on the field expert experience. The realization of good performance of the target detection algorithm based on deep learning requires that a user uploads data to a data center for centralized training, however, routing inspection image data may contain national road infrastructure and personal privacy information of the user, which causes a serious data island phenomenon and hinders the further development of the deep learning technology in the field of power routing inspection. With the increasingly strict privacy information protection and data security control and the increasingly remarkable phenomenon of data islanding, the traditional method relying on a large number of training data samples for centralized training cannot completely meet the regulations of national power grid on privacy protection and data security. At present, the defect detection of the power inspection image faces a plurality of challenges in the application process, and can be mainly summarized into the following 3 aspects:

(1) the training cost problem. The power inspection image defect detection algorithm usually adopts a centralized training mode, namely all marked data samples are collected, a server with better performance is searched for long-time training, but the mode has higher requirements on the performance and storage of the training server, and especially when large-scale image data are trained, the training time cost is very high.

(2) Data islanding problems. The data isolated island means that the data of the power inspection image are often stored in each power company in a scattered mode, and the data are independently stored and maintained in different power companies to form a physical isolated island. Because the power data usually has the confidentiality requirement, the data among different departments are difficult to share and are mutually independent, and the available scale and the value of the data are greatly limited by the data island problem.

(3) Privacy protection issues. In the unmanned aerial vehicle inspection process, because the image data captured by the camera contains a large amount of private information such as national infrastructure road information and user houses, the leakage of safety and private information can be caused when the target is detected. With the promulgation of Chinese data security laws and the enhancement of personal protection consciousness on private information, the existing target detection method cannot completely meet the requirements of privacy protection and data security sharing. .

Disclosure of Invention

The invention aims to provide a power inspection image defect detection method based on federal learning and self-adaptive difference.

In order to solve the technical problems, the technical scheme adopted by the invention is as follows:

a power inspection image defect detection method based on federal learning and self-adaptive difference comprises the following steps:

step 1: constructing a federal learning scene: the system comprises a central server and more than K clients, wherein K is more than 1; the central server is in bidirectional connection with each client; each client receives the model issued by the central server; the model comprises an input layer, an output layer and more than one intermediate layer; the central server sets an initial model parameter, an iteration time t equal to 0 and a preset detection accuracy; the central server configures the initial model parameters as current global model parameters;

step 2: data preprocessing: the data comprises a normal state image set and more than one fault state image set, and each image in each image set is respectively marked with a category label, an upper left corner abscissa, an upper left corner ordinate, a lower right corner abscissa and a lower right corner ordinate;

and step 3: the central server data issues training data: the central server divides the image data into K groups according to a non-independent and same-distribution mode, and issues a group of data to K clients as local training data of each client; (ii) a

And 4, step 4: the data issuing model parameters of the central server are as follows: the central server transmits the current model parameters and the iteration times to each client as the initial model parameters of each client

K is the number of clients;

and 5: client training model parameters: adding 1 to the training times, training the learning model by using local data to obtain local model parameters by each client, finishing the local training when the local model reaches a preset convergence condition or termination condition, and updating the local model parameters

Is the learning rate of the t-th iteration of client i,

is the model gradient of the t-th iteration of the client i;

and 6: uploading local model parameters: each client adds Gaussian noise meeting the difference privacy to the local model parameter of each client, and uploads the local model parameter with the noise to the central server;

and 7: and (3) global model parameter aggregation: the central server carries out weighted aggregation on the contribution weight of the global model based on each client model parameter to obtain an updated global model parameter, and the method comprises the following specific steps:

step 7-1: calculating the aggregate update contribution weight of each client model parameter to the global model;

step 7-2: after receiving the model parameters uploaded by the client, the server calculates to obtain contribution weights based on 7-1, and weights and aggregates the model parameters to obtain updated global model parameters;

and step 8: calculating the model total loss of the server: the central server issues the aggregated global model parameters to each client, each client calculates the local loss under the aggregated global model parameters and uploads the local loss to the central server, and the central server calculates the total model loss of the server, and the calculation method comprises the following steps:

wherein n is _i For the ith client data sample number,

is the sum of client data samples, F _i (Z _i,t ) Local loss, H, for client i, t round iteration model Z _i,t Weight f (Z) of model parameter contributing to global model aggregation after t-th iteration local difference privacy disturbance for client i _t ) Aggregating the total loss of the model for the t-th iteration of the central server;

judging whether the total model loss of the server reaches the preset detection accuracy or is greater than the training times; if yes, turning to step 9; otherwise, turning to step 4;

and step 9: updating client model parameters: the central server sends the optimal model parameters to each client, and each client updates the local model parameters with the optimal model parameters.

Further, the step 6 comprises the following specific steps:

step 6-1: each client side utilizes a local model to perform feature extraction on a group of input images to obtain a corresponding feature matrix;

step 6-2: grouping the obtained feature matrixes according to 3 channels of red, green and blue, and performing ReLU activation function operation on the feature matrix of each channel to obtain the feature matrixes of red, green and blue;

step 6-3: hadamard product result matrix W for red, green, and blue feature matrices _M Elements greater than 0 add gaussian noise:

W′ _M ＝W _M +N‘

w 'in the formula' _M Adding Gaussian noise into the feature matrix; n' is a noise matrix;

step 6-4: client-side to client-side i t th iteration local model parameters

Adding Gaussian noise N:

wherein, W' _t,L Adding model parameters of Gaussian noise to the local client, wherein the Gaussian noise satisfies that the mean value is 0 and the covariance is ((delta f sigma) ² I) I is the identity matrix, Δ f is the maximum variation range of the model parameters, and σ is

Where the size of e can be determined at the user's discretion, a smaller e indicates a higher level of privacy protection.

Compared with the prior art, the invention at least has the following beneficial effects:

(1) on the premise of privacy protection and data safety sharing, the method ensures that all participants perform collaborative training, and ensures the high precision of the power inspection image defect detection;

(2) according to the method, a differential privacy technology is introduced on the basis of federal learning, the data storage and model training stage of deep learning is transferred to the local, a local user only interacts model parameters with a central server, and meanwhile, a client side adaptively controls noise addition according to data characteristics to avoid reversely deducing pixel-level images and label information, so that privacy protection and data safety are effectively guaranteed;

(3) the knowledge distillation strategy is adopted, the knowledge learned by the global model is transferred to the local model learning, and the convergence of the local model is accelerated, so that the communication times are effectively reduced, and the communication cost is reduced;

(4) according to the method, the position for adding noise is selected in a self-adaptive mode according to a local training data sample, and the negative influence of noise relief on the accuracy of the model is utilized;

(5) according to the method, an evaluation mechanism based on the contribution of the client model parameters is adopted, the contribution capacity of each client model training to the global model parameter aggregation is effectively evaluated, the global model parameters are subjected to weighted aggregation aiming at the model parameter contribution weight, the influence of malicious equipment on the accuracy of a federated learning aggregation model is effectively reduced, and the federated learning is ensured to be more robust.

Drawings

FIG. 1 is a flow chart of the present invention;

FIG. 2 is a schematic diagram of a federated learning architecture in accordance with an embodiment of the present invention;

fig. 3 is a schematic diagram of uploading local model parameters and adding noise in the embodiment of the present invention.

Detailed Description

Example 1

Step 1: constructing a federal learning scene: the system comprises a central server and more than K clients, wherein K is more than 1; the central server is in bidirectional connection with each client; each client receives the model issued by the central server; the model comprises an input layer, an output layer and more than one intermediate layer; the central server sets an initial model parameter, an iteration time t equal to 0 and a preset detection accuracy; the central server configures the initial model parameters as current global model parameters;

step 2: data preprocessing: the data comprises a normal state image set and more than one fault state image set, and each image in each image set is respectively marked with a category label, an upper left corner abscissa, an upper left corner ordinate, a lower right corner abscissa and a lower right corner ordinate;

and step 3: the central server data issues training data: the central server divides the image data into K groups according to a non-independent and same-distribution mode, and issues a group of data to K clients as local training data of each client; (ii) a

And 4, step 4: the data issuing model parameters of the central server are as follows: the central server transmits the current model parameters and the iteration times to each client as the initial model parameters of each client

K is the number of clients;

and 5: client training model parameters: adding 1 to the training times, training the learning model by using local data to obtain local model parameters by each client, finishing the local training when the local model reaches a preset convergence condition or termination condition, and updating the local model parameters

Is the learning rate of the t-th iteration of client i,

is the model gradient of the t-th iteration of the client i;

step 6: uploading local model parameters: each client adds Gaussian noise meeting the difference privacy to the local model parameter, and uploads the local model parameter with noise to the central server;

and 7: and (3) global model parameter aggregation: the central server carries out weighted aggregation on the contribution weight of the global model based on each client model parameter to obtain an updated global model parameter, and the method comprises the following specific steps:

step 7-1: calculating the aggregate update contribution weight of each client model parameter to the global model;

step 7-2: after receiving the model parameters uploaded by the client, the server calculates to obtain contribution weights based on 7-1, and weights and aggregates the model parameters to obtain updated global model parameters;

and 8: calculating the model total loss of the server: the central server issues the aggregated global model parameters to each client, each client calculates the local loss under the aggregated global model parameters and uploads the local loss to the central server, and the central server calculates the total model loss of the server, wherein the calculation method comprises the following steps:

wherein n is _i For the ith client data sample number,

is the sum of client data samples, F _i (Z _i,t ) Local loss, H, for the client i, the t-th round iteration model Z _i,t Weight f (Z) of model parameter contributing to global model aggregation after t-th iteration local difference privacy disturbance for client i _t ) Aggregating the total loss of the model for the t-th iteration of the central server;

judging whether the total model loss of the server reaches the preset detection accuracy or is greater than the training times; if yes, turning to step 9; otherwise, turning to step 4;

and step 9: updating client model parameters: the central server sends the optimal model parameters to each client, and each client updates the local model parameters with the optimal model parameters.

Further, the step 6 includes the following specific steps, as shown in fig. 3:

step 6-1: each client side utilizes a local model to perform feature extraction on a group of input images to obtain a corresponding feature matrix;

step 6-2: grouping the obtained feature matrixes according to 3 channels of red, green and blue, and performing ReLU activation function operation on the feature matrix of each channel to obtain the feature matrixes of red, green and blue;

step 6-3: hadamard product result matrix W for red, green, and blue feature matrices _M Elements greater than 0 add gaussian noise:

W′ _M ＝W _M +N‘

w 'in the formula' _M Adding Gaussian noise into the feature matrix; n' is a noise matrix;

step 6-4: client-side to client-side ith iteration local model parameters

Adding Gaussian noise N:

wherein, W' _t,L Adding model parameters of Gaussian noise to the local client, wherein the Gaussian noise satisfies that the mean value is 0 and the covariance is ((delta f sigma) ² I) I is the identity matrix, Δ f is the maximum variation range of the model parameters, and σ is

The size of e can be determined by the user, and a smaller e represents a higher privacy protection level, and the value in this embodiment is 1.

Further, the step 7 comprises the following specific steps:

7-1, calculating contribution weights of all client model parameters to global model aggregation updating, wherein the specific calculation method is as formulas (4) - (9).

Wherein (4) is an element [ a ] _ji(t) ]Representing the change situation of j-th dimension model parameters in the ith client in the tth iteration, (j is more than or equal to 1 and is less than or equal to m, i is more than or equal to 1 and is less than or equal to k), wherein m is the dimension of the model parameters, and k is the number of clients participating in the federal training in the tth iteration;

(5) in the formula, the element [ b _ji(t) ]And representing the mean value of the change of j-th dimension model parameters in k groups of clients in the t-th iteration, (j is more than or equal to 1 and less than or equal to m, i is more than or equal to 1 and less than or equal to k), wherein m is the dimension of the model parameters, and k is the number of clients participating in the federal training in the t-th iteration.

D _m×k(t) ＝[d _ji(t) ]＝A _m×k(t) -B _m×k(t) (6)

The expressions (6) to (7) are only used for deriving the calculation to obtain the expression (8) e _iz(t) Without other special meanings.

Assuming that initial contribution weights of models uploaded by local training of each client to global model aggregation are the same, the initial contribution weights are all

k is the number of clients participating in the federal training in the t round.

H in formula (8) _i,t Representing the contribution weight of the model uploaded by the client i in the t-th iteration to the global model aggregation, SUM (e) _iz(t) <0) And representing whether uploaded model parameters of the client i and the client z in the t-th iteration are mutually restricted to the global model aggregation contribution. Wherein k is the number of clients participating in the federated training in the t-th iteration process, (1 is not less than i, z is not less than k and i is not equal to z),

The calculation formula (2) is shown as (9).

After receiving the model parameters uploaded by the client, the server obtains a contribution weight based on calculation, weights and aggregates the model parameters to obtain new global model parameters, and updates the global model parameters as shown in formula (10):

wherein, W' _t,G Is the model parameter of the central server after the t-th iteration update, k is the number of clients participating in the federal training, W' _i,t,L The model parameter H after the local privacy disturbance of the local client i after the t-th iteration is obtained _i,t And (4) carrying out local difference privacy disturbance on the model parameter of the t-th iteration client i to obtain a contribution weight value of the global model aggregation.

The image sample labeling is used for carrying out a series of operations on the collected image, then a txt/xml file with the same name as the image is generated, and target information is recorded in the txt/xml file and comprises the following steps: category label, upper left abscissa, upper left ordinate, lower right abscissa, and lower right ordinate. The high-resolution image data set shot by the unmanned aerial vehicle in the transmission line inspection process comprises 4 types of data sets, and 1742 sample labels. Wherein bolt normal sample label ls-zc is 1286, bolt nut missing sample label ls-qlm is 59, bolt pin missing sample label ls-qxd is 174, and bolt washer missing sample label ls-qdp is 223. The data set division is to distribute the image data to the following K clients in a non-independent and equally-distributed mode.

The present embodiment employs a pre-trained bolt defect detection model as the initial global model of the central server. And transmitting the initial global model to all clients participating in training. The initial global model is obtained by training images marked on the central server until the number of model iterations or the accuracy of the model.

And the client side trains by using the respective local power inspection image data, finishes local training when the model converges or reaches a set termination condition, and updates the local model. The knowledge distillation strategy is adopted in the local training process of the model, so that the convergence speed of the local model is increased, the communication times are reduced, and the communication cost is effectively reduced. The training process of the knowledge distillation strategy is divided into the following 2 steps: firstly, selecting an intermediate layer of a central server model as a basis, and only performing supervised learning on the intermediate layer of a local model to promote the local model to rapidly learn the intermediate feature expression capability of the central server model. And secondly, training all model layers of the local model.

The client adds noise to the trained model parameters through a self-adaptive local differential privacy strategy, and uploads the model parameters with the noise to the server.

Claims (2)

1. A power inspection image defect detection method based on federal learning and self-adaptive difference is characterized by comprising the following steps:

step 1: constructing a federal learning scene: the system comprises a central server and more than K clients, wherein K is more than 1; the central server is in bidirectional connection with each client; each client receives the model issued by the central server; the model comprises an input layer, an output layer and more than one intermediate layer; the central server sets initial model parameters, the iteration times t is 0 and the detection accuracy is preset; the central server configures the initial model parameters as current global model parameters;

step 2: data preprocessing: the data comprises a normal state image set and more than one fault state image set, and each image in each image set is respectively marked with a category label, an upper left corner abscissa, an upper left corner ordinate, a lower right corner abscissa and a lower right corner ordinate;

and step 3: the central server data issues training data: the central server divides the image data into K groups according to a non-independent and same-distribution mode, and issues a group of data to K clients as local training data of each client; (ii) a

And 4, step 4: the data issuing model parameters of the central server are as follows: the central server transmits the current model parameters and the iteration times to each client as the initial model parameters of each client

K is the number of clients;

and 5: client training model parameters: adding 1 to the training times, training the learning model by using local data to obtain local model parameters by each client, finishing the local training when the local model reaches a preset convergence condition or termination condition, and updating the local model parameters

Is the learning rate of the t-th iteration of client i,

is the model gradient of the t-th iteration of the client i;

step 6: uploading local model parameters: each client adds Gaussian noise meeting the difference privacy to the local model parameter of each client, and uploads the local model parameter with the noise to the central server;

and 7: and (3) global model parameter aggregation: the central server carries out weighted aggregation on the contribution weight of the global model based on each client model parameter to obtain an updated global model parameter, and the method comprises the following specific steps:

step 7-1: calculating the aggregate update contribution weight of each client model parameter to the global model;

step 7-2: after receiving the model parameters uploaded by the client, the server calculates to obtain contribution weights based on 7-1, and weights and aggregates the model parameters to obtain updated global model parameters;

and 8: calculating the model total loss of the server: the central server issues the aggregated global model parameters to each client, each client calculates the local loss under the aggregated global model parameters and uploads the local loss to the central server, and the central server calculates the total model loss of the server, and the calculation method comprises the following steps:

wherein n is _i For the ith client data sample number,

is the sum of client data samples, F _i (Z _i,t ) Local loss, H, for client i, t round iteration model Z _i,t Weight f (Z) of model parameter contributing to global model aggregation after t-th iteration local difference privacy disturbance for client i _t ) Aggregating the total loss of the model for the t-th iteration of the central server;

judging whether the total model loss of the server reaches the preset detection accuracy or is greater than the training times; if yes, turning to step 9; otherwise, turning to step 4;

and step 9: updating client model parameters: the central server sends the optimal model parameters to each client, and each client updates the local model parameters with the optimal model parameters.

2. The power inspection image defect detection method based on federal learning and adaptive difference as claimed in claim 1, wherein the step 6 comprises the following specific steps:

step 6-1: each client side utilizes a local model to perform feature extraction on a group of input images to obtain a corresponding feature matrix;

step 6-2: grouping the obtained feature matrixes according to 3 channels of red, green and blue, and performing ReLU activation function operation on the feature matrix of each channel to obtain the feature matrixes of red, green and blue;

step 6-3: hadamard product result matrix W for red, green, and blue feature matrices _M Elements greater than 0 add gaussian noise:

W′ _M ＝W _M +N’

w 'in the formula' _M Adding Gaussian noise into the feature matrix; n' is a noise matrix;

step 6-4: client-side to client-side ith iteration local model parameters

Adding Gaussian noise N:

wherein, W' _t,L Adding model parameters of Gaussian noise to the local client, wherein the Gaussian noise satisfies that the mean value is 0 and the covariance is ((delta f sigma) ² i) I is the identity matrix, Δ f is the maximum variation range of the model parameters, and σ is

Where the size of e can be determined at the user's discretion, a smaller e indicates a higher level of privacy protection.

Images