CN117094031B

CN117094031B - Industrial digital twin data privacy protection method and related medium

Info

Publication number: CN117094031B
Application number: CN202311332704.4A
Authority: CN
Inventors: 徐雪松; 陈晓红; 许冠英; 范国滨; 张震; 张新玉
Original assignee: Xiangjiang Laboratory
Current assignee: Xiangjiang Laboratory
Priority date: 2023-10-16
Filing date: 2023-10-16
Publication date: 2024-02-06
Anticipated expiration: 2043-10-16
Also published as: CN117094031A

Abstract

The invention discloses an industrial digital twin data privacy protection method and related media, comprising the following steps: determining the industrial equipment participating in training based on the parameter information of each industrial equipment, and taking the parameter information of each industrial equipment participating in training as a group of training data; grouping training models based on training data to obtain a plurality of groups of basic models; splitting the basic models of each group according to a preset splitting rule to obtain a plurality of sub-models, and distributing the sub-models in the group; adopting a federal split learning mode, selecting industrial equipment participating in federal learning as target equipment, and determining a group and a sub-model corresponding to the target equipment; the training data is adopted on each target device to carry out local training on the sub-model, global aggregation is carried out on the training results, global training results are obtained, the global training results are distributed to all industrial devices, and the performance and effect of industrial digital twin are improved by adopting the method and the device.

Description

Industrial digital twin data privacy protection method and related medium

Technical Field

The present invention relates to the field of data processing, and in particular, to a method, an apparatus, a computer device, and a medium for protecting privacy of industrial digital twin data.

Background

Along with industrial digital twinning, the technology for realizing real-time simulation and optimization of an industrial system by utilizing a physical model, sensor data, cloud computing and big data technology. Industrial digital twinning can improve the efficiency, safety and reliability of industrial systems, and also brings challenges for data privacy protection. Because the task of industrial digital twinning involves multiple industrial devices, each with its own data privacy requirements and interest appeal, there is a need for a method that can achieve data sharing and collaborative analysis while guaranteeing data privacy.

However, due to limited resource capacity of industrial equipment, it is difficult to complete the initialization model task through self resources, so that the utilization of the resources is insufficient, and the processing efficiency of industrial digital twin data is low.

Disclosure of Invention

The embodiment of the invention provides an industrial digital twin data privacy protection method, an industrial digital twin data privacy protection device, computer equipment and a storage medium, so as to improve the efficiency of industrial digital twin data processing and improve the industrial digital twin performance.

In order to solve the above technical problems, an embodiment of the present application provides an industrial digital twin data privacy protection method, including:

Determining the industrial equipment participating in training based on the parameter information of each industrial equipment, and taking the parameter information of each industrial equipment participating in training as a group of training data;

grouping training models based on the training data to obtain a plurality of groups of basic models;

splitting the basic models of each group according to a preset splitting rule to obtain a plurality of sub-models, and distributing the sub-models in the group;

adopting a federal split learning mode, selecting industrial equipment participating in federal learning as target equipment, and determining a group and a sub-model corresponding to the target equipment;

and carrying out local training on the sub-models by adopting the training data on each target device, carrying out global aggregation based on the training results to obtain global training results, and distributing the global training results to all industrial devices.

Optionally, the determining the industrial device participating in training based on the parameter information of each industrial device includes:

collecting parameter information of each industrial device;

determining data similarity, capability similarity and spatial similarity of each industrial device based on the parameter information;

Based on the data similarity, the capability similarity, and the spatial similarity, an industrial device participating in the training is determined.

Optionally, the assigning the sub-model in the group includes:

initializing allocation of each parameter;

the allocation index defining each parameter is:

wherein,represent the firstiThe individual industrial equipment is allocated to the firstjThe number of parameters of a sub-network is proportional to the total number of parameters of the sub-network, < >>And->Respectively represent the firstiCapability index and real-time factor of individual industrial equipment, < >>Represent the firstjIndustrial equipment set contained in group to which sub-network belongs,/->Represent the firstjThe size of the sub-network.

The allocation is updated iteratively such that after each update the loss function of the sub-network decreases until a convergence condition is reached or a maximum number of iterations is reached.

Optionally, the locally training the sub-model using the training data on each of the target devices includes:

when each industrial device is trained locally, calculating a gradient according to a loss function of a local data set by using a random gradient descent algorithm, and updating parameters of a sub-model of the gradient;

when each industrial device is trained locally, a dynamic learning rate adjustment strategy is used, and the magnitude of the learning rate is dynamically adjusted according to the magnitude and distribution of a local data set and the complexity of a sub-model thereof so as to ensure the convergence and stability of training;

When each industrial device is trained locally, an early-stopping strategy is used, whether the optimal training effect is achieved is judged according to the performance of the local data set on the verification set, and if the optimal training effect is not improved obviously, the local training is finished in advance, so that the overfitting is avoided.

Optionally, the performing global aggregation based on the training result, and obtaining the global training result includes:

selecting a server node as a coordinator of global aggregation and taking charge of collecting and distributing parameters of the sub-model;

after the local training is finished, each industrial device encrypts the parameters of the sub-model of the industrial device and sends the encrypted parameters to the central node;

after collecting all sub-model parameters of the industrial equipment, the server node uses a weighted average method to conduct global aggregation.

In order to solve the above technical problem, an embodiment of the present application further provides an industrial digital twin data privacy protection device, including:

the data screening module is used for determining the industrial equipment participating in training based on the parameter information of each industrial equipment, and taking the parameter information of each industrial equipment participating in training as a group of training data;

the model grouping module is used for grouping the training models based on the training data to obtain a plurality of groups of basic models;

The model distribution module is used for splitting the basic models of each group according to a preset splitting rule to obtain a plurality of sub-models, and distributing the sub-models in the group;

the splitting learning module is used for selecting industrial equipment participating in federal learning as target equipment by adopting a federal splitting learning mode, and determining a group and a sub-model corresponding to the target equipment;

and the training module is used for carrying out local training on the sub-models by adopting the training data on each target device, carrying out global aggregation based on the training results to obtain global training results, and distributing the global training results to all industrial devices.

Optionally, the data screening module includes:

the data acquisition unit is used for acquiring parameter information of each industrial device;

a similarity determining unit for determining data similarity, capability similarity and spatial similarity of each industrial equipment based on the parameter information;

and the participation object confirmation unit is used for determining industrial equipment participating in training based on the data similarity, the capability similarity and the spatial similarity.

Optionally, the model allocation module includes:

An initializing unit for initializing allocation of each parameter;

an index determining unit for defining an allocation index of each parameter as:

wherein,represent the firstiThe individual industrial equipment is allocated to the firstjThe number of parameters of a sub-network is proportional to the total number of parameters of the sub-network, < >>And->Respectively represent the firstiCapability index and real-time factor of individual industrial equipment, < >>Represent the firstjIndustrial equipment set contained in group to which sub-network belongs,/->Represent the firstjThe size of the sub-network;

and the iterative training unit is used for iteratively updating the allocation, so that the loss function of the sub-network is reduced after each updating until a convergence condition is reached or the maximum iteration number is reached.

Optionally, the training module includes:

a first updating unit, which is used for calculating gradients according to the loss function of the local data set by using a random gradient descent algorithm when each industrial device is trained locally, and updating the parameters of the sub-model;

the local training unit is used for dynamically adjusting the learning rate according to the size and the distribution of a local data set and the complexity of a sub-model thereof by using a dynamic learning rate adjustment strategy when each industrial device is trained locally so as to ensure the convergence and the stability of training;

And the iterative training unit is used for judging whether the optimal training effect is achieved or not according to the performance of the local data set of each industrial device on the verification set by using an early-stopping strategy when each industrial device is trained locally, and if the optimal training effect is not improved obviously, finishing the local training in advance so as to avoid over fitting.

Optionally, the training module further comprises:

the node selection unit is used for selecting a server node as a coordinator of global aggregation and is responsible for collecting and distributing parameters of the sub-model;

the encryption transmission unit is used for encrypting the parameters of the sub-model of each industrial device after the local training is finished and transmitting the encrypted parameters to the central node;

and the global aggregation unit is used for carrying out global aggregation by using a weighted average method after the server node collects all the sub-model parameters of the industrial equipment.

In order to solve the above technical problem, the embodiments of the present application further provide a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the steps of the above industrial digital twin data privacy protection method are implemented when the processor executes the computer program.

To solve the above technical problem, embodiments of the present application further provide a computer readable storage medium storing a computer program, where the computer program implements the steps of the industrial digital twin data privacy protection method described above when executed by a processor.

According to the industrial digital twin data privacy protection method, the device, the computer equipment and the storage medium provided by the embodiment of the invention, the industrial equipment participating in training is determined based on the parameter information of each industrial equipment, and the parameter information of each industrial equipment participating in training is used as a group of training data; grouping training models based on training data to obtain a plurality of groups of basic models; splitting the basic models of each group according to a preset splitting rule to obtain a plurality of sub-models, and distributing the sub-models in the group; adopting a federal split learning mode, selecting industrial equipment participating in federal learning as target equipment, and determining a group and a sub-model corresponding to the target equipment; and carrying out local training on the sub-models by adopting training data on each target device, carrying out global aggregation on the basis of the training results to obtain global training results, and distributing the global training results to all industrial devices. By splitting different initialization models into different sub-models and assigning the sub-models to different industrial devices, each industrial device has sufficient capacity to participate in training. The split idea is utilized, the advantage of deployment of federal learning in a large-scale scene is combined, the multiparty distributed cooperative training is realized while the data privacy is protected, the performance and the effect of industrial digital twin are improved, and the digitizing capability of enterprises is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;

FIG. 2 is a flow chart of one embodiment of an industrial digital twin data privacy protection method of the present application;

FIG. 3 is a schematic structural view of one embodiment of an industrial digital twin data privacy preserving device according to the present application;

FIG. 4 is a schematic structural diagram of one embodiment of a computer device according to the present application.

Detailed Description

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description and claims of the present application and in the description of the figures above are intended to cover non-exclusive inclusions. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Referring to fig. 1, as shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like.

The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.

It should be noted that, the method for protecting the privacy of the industrial digital twin data provided by the embodiment of the application is executed by a server, and accordingly, the device for protecting the privacy of the industrial digital twin data is arranged in the server.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. Any number of terminal devices, networks and servers may be provided according to implementation requirements, and the terminal devices 101, 102 and 103 in the embodiments of the present application may specifically correspond to application systems in actual production.

Referring to fig. 2, fig. 2 shows an industrial digital twin data privacy protection method according to an embodiment of the present invention, and the method is applied to the server in fig. 1 for illustration, and is described in detail as follows:

s201: and determining the industrial equipment participating in the training based on the parameter information of each industrial equipment, and taking the parameter information of each industrial equipment participating in the training as a group of training data.

In one embodiment, determining the industrial equipment to participate in the training based on the parameter information of each industrial equipment includes:

collecting parameter information of each industrial device;

based on the data similarity, the capacity similarity, and the spatial similarity, an industrial device participating in the training is determined.

Specifically, determining the data similarity, the capacity similarity, and the spatial similarity of the respective industrial devices based on the parameter information includes:

sub-step 1 data similarity of a metrology device

(1) And (3) data acquisition: respectively acquiring time sequence data of n industrial devices, and marking as:，。

(2) Distance matrix construction, namely calculating the distance of data of any two industrial devices p and q at each time point Form->A distance matrix of a size.

(3) Optimal path computation at distance matrixAnd calculating a best matching path by using dynamic programming.

(4) Path distance accumulation, namely accumulating the distances on the optimal path to obtain a dynamic time scaling (DTW) distance between industrial equipment p and q:。

(5) Similarity matrix construction, namely calculating allConstructing a +.>Similarity matrix of (2)>。

(6) Setting a similarity threshold:

1) Means based on similarity between industrial devicesAnd standard deviation->Calculating candidate threshold +.>；

a. Calculating a mean of similarities between industrial devices：

；

b. Calculating a mean of similarities between industrial devices：

；

Summing the similarity of the upper triangle or the lower triangle to obtain an average valueThen calculate the average of the sum of squares of all similarity means as variance, and then calculate the square root of variance to get standard deviation +.>。

d. Calculating candidate threshold values。

2) Calculating elbow similarity threshold according to characteristic value curve of similarity matrix of industrial equipment：

a. Pair matrixPerforming eigenvalue decomposition to obtain eigenvectors and eigenvalues;

b. the obtained characteristic values are sequenced from the big to the small and recorded as；

c. Drawing a characteristic value curve in a coordinate system, wherein the horizontal axis is the index of the characteristic value The vertical axis is characteristic value->；

d. Finding an elbow in the graph, namely, turning the graph from steep to gentle turning points;

e. the abscissa of the turning point is the index of the corresponding characteristic value；

f. Then characteristic valueI.e. is a defined similarity threshold +.>。

3) Empirically calculating the range of similarity thresholds：

a. Is provided withA similar threshold value provided by the individual history cases or expert:>；

b. range representation of empirical threshold

The method comprises the following steps:;/>. Wherein,is a tolerance parameter that represents the tolerance of a threshold value for one of the infinit and the infinit.

c. The empirical range of similarity thresholds is:。

4) Calculating the median of three candidate thresholds。

5) Will medianLimited to empirical ranges as the final similarity threshold:

。

(6) When (when)Industrial equipment is considered to have data similarity.

(7) Statistics of the number of similar industrial devices scanning matrixStatistics of->The number of elements of (1) to obtain the number of similar devices +.>。

Sub-step 2 measures the capability similarity of the device in the following manner:

(1) Calculating the functional similarity of industrial equipment:

the functional similarity of industrial equipment refers to the degree of matching between functions, and is expressed by the intersection and union between the functional sets of industrial equipment. The functional similarity F of the industrial devices a and B is calculated as follows:

Wherein,the number of elements representing the intersection of the functional sets of industrial equipment A and B, < >>Representing the number of elements of the union of the functional sets of industrial equipment a and B.

(2) Calculating the parameter similarity of industrial equipment:

the parameter similarity refers to the degree of difference between the parameters of the industrial equipment, expressed as euclidean distance between the parameter sets of the industrial equipment. The parameter similarity P of devices a and B can be calculated by the following equation:

。

(3) Calculating the performance similarity of industrial equipment:

the performance similarity refers to the degree of comparison between the performance of industrial equipment, and is expressed by the correlation coefficient between the performance indexes of the industrial equipment. The performance similarity Q of industrial devices a and B can be calculated by the following formula:

wherein,indicates the number of performance indexes>And->Indicating industrial installation A and B in +.>The values of the performance indexes +.>And->The average of all performance indicators for industrial equipment a and B is shown, respectively.

(4) Calculating maintenance similarity of industrial equipment:

maintenance similarity refers to the degree of agreement between maintenance of industrial equipment, expressed as a ratio between maintenance costs of industrial equipment. The maintenance similarity M of the industrial devices A and B can be calculated by the following formula:

wherein,and->Representing the maintenance costs of the industrial equipment a and B, respectively, over a certain period of time.

(5) Calculating the process similarity of industrial equipment:

process similarity refers to the degree of compatibility between processes of an industrial facility, represented by the largest conjugate subgraph of the industrial facility process flow diagram. The process similarity T of industrial equipment a and B can be calculated by the following formula:

wherein,for the number of sub-nodes, < >>And->A flow chart node.

(6) Computing capability similarity of industrial devices:

。

(7) Number of industrial devices with similar statistical capabilities:

setting a threshold value of capability similarity, such as 0.8, and then traversing all combinations of industrial devices to calculate their capability similarity, if greater than or equal to the threshold valueThey are considered similar in capacity, otherwise they are considered dissimilar in capacity. Finally, counting the number of all industrial equipment with similar capability。

Substep 3 measures the spatial similarity of the device (kruzkarl index):

(1) Determining a space subarea: the industrial plant A comprisesPersonal space subregion->Industrial plant B comprises->Personal space subregion->. Each sub-region may be determined by its vertex coordinates.

Such as，/>。

(2) Calculating the similarity of the area proportion of the space subareas:

wherein,representing the calculation of the sub-area.

(3) Calculating polygon intersection area:

Wherein,intersection polygonal area representing the area to be calculated, +.>Representing the number of vertices of this intersection polygon, +.>, />Representing the coordinates of each vertex in the intersection polygon.

(4) Calculating the spatial similarity of industrial equipment:

。

(5) Counting the number of spatially similar industrial devices:

a threshold of spatial similarity is set, such as 0.8, and then a combination of all industrial devices is traversed to calculate their spatial similarity, if greater than or equal to the threshold, they are considered spatially similar, otherwise they are considered spatially dissimilar. Finally, counting the number of all spatially similar industrial plants。

Sub-step 4 determines the total number of devices in the whole:

(1) Calculating an arithmetic mean value:

；

(2) Calculating standard deviation:

；

(3) Deletion deviation exceeds a standard deviationOther similar number of devices;

(4) Taking the median m for the number of the rest similar devices;

(5) Determining the medianThe total number of industrial devices involved in training.

S202: based on the training data, the training models are grouped to obtain groups of base models.

Specifically, in this embodiment, different grouping strategies are formulated for different models according to training data, which is specifically as follows:

(1) Determination of a reference model:

wherein,is a set of all optional models, +.>Is all possible data sets and task sets, < +.>Is an evaluation model->In data set and task->Function of Performance->Is a reference model.

(2) Initializing model classification includes:

high performance models over baseline models:

low performance model below baseline model:

wherein,is a set of all high performance models,>is a set of all low performance models.

(3) Model complexity assessment:

1) Number of parameters of the model (expressed as the sum of the number of elements of all tensors in the model):

wherein,is the number of tensors in the model, +.>Is the dimension of tensor, +.>Is->The tensor is at->Size in the individual dimensions.

2) The computational effort of the model (expressed as the sum of the number of all multiplications and additions in the model):

wherein,is the number of multiplication and addition operations in the model, +.>And->Let alone->The number of operands involved in the individual multiply and add operations.

3) Training time of model (represented by the average time the model spends on each training batch (batch) multiplied by the total number of training batches):

wherein,is the average time spent on each training batch, +. >Is the total number of training batches.

4) Model complexity computation

Wherein,representing the number of model parameters, reflecting the structural complexity of the model, in general, the more parameters, the more complex the model, and the easier the model is to be overfitted; />Representing the calculation amount of the model, reflecting the calculation complexity of the model, and generally, the larger the calculation amount is, the more time-consuming the model is, and the more difficult the model is to be deployed; />The model training time is represented and reflects the convergence rate of the model, and in general, the longer the training time is, the more difficult the model is to optimize and the more unstable the model is.

, />,/>The contributions of the number of model parameters, the calculated amount and the training time to the information entropy are respectively represented, the influence degree of each complexity index on the model performance is reflected, and in general, the larger the information gain is, the more important the index is. />, />, />The specific calculation mode is as follows:

a. defining an information entropy function:

wherein,, />，/>respectively indicate->The proportion of the total complexity. The larger the information entropy, the more uniform the complexity distribution, and the more balanced the model. The smaller the information entropy, the more non-uniform the complexity distribution, and the more oblique the model.

b. Calculating the contribution of each complexity index to the information entropy using the information gain:

。

(4) Model complexity grouping:

1) The high performance model and the low performance model are divided into subgroups based on the complexity and performance of the model. The models within each subgroup have similar complexity and performance.

High performance model grouping:

low performance model grouping:

wherein,is the first>Subgroup (s)/(s)>Is the +.>Subgroup (s)/(s)>, />, />, , />,/> , />,/>Is the threshold value of the packet, ">And->Is the number of packets.

2) For high performance models, the present embodiment is based onIs divided into +.>Equidistant or equidistant intervals, i.e.

。

Then, calculate the average performance of the model over the dataset for each intervalAnd according to->Is divided into +.>Equidistant or equidistant intervals, i.e.

。

3) For low performance models, the present embodiment uses a similar approach as described above, except thatAnd->Replaced by->, />, />, />Finally, the +.A. of the high performance model and the low performance model are obtained>Suo and->Subgroups, and complexity and performance range of each subgroup.

S203: splitting the basic models of each group according to a preset splitting rule to obtain a plurality of sub-models, and distributing the sub-models in the group.

Specifically, for different types of model groups, different splits are performed:

(1) Splitting rules within the high performance model set:

a. for each modelCalculate its position in the dataset +.>Prediction error->。

b. For each modelAccording to its prediction error->) And complexity->Determining the number of split target submodels +.>。/>

c. For each modelSplitting it into +.>A sub-model such that each subThe model has lower prediction error and lower complexity on its corresponding subset of data.

d. For each sub-model, evaluate it in the datasetPerformance on->And reassigns it to the appropriate high performance model group.

The mathematical form is as follows:

wherein,for measuring model->Utility functions of (2); />Is a regularization parameter used for controlling the weight between the calculation cost and the accuracy; />Is a model->Sub-model set obtained after splitting, +.>Is the first part obtained by a clustering algorithm>The submodel satisfies the following conditions:

；/> 。

in summary, the present embodiment uses a sub-model setTo approximate the original model->. Finally, usingRepresentation of submodel->In dataset +.>The lower the accuracy, that is, the lower the error rate, the higher the accuracy.

(2) Splitting rules within the low performance model set:

a. for each modelCalculate its position in the dataset +. >Prediction error->。

b. For each modelAccording to its prediction error->And complexity->It is determined whether it needs to be merged or deleted.

c. For the model to be combined, combining the model with other similar models by using a combining algorithm to form a new model, so thatNew model in datasetWith higher prediction accuracy and lower complexity.

d. For models that need to be deleted, it is removed directly from the low performance model grouping.

e. For each new or retained model, evaluate it in the datasetPerformance on->And reassigns it to the appropriate low performance model group.

The mathematical form is as follows:

if it isOr->Then->The combination or deletion is needed;

if it isAnd->Then->No merging or deletion is required;

（）

if it isIf merge is needed,)>The method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>Is in combination with->Similar other models requiring merging, +.>Is a merging algorithm.

In this embodiment, a local optimal submodel allocation mechanism is also designed, including the following sub-steps:

substep 1 capability assessment based on local device history information:

to evaluate the computing power and communication power of each industrial device (note: the total number of participating training industrial devices and individuals obtained here and following the industrial devices as a result of step 1), the present embodiment uses its historical information including average computing speed, average communication rate and average communication delay. The capability index defining each industrial device is:

Wherein,is an industrial plant->Average calculation speed of>Is the average calculation speed of all devices, +.>Is an industrial plant->Average communication rate,/, of (a)>Is the average communication rate of all industrial devices, +.>Is a device->Average communication delay,/, of (a)>Is the average communication delay of all industrial devices, +.>, />, />Is a weight coefficient, satisfy->. The higher the capacity index, the more comprehensive the industrial plant.

Sub-step 2 capability assessment based on local device real-time information:

since the state of the local device may change over time, such as power, CPU occupancy, memory occupancy, etc., we need to dynamically adjust the capabilities of each device based on real-time information. We define the real-time factor for each device to be:

wherein,is an industrial plant->Residual quantity of (2),/>Is the average remaining capacity of all industrial equipments, +.>Is an industrial plant->CPU occupancy of>Is the average CPU idle rate of all industrial devices, < >>Is an industrial plant->Memory occupancy of->Is the average memory idle rate of all industrial devices, +.>, />, />Is a weight coefficient, satisfy->. The higher the real-time factor, the better the current state of the device.

Substep 3, fusing inter-group submodel allocation for history and real-time information capability assessment:

In order to balance the load of each industrial plant and make full use of its limited resources, all the sub-models need to be assigned to different industrial plants so that the time required for each industrial plant to train and transmit the sub-models is as equal as possible.

1) Dividing industrial equipment into groups ofExpress the number of groups, use->Indicate->The individual groups comprise a collection of industrial devices. Use->Indicate->The number of submodels assigned to the individual groups is +.>Indicate->The>And (5) sub-models. Use->Indicate->Personal local device training->The time required for the submodel is +.>Indicate->Uploading or downloading of personal local device->The time required for the sub-model.

2) Optimization problem definition:

/>

wherein,is the total number of submodels, +.>Is->The number of samples contained by the sub-model, +.>Is->The weight size of the sub-model.

3) Solving an optimization problem:

a. for each groupCalculate each submodel +.>Sum of training and transmission time of (i) i.eAnd all submodels are according to +.>Is arranged in a descending order of (a). Use->Indicate->Total time of the individual group, i.e.)>And initialize +.>。

b. For each groupInitializing->And sequentially selecting one sub-model from the ordered list of sub-models to be assigned to the group until the total time of the group reaches or exceeds +. >Until that point. The sub-model to which the group is assigned is deleted from the list and updated +.>And->Is a value of (2).

c. For each groupCheck whether there isAnother group->Such that any two sub-models in two groups are swappedAnd->The total time of both groups can be reduced. I.e. whether or not there is +.>The method meets the following conditions:

if such an exchange exists, the execution is performed and the list of total time and submodels for both groups is updated. This step is repeated until no more exchanges can be performed.

d. Outputting each groupNumber of assigned submodels->And submodel list->And each local device +.>Total time required for training and transmitting sub-models +.>。

Within each group, parameters of the sub-network are assigned to the industrial devices within the group so that each industrial device can participate in the training of the sub-network. The present embodiment uses an intra-group model allocation mechanism based on gradient descent (gradient descent) to allocate each parameter to one device. Specifically, the allocation of each parameter is initialized first, and then updated iteratively, such that each update is followed by an allocation scheme that enables the loss function (loss function) of the subnetwork to be reduced.

In one embodiment, assigning sub-models within a group includes:

Initializing allocation of each parameter;

the allocation index defining each parameter is:

/>

The higher the allocation index, the greater the number of parameters that the industrial equipment allocates to the sub-network.

And updating the distribution index of each parameter by using a gradient descent method, so that the loss function of the sub-network is descended most after each update.

This process is repeated until a convergence condition is reached or a maximum number of iterations is reached.

S204: and selecting industrial equipment participating in federal learning as target equipment by adopting a federal split learning mode, and determining a group and a sub-model corresponding to the target equipment.

Since the split learning training mechanism has a relay problem and cannot be deployed in a large-scale distributed scenario, the present embodiment proposes the following sub-steps to solve this problem.

Split learning is a distributed machine learning method, which can protect data privacy and reduce communication overhead. In split learning, each sub-model waits for the training of the previous sub-model to be completed before training itself, so that the waiting time and the waiting energy consumption of the relay exist.

In split learning, each sub-model needs to receive input from the previous sub-model and send the input to the next sub-model. Therefore, each industrial device needs to perform data transmission and data processing. The purpose of this is to ensure continuity and consistency throughout the network, as well as to reduce the amount of data that each device needs to store and calculate. Data transmission and data processing are two important links in split learning, and can affect the training efficiency and energy consumption of a network. Data transmission includes receiving data from a previous industrial device and transmitting data to a next device, and data processing includes forward and reverse propagating the received data. By usingRepresenting slave industrial plant->To industrial plant->The time required for transmitting one bit of data is +.>Representing industrial plant->The time required for processing one bit of data is +.>Representing sub-models- >To sub-model->The amount of data (bits) transmitted, use +.>Representing sub-models->To sub-model->The amount of data (bits) transmitted, use +.>Representing industrial plant->The energy (joule) required to process one bit of data, using +.>Representing slave device->To the device->The energy (joules) required to transmit one bit of data. Then, the total time delay and total energy consumption generated by each industrial device in split learning can be calculated as follows:

1) For the first industrial equipmentIt only needs to be directed to the second industrial installation +.>And sending data and processing own input data. Thus, the first and second substrates are bonded together,

the total time delay is:

the total energy consumption is:

。/>

2) For the last industrial plantIt only needs to be from +.>Personal industrial installation->And receiving data and processing own output data. Thus, the first and second substrates are bonded together,

the total time delay is:

the total energy consumption is:

。

3) For intermediate industrial plantsIt needs to be from->Personal industrial installation->Receive data to->Personal industrial installation->And sending data and processing own intermediate data. Thus, the first and second substrates are bonded together,

the total time delay is:

the total energy consumption is:

。

4) Calculating the overall time delay:

。

5) Calculating the overall energy consumption:

。

6) Optimization problem definition:

the industrial equipment problem of selecting to participate in split learning is converted into a multi-objective optimization problem, namely, relay time delay and relay energy consumption are minimized at the same time under the condition that a certain constraint condition is met.

Wherein,indicating the number of all available devices.

7) Solving an optimization problem:

to solve this optimization problem, the present embodiment provides a number of multi-objective optimization algorithms, such as non-dominant ordered genetic algorithm (NSGA-II), multi-objective particle swarm optimization algorithm (MOPSO), or multi-objective simulated annealing algorithm (MOSA). These algorithms can balance the two objective functions to some extent and find a set of non-inferior solutions, i.e., improve the solution on one objective function without degrading the solution on the other objective function. Specifically, according to specific application scenes and requirements, a proper solution can be selected from the non-inferior solution set to be used as a final industrial equipment set participating in split learning.

For example, a non-dominant order genetic algorithm (NSGA-II) is adopted for solving, and the algorithm is an evolutionary algorithm based on multi-objective optimization, so that the pareto optimal solution set can be effectively searched. The main steps of NSGA-II are as follows:

1) Initializing a population: randomly generating a certain number of initial solutions, each corresponding to an industrial equipment selection schemeI.e. a length of +.>Binary vector of (2), wherein->The number of elements 1 represents +.>The individual industrial devices participate in split learning, with a 0 indicating no participation. And calculating the satisfaction of the objective function value and the constraint condition of each solution.

2) Non-dominant ordering: solutions in a population are layered in a non-dominant relationship, i.e., if one solution is not inferior to another solution on all objective functions, it is said that it does not dominate another solution. Solutions in the same layer are not mutually exclusive, and a smaller number of layers indicates a closer approach to the pareto front. Each solution is assigned a rank value that indicates the layer number in which it resides.

3) Calculating the degree of congestion: to maintain diversity in the population, it is necessary to calculate the crowding level of each solution, i.e., its density in the objective function space. The greater the degree of crowding means that the more sparse the solution around the solution, the more likely it is to be preserved. The method for calculating the crowding degree is to arrange solutions in the same layer according to ascending order for each objective function, then calculate the sum of the distances between adjacent solutions at two sides of each solution, and the crowding degree of the solutions on the boundary is set to infinity.

4) Selecting: adopting a binary tournament selection method, namely randomly selecting two solutions from a population for comparison, and selecting one solution if the grade value of the solution is smaller than that of the other solution; if the rank values of the two solutions are equal, a solution with a higher degree of congestion is selected. This process is repeated until a sufficient number of parents are selected.

5) Crossover and mutation: and performing crossover and mutation operations on the parent to generate offspring. The crossover operation refers to randomly selecting two parents, and exchanging part of genes (binary digits) of the parents according to a certain probability to generate two offspring. Mutation refers to the creation of new offspring by changing some of their genes (binary bits) with a certain probability for each offspring.

6) And (3) environment selection: the parent and offspring are combined into a new population, non-dominant ranking and congestion degree calculation are performed on the new population, and then the next generation is selected according to the following rules: firstly, selecting all solutions in a layer with the minimum grade value; if the layer has exceeded the population size, then the previous solutions are selected in descending order of congestion level; if the layer does not reach the population size, continuing to select all solutions in the next layer; this process is repeated until a sufficient number of next generations are selected.

7) Termination condition: judging whether a preset evolution algebra or convergence degree is reached, if so, stopping the algorithm and outputting the solution of the optimal layer in the current population as a pareto optimal solution set; if not, returning to the step 4) to continue evolution.

S205: and carrying out local training on the sub-models by adopting training data on each target device, carrying out global aggregation on the basis of the training results to obtain global training results, and distributing the global training results to all industrial devices.

In one embodiment, locally training the sub-model with training data on each target device includes:

The specific detailed process is as follows:

parameter definition: industrial equipmentThe upper submodel is->Industrial plant->Local data set on isWherein->For its number of samples; industrial plant->The loss function is thatWherein->A loss function for a single sample; industrial plant->The learning rate is->The method comprises the steps of carrying out a first treatment on the surface of the Industrial plant->The verification set on is->Wherein->For its number of samples; industrial plant->The manifestation on the verification set of the upper is +.>。

Industrial equipmentThe above local training mechanism is as follows:

A. initialization of

B. For the followingThe following steps are repeated:

a. calculating the gradient:；

b. updating parameters:；

c. and (3) adjusting the learning rate: ；

d. judging early stop: if it isStopping the local training and outputting +.>。

Wherein,for maximum number of local training rounds, < ->For early stop threshold->The function is adjusted for the learning rate.

In a specific embodiment, performing global aggregation based on the training results, where obtaining the global training results includes:

Specifically, after all the selected devices complete local training, the sub-models on each device need to be globally aggregated, that is, parameters of each sub-model are combined to obtain a complete initialization model. In order to ensure the aggregation effect and security, the following global aggregation strategy is adopted in this embodiment:

1) One server node is selected as a coordinator of the global aggregation and is responsible for collecting and distributing parameters of the sub-model.

2) After the local training is finished, each industrial device encrypts the parameters of the sub-model of the industrial device and sends the encrypted parameters to the central node.

3) After collecting all sub-model parameters of the industrial equipment, the server node uses a weighted average method to carry out global aggregation, namely. Wherein (1)>Is the initialized model parameters obtained after global aggregation, < >>Is->Sub-model parameters of individual industrial plants, +.>Is->Weights of individual industrial devices;

4) And the server node decrypts the initialization model parameters obtained after the global aggregation and then sends the decrypted initialization model parameters to all industrial equipment to serve as initial values of the next round of local training.

In this embodiment, based on the parameter information of each industrial device, determining the industrial device participating in the training, and taking the parameter information of each industrial device participating in the training as a set of training data; grouping training models based on training data to obtain a plurality of groups of basic models; splitting the basic models of each group according to a preset splitting rule to obtain a plurality of sub-models, and distributing the sub-models in the group; adopting a federal split learning mode, selecting industrial equipment participating in federal learning as target equipment, and determining a group and a sub-model corresponding to the target equipment; and carrying out local training on the sub-models by adopting training data on each target device, carrying out global aggregation on the basis of the training results to obtain global training results, and distributing the global training results to all industrial devices. By splitting different initialization models into different sub-models and assigning the sub-models to different industrial devices, each industrial device has sufficient capacity to participate in training. The split idea is utilized, the advantage of deployment of federal learning in a large-scale scene is combined, the multiparty distributed cooperative training is realized while the data privacy is protected, the performance and the effect of industrial digital twin are improved, and the digitizing capability of enterprises is improved.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.

Fig. 3 shows a schematic block diagram of an industrial digital twin data privacy protecting apparatus in one-to-one correspondence with the industrial digital twin data privacy protecting method of the above embodiment. As shown in fig. 3, the industrial digital twin data privacy protecting apparatus includes a data screening module 31, a model grouping module 32, a model assigning module 33, a split learning module 34, and a training module 35. The functional modules are described in detail as follows:

a data screening module 31, configured to determine, based on the parameter information of each industrial device, the industrial device involved in training, and use the parameter information of each industrial device involved in training as a set of training data;

a model grouping module 32, configured to group training models based on training data, so as to obtain a plurality of groups of basic models;

the model allocation module 33 is configured to split the basic models of each group according to a preset splitting rule to obtain a plurality of sub-models, and allocate the sub-models in the group;

The splitting learning module 34 is configured to select an industrial device participating in federal learning as a target device by adopting a federal splitting learning manner, and determine a group and a sub-model corresponding to the target device;

the training module 35 is configured to perform local training on the sub-models by using training data on each target device, perform global aggregation based on the training results, obtain global training results, and distribute the global training results to all industrial devices.

Optionally, the data filtering module 31 includes:

Optionally, the model assignment module 33 includes:

an initializing unit for initializing allocation of each parameter;

/>

wherein,represent the firstiThe individual industrial equipment is allocated to the firstjThe number of parameters of a sub-network is proportional to the total number of parameters of the sub-network, < > >And->Respectively represent the firstiCapability index and real-time factor of individual industrial equipment, < >>Represent the firstjIndustrial equipment set contained in group to which sub-network belongs,/->Represent the firstjThe size of the sub-network;

Optionally, the training module 35 includes:

Optionally, the training module further comprises:

For specific limitations on the industrial digital twin data privacy protection device, reference may be made to the above limitations on the industrial digital twin data privacy protection method, and no further description is given here. The various modules in the industrial digital twin data privacy protection device described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 4, fig. 4 is a basic structural block diagram of a computer device according to the present embodiment.

The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It is noted that only a computer device 4 having a component connection memory 41, a processor 42, a network interface 43 is shown in the figures, but it is understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.

The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.

The memory 41 includes at least one type of readable storage medium including flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or D interface display memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 4. Of course, the memory 41 may also comprise both an internal memory unit of the computer device 4 and an external memory device. In this embodiment, the memory 41 is typically used to store an operating system and various application software installed on the computer device 4, such as program code for protecting privacy of industrial digital twin data. Further, the memory 41 may be used to temporarily store various types of data that have been output or are to be output.

The processor 42 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute the program code stored in the memory 41 or process data, such as the program code for industrial digital twin data privacy protection.

The network interface 43 may comprise a wireless network interface or a wired network interface, which network interface 43 is typically used for establishing a communication connection between the computer device 4 and other electronic devices.

The present application also provides another embodiment, namely, a computer readable storage medium storing an interface display program executable by at least one processor to cause the at least one processor to perform the steps of the industrial digital twin data privacy preserving method as described above.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method described in the embodiments of the present application.

It is apparent that the embodiments described above are only some embodiments of the present application, but not all embodiments, the preferred embodiments of the present application are given in the drawings, but not limiting the patent scope of the present application. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a more thorough understanding of the present disclosure. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing, or equivalents may be substituted for elements thereof. All equivalent structures made by the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the protection scope of the application.

Claims

1. An industrial digital twin data privacy protection method, characterized in that the industrial digital twin data privacy protection method comprises the following steps:

carrying out local training on the sub-models by adopting the training data on each target device, carrying out global aggregation based on training results to obtain global training results, and distributing the global training results to all industrial devices;

wherein said assigning the sub-models within a group comprises:

initializing allocation of each parameter;

the allocation index defining each parameter is:

wherein P is _ij Representing the ratio of the number of parameters allocated to the jth sub-network by the ith industrial equipment to the total number of parameters of the sub-network, C _i And R is _i Respectively representing the capability index and the real-time factor of the ith industrial equipment, G _j Representing a set of industrial devices contained in a group to which the jth subnetwork belongs, S _j Indicating the size of the jth subnetwork;

2. The industrial digital twin data privacy protection method of claim 1, wherein the determining industrial devices involved in training based on the parameter information of each industrial device comprises:

collecting parameter information of each industrial device;

3. The method of claim 1, wherein the employing the training data on each of the target devices to locally train a sub-model comprises:

4. The method of claim 1, wherein the performing global aggregation based on the training results to obtain global training results comprises:

5. An industrial digital twin data privacy protection device, characterized in that the industrial digital twin data privacy protection device comprises:

the training module is used for carrying out local training on the sub-models by adopting the training data on each target device, carrying out global aggregation on the basis of the training results to obtain global training results, and distributing the global training results to all industrial devices;

wherein the model assignment module comprises:

an initializing unit for initializing allocation of each parameter;

6. The industrial digital twin data privacy protection device of claim 5, wherein the data screening module comprises:

7. The industrial digital twin data privacy protection device of claim 5, wherein the training module comprises:

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the industrial digital twin data privacy protection method of any of claims 1 to 4 when the computer program is executed.

9. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the industrial digital twin data privacy protection method of any of claims 1 to 4.