CN116739038A

CN116739038A - Data processing method and device, electronic equipment and computer readable storage medium

Info

Publication number: CN116739038A
Application number: CN202310303482.7A
Authority: CN
Inventors: 何向南; 吴剑灿; 王翔; 杨益; 隋永铎
Original assignee: University of Science and Technology of China USTC
Current assignee: University of Science and Technology of China USTC
Priority date: 2023-03-24
Filing date: 2023-03-24
Publication date: 2023-09-12
Anticipated expiration: 2043-03-24
Also published as: CN116739038B

Abstract

The disclosure provides a data processing method and device, electronic equipment and a computer readable storage medium, which can be applied to the fields of machine learning and data mining. The data processing method comprises the following steps: in response to receiving a data processing request from a target user, acquiring data to be processed indicated by the data processing request from a data source; inputting the image data to be processed into a target image neural network, and outputting an image data processing result; the target graph neural network is trained by: determining residual sample diagram data according to the original sample diagram data and the sample diagram data to be forgotten; based on the forgetting type, determining an original gradient vector, a disturbance gradient vector and a sea plug matrix according to the original sample map data, the residual sample map data, the original network parameters and the original loss function; and carrying out parameter adjustment on the original network parameters according to the original gradient vector, the disturbance gradient vector and the sea plug matrix to obtain the target graph neural network.

Description

Data processing method and device, electronic equipment and computer readable storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence, machine learning, and data mining, and more particularly, to a training method, a data processing method and apparatus, an electronic device, a computer readable storage medium, and a computer program product for a graph neural network.

Background

With the development of internet technology, artificial intelligence technology has been widely applied to processing scenes with massive graph structure data, such as recommendation systems and social networks. For example, graph neural networks (Graph Neural Network, GNN) may be used to learn graph structure data in order to extract and discover features and patterns in the graph structure data.

After uploading the data, the user may require that the use of a portion of the private data be revoked from the deployed neural network, i.e., the network parameters of the deployed neural network may be changed according to the private data.

In the process of implementing the disclosed concept, the inventor finds that at least the following problems exist in the related art: because the private data processing method based on the graph neural network in the related technology easily affects the graph structure data and has the condition that the private data is not forgotten thoroughly, the efficiency and the accuracy of the private data processing based on the graph neural network cannot be effectively ensured, and the safety of the private data of a user cannot be ensured.

Disclosure of Invention

In view of this, the present disclosure provides a data processing method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

According to one aspect of the present disclosure, there is provided a data processing method including:

in response to receiving a data processing request from a target user, obtaining to-be-processed data indicated by the data processing request from a data source, wherein the to-be-processed data comprises to-be-processed graph data, and the to-be-processed graph data is associated with the target user;

inputting the map data to be processed into a target map neural network, and outputting a map data processing result;

the target graph neural network is trained by the following modes:

acquiring original network parameters, an original loss function, original sample graph data and sample graph data to be forgotten, wherein the original network parameters correspond to an original graph neural network, and the sample graph data to be forgotten has a forgetting type;

determining residual sample diagram data according to the original sample diagram data and the sample diagram data to be forgotten;

determining an original gradient vector, a disturbance gradient vector and a sea plug matrix according to the original sample map data, the residual sample map data, the original network parameters and the original loss function based on the forgetting type; and

And carrying out parameter adjustment on the original network parameters according to the original gradient vector, the disturbance gradient vector and the sea plug matrix to obtain the target graph neural network.

According to an embodiment of the present disclosure, the original gradient vector includes at least one original gradient sub-vector, and the perturbation gradient vector includes at least one perturbation gradient sub-vector.

According to an embodiment of the disclosure, performing parameter adjustment on the original network parameter according to the original gradient vector, the disturbance gradient vector, and the sea plug matrix to obtain the target graph neural network includes:

determining a first target vector according to the at least one original gradient sub-vector;

determining a second target vector according to the at least one disturbance gradient sub-vector;

determining a target difference vector according to the first target vector and the second target vector;

determining an approximate matrix according to the sea plug matrix and the target difference vector;

determining optimized network parameters according to the approximate matrix; and

and generating the target graph neural network according to the optimized network parameters.

According to an embodiment of the present disclosure, the determining the approximation matrix according to the sea plug matrix and the target difference vector includes:

Constructing a recursive equation according to the sea-plug matrix and the target difference vector, wherein the recursive equation comprises a product between an inverse matrix corresponding to the sea-plug matrix and the target difference vector; and

and carrying out iteration processing on the recursion equation until a preset ending condition is met, so as to obtain the approximate matrix.

According to an embodiment of the present disclosure, in a case where the forgetting type belongs to the node forgetting, the original sample graph data includes a original nodes, the sample graph data to be forgotten includes M nodes to be forgotten, and the remaining sample graph data includes B remaining nodes.

According to an embodiment of the present disclosure, the determining the original gradient vector, the disturbance gradient vector, and the sea plug matrix based on the forgetting type according to the original sample map data, the residual sample map data, the original network parameters, and the original loss function includes:

determining the original gradient vector according to the original network parameters and the A original nodes;

determining the disturbance gradient vector according to the original network parameters and the B residual nodes;

and determining the sea plug matrix according to the original loss function, the original network parameters and the A original nodes.

According to an embodiment of the present disclosure, determining the first target vector according to the at least one original gradient sub-vector includes:

n neighbor nodes corresponding to the M nodes to be forgotten are determined;

determining original gradient subvectors corresponding to the N neighbor nodes according to the original network parameters, the M nodes to be forgotten and the N neighbor nodes corresponding to the M nodes to be forgotten respectively;

and determining the first target vector according to the original gradient sub-vectors corresponding to the M nodes to be forgotten and the N neighbor nodes.

According to an embodiment of the present disclosure, determining the second target vector according to the at least one disturbance gradient sub-vector includes:

according to the network parameters to be adjusted and N neighbor nodes corresponding to the M nodes to be forgotten, determining disturbance gradient sub-vectors corresponding to the N neighbor nodes; and

and determining the second target vector according to disturbance gradient sub-vectors corresponding to the N neighbor nodes respectively.

According to an embodiment of the present disclosure, A, B, M is a positive integer greater than 1 and N is a positive integer.

According to an embodiment of the present disclosure, determining the optimized network parameters according to the approximation matrix includes:

The above-mentioned optimized network parameters are determined according to the following formula:

wherein ,characterizing the optimized network parameters, θ characterizing the original network parameters, H _θ The sea plug matrix is characterized in that,characterizing an inverse matrix to the sea plug matrix;

wherein g represents the original sample graph data, Δg represents the sample graph data to be forgotten, g\Δg represents the residual sample graph data, z _i Characterizing the sample graph data, deltav characterizing the nodes to be forgotten,characterizing neighbor nodes corresponding to nodes to be forgotten, f _g (z _i ) Characterization of the raw graph neural network pair z trained on the raw sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) characterization of z on the remaining sample map data using the raw network parameters described above _i And (b) prediction result, y _i Characterization z _i Is a label of (2);

wherein ,characterizing the original gradient subvector, +.>Characterizing the first target vector,/for the first object>Characterizing the disturbance gradient sub-vector described above,characterizing the second target vector;

the approximate matrix corresponding to the product result.

According to an embodiment of the present disclosure, in a case where the forgetting type belongs to the edge forgetting, the original sample graph data includes C original nodes, the sample graph data to be forgotten includes X edges to be forgotten, and the remaining sample graph data includes D remaining nodes.

determining the original gradient vector according to the original network parameters and the C original nodes;

determining the disturbance gradient vector according to the original network parameters and the D remaining nodes;

and determining the sea plug matrix according to the original loss function, the original network parameters and the C original nodes.

According to an embodiment of the present disclosure, determining the first target vector according to the above-described at least one original gradient sub-vector includes:

determining Y endpoints corresponding to the X edges to be forgotten respectively;

for each of the X edges to be forgotten, determining Z neighbor nodes corresponding to the Y endpoints respectively;

determining original gradient sub-vectors corresponding to the X edges to be forgotten according to the original network parameters, the Y endpoints corresponding to the X edges to be forgotten and the Z neighbor nodes corresponding to the X edges to be forgotten;

And determining the first target vector according to the original gradient sub-vectors corresponding to the X edges to be forgotten.

according to network parameters to be adjusted, the Y endpoints respectively corresponding to the X edges to be forgotten and the Z neighbor nodes respectively corresponding to the X edges to be forgotten, determining disturbance gradient sub-vectors respectively corresponding to the X edges to be forgotten; and

and determining the second target vector according to disturbance gradient sub-vectors corresponding to the X edges to be forgotten.

According to an embodiment of the present disclosure, C, D, X, Y is a positive integer greater than 1 and Z is a positive integer.

wherein g represents the original sample graph data, Δg represents the sample graph data to be forgotten, g\Δg represents the residual sample graph data, z _i Characterizing the data of the sample graph, wherein delta epsilon characterizes the edges to be forgotten,characterizing neighbor nodes corresponding to the endpoints of Δε, f _g (z _i ) Characterization of the raw graph neural network pair z trained on the raw sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) characterization of z on the remaining sample map data using the raw network parameters described above _i And (b) prediction result, y _i Characterization z _i Is a label of (2);

wherein ,characterizing the original gradient subvector, +.>Characterizing the first target vector,/for the first object>Characterizing the disturbance gradient subvector, +.>Characterizing the second target vector;

wherein ,characterizing the above-mentioned target difference vector,characterizing the approximation matrix corresponding to the product result.

According to an embodiment of the present disclosure, in a case where the forgetting type belongs to the feature forgetting, the original sample graph data includes E original nodes, the sample graph data to be forgotten includes P features to be forgotten, and the remaining sample graph data includes F remaining nodes.

Determining the original gradient vector according to the original network parameters and the E original nodes;

determining the disturbance gradient vector according to the original network parameters and the F residual nodes;

and determining the sea plug matrix according to the original loss function, the original network parameters and the E original nodes.

determining nodes to be forgotten, which correspond to the P features to be forgotten respectively;

q neighbor nodes corresponding to the P nodes to be forgotten are determined;

determining original gradient subvectors corresponding to the P nodes to be forgotten according to the original network parameters, the P nodes to be forgotten and the Q neighbor nodes corresponding to the P nodes to be forgotten respectively;

and determining the first target vector according to the original gradient sub-vectors corresponding to the M nodes to be forgotten.

according to network parameters to be adjusted, the P nodes to be forgotten and the Q neighbor nodes corresponding to the P nodes to be forgotten, determining disturbance gradient sub-vectors corresponding to the P features to be forgotten; and

And determining the second target vector according to disturbance gradient sub-vectors corresponding to the M nodes to be forgotten.

According to an embodiment of the present disclosure, E, F, P is a positive integer greater than 1 and Q is a positive integer.

wherein g represents the original sample graph data, Δg represents the sample graph data to be forgotten, g\Δg represents the residual sample graph data, z _i Characterizing the sample map data, Δx characterizing the forgetting feature,characterizing neighbor nodes corresponding to the endpoints of Δε, f _g (z _i ) Characterization of the raw graph neural network pair z trained on the raw sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) characterization of z on the remaining sample map data using the raw network parameters described above _i And (b) prediction result, y _i Characterization z _i Is a label of (2);

wherein ,characterizing the original gradient subvector, +.>Characterizing the first target vector,/for the first object >Characterizing the disturbance gradient sub-vector described above,characterizing the second target vector;

wherein ,characterizing the above target difference vector,>characterizing the approximation matrix corresponding to the product result.

According to another aspect of the present disclosure, there is provided a data processing apparatus comprising:

the first acquisition module is used for responding to a data processing request received from a target user and acquiring to-be-processed data indicated by the data processing request from a data source, wherein the to-be-processed data comprises to-be-processed graph data, and the to-be-processed graph data is associated with the target user;

the processing module is used for inputting the graph data to be processed into the target graph neural network and outputting a graph data processing result;

according to an embodiment of the present disclosure, an apparatus for training the above target graph neural network includes:

the second acquisition module is used for acquiring original network parameters, an original loss function, original sample graph data and to-be-forgotten sample graph data, wherein the original network parameters correspond to an original graph neural network, and the to-be-forgotten sample graph data has a forgetting type;

the first determining module is used for determining residual sample diagram data according to the original sample diagram data and the sample diagram data to be forgotten;

The second determining module is used for determining an original gradient vector, a disturbance gradient vector and a sea plug matrix according to the original sample map data, the residual sample map data, the original network parameters and the original loss function based on the forgetting type; and

and the adjusting module is used for carrying out parameter adjustment on the original network parameters according to the original gradient vector, the disturbance gradient vector and the sea plug matrix to obtain the target graph neural network.

According to another aspect of the present disclosure, there is provided an electronic device including:

one or more processors;

a memory for storing one or more instructions,

wherein the one or more instructions, when executed by the one or more processors, cause the one or more processors to implement a method as described in the present disclosure.

According to another aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to implement a method as described in the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising computer executable instructions which, when executed, are adapted to carry out the method as described in the present disclosure.

According to the embodiment of the disclosure, since the residual sample map data is determined according to the original sample map data and the sample map data to be forgotten, the determined disturbance loss function considers the dependency relationship between the nodes in the original map neural network according to the original network parameters and the residual sample map data based on the forgetting type of the sample map data to be forgotten. On the basis, the target graph neural network is obtained by training by utilizing the training method of the graph neural network, so that the technical problems that the privacy data processing method based on the graph neural network is easy to influence graph structure data and privacy data forgets incompletely in the related art are at least partially overcome, and the accuracy of data processing is improved by utilizing the target graph neural network to process the graph data to be processed associated with the target user, so that the data security is improved.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments thereof with reference to the accompanying drawings in which:

FIG. 1 schematically illustrates a flow chart of a data processing method according to an embodiment of the present disclosure;

FIG. 2 schematically illustrates a flow chart of a method for parameter adjustment of original network parameters to obtain a target graph neural network based on original gradient vectors, perturbation gradient vectors, and sea plug matrices, in accordance with an embodiment of the present disclosure;

FIG. 3A schematically illustrates an example schematic diagram of a process of determining a target difference vector according to an embodiment of the disclosure;

FIG. 3B schematically illustrates an example schematic diagram of a process of determining an approximate matrix according to an embodiment of the disclosure;

FIG. 4 schematically illustrates an example schematic diagram of a process of determining a target graph neural network, according to an embodiment of the disclosure;

FIG. 5A schematically illustrates a schematic of experimental results according to an embodiment of the present disclosure;

FIG. 5B schematically illustrates a schematic diagram of experimental results according to another embodiment of the disclosure;

FIG. 6A schematically illustrates a schematic diagram of experimental results according to another embodiment of the present disclosure;

FIG. 6B schematically illustrates a schematic diagram of experimental results according to another embodiment of the disclosure;

FIG. 7 schematically illustrates a block diagram of a data processing apparatus according to an embodiment of the present disclosure; and

fig. 8 schematically illustrates a block diagram of an electronic device adapted to implement a data processing method according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a formulation similar to at least one of "A, B or C, etc." is used, in general such a formulation should be interpreted in accordance with the ordinary understanding of one skilled in the art (e.g. "a system with at least one of A, B or C" would include but not be limited to systems with a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related personal information of the user all conform to the regulations of related laws and regulations, necessary security measures are taken, and the public order harmony is not violated.

In the technical scheme of the disclosure, the authorization or consent of the user is obtained before the personal information of the user is obtained or acquired.

The data forgetting method of the graph neural network can comprise at least one of the following: a data forgetting method based on retraining, a data forgetting method based on subgraph and a data forgetting method based on influence function.

The retraining-based data forgetting method can refer to retraining the graph neural network directly with the remaining data. However, this method is difficult to be practically applied because of its high computational cost.

The method for forgetting data based on the subgraphs can be used for dividing the original input graph into a plurality of subgraphs with balanced sizes, respectively training each subgraph, and then aggregating each subgraph to obtain the whole output. After the forgetting requirement is obtained, only the subgraphs corresponding to the forgetting data can be changed, the subgraphs are retrained, and then the whole aggregation operation is carried out. However, since the graph splitting operation of this method affects the graph structure data, the model performance is lowered.

The data forgetting method based on the influence function can refer to estimating the change of the model parameters before and after the data forgetting by introducing the influence function, and the influence function can be understood as weighting the data to estimate the influence of the tiny disturbance of the forgetting data on the model parameters. However, since the derivation of the influence function in the method is based on independent assumption among data samples, that is, changing one data sample does not influence the representation of other samples, in a real scene, because the central node of the graph neural network continuously acquires characterization information from the neighbors to update, complex interdependence exists among nodes, the independent assumption is difficult to be established, and further, the situation that data forgets incompletely exists.

In order to at least partially solve the technical problems in the related art, the present disclosure provides a data processing method and apparatus, an electronic device, and a computer readable storage medium, which can be applied to the fields of machine learning and data mining. The data processing method comprises the following steps: in response to receiving a data processing request from a target user, acquiring to-be-processed data indicated by the data processing request from a data source, wherein the to-be-processed data comprises to-be-processed graph data, and the to-be-processed graph data is associated with the target user; and inputting the graph data to be processed into a target graph neural network, and outputting a graph data processing result. The target graph neural network is trained by: acquiring original network parameters, an original loss function, original sample graph data and sample graph data to be forgotten, wherein the original network parameters correspond to an original graph neural network, and the sample graph data to be forgotten has a forgetting type; determining residual sample diagram data according to the original sample diagram data and the sample diagram data to be forgotten; based on the forgetting type, determining an original gradient vector, a disturbance gradient vector and a sea plug matrix according to the original sample map data, the residual sample map data, the original network parameters and the original loss function; and according to the original gradient vector, the disturbance gradient vector and the sea plug matrix, carrying out parameter adjustment on the original network parameters to obtain the target graph neural network.

Fig. 1 schematically shows a flow chart of a data processing method according to an embodiment of the present disclosure.

As shown in fig. 1, the data processing method 100 includes operations S110 to S120.

In response to receiving the data processing request from the target user, the data to be processed indicated by the data processing request is obtained from the data source, wherein the data to be processed includes pending diagram data associated with the target user in operation S110.

In operation S120, the map data to be processed is input into the target map neural network, and the map data processing result is output.

According to embodiments of the present disclosure, the target graph neural network may be trained by: and acquiring original network parameters, an original loss function, original sample graph data and sample graph data to be forgotten. The original network parameters correspond to the original graph neural network, and the sample graph data to be forgotten has a forgetting type. And determining residual sample graph data according to the original sample graph data and the sample graph data to be forgotten. Based on the forgetting type, an original gradient vector, a disturbance gradient vector and a sea plug matrix are determined according to the original sample map data, the residual sample map data, the original network parameters and the original loss function. And carrying out parameter adjustment on the original network parameters according to the original gradient vector, the disturbance gradient vector and the sea plug matrix to obtain the target graph neural network.

According to the embodiment of the disclosure, the training method of the graph neural network can be utilized to process the original network parameters, the original loss function, the original sample graph data and the sample graph data to be forgotten to obtain the target graph neural network. After receiving a data processing request from a target user, the data to be processed indicated by the data processing request may be obtained from a data source. The data source may include at least one of: local databases, cloud databases, and network resources. For example, a data interface may be invoked, with which to obtain the data indicated by the data processing request from the data source to be processed. The data to be processed may include map data to be processed. After the data to be processed is obtained, the target graph neural network can be utilized to process the data of the graph to be processed associated with the target user, and a graph data processing result is obtained.

According to embodiments of the present disclosure, the raw sample data may include raw sample map data. The raw sample map data may refer to sample map data used to derive raw network parameters corresponding to the raw map neural network. The original sample graph data may include sample user graph data associated with each of the at least one sample user and an image tag corresponding to the sample user graph data.

According to an embodiment of the present disclosure, the original sample map data may include at least one of sample image information of the sample user, sample text information of the sample user, and sample audio information of the sample user. In the case where the original sample map data includes sample image information, the target map neural network may be applied to the field of image processing. In the case where the original sample map data includes sample text information, the target map neural network may be applied to the field of text processing. In the case where the original sample map data includes sample audio information, the target map neural network may be applied to the field of speech processing.

According to embodiments of the present disclosure, sample graph data to be forgotten may refer to sample graph data corresponding to a forgetting requirement of a sample user. The remaining sample map data may be determined from the original sample map data and the sample map data to be forgotten. The remaining sample map data may refer to sample map data for deriving optimized network parameters corresponding to the target map neural network.

According to embodiments of the present disclosure, the raw graph neural network may be derived by training a predetermined graph neural network (Graph Neural Network, GNN) with raw sample graph data. The model structure of the neural network of the predetermined graph can be configured according to actual service requirements, which is not limited herein. The predetermined graph neural network may include at least one of: a reservation map convolution network (GraphConvolutional Network, GCN), a reservation map Auto-encoder (GAE), a reservation map generation network (Graph Generative Network, GGN), a reservation map looping network (Graph Recurrent Network, GRN), and a reservation map attention network (Graph Attention Network, GAT). The training mode of the neural network of the predetermined graph can be configured according to actual service requirements, and is not limited herein. For example, the training regimen may include at least one of: unsupervised training, supervised training, and semi-supervised training.

According to the embodiment of the disclosure, sample user data corresponding to each of at least one sample user may be input to a predetermined graph neural network, and a prediction result may be output. And obtaining an original loss function value by using the prediction result and the label corresponding to the sample user data based on the original loss function. And adjusting model parameters of the neural network of the preset graph according to the original loss function value until preset conditions are met. For example, model parameters of the predetermined graph neural network may be adjusted according to a back-propagation algorithm or a random gradient descent algorithm until a predetermined condition is satisfied. The model parameters obtained in the case that the predetermined condition is satisfied are determined as the original network parameters. A predetermined graph neural network obtained in the case that the predetermined condition is satisfied is determined as an original graph neural network.

According to embodiments of the present disclosure, the original network parameters may include at least one of: network node number, initial weight, minimum training rate, dynamic parameters, allowed error, iteration number, and Sigmoid parameters. The original loss function may be set according to the actual service requirement, which is not limited herein. For example, the original loss function may include at least one of: regression loss function, exponential loss function, square error loss function, absolute error loss function, cross entropy loss function, range loss function, and Huber loss function, two-class cross entropy loss function, multi-class loss function, and multi-class cross entropy loss function.

According to embodiments of the present disclosure, the sample graph data to be forgotten may be characterized by a sub graph to be forgotten. The to-be-forgotten subgraph may include to-be-forgotten nodes, to-be-forgotten edges, and to-be-forgotten features. The sample map data to be forgotten may be of a forgetting type. For example, the forgetting type may include at least one of: node forgetting corresponding to the node to be forgotten, edge forgetting corresponding to the edge to be forgotten, and feature forgetting corresponding to the feature to be forgotten. Based on the forgetting type, the dependency relationship of the nodes in the neural network of the original graph can be determined according to the original network parameters and the data of the residual sample graph, and then the disturbance loss function is determined.

According to the embodiment of the disclosure, since the residual sample map data is determined according to the original sample map data and the sample map data to be forgotten, based on the forgetting type of the sample map data to be forgotten, the determined original gradient vector, disturbance gradient vector and sea plug matrix consider the dependency relationship between the nodes in the original map neural network according to the original sample map data, the residual sample map data, the original network parameters and the original loss function. On the basis, the target graph neural network is obtained by training by utilizing the training method of the graph neural network, so that the technical problems that the privacy data processing method based on the graph neural network is easy to influence graph structure data and privacy data forgets incompletely in the related art are at least partially overcome, and the accuracy of data processing is improved by utilizing the target graph neural network to process the graph data to be processed associated with the target user, so that the data security is improved.

The data processing method 100 according to an embodiment of the present disclosure is further described below with reference to fig. 2, 3A, 3B, 4, 5A, 5B, 6A, and 6B.

Fig. 2 schematically illustrates a flowchart of a method for performing parameter adjustment on original network parameters to obtain a target graph neural network according to an embodiment of the present disclosure, based on an original gradient vector, a disturbance gradient vector, and a sea plug matrix.

As shown in fig. 2, performing parameter adjustment on the original network parameters according to the original gradient vector, the disturbance gradient vector and the sea plug matrix, and obtaining the target graph neural network may include operations S210 to S260.

In operation S210, a first target vector is determined from at least one original gradient sub-vector.

In operation S220, a second target vector is determined from the at least one disturbance gradient sub-vector.

In operation S230, a target difference vector is determined from the first target vector and the second target vector.

In operation S240, an approximation matrix is determined according to the sea plug matrix and the target difference vector.

In operation S250, optimized network parameters are determined from the approximation matrix.

In operation S260, a target graph neural network is generated according to the optimized network parameters.

According to an embodiment of the present disclosure, the original gradient vector may include at least one original gradient sub-vector. The disturbance gradient vector may comprise at least one disturbance gradient sub-vector.

According to embodiments of the present disclosure, a sea plug matrix (Hessian matrix) may refer to a square matrix consisting of the second partial derivatives of real valued functions of the argument as a vector. The sea plug matrix corresponding to the original gradient vector may be determined from a Jacobian matrix (Jacobian matrix) of the original gradient vector to the independent variables.

According to embodiments of the present disclosure, an approximation matrix corresponding to a product between an inverse matrix of the sea plug matrix and the perturbation gradient vector may be determined from the sea plug matrix and the perturbation gradient vector. The manner of determining the approximation matrix may be set according to actual service requirements, and is not limited herein. For example, the manner in which the approximation matrix is determined may include at least one of: an approximation matrix is determined based on HVPs (Hessian-vector Products), an approximation matrix is determined based on conjugate gradients (Conjugate gradients), and an approximation matrix is determined based on random estimates (Stochastic estimation).

According to embodiments of the present disclosure, the original gradient vector may be determined from the original network parameters and the original sample map data. The perturbation gradient vector may be determined from the original network parameters and the original sample map data.

According to an embodiment of the present disclosure, after obtaining the at least one original gradient sub-vector, a first target vector may be determined from the at least one original gradient sub-vector. After obtaining the at least one disturbance gradient sub-vector, vector summation may be performed on the at least one disturbance gradient sub-vector to obtain a second target vector.

According to an embodiment of the present disclosure, the candidate optimized network parameters may be represented by the following formula (1), for example.

wherein ,characterizing the original loss function, e characterizing the size of the disturbance, +.> For example, the following formulas (2) to (4) can be used.

According to the embodiment of the disclosure, according to the formulas (2) to (4), when e=1, node dependence of the characterization level introduced by the neighbor aggregation policy of the graph neural network can be eliminated, so as to obtain the optimized network parameters as shown in the following formula (5).

wherein ,H_θ A sea plug matrix characterizing the original loss function with respect to the original network parameters,characterizing a disturbance loss function->Gradient vector with respect to original network parameters +.>And (5) characterizing and optimizing network parameters.

Fig. 3A schematically illustrates an example schematic diagram of a process of determining a target difference vector according to an embodiment of the disclosure.

As shown in fig. 3A, in 300A, the original gradient vector 301 may include at least one original gradient sub-vector. The perturbation gradient vector 302 may include at least one perturbation gradient sub-vector.

The at least one original gradient sub-vector may comprise an original gradient sub-vector 301_1, an original gradient sub-vector 301_2, …, an original gradient sub-vector 301_a, …, an original gradient sub-vector 301_a. A may be an integer greater than or equal to 1, a ε {1,2, …, (A-1), A }.

The first target vector 303 may be determined from the original perturbation gradient sub-vector 301_1, the original perturbation gradient sub-vectors 301_2, …, the original perturbation gradient sub-vectors 301_a, …, the original perturbation gradient sub-vector 301_a.

The at least one perturbation gradient sub-vector may include perturbation gradient sub-vector 302_1, perturbation gradient sub-vectors 302_2, …, perturbation gradient sub-vectors 302_b, …, perturbation gradient sub-vector 302_b. B may be an integer greater than or equal to 1, B ε {1,2, …, (B-1), B }.

The second target vector 304 may be determined from the perturbation gradient sub-vector 302_1, the perturbation gradient sub-vectors 302_2, …, the perturbation gradient sub-vectors 302_b, …, the perturbation gradient sub-vector 302_b.

After the first target vector 303 and the second target vector 304 are obtained, a target difference vector 305 may be determined from the first target vector 303 and the second target vector 304.

According to an embodiment of the present disclosure, operation S240 may include the following operations.

And constructing a recursive equation according to the sea-plug matrix and the target difference vector, wherein the recursive equation comprises the product between an inverse matrix corresponding to the sea-plug matrix and the target difference vector. And carrying out iterative processing on the recursion equation until a preset ending condition is met, so as to obtain an approximate matrix.

According to embodiments of the present disclosure, the matrix form of the taylor expansion may be determined from the original gradient vector and the sea plug matrix. The taylor expansion is determined from a matrix form of the taylor expansion. For example, the method can be based on the first t item of Taylor expansion A recursive equation corresponding to the sea plug matrix is constructed, and the recursive equation can be represented by the following equation (6).

According to embodiments of the present disclosure, H _j-1 The sea plug matrix can be characterized as,can characterize the inverse matrix corresponding to the sea plug matrix, v can characterize the target difference vector,/>The product between the inverse matrix corresponding to the sea plug matrix and the target difference vector may be characterized and λ may characterize the scaling super-parameter.

According to an embodiment of the present disclosure, by performing iterative processing on equation (6), an approximate matrix corresponding to the inverse matrix of the sea plug matrix may be obtained. The calculation complexity of the inverse matrix of the sea plug matrix can be calculated by O (|theta|) ³ ) To O (|θ|) to reduce the computational cost.

Fig. 3B schematically illustrates an example schematic diagram of a process of determining an approximate matrix according to an embodiment of the disclosure.

As shown in fig. 3B, in 300B, a product 308 between an inverse matrix corresponding to the sea plug matrix and the target difference vector may be determined from the sea plug matrix 306 and the target difference vector 307. After obtaining the product 308, a recursive equation 309 may be constructed from the product 308. After the recursive equation 309 is obtained, operation S310 may be performed.

In operation S310, is the iterative processing result satisfying a predetermined end condition? If not, iterative processing of the recursive equation 309 may continue. If so, an approximation matrix 310 may be obtained.

According to an embodiment of the present disclosure, in case the forgetting type belongs to node forgetting, determining the original gradient vector, the disturbance gradient vector and the sea plug matrix from the original sample map data, the remaining sample map data, the original network parameters and the original loss function based on the forgetting type may comprise the following operations.

And determining an original gradient vector according to the original network parameters and the A original nodes. And determining disturbance gradient vectors according to the original network parameters and the B remaining nodes. And determining the sea plug matrix according to the original loss function, the original network parameters and the A original nodes.

According to an embodiment of the present disclosure, the raw sample graph data may include a raw nodes. The sample graph data to be forgotten may include M nodes to be forgotten. The remaining sample graph data may include B remaining nodes. M may be a positive integer greater than 1 and N may be a positive integer.

According to an embodiment of the present disclosure, operation S210 may include the following operations.

N neighbor nodes corresponding to the M nodes to be forgotten are determined. And determining original gradient subvectors corresponding to the N neighbor nodes according to the original network parameters, the M nodes to be forgotten and the N neighbor nodes corresponding to the M nodes to be forgotten. And determining a first target vector according to the original gradient sub-vectors corresponding to the M nodes to be forgotten and the N neighbor nodes.

According to an embodiment of the present disclosure, operation S220 may include the following operations.

And determining disturbance gradient sub-vectors corresponding to the N neighbor nodes according to the network parameters to be adjusted and the N neighbor nodes corresponding to the M nodes to be forgotten. And determining a second target vector according to the disturbance gradient sub-vectors corresponding to the N neighbor nodes respectively. A. B, M can be a positive integer greater than 1 and N can be a positive integer.

In accordance with an embodiment of the present disclosure, in the case where the forgetting type belongs to node forgetting, operation S250 may include the following operations.

Determining optimized network parameters according to the following formula (7):

in accordance with an embodiment of the present disclosure,can characterize the optimized network parameters, theta can characterize the original network parameters, H _θ Sea plug matrix can be characterized, +.>The inverse of the sea plug matrix can be characterized.

According to embodiments of the present disclosure, g may characterize the original sampleThe graph data, Δg can represent the graph data of the sample to be forgotten, g\Δg can represent the graph data of the residual sample, z _i Sample graph data may be characterized, Δv may characterize the nodes to be forgotten,can characterize neighbor nodes corresponding to the nodes to be forgotten, f _g (zi) can characterize the original graph neural network pair z trained on the original sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) can characterize the z over the remaining sample map data using the original network parameters _i And (b) prediction result, y _i Can characterize z _i Is a label of (a).

In accordance with an embodiment of the present disclosure,the original gradient sub-vector can be characterized,the first target vector can be characterized, +.>The perturbation gradient subvector, +.>The second target vector may be characterized.

In accordance with an embodiment of the present disclosure,can characterize the target difference vector, ">An approximation matrix corresponding to the product result may be characterized.

According to an embodiment of the present disclosure, the above-mentioned approximation matrix corresponding to the product result may be calculated by performing an iterative equation.

According to an embodiment of the present disclosure, in the case where the forgetting type belongs to edge forgetting, determining the original gradient vector, the disturbance gradient vector, and the sea plug matrix from the original sample map data, the remaining sample map data, the original network parameters, and the original loss function based on the forgetting type may include the following operations.

And determining an original gradient vector according to the original network parameters and the C original nodes. And determining disturbance gradient vectors according to the original network parameters and the D remaining nodes. And determining the sea plug matrix according to the original loss function, the original network parameters and the C original nodes.

According to an embodiment of the present disclosure, the original sample graph data may include C original nodes, the sample graph data to be forgotten may include X edges to be forgotten, and the remaining sample graph data may include D remaining nodes. X, Y can be a positive integer greater than 1 and Z can be a positive integer.

Y endpoints corresponding to each of the X edges to be forgotten are determined. And determining Z neighbor nodes corresponding to the Y endpoints respectively for each of the X edges to be forgotten. And determining original gradient subvectors corresponding to the X edges to be forgotten according to the original network parameters, Y endpoints corresponding to the X edges to be forgotten and Z neighbor nodes corresponding to the X edges to be forgotten. And determining a first target vector according to the original gradient sub-vectors corresponding to the X edges to be forgotten.

And determining disturbance gradient sub-vectors corresponding to the X edges to be forgotten according to the network parameters to be regulated, Y endpoints corresponding to the X edges to be forgotten and Z neighbor nodes corresponding to the X edges to be forgotten. And determining a second target vector according to the disturbance gradient sub-vectors corresponding to the X edges to be forgotten.

According to embodiments of the present disclosure, C, D, X, Y can be a positive integer greater than 1 and Z can be a positive integer.

In accordance with an embodiment of the present disclosure, in the case where the forgetting type belongs to edge forgetting, operation S250 may include the following operations.

Determining optimized network parameters according to the following formula (8):

According to embodiments of the present disclosure, g may characterize raw sample map data, Δg may characterize sample map data to be forgotten, g\Δg may characterize remaining sample map data, z _i Sample graph data may be characterized, delta epsilon may be characterized for edges to be forgotten,neighbor nodes corresponding to endpoints of Δε, f may be characterized _g (z _i ) Original graph neural network pair z trained on original sample graph data can be characterized _i F is the predicted result of (f) _g (z _i G\Δg) can characterize the z over the remaining sample map data using the original network parameters _i And (b) prediction result, y _i Can characterize z _i Is a label of (a).

In accordance with an embodiment of the present disclosure,the original gradient sub-vector can be characterized,the first target vector can be characterized, +.>The perturbation gradient subvector, +. >Can characterize the second targetVector.

According to an embodiment of the present disclosure, the above-mentioned approximation matrix corresponding to the product result may be calculated by performing an iterative equation. .

According to an embodiment of the present disclosure, in case the forgetting type belongs to the feature forgetting, determining the original gradient vector, the disturbance gradient vector and the sea plug matrix from the original sample map data, the remaining sample map data, the original network parameters and the original loss function based on the forgetting type may comprise the following operations.

And determining an original gradient vector according to the original network parameters and E original nodes. And determining disturbance gradient vectors according to the original network parameters and F residual nodes. And determining the sea plug matrix according to the original loss function, the original network parameters and E original nodes. According to an embodiment of the present disclosure, the raw sample graph data may include E raw nodes. The to-be-forgotten sample map data may include P to-be-forgotten features. The remaining sample graph data may include F remaining nodes.

And determining to-be-forgotten nodes corresponding to the P to-be-forgotten features respectively. Q neighbor nodes corresponding to the P nodes to be forgotten are determined. And determining original gradient subvectors corresponding to the P nodes to be forgotten according to the original network parameters, the P nodes to be forgotten and Q neighbor nodes corresponding to the P nodes to be forgotten respectively. And determining a first target vector according to the original gradient sub-vectors corresponding to the M nodes to be forgotten.

And determining disturbance gradient sub-vectors corresponding to the P nodes to be forgotten according to the network parameters to be regulated, the P nodes to be forgotten and Q neighbor nodes corresponding to the P nodes to be forgotten respectively. And determining a second target vector according to the disturbance gradient sub-vectors corresponding to the M nodes to be forgotten.

According to embodiments of the present disclosure, E, F, P may be a positive integer greater than 1 and Q may be a positive integer.

In accordance with an embodiment of the present disclosure, in the case where the forgetting type belongs to the feature forgetting, operation S250 may include the following operations.

Determining optimized network parameters according to the following formula (9):

in accordance with an embodiment of the present disclosure,characterizing the optimized network parameters, θ characterizing the original network parameters, H _θ Characterization of the sea plug matrix,>the inverse of the sea plug matrix is characterized.

According to an embodiment of the present disclosure, g characterizes the original sample map data, Δg characterizes the sample map data to be forgotten, g\Δg characterizes the remaining sample map data, z _i Characterizing the sample graph data, Δx characterizing the feature to be forgotten,characterizing neighbor nodes corresponding to endpoints of Δε, f _g (z _i ) Characterization of raw graph neural network pair z trained on raw sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) characterization versus z on the remaining sample map data using the original network parameters _i And (b) prediction result, y _i Characterization z _i Is a label of (a).

In accordance with an embodiment of the present disclosure,the original gradient sub-vector is characterized,characterizing a first target vector, ">The disturbance gradient sub-vector is characterized,the second target vector is characterized.

In accordance with an embodiment of the present disclosure,characterizing a target difference vector, ">An approximation matrix corresponding to the product result is characterized.

Fig. 4 schematically illustrates an example schematic diagram of a process of determining a target graph neural network in accordance with an embodiment of the disclosure.

As shown in fig. 4, in 400, in response to receiving sample map data 401 to be forgotten from a sample user, remaining sample map data 403 may be determined from original sample map data 402 and sample map data 401 to be forgotten.

The original gradient vector 405 may be determined from the original sample map data 402 and the original network parameters 404. The original gradient vector 405 may include at least one original gradient sub-vector. The first target vector 407 may be determined from at least one original gradient sub-vector.

A perturbation gradient vector 406 may be determined from the remaining sample map data 403 and the original network parameters 404. The perturbation gradient vector 406 may include at least one perturbation gradient sub-vector. The second target vector 408 may be determined from at least one disturbance gradient sub-vector.

After the first target vector 407 and the second target vector 408 are obtained, a target difference vector 410 may be determined from the first target vector 407 and the second target vector 408. The sea plug matrix 411 may be determined from the raw sample map data 402, the raw network parameters 404, and the raw loss function 409.

After the target difference vector 410 and the sea plug matrix 411 are obtained, an approximation matrix 412 may be determined from the target difference vector 410 and the sea plug matrix 411. The optimized network parameters 413 may be determined from the approximation matrix 412.

In accordance with an embodiment of the present disclosure, experiments were performed using the public map dataset Cora, citeseer and Coauthor-CS. The statistics of the public map dataset are shown in table 1 below.

TABLE 1

Data set	Node count	Edge number	Feature number	Class number
					Cora	2708	5429	1433	7
Citeseer	3327	4732	3703	6
					Coauthor-CS	18333	163788	6805	15

According to an embodiment of the present disclosure, experiments are performed by taking the Graph neural network GCN, GAT, SGC and GIN, the retraining-based data forgetting method, the subgraph-based data forgetting method (BLPA and BEKM) and the method for obtaining the target Graph neural network (i.e., IF4 Graph) through training provided in the embodiment of the present disclosure as examples, and the experimental results are shown in table 2 below.

TABLE 2

According to the embodiment of the disclosure, as can be seen from table 2, the method for obtaining the target graph neural network through training provided by the embodiment of the disclosure does not need to retrain a model, and only needs to perform influence estimation on the affected subgraph to generate optimized network parameters, thereby reducing the calculation time cost.

According to the embodiment of the disclosure, each forgetting algorithm is compared by taking a continuous edge forgetting task with a forgetting rate of 5% as an example, and an F1 score comprehensively considering two precision indexes of recall rate and accuracy rate is adopted as an evaluation index of model precision, and experimental results are shown in the following table 3.

TABLE 3 Table 3

According to the embodiment of the disclosure, as can be seen from table 3, the method for obtaining the target graph neural network through training provided by the embodiment of the disclosure can avoid the change of the graph topology structure caused by dividing the data subsets, thereby guaranteeing the accuracy of the graph neural network.

Fig. 5A schematically illustrates a schematic of experimental results according to an embodiment of the present disclosure.

Fig. 5B schematically illustrates a schematic diagram of experimental results according to another embodiment of the present disclosure.

As shown in fig. 5A and 5B, by adding noise edges with different proportions in the training diagram, two nodes connected by the noise edges have different class labels, the noise proportions are set to be 10%, 40%, 70% and 100%, and an F1 score comprehensively considering two precision indexes of recall rate and accuracy rate is adopted as an evaluation index of model precision. Challenge experiments were performed on the Cora dataset for the GCN model, SGC model, and method of training the target graph neural network provided by embodiments of the present disclosure.

According to embodiments of the present disclosure, as noise margin increases, the prediction accuracy of the model obtained by direct training continues to decrease. Compared with the traditional IF method, the method for training the target graph neural network can remove negative effects of noise data more effectively, and therefore higher prediction accuracy is achieved.

According to the embodiment of the disclosure, the method for obtaining the target graph neural network through training provided by the embodiment of the disclosure forgets information more thoroughly, not only directly affected nodes and connected edges are considered, but also changes of affected neighbor nodes are considered, namely effects of the directly affected nodes and affected nodes in higher-order neighbors thereof on model parameter changes are comprehensively considered, and therefore the forgotten effectiveness is ensured.

Fig. 6A schematically illustrates a schematic diagram of experimental results according to another embodiment of the present disclosure.

Fig. 6B schematically illustrates a schematic diagram of experimental results according to another embodiment of the present disclosure.

As shown in fig. 6A and 6B, for the node forgetting task and the feature forgetting task, by setting forgetting proportions of 10%, 20%, 30%, 40% and 50%, an F1 score comprehensively considering two precision indexes of recall rate and accuracy rate is adopted as an evaluation index of model precision. Prediction accuracy experiments were performed on the Citeseer dataset for the GCN model, the SGC model, and the method for training to obtain the target graph neural network provided by the embodiments of the present disclosure.

According to the embodiment of the disclosure, the method for training the target graph neural network provided by the embodiment of the disclosure can achieve almost the same effect as the retraining method in terms of the performance of model accuracy, so that the method for training the target graph neural network provided by the embodiment of the disclosure has universal applicability.

The above is only an exemplary embodiment, but not limited thereto, and other training methods of the graph neural network known in the art may be also included as long as the accuracy and the data security of the data processing are improved.

Fig. 7 schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure.

As shown in fig. 7, the data processing apparatus 700 may include a first acquisition module 710 and a processing module 720.

The first obtaining module 710 is configured to obtain, in response to receiving a data processing request from a target user, to-be-processed data indicated by the data processing request from a data source, where the to-be-processed data includes to-be-processed graph data, and the to-be-processed graph data is associated with the target user.

And the processing module 720 is used for inputting the graph data to be processed into the target graph neural network and outputting a graph data processing result.

According to an embodiment of the present disclosure, an apparatus for training a target graph neural network may include a second acquisition module, a first determination module, a second determination module, and an adjustment module.

The second acquisition module is used for acquiring original network parameters, an original loss function, original sample graph data and to-be-forgotten sample graph data, wherein the original network parameters correspond to an original graph neural network, and the to-be-forgotten sample graph data has a forgetting type.

And the first determining module is used for determining residual sample graph data according to the original sample graph data and the sample graph data to be forgotten.

And the second determining module is used for determining an original gradient vector, a disturbance gradient vector and a sea plug matrix according to the original sample map data, the residual sample map data, the original network parameters and the original loss function based on the forgetting type.

According to an embodiment of the present disclosure, the original gradient vector comprises at least one original gradient sub-vector and the perturbation gradient vector comprises at least one perturbation gradient sub-vector.

According to an embodiment of the present disclosure, the adjustment module may include a first determination sub-module, a second determination sub-module, a third determination sub-module, a fourth determination sub-module, a fifth determination sub-module, and a generation sub-module.

The first determining sub-module is used for determining a first target vector according to at least one original gradient sub-vector.

And the second determining sub-module is used for determining a second target vector according to the at least one disturbance gradient sub-vector.

And the third determining submodule is used for determining a target difference vector according to the first target vector and the second target vector.

And the fourth determination submodule is used for determining an approximate matrix according to the sea plug matrix and the target difference vector.

And a fifth determining submodule for determining optimized network parameters according to the approximate matrix.

And the generating sub-module is used for generating a target graph neural network according to the optimized network parameters.

According to an embodiment of the present disclosure, the fourth determination submodule may include a building unit and an iterative processing unit.

And the construction unit is used for constructing a recursive equation according to the sea plug matrix and the target difference vector, wherein the recursive equation comprises the product between the inverse matrix corresponding to the sea plug matrix and the target difference vector.

And the iteration processing unit is used for carrying out iteration processing on the recursion equation until a preset ending condition is met, so as to obtain an approximate matrix.

According to an embodiment of the present disclosure, in a case where the forgetting type belongs to node forgetting, the original sample graph data includes a number a of original nodes, the sample graph data to be forgotten includes M number M of nodes to be forgotten, and the remaining sample graph data includes B number of remaining nodes.

According to an embodiment of the present disclosure, the second determination module may include a sixth determination sub-module, a seventh determination sub-module, and an eighth determination sub-module.

And the sixth determining submodule is used for determining an original gradient vector according to the original network parameters and the A original nodes.

And a seventh determination submodule, configured to determine a disturbance gradient vector according to the original network parameter and the B remaining nodes.

And the eighth determination submodule is used for determining the sea plug matrix according to the original loss function, the original network parameters and the A original nodes.

According to an embodiment of the present disclosure, the first determination sub-module may include a first determination unit, a second determination unit, and a third determination unit.

And the first determining unit is used for determining N neighbor nodes corresponding to the M nodes to be forgotten respectively.

The second determining unit is used for determining original gradient subvectors corresponding to the N neighbor nodes according to the original network parameters, the M nodes to be forgotten and the N neighbor nodes corresponding to the M nodes to be forgotten.

And the third determining unit is used for determining a first target vector according to the original gradient sub-vectors corresponding to the M nodes to be forgotten and the N neighbor nodes.

According to an embodiment of the present disclosure, the second determination sub-module may include a fourth determination unit and a fifth determination unit.

And the fourth determining unit is used for determining disturbance gradient sub-vectors corresponding to the N neighbor nodes according to the network parameters to be adjusted and the N neighbor nodes corresponding to the M nodes to be forgotten.

And the fifth determining unit is used for determining a second target vector according to the disturbance gradient sub-vectors corresponding to the N neighbor nodes respectively.

According to embodiments of the present disclosure, the optimized network parameters may be determined according to the following formula:

wherein ,characterizing the optimized network parameters, θ characterizing the original network parameters, H _θ Characterization of the sea plug matrix,>the inverse of the sea plug matrix is characterized.

Wherein g represents original sample graph data, Δg represents sample graph data to be forgotten, g\Δg represents residual sample graph data, and z _i Characterizing the sample graph data, deltav characterizing the nodes to be forgotten,characterizing neighbor nodes corresponding to nodes to be forgotten, f _g (z _i ) Characterization of raw graph neural network pair z trained on raw sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) characterization versus z on the remaining sample map data using the original network parameters _i And (b) prediction result, y _i Characterization z _i Is a label of (a).

wherein ,characterizing the original gradient subvector,>characterizing a first target vector, ">Characterizing disturbance gradient subvectors, ">The second target vector is characterized.

wherein ,the target difference vector is characterized by the fact that,an approximation matrix corresponding to the product result is characterized.

According to an embodiment of the present disclosure, in a case where the forgetting type belongs to edge forgetting, the original sample graph data includes C original nodes, the sample graph data to be forgotten includes X edges to be forgotten, and the remaining sample graph data includes D remaining nodes.

According to an embodiment of the present disclosure, the second determination module may include a ninth determination sub-module, a tenth determination sub-module, and an eleventh determination sub-module.

And a ninth determination submodule, configured to determine an original gradient vector according to the original network parameter and the C original nodes.

And a tenth determination submodule, configured to determine a disturbance gradient vector according to the original network parameter and the D remaining nodes.

An eleventh determination submodule is used for determining the sea plug matrix according to the original loss function, the original network parameters and the C original nodes.

According to an embodiment of the present disclosure, the first determination sub-module may include a sixth determination unit, a seventh determination unit, an eighth determination unit, and a ninth determination unit.

And a sixth determining unit, configured to determine Y endpoints corresponding to the X edges to be forgotten, respectively.

A seventh determining unit, configured to determine, for each of the X edges to be forgotten, Z neighbor nodes corresponding to the Y endpoints respectively.

And the eighth determining unit is used for determining original gradient subvectors corresponding to the X edges to be forgotten according to the original network parameters, Y endpoints corresponding to the X edges to be forgotten and Z neighbor nodes corresponding to the X edges to be forgotten.

And the ninth determining unit is used for determining a first target vector according to the original gradient sub-vectors corresponding to the X edges to be forgotten.

According to an embodiment of the present disclosure, the second determination sub-module may include a tenth determination unit and an eleventh determination unit.

And the tenth determining unit is used for determining disturbance gradient sub-vectors corresponding to the X edges to be forgotten according to the network parameters to be adjusted, Y endpoints corresponding to the X edges to be forgotten and Z neighbor nodes corresponding to the X edges to be forgotten.

And the eleventh determining unit is used for determining a second target vector according to the disturbance gradient sub-vectors corresponding to the X edges to be forgotten.

Wherein g represents original sample graph data, Δg represents sample graph data to be forgotten, g\Δg represents residual sample graph data, and z _i Characterizing the data of the sample graph, delta epsilon characterizing the edges to be forgotten, Characterizing neighbor nodes corresponding to endpoints of Δε, f _g (z _i ) Characterizing an original graph trained on original sample graph dataNeural network pair z _i F is the predicted result of (f) _g (z _i G\Δg) characterization versus z on the remaining sample map data using the original network parameters _i And (b) prediction result, y _i Characterization z _i Is a label of (a).

According to an embodiment of the present disclosure, in the case where the forgetting type belongs to feature forgetting, the original sample graph data includes E original nodes, the sample graph data to be forgotten includes P features to be forgotten, and the remaining sample graph data includes F remaining nodes.

According to an embodiment of the present disclosure, the second determination module may include a twelfth determination sub-module, a thirteenth determination sub-module, and a fourteenth determination sub-module.

A twelfth determination submodule, configured to determine an original gradient vector according to the original network parameter and the E original nodes.

A thirteenth determination submodule is configured to determine a disturbance gradient vector according to the original network parameter and the F remaining nodes.

A fourteenth determination submodule, configured to determine a sea plug matrix according to the original loss function, the original network parameter, and the E original nodes.

According to an embodiment of the present disclosure, the first determination sub-module may include a twelfth determination unit, a thirteenth determination unit, a fourteenth determination unit, and a fifteenth determination unit.

And the twelfth determining unit is used for determining the nodes to be forgotten, which correspond to the P features to be forgotten respectively.

And a thirteenth determining unit, configured to determine Q neighbor nodes corresponding to the P nodes to be forgotten, respectively.

The fourteenth determining unit is used for determining original gradient subvectors corresponding to the P nodes to be forgotten according to the original network parameters, the P nodes to be forgotten and Q neighbor nodes corresponding to the P nodes to be forgotten respectively.

And the fifteenth determining unit is used for determining a first target vector according to the original gradient sub-vectors corresponding to the M nodes to be forgotten.

According to an embodiment of the present disclosure, the second determination sub-module may include a sixteenth determination unit and a seventeenth determination unit.

The sixteenth determining unit is used for determining disturbance gradient subvectors corresponding to the P to-be-forgotten features according to the network parameters to be adjusted, the P to-be-forgotten nodes and Q neighbor nodes corresponding to the P to-be-forgotten nodes respectively.

Seventeenth determining unit, configured to determine a second target vector according to disturbance gradient subvectors corresponding to M nodes to be forgotten.

wherein ,characterization of optimized network parameters, θ characterizes the original networkParameters, H _θ Characterization of the sea plug matrix,>the inverse of the sea plug matrix is characterized. />

Wherein g represents original sample graph data, Δg represents sample graph data to be forgotten, g\Δg represents residual sample graph data, and z _i Characterizing the sample graph data, Δx characterizing the feature to be forgotten,characterizing neighbor nodes corresponding to endpoints of Δε, f _g (zi) characterization of the original graph neural network pair z trained on the original sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) characterization versus z on the remaining sample map data using the original network parameters _i And (b) prediction result, y _i Characterization z _i Is a label of (a).

Any number of modules, sub-modules, units, sub-units, or at least some of the functionality of any number of the sub-units according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented as split into multiple modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or in any other reasonable manner of hardware or firmware that integrates or encapsulates the circuit, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be at least partially implemented as computer program modules, which when executed, may perform the corresponding functions.

For example, any number of the acquisition module 710 and the processing module 720 may be combined in one module/unit/sub-unit or any one of the modules/units/sub-units may be split into a plurality of modules/units/sub-units. Alternatively, at least some of the functionality of one or more of these modules/units/sub-units may be combined with at least some of the functionality of other modules/units/sub-units and implemented in one module/unit/sub-unit. According to embodiments of the present disclosure, at least one of the acquisition module 710 and the processing module 720 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware, such as any other reasonable way of integrating or packaging the circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, at least one of the acquisition module 710 and the processing module 720 may be at least partially implemented as computer program modules, which when executed, may perform the corresponding functions.

It should be noted that, in the embodiments of the present disclosure, the data processing apparatus portion corresponds to the data processing method portion in the embodiments of the present disclosure, and the description of the data processing apparatus portion specifically refers to the data processing method portion and is not described herein.

Fig. 8 schematically illustrates a block diagram of an electronic device adapted to implement a data processing method according to an embodiment of the disclosure. The electronic device shown in fig. 8 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.

As shown in fig. 8, a computer electronic device 800 according to an embodiment of the present disclosure includes a processor 801 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 809 into a Random Access Memory (RAM) 803. The processor 801 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 801 may also include on-board memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the disclosure.

In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 802 and/or the RAM 803. Note that the program may be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, the electronic device 800 may also include an input/output (I/O) interface 805, the input/output (I/O) interface 805 also being connected to the bus 804. The electronic device 800 may also include one or more of the following components connected to the I/O interface 805: an input portion 806 including a keyboard, mouse, etc.; an output portion 807 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 808 including a hard disk or the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed.

According to embodiments of the present disclosure, the method flow according to embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 802 and/or RAM 803 and/or one or more memories other than ROM 802 and RAM 803 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program comprising program code for performing the methods provided by the embodiments of the present disclosure, the program code for causing an electronic device to implement the data processing methods provided by the embodiments of the present disclosure when the computer program product is run on the electronic device.

The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed, and downloaded and installed in the form of a signal on a network medium, and/or from a removable medium 811 via a communication portion 809. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1. A data processing method, comprising:

in response to receiving a data processing request from a target user, acquiring to-be-processed data indicated by the data processing request from a data source, wherein the to-be-processed data comprises to-be-processed graph data, and the to-be-processed graph data is associated with the target user;

inputting the graph data to be processed into a target graph neural network, and outputting a graph data processing result;

the target graph neural network is trained by the following modes:

Determining residual sample map data according to the original sample map data and the sample map data to be forgotten;

and according to the original gradient vector, the disturbance gradient vector and the sea plug matrix, carrying out parameter adjustment on the original network parameters to obtain the target graph neural network.

2. The method of claim 1, wherein the original gradient vector comprises at least one original gradient sub-vector and the perturbation gradient vector comprises at least one perturbation gradient sub-vector;

the step of performing parameter adjustment on the original network parameters according to the original gradient vector, the disturbance gradient vector and the sea plug matrix to obtain the target graph neural network comprises the following steps:

determining a first target vector from the at least one original gradient sub-vector;

determining optimized network parameters according to the approximation matrix; and

3. The method of claim 2, wherein the determining an approximation matrix from the sea plug matrix and the target difference vector comprises:

and carrying out iterative processing on the recursion equation until a preset ending condition is met, so as to obtain the approximate matrix.

4. A method according to claim 2 or 3, wherein, in case the forgetting type belongs to the node forgetting, the original sample graph data comprises a number of original nodes, the sample graph data to be forgotten comprises M number of nodes to be forgotten, and the remaining sample graph data comprises B number of remaining nodes;

the determining, based on the forgetting type, an original gradient vector, a disturbance gradient vector, and a sea plug matrix according to the original sample map data, the residual sample map data, the original network parameters, and the original loss function includes:

determining the sea plug matrix according to the original loss function, the original network parameters and the A original nodes;

said determining a first target vector from said at least one original gradient sub-vector comprises:

n neighbor nodes corresponding to the M nodes to be forgotten respectively are determined;

determining the first target vector according to original gradient sub-vectors corresponding to the M nodes to be forgotten and N neighbor nodes respectively;

said determining a second target vector from said at least one disturbance gradient sub-vector comprises:

according to network parameters to be adjusted and N neighbor nodes corresponding to the M nodes to be forgotten, determining disturbance gradient sub-vectors corresponding to the N neighbor nodes; and

Determining the second target vector according to disturbance gradient sub-vectors corresponding to the N neighbor nodes respectively;

wherein A, B, M is a positive integer greater than 1, and N is a positive integer.

5. The method of claim 4, wherein said determining optimized network parameters from said approximation matrix comprises:

determining the optimized network parameters according to the following formula:

wherein ,characterizing the optimized network parameters, θ characterizing the original network parameters, H _θ Characterizing the sea plug matrix,>characterizing an inverse of the sea plug matrix;

wherein g represents the original sample map data, Δg represents the sample map data to be forgotten, g\Δg represents the residual sample map data, z _i Characterizing the sample graph data, Δv characterizing the nodes to be forgotten,characterizing neighbor nodes corresponding to nodes to be forgotten, f _g (z _i ) Characterizing the original graph neural network pair z trained on the original sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) characterization of z over the remaining sample map data using the raw network parameters _i And (b) prediction result, y _i Characterization z _i Is a label of (2);

wherein ,characterizing the original gradient sub-vector,>characterizing said first target vector,/for >The perturbation gradient sub-vector is characterized,characterizing the second target vector;

wherein ,the target difference vector is characterized in that,characterizing the approximation matrix corresponding to the product result.

6. A method according to claim 2 or 3, wherein, in case the forgetting type belongs to the edge forgetting, the original sample graph data comprises C original nodes, the sample graph data to be forgotten comprises X edges to be forgotten, and the remaining sample graph data comprises D remaining nodes;

determining the disturbance gradient vector according to the original network parameters and the D residual nodes;

determining the sea plug matrix according to the original loss function, the original network parameters and the C original nodes;

determining Z neighbor nodes corresponding to the Y endpoints respectively for each to-be-forgotten edge in the X to-be-forgotten edges;

determining the first target vector according to the original gradient sub-vectors corresponding to the X edges to be forgotten respectively;

according to network parameters to be adjusted, Y endpoints respectively corresponding to X edges to be forgotten and Z neighbor nodes respectively corresponding to the X edges to be forgotten, determining disturbance gradient sub-vectors respectively corresponding to the X edges to be forgotten; and

determining the second target vector according to disturbance gradient sub-vectors corresponding to the X edges to be forgotten respectively;

wherein C, D, X, Y is a positive integer greater than 1, and Z is a positive integer.

7. The method of claim 6, wherein said determining optimized network parameters from said approximation matrix comprises:

wherein g represents the original sample map data, Δg represents the sample map data to be forgotten, g\Δg represents the residual sample map data, z _i Characterizing the sample graph data, delta epsilon characterizing the edges to be forgotten,characterization of the delta epsilon from the delta epsilonNeighbor node corresponding to endpoint, f _g (z _i ) Characterizing the original graph neural network pair z trained on the original sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) characterization of z over the remaining sample map data using the raw network parameters _i And (b) prediction result, y _i Characterization z _i Is a label of (2);

wherein ,characterizing the original gradient sub-vector,>characterizing said first target vector,/for>Characterizing the perturbation gradient sub-vector,>characterizing the second target vector;

8. A method according to claim 2 or 3, wherein, in case the forgetting type belongs to the feature forgetting, the original sample graph data comprises E original nodes, the sample graph data to be forgotten comprises P features to be forgotten, and the remaining sample graph data comprises F remaining nodes;

determining the sea plug matrix according to the original loss function, the original network parameters and the E original nodes;

q neighbor nodes corresponding to the P nodes to be forgotten respectively are determined;

determining the first target vector according to the original gradient sub-vectors corresponding to the M nodes to be forgotten respectively;

determining the second target vector according to disturbance gradient sub-vectors corresponding to the M nodes to be forgotten respectively;

wherein E, F, P is a positive integer greater than 1, and Q is a positive integer.

9. The method of claim 8, wherein the determining optimized network parameters from the approximation matrix comprises:

wherein g represents the original sample map data, Δg represents the sample map data to be forgotten, g\Δg represents the residual sample map data, z _i Characterizing the sample map data, Δx characterizing the to-be-forgotten feature,characterizing neighbor nodes corresponding to the endpoints of Δε, f _g (z _i ) Characterizing the original graph neural network pair z trained on the original sample graph data _i F is the predicted result of (f) _g (z _i G\Δg) characterization of z over the remaining sample map data using the raw network parameters _i And (b) prediction result, y _i Characterization z _i Is a label of (2);

10. A data processing apparatus comprising:

the first acquisition module is used for responding to a data processing request received from a target user, and acquiring to-be-processed data indicated by the data processing request from a data source, wherein the to-be-processed data comprises to-be-processed graph data, and the to-be-processed graph data is associated with the target user;

the processing module is used for inputting the graph data to be processed into a target graph neural network and outputting a graph data processing result;

the device for training and obtaining the target graph neural network comprises the following components: