CN111353534B - Graph data category prediction method based on adaptive fractional order gradient - Google Patents

Graph data category prediction method based on adaptive fractional order gradient Download PDF

Info

Publication number
CN111353534B
CN111353534B CN202010122396.2A CN202010122396A CN111353534B CN 111353534 B CN111353534 B CN 111353534B CN 202010122396 A CN202010122396 A CN 202010122396A CN 111353534 B CN111353534 B CN 111353534B
Authority
CN
China
Prior art keywords
fractional order
neural network
layer
graph data
graph
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN202010122396.2A
Other languages
Chinese (zh)
Other versions
CN111353534A (en
Inventor
骆春波
刘子健
罗杨
孙文健
王亚宁
吴佳
刘翔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN202010122396.2A priority Critical patent/CN111353534B/en
Publication of CN111353534A publication Critical patent/CN111353534A/en
Application granted granted Critical
Publication of CN111353534B publication Critical patent/CN111353534B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a graph data category prediction method based on adaptive fractional order gradient, which comprises the steps of carrying out normalization processing on an adjacency matrix A of graph data to obtain a normalized adjacency matrix; initializing the network weight according to the preset number of layers of the network; feeding the feature matrixes H of all the nodes in the graph and the adjacency matrix A of the graph into a graph network, and calculating a mean square error loss function; updating the weight parameters and the fractional order according to the mean square error loss function to obtain a graph neural network of the self-adaptive fractional order gradient; the method solves the technical problem that the existing graph neural network method falls into local optimum, thereby obtaining an ideal optimization result.

Description

Graph data category prediction method based on adaptive fractional order gradient
Technical Field
The invention belongs to the field of graph signal processing, and particularly relates to a graph data category prediction method based on adaptive fractional order gradient.
Background
Deep learning has received a great deal of attention in dealing with non-euclidean structural data represented by graphs. In a broad sense, images, video, manifolds, etc. are different representations of a graph. Therefore, graph signal processing provides an important tool for the research fields of social science, bioinformatics, physical systems, knowledge graphs and the like.
As a non-european data processing technique, Graph Neural Networks (GNNs) take data features and adjacency relations as inputs, and focus on tasks such as node classification, edge prediction, graph clustering, and the like. The graph convolutional neural networks (GCNs) apply convolution operation to the graph networks, high-dimensional information of the graph space is extracted and aggregated, and therefore the semi-supervised node classification task is completed. These methods are usually based on the iterative update of parameters by gradient descent methods, and all have the following disadvantages: due to the non-convex loss function, the network is easy to fall into local optimum, and an ideal result cannot be obtained; the first order search method converges more slowly than the first order search method.
Disclosure of Invention
Aiming at the defects in the prior art, the graph data category prediction method based on the adaptive fractional order gradient solves the problems that the network is easy to fall into local optimum and the convergence rate of the first-order search method is slower than that of the high-order search method in the existing graph neural network method.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that: a map data category prediction method based on adaptive fractional order gradient comprises the following steps:
s1, normalizing the adjacency matrix A of the graph data to obtain a normalized adjacency matrix
Figure BDA0002393348210000011
S2, establishing a five-layer neural network structure, initializing weight of each layer of the network, and initializing iteration times t, fractional order parameters and fractional orders, wherein t is 0;
s3, adjacent matrix to be normalized
Figure BDA0002393348210000021
Inputting the characteristic H of each node of the graph data into a five-layer neural network structure to obtain the prediction category of each node of the graph data;
s4, calculating a mean square error loss function value according to the prediction type of each node and the target type of each node of the graph data;
s5, judging whether the mean square error loss function value is smaller than a threshold value, if so, obtaining an optimized five-layer neural network, if not, updating the weight parameters by adopting a fractional order gradient descent method, updating the fractional order according to the fractional order parameters, increasing the value of t by 1, and jumping to the step S3;
and S6, processing the graph data by adopting the five-layer neural network after optimization to obtain the final prediction category of each node of the graph data.
Further: the step S1 includes the following steps:
s11, carrying out row summation or column summation on the adjacent matrix A to obtain a degree matrix D;
s12, adding the adjacent matrix A and the unit matrix I, and then carrying out left multiplication and right multiplication with the 0.5 power of the degree matrix D to obtain a normalized adjacent matrix
Figure BDA0002393348210000022
Further: adjacency matrix normalized in step S12
Figure BDA0002393348210000023
Comprises the following steps:
Figure BDA0002393348210000024
further: the five-layer neural network in the step S2 sequentially includes: an input layer, a 3-layer hidden layer and an output layer.
Further: in step S3, the input/output relationship of each layer of neural network in the five-layer neural network structure is:
Figure BDA0002393348210000025
wherein the content of the first and second substances,
Figure BDA00023933482100000211
is the input of the l-th layer neural network in the t-th iteration process,
Figure BDA0002393348210000027
is the output of the l-th layer neural network in the t-th iteration process,
Figure BDA0002393348210000028
is the input to the first layer neural network during iteration 0,
Figure BDA0002393348210000029
Figure BDA00023933482100000210
is the weight of the l-1 layer neural network in the t iteration process, 1<l is less than or equal to 5, and e is a natural logarithm.
Further: in step S5, the formula for updating the weight parameter by the fractional gradient descent method is:
Figure BDA0002393348210000031
wherein η is the step size factor, γ is the mean square error loss function, Γ represents the Hadamard product, Γ (·) represents the gamma function, νt-1Is the fractional order of t-1 iterations.
Further: in step S5, the calculation formula for updating the fractional order according to the fractional order parameter is:
Figure BDA0002393348210000032
wherein α is a fractional order parameter.
The invention has the beneficial effects that: compared with the traditional first-order search method, the method has higher convergence rate; because the order itself is time-varying, the order of the iteration itself is continuously updated in the process of the iteration, and the order does not fall into the current fractional order extreme point. Moreover, as the iteration process proceeds, the order gradually approaches 1, thereby ensuring the finally reached solution of the first-order problem, i.e. the ideal solution of the problem.
The method combines the advantages of fractional calculus non-local characteristics and good memorability and the design of time-varying adaptive order, so that the method can obtain more ideal optimization results.
Drawings
FIG. 1 is a flow chart of a method for predicting image data classes based on adaptive fractional order gradients.
FIG. 2 is a variation curve of a fractional order iteration process;
FIG. 3 is a comparison graph of the processing result of the method and the neural network GNN for the semi-supervised node classification dataset Cora;
FIG. 4 is a comparison graph of the processing results of the method and the neural network GNN of the graph for the semi-supervised node classification dataset Citeseer;
fig. 5 is a comparison graph of the processing result of the semi-supervised node classification data set Pubmed by the method and the neural network GNN.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
As shown in fig. 1, a method for predicting a class of map data based on an adaptive fractional order gradient includes the following steps:
s1, normalizing the adjacency matrix A of the graph data to obtain a normalized adjacency matrix
Figure BDA00023933482100000410
The step S1 includes the following steps:
s11, carrying out row summation or column summation on the adjacent matrix A to obtain a degree matrix D;
s12, adding the adjacent matrix A and the unit matrix I, and then carrying out left multiplication and right multiplication with the 0.5 power of the degree matrix D to obtain a normalized adjacent matrix
Figure BDA0002393348210000041
Adjacency matrix normalized in step S12
Figure BDA0002393348210000042
Comprises the following steps:
Figure BDA0002393348210000043
s2, establishing a five-layer neural network structure, initializing weight of each layer of the network, and initializing iteration times t, fractional order parameters and fractional orders, wherein t is 0;
the five-layer neural network in the step S2 sequentially includes: an input layer, a 3-layer hidden layer and an output layer.
S3, adjacent matrix to be normalized
Figure BDA0002393348210000044
Inputting the characteristic H of each node of the graph data into a five-layer neural network structure to obtain the prediction category of each node of the graph data;
in step S3, the input/output relationship of each layer of neural network in the five-layer neural network structure is:
Figure BDA0002393348210000045
wherein the content of the first and second substances,
Figure BDA0002393348210000046
is the input of the l-th layer neural network in the t-th iteration process,
Figure BDA0002393348210000047
is the output of the l-th layer neural network in the t-th iteration process,
Figure BDA0002393348210000048
is the input to the first layer neural network during iteration 0,
Figure BDA0002393348210000049
Figure BDA0002393348210000051
is the t-th iterationWeight of layer 1-1 neural network in generation process, 1<l is less than or equal to 5, and e is a natural logarithm.
S4, calculating a mean square error loss function value according to the prediction type of each node and the target type of each node of the graph data;
s5, judging whether the mean square error loss function value is smaller than a threshold value, if so, obtaining an optimized five-layer neural network, if not, updating the weight parameters by adopting a fractional order gradient descent method, updating the fractional order according to the fractional order parameters, increasing the value of t by 1, and jumping to the step S3;
in step S5, the formula for updating the weight parameter by the fractional gradient descent method is:
Figure BDA0002393348210000052
wherein η is the step size factor, γ is the mean square error loss function, Γ represents the Hadamard product, Γ (-) represents the gamma function, vt-1Is the fractional order of t-1 iterations.
In step S5, the calculation formula for updating the fractional order according to the fractional order parameter is:
Figure BDA0002393348210000053
wherein α is a fractional order parameter.
And S6, processing the graph data by adopting the five-layer neural network after optimization to obtain the final prediction category of each node of the graph data.
Fig. 2 shows the variation curve of the adaptive fractional order of the present invention at different initial preset values with the iterative process.
The invention has the beneficial effects that: compared with the traditional first-order search method, the method has higher convergence rate; because the order itself is time-varying, the order of the iteration itself is continuously updated in the process of the iteration, and the order does not fall into the current fractional order extreme point. Moreover, as the iteration process proceeds, the order gradually approaches 1, thereby ensuring the finally reached solution of the first-order problem, i.e. the ideal solution of the problem.
The method combines the advantages of fractional calculus non-local characteristics and good memorability and the design of time-varying adaptive order, so that the method can obtain more ideal optimization results. As shown in FIGS. 3-5, the box line graphs in the graphs are the accuracy of the prediction categories obtained by the method, and it can be seen from the graphs that the accuracy of the prediction categories of the method is generally superior to the accuracy obtained by using the graph neural network GNN.

Claims (5)

1. A graph data category prediction method based on adaptive fractional order gradient is characterized by comprising the following steps:
s1, normalizing the adjacency matrix A of the graph data to obtain a normalized adjacency matrix
Figure FDA0002752372220000011
S2, establishing a five-layer neural network structure, initializing weight of each layer of the network, and initializing iteration times t, fractional order parameters and fractional orders, wherein t is 0;
s3, adjacent matrix to be normalized
Figure FDA0002752372220000012
Inputting the characteristic H of each node of the graph data into a five-layer neural network structure to obtain the prediction category of each node of the graph data; s4, calculating a mean square error loss function value according to the prediction type of each node and the target type of each node of the graph data;
s5, judging whether the mean square error loss function value is smaller than a threshold value, if so, obtaining an optimized five-layer neural network, if not, updating the weight parameters by adopting a fractional order gradient descent method, updating the fractional order according to the fractional order parameters, increasing the value of t by 1, and jumping to the step S3;
in step S5, the formula for updating the weight parameter by the fractional gradient descent method is:
Figure FDA0002752372220000013
wherein η is the step size factor, γ is the mean square error loss function, Γ represents the Hadamard product, Γ (-) represents the gamma function, vt-1Is the fractional order of the t-1 iteration process;
in step S5, the calculation formula for updating the fractional order according to the fractional order parameter is:
Figure FDA0002752372220000014
wherein alpha is a fractional order parameter; and S6, processing the graph data by adopting the five-layer neural network after optimization to obtain the final prediction category of each node of the graph data.
2. The adaptive fractional order gradient-based map data category prediction method of claim 1, wherein the step S1 comprises the following steps:
s11, carrying out row summation or column summation on the adjacent matrix A to obtain a degree matrix D;
s12, adding the adjacent matrix A and the unit matrix I, and then carrying out left multiplication and right multiplication with the 0.5 power of the degree matrix D to obtain a normalized adjacent matrix
Figure FDA0002752372220000021
3. The adaptive fractional order gradient-based graph data class prediction method according to claim 2, wherein the adjacency matrix normalized in step S12
Figure FDA0002752372220000022
Comprises the following steps:
Figure FDA0002752372220000023
4. the adaptive fractional order gradient-based map data category prediction method of claim 1, wherein the five-layer neural network in step S2 sequentially comprises: an input layer, a 3-layer hidden layer and an output layer.
5. The adaptive fractional order gradient-based map data category prediction method of claim 1, wherein the input-output relationship of each layer of neural network in the five-layer neural network structure in step S3 is as follows:
Figure FDA0002752372220000024
wherein the content of the first and second substances,
Figure FDA0002752372220000025
is the input of the l-th layer neural network in the t-th iteration process,
Figure FDA0002752372220000026
is the output of the l-th layer neural network in the t-th iteration process,
Figure FDA0002752372220000027
is the input to the first layer neural network during iteration 0,
Figure FDA0002752372220000028
Figure FDA0002752372220000029
is the weight of the l-1 layer neural network in the t iteration process, 1<l is less than or equal to 5, and e is a natural logarithm.
CN202010122396.2A 2020-02-27 2020-02-27 Graph data category prediction method based on adaptive fractional order gradient Expired - Fee Related CN111353534B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010122396.2A CN111353534B (en) 2020-02-27 2020-02-27 Graph data category prediction method based on adaptive fractional order gradient

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010122396.2A CN111353534B (en) 2020-02-27 2020-02-27 Graph data category prediction method based on adaptive fractional order gradient

Publications (2)

Publication Number Publication Date
CN111353534A CN111353534A (en) 2020-06-30
CN111353534B true CN111353534B (en) 2021-01-26

Family

ID=71197169

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010122396.2A Expired - Fee Related CN111353534B (en) 2020-02-27 2020-02-27 Graph data category prediction method based on adaptive fractional order gradient

Country Status (1)

Country Link
CN (1) CN111353534B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112183234A (en) * 2020-09-10 2021-01-05 北京华跃信息技术有限公司 Situation perception method and device based on graph neural network
CN112149808B (en) * 2020-09-28 2022-10-14 上海交通大学 Method, system and medium for expanding stand-alone graph neural network training to distributed training
US20220383127A1 (en) * 2021-06-01 2022-12-01 Basmah ALTAF Methods and systems for training a graph neural network using supervised contrastive learning

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103077381A (en) * 2013-01-08 2013-05-01 郑州威科姆科技股份有限公司 Monocular dynamic hand gesture recognition method on basis of fractional Fourier transformation
CN106407932B (en) * 2016-09-20 2019-05-28 中国石油大学(华东) Handwritten Digit Recognition method based on fractional calculus Yu generalized inverse neural network
US10210616B2 (en) * 2017-04-27 2019-02-19 Hong Kong Applied Science And Technology Research Institute Co., Ltd. Kernal approximation on fractional differential operator for edge detection
CN108764472A (en) * 2018-05-18 2018-11-06 南京信息工程大学 Convolutional neural networks fractional order error back propagation method
CN108764320A (en) * 2018-05-21 2018-11-06 深圳信息职业技术学院 Feature extracting method based on fractional order feature line analysis

Also Published As

Publication number Publication date
CN111353534A (en) 2020-06-30

Similar Documents

Publication Publication Date Title
CN113905391B (en) Integrated learning network traffic prediction method, system, equipment, terminal and medium
CN111353534B (en) Graph data category prediction method based on adaptive fractional order gradient
WO2020019236A1 (en) Loss-error-aware quantization of a low-bit neural network
CN107832787B (en) Radar radiation source identification method based on bispectrum self-coding characteristics
WO2022121289A1 (en) Methods and systems for mining minority-class data samples for training neural network
CN110288030B (en) Image identification method, device and equipment based on lightweight network model
CN112508085B (en) Social network link prediction method based on perceptual neural network
CN112966114B (en) Literature classification method and device based on symmetrical graph convolutional neural network
CN110175628A (en) A kind of compression algorithm based on automatic search with the neural networks pruning of knowledge distillation
CN110799995A (en) Data recognizer training method, data recognizer training device, program, and training method
WO2022105108A1 (en) Network data classification method, apparatus, and device, and readable storage medium
WO2022252455A1 (en) Methods and systems for training graph neural network using supervised contrastive learning
CN110796485A (en) Method and device for improving prediction precision of prediction model
Liu et al. EACP: An effective automatic channel pruning for neural networks
CN113822419A (en) Self-supervision graph representation learning operation method based on structural information
WO2021253938A1 (en) Neural network training method and apparatus, and video recognition method and apparatus
CN111967528B (en) Image recognition method for deep learning network structure search based on sparse coding
CN115599918B (en) Graph enhancement-based mutual learning text classification method and system
CN111291193B (en) Application method of knowledge graph in zero-time learning
CN116992151A (en) Online course recommendation method based on double-tower graph convolution neural network
CN116303386A (en) Intelligent interpolation method and system for missing data based on relational graph
CN115294402B (en) Semi-supervised vehicle classification method based on redundancy elimination multi-stage hybrid training
CN111753995A (en) Local interpretable method based on gradient lifting tree
CN115619563A (en) Stock price analysis method based on neural network
Xia et al. Efficient synthesis of compact deep neural networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20210126