CN111209398A - Text classification method and system based on graph convolution neural network - Google Patents

Text classification method and system based on graph convolution neural network Download PDF

Info

Publication number
CN111209398A
CN111209398A CN201911393728.4A CN201911393728A CN111209398A CN 111209398 A CN111209398 A CN 111209398A CN 201911393728 A CN201911393728 A CN 201911393728A CN 111209398 A CN111209398 A CN 111209398A
Authority
CN
China
Prior art keywords
text
graph
neural network
text classification
similarity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911393728.4A
Other languages
Chinese (zh)
Other versions
CN111209398B (en
Inventor
李建欣
邵明来
周佰聪
孙佩源
邰振赢
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN201911393728.4A priority Critical patent/CN111209398B/en
Publication of CN111209398A publication Critical patent/CN111209398A/en
Application granted granted Critical
Publication of CN111209398B publication Critical patent/CN111209398B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention designs a text classification method and a text classification system based on a graph convolution neural network through a graph neural network modeling method, wherein the method mainly comprises the following three steps: inputting a text and constructing a similarity graph; constructing a deep learning model; and forming a text classification method overall model by the similarity graph and the deep learning model, and performing text classification. By the method, based on the text similarity graph, the graph convolution network and the graph attention network, an efficient text classification method is provided, namely, only a source text needs to be input, the text can be automatically classified, and a system is formed by combining hardware on the basis of a model.

Description

Text classification method and system based on graph convolution neural network
Technical Field
The invention relates to the field of neural networks, in particular to a text classification method based on a graph convolution neural network.
Background
Recently, a great deal of research has been focused on data of graph structures, which are data structures that can be used in many ways. Graph Convolution Networks (GCNs) provide us with a very efficient way to analyze graph structure data. It is a very powerful model using neighborhood information, and this special convolution has found wide application in research. It has done much work in several ways to date. Recent work has focused on the following four aspects: community detection using graphical methods, malware detection, object or significance detection in video or pictures, and internet security. There are several other aspects, such as computer vision and research into social networks. In this study, we mainly applied this model to the classification of text.
In the invention, a simplified graph convolution network and a graph attention network are compounded, the accuracy of classification is improved by utilizing the graph attention network while the calculation consumption of the graph convolution neural network is reduced, and a new method is introduced to construct a graph based on the similarity between graph nodes. Then, we import the graph into the network to complete text classification.
Disclosure of Invention
The text representation of the main problems of the existing text classification algorithm is high-dimensional and high-sparse, the feature expression capability is weak, in addition, the feature engineering needs to be carried out manually, and the cost is high. The deep learning initially succeeds in image and voice greatly, and accordingly promotes the development of the deep learning on NLP, so that the deep learning model has good effect on text classification.
In order to achieve the purpose, the invention adopts the following technical scheme:
a text classification method based on a graph convolution neural network comprises the following steps:
the method comprises the following steps: inputting texts, constructing a similarity graph, constructing an adjacency matrix in a multi-dimensional tuple database with each text having basic characteristics based on a plurality of text data sets, defining each text as a node, determining the existence of an edge between two nodes according to the similarity between the two nodes, and generating an edge between the two nodes if the similarity is greater than a specific value so as to classify the nodes in the graph in a binary mode;
step two: constructing a deep learning model by using a simplified graph convolution neural network and a graph attention neural network and combining a Softmax function;
step three: and forming a text classification method overall model by the similarity graph and the deep learning model, and outputting the classified text, wherein the text classification method overall model is realized by using the simplified graph convolution neural network and the graph attention network and combining a Softmax function.
In the first step, the similarity calculation method between two nodes is as follows:
Figure BDA0002345705970000021
in the similarity calculation mode between the two nodes: γ is defined as:
Figure BDA0002345705970000022
further, for quantitative measurements, γ is defined as
Figure BDA0002345705970000023
Definition of dist (A)v,Aw) As follows:
Figure BDA0002345705970000024
the simplified graph convolution neural network calculation mode is defined as:
Figure BDA0002345705970000025
the graph attention network is defined as:
Figure BDA0002345705970000026
α thereinijIs defined as:
Figure BDA0002345705970000027
eijis defined as:
Figure BDA0002345705970000031
the overall model of the text classification method is as follows:
Figure BDA0002345705970000032
Figure BDA0002345705970000033
a system for text classification based on a graph-convolution neural network, comprising:
the information input module is used for normalizing the source text acquired by the external database and then importing the source text;
the text classification module based on the graph convolution neural network is used for classifying the input source text by applying the text classification method based on the graph convolution neural network;
and the information output module is used for outputting the text classification result generated in the text classification module based on the atlas neural network in a new database form.
According to the technical scheme, a novel model called ASGCN is provided to classify different texts. In this model, we combine a simplified graph convolution network and a graph attention network to get better classification results. Furthermore, we propose a new way of constructing a graph so that it can better fit the proposed model and provide us with better results, thus enabling the following effects to be achieved:
1. pre-judging the text category through the correlation between the texts;
2. a new model based on the graph neural network is constructed to improve the classification precision.
Detailed Description
The following is a preferred embodiment of the present invention, and the technical solution of the present invention is further described, but the present invention is not limited to this embodiment.
The text classification method based on the graph convolution neural network in the embodiment mainly comprises three steps:
the method comprises the following steps: inputting a text and constructing a similarity graph;
step two: constructing a deep learning model;
step three: and forming a text classification method overall model by the similarity graph and the deep learning model, and outputting a classification result.
To implement this method, a python runtime environment is deployed and configured.
The method comprises the following steps: inputting a text and constructing a similarity graph;
as with the pixel neighborhood system processed using CNNs, the GCN network will pass information for each node to its neighborhood rather than processing each feature separately. This is why we have to construct a well organized graph to be able to better reveal the context between the texts. Also, a deep learning model is required for classification. Therefore, it is very important to select measures that can explain the similarity between nodes. Our method is based on a data set of N texts. Each text in the database has a d-dimensional tuple containing the base features. The construction of the adjacency matrix will use all this information and the result will be G ═ V, E. V is a set of nodes, each node representing a text. All nodes in the training set and the test set are included, and the number of elements in V is N. E is the set of edges in the graph, the existence of an edge between two nodes being determined by the similarity between the two nodes. Our goal is to perform binary classification of the nodes in the graph. We treat each text as a node in the graph, called ni. One text contains a set M with H-type features in M, i.e., M ═ Mh}. The similarity between nodes in the graph is defined as follows:
Figure BDA0002345705970000041
wherein Sim (A)v,Aw) Representing the similarity between node v and node w. The more similar the two nodes, the larger the value. In this formula, Mh(v) Representing the h-th feature of the v-th text. In processing the classification information, γ is defined as:
Figure BDA0002345705970000042
for quantitative measures, the definition of γ is somewhat different:
Figure BDA0002345705970000043
θ is a threshold that affects the result. Finally, we define dist (A)v,Aw) As follows:
Figure BDA0002345705970000044
in this formula, σ determines the width of the kernel, and x (v) is the feature vector of the v-th object. ρ represents a correlation distance. And the existence of an edge between two nodes is determined by the value of the similarity. If the similarity is greater than λ, an edge will be generated between these nodes. Our graph is composed of these edges. The adjacency matrix of the figure is denoted a.
Step two: constructing a deep learning model;
a simplified graph convolutional neural network and a graph attention neural network are used in our model. The structure of the simplified graph convolutional neural network is as follows:
Figure BDA0002345705970000051
wherein
Figure RE-GDA0002400755900000052
Represents the output of the convolutional layer, Θ is the trained parameter, X represents the input matrix of the network, consisting of the feature vectors of each text, i.e.: x ═ X1，...，xn]T. And S represents a normalized adjacency matrix with self-loop, and the expression is
Figure RE-GDA0002400755900000053
Wherein
Figure RE-GDA0002400755900000054
Figure RE-GDA0002400755900000055
Is that
Figure RE-GDA0002400755900000056
The degree matrix of (c).
This formula can be simplified as:
Figure BDA0002345705970000057
the structure of the graph attention network is as follows: in this layer, for each vector fed into the layer, the following formula will apply:
Figure BDA0002345705970000058
wherein the content of the first and second substances,
Figure BDA0002345705970000059
is the feature vector of the jth node after convolutional layer processing, and W is the training parameter. N is a radical ofiHere the neighborhood of the ith node in the figure, and αijIs defined as:
Figure BDA00023457059700000510
wherein, αijIs defined as:
Figure BDA00023457059700000511
step three: and forming a text classification method overall model by the similarity graph and the deep learning model, and outputting a classification result.
In summary, the general formula of this model can be written as:
Figure BDA0002345705970000061
Figure BDA0002345705970000062
and outputting the result of text classification by the model to the outside to obtain the required text classification result.

Claims (6)

1. A text classification method based on a graph convolution neural network is characterized in that: the method comprises the following steps:
the method comprises the following steps: inputting text and constructing a similarity graph. In a dataset containing a plurality of texts, each text has a multi-dimensional tuple comprising basic feature keywords. Constructing an adjacency matrix based on the text data set, defining each text as a node, determining the existence of an edge between two nodes by the similarity between the two nodes, and if the similarity is greater than a certain threshold value, generating an edge between the nodes so as to prejudge the node category in the graph and construct a similarity graph;
step two: constructing a deep learning model by using a simplified graph convolution neural network and a graph attention neural network and combining a Softmax function;
step three: and forming a text classification method overall model by the similarity graph and the deep learning model, and outputting the classified text, wherein the text classification method overall model is realized by using the simplified graph convolution neural network and the graph attention network and combining Softmax function calculation.
2. The method of claim 1, wherein the text classification method based on the atlas neural network is characterized in that: in the construction step of the similarity graph, the similarity calculation mode between two nodes is as follows:
Figure FDA0002345705960000011
3. the method of claim 2, wherein the text classification method based on the atlas neural network is characterized in that: in the similarity calculation mode between the two nodes:
γ is defined as:
Figure FDA0002345705960000012
for quantitative measurements, γ is defined as
Figure FDA0002345705960000013
Definition of dist (A)v,Aw) As follows:
Figure FDA0002345705960000014
4. the method of claim 3, wherein the text classification method based on the atlas neural network is characterized in that: the simplified graph convolution neural network calculation mode is defined as:
Figure FDA0002345705960000021
the graph attention network is defined as:
Figure FDA0002345705960000022
α thereinijIs defined as:
Figure FDA0002345705960000023
eijis defined as:
Figure FDA0002345705960000024
5. the method of claim 4, wherein the text classification method based on the atlas neural network is characterized in that: the overall model of the text classification method is as follows:
Figure FDA0002345705960000025
Figure FDA0002345705960000026
6. a text classification system based on a graph convolution neural network is characterized in that: the method comprises the following steps:
the information input module is used for standardizing and importing the source text acquired by the external database;
a text classification module based on the atlas neural network, which applies the text classification method based on the atlas neural network in any one of claims 1-5 to classify the input source text;
and the information output module is used for outputting the text classification result generated in the text classification module based on the atlas neural network in a new database form.
CN201911393728.4A 2019-12-30 2019-12-30 Text classification method and system based on graph convolution neural network Active CN111209398B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911393728.4A CN111209398B (en) 2019-12-30 2019-12-30 Text classification method and system based on graph convolution neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911393728.4A CN111209398B (en) 2019-12-30 2019-12-30 Text classification method and system based on graph convolution neural network

Publications (2)

Publication Number Publication Date
CN111209398A true CN111209398A (en) 2020-05-29
CN111209398B CN111209398B (en) 2023-01-17

Family

ID=70786507

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911393728.4A Active CN111209398B (en) 2019-12-30 2019-12-30 Text classification method and system based on graph convolution neural network

Country Status (1)

Country Link
CN (1) CN111209398B (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111737474A (en) * 2020-07-17 2020-10-02 支付宝(杭州)信息技术有限公司 Method and device for training business model and determining text classification category
CN112085161A (en) * 2020-08-20 2020-12-15 清华大学 Graph neural network method based on random information transmission
CN112163069A (en) * 2020-09-27 2021-01-01 广东工业大学 Text classification method based on graph neural network node feature propagation optimization
CN112687328A (en) * 2021-03-12 2021-04-20 北京贝瑞和康生物技术有限公司 Method, apparatus and medium for determining phenotypic information of clinical descriptive information
CN112766376A (en) * 2021-01-20 2021-05-07 重庆邮电大学 Multi-label eye fundus image identification method based on GACNN
CN113434668A (en) * 2021-05-18 2021-09-24 湘潭大学 Deep learning text classification method and system based on model fusion
WO2022257578A1 (en) * 2021-06-07 2022-12-15 京东科技信息技术有限公司 Method for recognizing text, and apparatus
US11562028B2 (en) 2020-08-28 2023-01-24 International Business Machines Corporation Concept prediction to create new intents and assign examples automatically in dialog systems
CN116226388A (en) * 2023-05-08 2023-06-06 浪潮电子信息产业股份有限公司 Literature classification method, graphic neural network training method and related components

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170308790A1 (en) * 2016-04-21 2017-10-26 International Business Machines Corporation Text classification by ranking with convolutional neural networks
CN109977232A (en) * 2019-03-06 2019-07-05 中南大学 A kind of figure neural network visual analysis method for leading figure based on power
US20190251480A1 (en) * 2018-02-09 2019-08-15 NEC Laboratories Europe GmbH Method and system for learning of classifier-independent node representations which carry class label information
CN110263799A (en) * 2019-06-26 2019-09-20 山东浪潮人工智能研究院有限公司 A kind of image classification method and device based on the study of depth conspicuousness similar diagram
CN110472003A (en) * 2019-08-08 2019-11-19 东北大学 Social networks text emotion fine grit classification method based on figure convolutional network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170308790A1 (en) * 2016-04-21 2017-10-26 International Business Machines Corporation Text classification by ranking with convolutional neural networks
US20190251480A1 (en) * 2018-02-09 2019-08-15 NEC Laboratories Europe GmbH Method and system for learning of classifier-independent node representations which carry class label information
CN109977232A (en) * 2019-03-06 2019-07-05 中南大学 A kind of figure neural network visual analysis method for leading figure based on power
CN110263799A (en) * 2019-06-26 2019-09-20 山东浪潮人工智能研究院有限公司 A kind of image classification method and device based on the study of depth conspicuousness similar diagram
CN110472003A (en) * 2019-08-08 2019-11-19 东北大学 Social networks text emotion fine grit classification method based on figure convolutional network

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111737474A (en) * 2020-07-17 2020-10-02 支付宝(杭州)信息技术有限公司 Method and device for training business model and determining text classification category
US11216620B1 (en) 2020-07-17 2022-01-04 Alipay (Hangzhou) Information Technology Co., Ltd. Methods and apparatuses for training service model and determining text classification category
CN112085161B (en) * 2020-08-20 2022-12-13 清华大学 Graph neural network method based on random information transmission
CN112085161A (en) * 2020-08-20 2020-12-15 清华大学 Graph neural network method based on random information transmission
US11562028B2 (en) 2020-08-28 2023-01-24 International Business Machines Corporation Concept prediction to create new intents and assign examples automatically in dialog systems
CN112163069A (en) * 2020-09-27 2021-01-01 广东工业大学 Text classification method based on graph neural network node feature propagation optimization
CN112163069B (en) * 2020-09-27 2024-04-12 广东工业大学 Text classification method based on graph neural network node characteristic propagation optimization
CN112766376A (en) * 2021-01-20 2021-05-07 重庆邮电大学 Multi-label eye fundus image identification method based on GACNN
CN112687328A (en) * 2021-03-12 2021-04-20 北京贝瑞和康生物技术有限公司 Method, apparatus and medium for determining phenotypic information of clinical descriptive information
CN113434668B (en) * 2021-05-18 2022-05-20 湘潭大学 Deep learning text classification method and system based on model fusion
CN113434668A (en) * 2021-05-18 2021-09-24 湘潭大学 Deep learning text classification method and system based on model fusion
WO2022257578A1 (en) * 2021-06-07 2022-12-15 京东科技信息技术有限公司 Method for recognizing text, and apparatus
CN116226388A (en) * 2023-05-08 2023-06-06 浪潮电子信息产业股份有限公司 Literature classification method, graphic neural network training method and related components

Also Published As

Publication number Publication date
CN111209398B (en) 2023-01-17

Similar Documents

Publication Publication Date Title
CN111209398B (en) Text classification method and system based on graph convolution neural network
US20220058426A1 (en) Object recognition method and apparatus, electronic device, and readable storage medium
CN110334742B (en) Graph confrontation sample generation method based on reinforcement learning and used for document classification and adding false nodes
JP7266674B2 (en) Image classification model training method, image processing method and apparatus
WO2021143267A1 (en) Image detection-based fine-grained classification model processing method, and related devices
CN109766557B (en) Emotion analysis method and device, storage medium and terminal equipment
CN105989288A (en) Deep learning-based malicious code sample classification method and system
Taesiri et al. Visual correspondence-based explanations improve AI robustness and human-AI team accuracy
US11637858B2 (en) Detecting malware with deep generative models
CN112418320A (en) Enterprise association relation identification method and device and storage medium
Xian et al. DeepEC: Adversarial attacks against graph structure prediction models
WO2023088174A1 (en) Target detection method and apparatus
CN111563373A (en) Attribute-level emotion classification method for focused attribute-related text
Petrosyan et al. Neural network integral representations with the ReLU activation function
CN112100377A (en) Text classification method and device, computer equipment and storage medium
Saitulasi et al. Deep Belief Network and Sentimental analysis for extracting on multi-variable Features to predict Stock market Performance and accuracy
Yu et al. Deep metric learning with dynamic margin hard sampling loss for face verification
Lin et al. Differential privacy protection over deep learning: An investigation of its impacted factors
Wu et al. Heterogeneous representation learning and matching for few-shot relation prediction
Bouke et al. BukaGini: a stability-aware Gini index feature selection algorithm for robust model performance
CN107070702B (en) User account correlation method and device based on cooperative game support vector machine
Wang et al. Instance-aware deep graph learning for multi-label classification
Chen et al. Understanding and improving adversarial transferability of vision transformers and convolutional neural networks
Wang et al. Variance of the gradient also matters: Privacy leakage from gradients
Wang Evaluation of four black-box adversarial attacks and some query-efficient improvement analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant