CN111209398B - Text classification method and system based on graph convolution neural network - Google Patents
Text classification method and system based on graph convolution neural network Download PDFInfo
- Publication number
- CN111209398B CN111209398B CN201911393728.4A CN201911393728A CN111209398B CN 111209398 B CN111209398 B CN 111209398B CN 201911393728 A CN201911393728 A CN 201911393728A CN 111209398 B CN111209398 B CN 111209398B
- Authority
- CN
- China
- Prior art keywords
- graph
- text
- neural network
- text classification
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 29
- 238000013136 deep learning model Methods 0.000 claims abstract description 12
- 238000004364 calculation method Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 2
- 238000005259 measurement Methods 0.000 claims description 2
- 239000013598 vector Substances 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 229910052739 hydrogen Inorganic materials 0.000 description 1
- 239000001257 hydrogen Substances 0.000 description 1
- 125000004435 hydrogen atom Chemical class [H]* 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000000547 structure data Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention designs a text classification method and a text classification system based on a graph convolution neural network through a graph neural network modeling method, wherein the method mainly comprises the following three steps: inputting a text and constructing a similarity graph; constructing a deep learning model; and forming a text classification method overall model by the similarity graph and the deep learning model, and performing text classification. Through the method, an efficient text classification method is provided based on the text similarity graph, the graph convolution network and the graph attention network, namely, the text can be automatically classified only by inputting a source text, and a system is formed by combining hardware on the basis of a model.
Description
Technical Field
The invention relates to the field of neural networks, in particular to a text classification method based on a graph convolution neural network.
Background
Recently, a great deal of research has been focused on data of a graph structure, which is a data structure that can be used in many ways. Graph-convolution networks (GCNs) provide us with a very efficient way to analyze graph structure data. It is a very powerful model using neighborhood information, and this special convolution has found wide application in research. It has done much work in several ways to date. Recent work has focused on the following four aspects: community detection using graphical methods, malware detection, object or significance detection in video or pictures, and internet security. There are several other aspects, such as computer vision and research into social networks. In this study, we mainly applied this model to the classification of text.
In the invention, a simplified graph convolution network and a graph attention network are compounded, the accuracy of classification is improved by utilizing the graph attention network while the calculation consumption of the graph convolution neural network is reduced, and a new method is introduced to construct a graph based on the similarity between graph nodes. Then, we import the graph into the network to complete text classification.
Disclosure of Invention
The text representation of the main problems of the existing text classification algorithm is high in dimension and sparsity, the feature expression capability is weak, in addition, the feature engineering needs to be carried out manually, and the cost is high. The deep learning initially succeeds in image and voice greatly, and accordingly promotes the development of the deep learning on NLP, so that the deep learning model has good effect on text classification.
In order to achieve the purpose, the invention adopts the following technical scheme:
a text classification method based on a graph convolution neural network comprises the following steps:
the method comprises the following steps: inputting texts, constructing a similarity graph, constructing an adjacency matrix in a multidimensional tuple database containing basic features in each text based on a plurality of text data sets, defining each text as a node, determining the existence of an edge between two nodes according to the similarity between the two nodes, and generating an edge between the nodes if the similarity is greater than a specific value so as to classify the nodes in the graph in a binary mode;
step two: constructing a deep learning model by using a simplified graph convolution neural network and a graph attention neural network and combining a Softmax function;
step three: and forming a text classification method overall model by the similarity graph and the deep learning model, and outputting the classified text, wherein the text classification method overall model is realized by using the simplified graph convolution neural network and the graph attention network and combining a Softmax function.
In the first step, the similarity calculation method between two nodes is as follows:
in the similarity calculation mode between the two nodes: γ is defined as:
further, for quantitative measurements, γ is defined as
Definition of dist (A) v ,A w ) As follows:
the simplified graph convolution neural network calculation mode is defined as:
the graph attention network is defined as:
wherein alpha is ij Is defined as:
e ij is defined as follows:
the overall model of the text classification method is as follows:
a system for text classification based on a graph-convolution neural network, comprising:
the information input module is used for importing the source text acquired by the external database after normalization;
the text classification module based on the graph convolution neural network is used for classifying the input source text by applying the text classification method based on the graph convolution neural network;
and the information output module is used for outputting the text classification result generated in the text classification module based on the atlas neural network in a new database form.
According to the technical scheme, a novel model called ASGCN is provided to classify different texts. In this model, we combine a simplified graph convolution network and a graph attention network to get better classification results. Furthermore, we propose a new way of constructing the graph, so that it can better fit the proposed model and provide us with better results, thus enabling the following effects to be achieved:
1. pre-judging the text category through the correlation relationship among the texts;
2. a new model based on the graph neural network is constructed to improve the classification precision.
Detailed Description
The following is a preferred embodiment of the present invention, and the technical solution of the present invention is further described, but the present invention is not limited to this embodiment.
The text classification method based on the graph convolution neural network in the embodiment mainly comprises the following three steps:
the method comprises the following steps: inputting a text and constructing a similarity graph;
step two: constructing a deep learning model;
step three: and forming a text classification method overall model by the similarity graph and the deep learning model, and outputting a classification result.
To implement this method, a python runtime environment is deployed and configured.
The method comprises the following steps: inputting a text and constructing a similarity graph;
as with the pixel neighborhood system processed using CNNs, the GCN network will pass information for each node to its neighborhood rather than processing each feature separately. This is why we have to construct a well organized graph to be able to better reveal the context between the texts. Also, a deep learning model is required for classification. Therefore, it is very important to select a measure capable of explaining the similarity between nodes. Our method is based on a dataset of N texts. Each text in the database has a d-dimensional tuple containing the base features. The construction of the adjacency matrix will use all this information and the result will be G = (V, E). V is a set of nodes, each node representing a text. All nodes in the training set and the test set are included, and the number of elements in V is N. E is the set of edges in the graph, and the existence of an edge between two nodes is determined by the similarity between the two nodes. Our goal is to binary classify the nodes in the graph. We consider each text as a node in the graph, called n i . One text contains a set M of H-type phenotype features, i.e., M = { M = { (M) } h }. The similarity between nodes in the graph is defined as follows:
wherein Sim (A) v ,A w ) Representing the similarity between node v and node w. The more similar the two nodes, the larger the value. In this formula, M h (v) Representing the h-th feature of the v-th text. In processing the classification information, γ is defined as:
for quantitative measures, the definition of γ is somewhat different:
θ is a threshold that affects the result. Finally, we define dist (A) v ,A w ) As follows:
in this formula, σ determines the width of the kernel, and x (v) is the feature vector of the v-th object. ρ represents the correlation distance. And the existence of an edge between two nodes is determined by the value of the similarity. If the similarity is greater than λ, an edge will be generated between these nodes. Our graph is composed of these edges. The adjacency matrix of the figure is denoted a.
Step two: constructing a deep learning model;
a simplified graph convolutional neural network and a graph attention neural network are used in our model. The structure of the simplified graph convolutional neural network is as follows:
wherein
Represents the output of the convolutional layer, Θ is the trained parameter, X represents the input matrix of the network, consisting of the feature vectors of each text, i.e.: x = [ X ] 1 ，...，x n ] T . And S represents a normalized adjacency matrix with self-loops, whose expression is
Wherein
Is that
The degree matrix of (c).
This formula can be simplified as:
the structure of the graph attention network is as follows: in this layer, for each vector fed into the layer, the following formula will apply:
wherein,
is the feature vector of the jth node after convolutional layer processing, and W is the training parameter. N is a radical of hydrogen i Here the neighborhood of the ith node in the figure, and alpha ij Is defined as:
wherein alpha is ij Is defined as:
step three: and forming a text classification method overall model by the similarity graph and the deep learning model, and outputting a classification result.
In summary, the general formula of this model can be written as:
and outputting the result of text classification by the model to the outside to obtain the required text classification result.
Claims (3)
1. A text classification method based on a graph convolution neural network is characterized in that: the method comprises the following steps:
the method comprises the following steps: inputting a text, constructing a similarity graph, wherein each text has a multi-dimensional tuple containing basic characteristic keywords in a data set containing a plurality of texts, constructing an adjacency matrix based on the text data set, defining each text as a node, determining the existence of an edge between two nodes according to the similarity between the two nodes, and if the similarity is greater than a certain threshold value, generating an edge between the nodes so as to prejudge the node category in the graph and construct the similarity graph;
step two: constructing a deep learning model by using a simplified graph convolution neural network and a graph attention neural network and combining a Softmax function;
step three: constructing a text classification method overall model by a similarity graph and a deep learning model, and outputting classified texts, wherein the text classification method overall model is realized by using the simplified graph convolution neural network and the graph attention network and combining Softmax function calculation; the simplified graph convolution neural network calculation mode is defined as:
the graph attention network is defined as:
wherein alpha is ij Is defined as follows:
e ij is defined as:
in the construction step of the similarity graph, the similarity calculation mode between two nodes is as follows:
in the similarity calculation mode between the two nodes:
γ is defined as:
for quantitative measurements, γ is defined as
Definition of dist (A) v ,A w ) As follows:
2. the method of claim 1, wherein the text classification method based on the atlas neural network is characterized in that:
the overall model of the text classification method is as follows:
3. a text classification system based on a graph convolution neural network is characterized in that: the method comprises the following steps:
the information input module is used for standardizing and importing the source text acquired by the external database;
a text classification module based on the atlas neural network, which applies the text classification method based on the atlas neural network in any one of claims 1-2 to classify the input source text;
and the information output module is used for outputting the text classification result generated in the text classification module based on the atlas neural network in a new database form.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911393728.4A CN111209398B (en) | 2019-12-30 | 2019-12-30 | Text classification method and system based on graph convolution neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911393728.4A CN111209398B (en) | 2019-12-30 | 2019-12-30 | Text classification method and system based on graph convolution neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111209398A CN111209398A (en) | 2020-05-29 |
| CN111209398B true CN111209398B (en) | 2023-01-17 |
Family
ID=70786507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911393728.4A Active CN111209398B (en) | 2019-12-30 | 2019-12-30 | Text classification method and system based on graph convolution neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111209398B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111737474B (en) * | 2020-07-17 | 2021-01-12 | 支付宝(杭州)信息技术有限公司 | Method and device for training business model and determining text classification category |
| CN112085161B (en) * | 2020-08-20 | 2022-12-13 | 清华大学 | Graph neural network method based on random information transmission |
| US11562028B2 (en) | 2020-08-28 | 2023-01-24 | International Business Machines Corporation | Concept prediction to create new intents and assign examples automatically in dialog systems |
| CN112163069B (en) * | 2020-09-27 | 2024-04-12 | 广东工业大学 | Text classification method based on graph neural network node characteristic propagation optimization |
| CN112766376A (en) * | 2021-01-20 | 2021-05-07 | 重庆邮电大学 | Multi-label eye fundus image identification method based on GACNN |
| CN112687328B (en) * | 2021-03-12 | 2021-08-31 | 北京贝瑞和康生物技术有限公司 | Method, apparatus and medium for determining phenotypic information of clinical descriptive information |
| CN113434668B (en) * | 2021-05-18 | 2022-05-20 | 湘潭大学 | Deep learning text classification method and system based on model fusion |
| CN115457531A (en) * | 2021-06-07 | 2022-12-09 | 京东科技信息技术有限公司 | Method and device for recognizing text |
| CN116226388B (en) * | 2023-05-08 | 2023-07-21 | 浪潮电子信息产业股份有限公司 | Literature classification method, graphic neural network training method and related components |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977232A (en) * | 2019-03-06 | 2019-07-05 | 中南大学 | A kind of figure neural network visual analysis method for leading figure based on power |
| CN110263799A (en) * | 2019-06-26 | 2019-09-20 | 山东浪潮人工智能研究院有限公司 | A kind of image classification method and device based on the study of depth conspicuousness similar diagram |
| CN110472003A (en) * | 2019-08-08 | 2019-11-19 | 东北大学 | Social networks text emotion fine grit classification method based on figure convolutional network |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170308790A1 (en) * | 2016-04-21 | 2017-10-26 | International Business Machines Corporation | Text classification by ranking with convolutional neural networks |
| US20190251480A1 (en) * | 2018-02-09 | 2019-08-15 | NEC Laboratories Europe GmbH | Method and system for learning of classifier-independent node representations which carry class label information |
-
2019
- 2019-12-30 CN CN201911393728.4A patent/CN111209398B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977232A (en) * | 2019-03-06 | 2019-07-05 | 中南大学 | A kind of figure neural network visual analysis method for leading figure based on power |
| CN110263799A (en) * | 2019-06-26 | 2019-09-20 | 山东浪潮人工智能研究院有限公司 | A kind of image classification method and device based on the study of depth conspicuousness similar diagram |
| CN110472003A (en) * | 2019-08-08 | 2019-11-19 | 东北大学 | Social networks text emotion fine grit classification method based on figure convolutional network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111209398A (en) | 2020-05-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111209398B (en) | Text classification method and system based on graph convolution neural network | |
| US12099577B2 (en) | Object recognition method and apparatus, electronic device, and readable storage medium | |
| US20200311467A1 (en) | Generating multi modal image representation for an image | |
| WO2021189922A1 (en) | Method and apparatus for generating user portrait, and device and medium | |
| CN109766557B (en) | Emotion analysis method and device, storage medium and terminal equipment | |
| WO2023088174A1 (en) | Target detection method and apparatus | |
| Petrosyan et al. | Neural network integral representations with the ReLU activation function | |
| CN110516950A (en) | A kind of risk analysis method of entity-oriented parsing task | |
| Saitulasi et al. | Deep Belief Network and Sentimental analysis for extracting on multi-variable Features to predict Stock market Performance and accuracy | |
| CN114154557A (en) | Cancer tissue classification method, apparatus, electronic device, and storage medium | |
| Yu et al. | Deep metric learning with dynamic margin hard sampling loss for face verification | |
| Lin et al. | Differential privacy protection over deep learning: An investigation of its impacted factors | |
| Wu et al. | Heterogeneous representation learning and matching for few-shot relation prediction | |
| CN112489689B (en) | Cross-database voice emotion recognition method and device based on multi-scale difference countermeasure | |
| CN111858999B (en) | Retrieval method and device based on segmentation difficult sample generation | |
| Wang et al. | Instance-aware deep graph learning for multi-label classification | |
| Ahan et al. | Social network analysis using data segmentation and neural networks | |
| Wang et al. | Variance of the gradient also matters: Privacy leakage from gradients | |
| Shim et al. | Fast and accurate interpretation of workload classification model | |
| CN116029760A (en) | Message pushing method, device, computer equipment and storage medium | |
| Wang et al. | Sequential safe feature elimination rule for l1-regularized regression with kullback–leibler divergence | |
| Ahmad et al. | Customer Personality Analysis for Churn Prediction Using Hybrid Ensemble Models and Class Balancing Techniques | |
| CN110457543B (en) | Entity resolution method and system based on end-to-end multi-view matching | |
| Tomar | A critical evaluation of activation functions for autoencoder neural networks | |
| Chin-Purcell et al. | Investigating accuracy disparities for gender classification using convolutional neural networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |