CN115858792B - Short text classification method and system for bidding project names based on graphic neural network - Google Patents
Short text classification method and system for bidding project names based on graphic neural network Download PDFInfo
- Publication number
- CN115858792B CN115858792B CN202310132159.8A CN202310132159A CN115858792B CN 115858792 B CN115858792 B CN 115858792B CN 202310132159 A CN202310132159 A CN 202310132159A CN 115858792 B CN115858792 B CN 115858792B
- Authority
- CN
- China
- Prior art keywords
- graph
- semantic
- sequence
- neural network
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of data processing, and discloses a method and a system for classifying short names of bid-inviting projects based on a graph neural network; the method comprises the following steps: acquiring the names of bid-inviting items to be classified; word segmentation processing is carried out on the text; inputting the word segmentation result into a trained short text classification model, and outputting a classification result; the trained short text classification model extracts a feature matrix of the semantic graph from the segmentation result; constructing a sequence chart of the word segmentation result to obtain a feature matrix of the sequence chart; extracting text features from the word segmentation result, and mapping the text features into feature matrixes of the semantic graphs and feature matrixes of the sequence graphs respectively to obtain mapped feature matrixes of the semantic graphs and mapped feature matrixes of the sequence graphs; realizing intra-graph propagation and inter-graph propagation of the semantic graph and the sequence graph, and classifying the short texts of the names of the bid-inviting items to obtain classification labels; the invention can solve the problems of low efficiency and serious waste of information resources of the existing manual labeling.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a method and a system for classifying short names of bid-drawing projects based on a graphic neural network.
Background
The statements in this section merely relate to the background of the present disclosure and may not necessarily constitute prior art.
With the rapid development of information technology and the perfection of bidding standardization system, besides secret items, bidding parties generally choose to issue bidding notices from bidding systems or websites, so that bidding parties bid and bid, and thus, trade is achieved rapidly.
The bidding system or the website can release massive bidding information every day, the updating speed is faster and faster, and the bidding content also relates to aspects of life. Sometimes, the bidder misses the item of his own heart instrument with little care. At present, in the face of massive bid information released by bid systems and websites, bidders often choose to arrange staff to view bid file notices of each bid system and website multiple times a day, and the staff or a department leader thereof screens and analyzes the bid file notices from numerous and numerous scattered bid file notices to judge bid information closely related to own main business.
Because the working capacity of the bidder staff and the familiarity degree of the bidder staff to the business are different, bid-inviting items and files screened by different staff are often different facing the same quantity of bid-inviting information; secondly, the nonstandard and ambiguous bid information classification and long-time work fatigue of bidders can greatly reduce the efficiency and accuracy of bid information resource allocation. Therefore, in the bid-bidding system based on informatization, the bidding party and the platform all invest a great deal of manpower, financial resources and material resources, but the efficiency of bid-bidding information resource allocation cannot be improved any more, and the information resource allocation efficiency is further reduced along with the increase of bid-bidding information release. Therefore, a system and a method for rapidly and accurately labeling and classifying bidding documents and notifying website-published bidding documents are developed, so that the problem of low allocation efficiency of massive bidding information resources is solved, and the need is felt.
Disclosure of Invention
In order to solve the defects of the prior art, the invention provides a method and a system for classifying short names of bidding projects based on a graph neural network; the text context information is obtained by using the bidirectional LSTM, and meanwhile, a double-graph structure (the double-graph structure refers to a semantic graph structure and a sequence graph structure) is constructed, information between different granularities in the text is captured by utilizing intra-graph propagation and inter-graph propagation of the graph neural network, isomerism of the information is overcome, so that higher-hierarchy structure information is obtained, automatic labeling and classification can be carried out on bidding projects, and the problems of low efficiency of existing manual labeling and serious waste of information resources are solved.
In a first aspect, the invention provides a method for classifying short text of a bid term name based on a graph neural network;
the method for classifying the short text of the name of the bidding project based on the graph neural network comprises the following steps:
acquiring the names of bid-inviting items to be classified;
preprocessing the names of the bid-inviting projects to be classified, and performing word segmentation on the preprocessed text;
inputting the word segmentation result into a trained short text classification model, and outputting a classification result;
wherein the trained short text classification model comprises:
extracting a feature matrix of the semantic graph from the segmentation result;
constructing a sequence diagram of the word segmentation result, and further obtaining an adjacent matrix of the sequence diagram; initializing an adjacent matrix of the sequence diagram to obtain a feature matrix of the sequence diagram;
extracting text features from the word segmentation result, and caching the text features into a memory bank;
mapping text features into feature matrixes of the semantic graphs and feature matrixes of the sequence graphs respectively to obtain mapped feature matrixes of the semantic graphs and mapped feature matrixes of the sequence graphs;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph, the mapped semantic graph feature matrix and the mapped sequence graph feature matrix, the intra-graph propagation of the semantic graph and the sequence graph is realized;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph and the intra-graph propagation result, inter-graph propagation of the semantic graph and the sequence graph is realized, and the inter-graph propagation result is classified to obtain a classification label.
In a second aspect, the invention provides a short text classification system of the bid term name based on a graphic neural network;
a graph neural network based short text classification system for bid term names, comprising:
an acquisition module configured to: acquiring the names of bid-inviting items to be classified;
a preprocessing module configured to: preprocessing the names of the bid-inviting projects to be classified, and performing word segmentation on the preprocessed text;
a classification module configured to: inputting the word segmentation result into a trained short text classification model, and outputting a classification result;
wherein the trained short text classification model comprises:
extracting a feature matrix of the semantic graph from the segmentation result;
constructing a sequence diagram of the word segmentation result, and further obtaining an adjacent matrix of the sequence diagram; initializing an adjacent matrix of the sequence diagram to obtain a feature matrix of the sequence diagram;
extracting text features from the word segmentation result, and caching the text features into a memory bank;
mapping text features into feature matrixes of the semantic graphs and feature matrixes of the sequence graphs respectively to obtain mapped feature matrixes of the semantic graphs and mapped feature matrixes of the sequence graphs;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph, the mapped semantic graph feature matrix and the mapped sequence graph feature matrix, the intra-graph propagation of the semantic graph and the sequence graph is realized;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph and the intra-graph propagation result, inter-graph propagation of the semantic graph and the sequence graph is realized, and the inter-graph propagation result is classified to obtain a classification label.
Compared with the prior art, the invention has the beneficial effects that:
the method solves the problem that the bidding party cannot accurately acquire bidding projects meeting the needs of the bidding party in real time from a plurality of bidding information, can analyze the demand trend of the bidding projects in the purchasing market, and effectively improves the resource allocation efficiency of the bidding party; according to the classification result of the invention, the bidder can reclassify the bidding purchasing information so as to realize the improvement of the differentiation degree of classifying different types of bidding information; the invention greatly reduces the manual marking cost of bidding information by a bidding party, effectively reduces the marking error of bidding items caused by manual subjectivity and improves the classification accuracy.
The bidirectional LSTM technology in the invention can efficiently acquire text context information, effectively acquire information between different granularities in the bidding information text and overcome the isomerism thereof by means of a constructed double-graph structure, namely by utilizing the intra-graph propagation and inter-graph propagation technologies of the graph neural network, and acquire higher-level structural information through fusion, thereby being more superior in processing short text classification like bidding project name class.
The invention also discloses a memory bank technology for caching the inductive output result. The technology utilizes a simplified graph rolling network (SGC) in a transduction network and a bidirectional long-short term memory network (LSTM) in a induction network to replace an embedded vector of a text node by a cached updated bidirectional LSTM characteristic, and performs transduction operation in the simplified graph rolling network, so that the technology is used for learning the global structural characteristic of a text, and the short text classification calculation of the names of bidding projects is realized more efficiently.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
FIG. 1 is a flow chart of a method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a network structure according to an embodiment of the present invention.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular forms also are intended to include the plural forms, and furthermore, it is to be understood that the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, such as, for example, processes, methods, systems, products or devices that comprise a series of steps or units, are not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or inherent to such processes, methods, products or devices.
Embodiments of the invention and features of the embodiments may be combined with each other without conflict.
All data acquisition in the embodiment is legal application of the data on the basis of meeting laws and regulations and agreements of users.
Example 1
The embodiment provides a labeling project name short text classification method based on a graph neural network;
as shown in fig. 1, the method for classifying short text of a bid term name based on a graph neural network comprises the following steps:
s101: acquiring the names of bid-inviting items to be classified;
s102: preprocessing the names of the bid-inviting projects to be classified, and performing word segmentation on the preprocessed text;
s103: inputting the word segmentation result into a trained short text classification model, and outputting a classification result;
s104: the trained short text classification model is used for extracting a feature matrix of the semantic graph from the word segmentation result; constructing a sequence diagram of the word segmentation result, and further obtaining an adjacent matrix of the sequence diagram; initializing an adjacent matrix of the sequence diagram to obtain a feature matrix of the sequence diagram; extracting text features from the word segmentation result, and caching the text features into a memory bank;
s105: the model is also used for mapping the text features into the feature matrix of the semantic graph and the feature matrix of the sequence graph respectively to obtain a mapped semantic graph feature matrix and a mapped sequence graph feature matrix;
s106: the model is also used for realizing intra-graph propagation of the semantic graph and the sequence graph based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph, the mapped semantic graph feature matrix and the mapped sequence graph feature matrix;
s107: the model is also used for realizing inter-graph propagation of the semantic graph and the sequence graph based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph and the intra-graph propagation result, and classifying the inter-graph propagation result to obtain a classification label.
Further, the feature matrix of the semantic graph is extracted from the word segmentation result, which specifically comprises the following steps:
extracting semantic features of each word and context information between the words, and constructing a semantic graph based on the semantic features of all the words and the context information between the words so as to obtain an adjacency matrix of the semantic graph; initializing the adjacent matrix of the semantic graph to obtain the feature matrix of the semantic graph.
Further, the semantic graph-based adjacency matrix, the mapped semantic graph feature matrix, the sequence graph adjacency matrix and the mapped sequence graph feature matrix realize intra-graph propagation of the semantic graph and the sequence graph, and specifically comprise:
inputting an adjacent matrix of the semantic graph and a mapped semantic graph feature matrix into a first graph neural network, and carrying out intra-graph propagation to obtain a first final representation after intra-graph propagation;
and inputting the adjacency matrix of the sequence diagram and the mapped sequence diagram feature matrix into a first diagram neural network to carry out intra-diagram propagation, so as to obtain a second final representation after intra-diagram propagation.
Further, the semantic graph-based adjacency matrix, the sequence graph-based adjacency matrix and the intra-graph propagation result realize inter-graph propagation of the semantic graph and the sequence graph, and specifically comprise:
inputting the first final representation after the intra-graph propagation and the single-graph adjacency matrix after the combination into a second graph neural network, and carrying out inter-graph propagation to obtain a first intermediate value after the inter-graph propagation; the combined single-graph adjacency matrix refers to the weighted summation result of the adjacency matrix of the semantic graph and the adjacency matrix of the sequence graph;
inputting the second final representation after the intra-graph propagation and the single-graph adjacency matrix after the combination into a second graph neural network, and carrying out inter-graph propagation to obtain a second intermediate value after the inter-graph propagation;
summing the first intermediate value transmitted between the graphs and the first final representation transmitted in the graphs to obtain a first final representation of heterogeneous information between the fused semantic graph and the sequential graph;
and summing the second intermediate value after the inter-graph propagation with the second final representation after the intra-graph propagation to obtain a second final representation of the heterogeneous information between the fused semantic graph and the sequential graph.
Further, the classifying the inter-graph propagation result to obtain a classification label specifically includes:
and merging the first final representation of the heterogeneous information between the fusion semantic graph and the sequential graph with the second final representation of the heterogeneous information between the fusion semantic graph and the sequential graph according to dimensions, inputting the merged data into a mean value pooling layer for processing, inputting an output result of the mean value pooling layer into a classification layer, and outputting a classification label.
Further, the preprocessing of the names of the bid projects to be classified specifically includes:
and cleaning the crawled names of the to-be-classified bid-inviting projects by adopting a regular expression.
Further, the word segmentation processing for the preprocessed text specifically includes:
and performing word segmentation on the preprocessed text by adopting a Jieba word segmentation mode.
Illustratively, considering the characteristic of short text of the name of the bidding project, to preserve the integrity of text semantics, only using a regular expression of remodule in Python to wash the crawled data, and then using a jieba () function to perform word segmentation on the washed text;
taking a crawled bid item name as an example: "Fushan region 100MW/200MWh centralized (shared) energy storage power station project total contractor (EPC) bid advertisement", the result after regular cleaning using remodule is "Fushan region centralized shared energy storage power station project total contractor bid advertisement". Word segmentation is carried out on the cleaned result, wherein the content is as follows: 'Fushan region centralized shared energy storage power station project engineering general contractual bulletin'.
Further, the short text classification model has the same network structure as the trained short text classification model.
Further, as shown in fig. 2, the trained short text classification model has a network structure specifically including:
three parallel branches: a first branch, a second branch, and a third branch;
the first branch comprises: the system comprises a first bidirectional long-short-term memory cyclic neural network, a semantic graph construction module, a first feature initialization module and a first mapping module which are connected in sequence;
the second branch comprises: the sequence diagram construction module, the second characteristic initialization module and the second mapping module are sequentially connected;
the third branch comprises: the second bidirectional long and short-term memory cyclic neural network and the characteristic cache module are connected in sequence;
the output end of the characteristic cache module is respectively connected with the input end of the first mapping module and the input end of the second mapping module; the output end of the first mapping module and the output end of the second mapping module are connected with the input end of the first graph neural network;
the output end of the first graph neural network is connected with the input end of the second graph neural network; the output end of the first graph neural network is connected with the input end of the adder; the output end of the second graph neural network is connected with the input end of the adder, and the output end of the adder is connected with the input end of the average value pooling layer; the output end of the averaging layer is connected with the input end of the classifying layer.
Further, the first bidirectional long-short-time memory cyclic neural network is used for capturing semantic features of each word and context information among the words in the segmented text.
Further, the second bidirectional long short-time memory cyclic neural network is used for capturing text characteristics of the segmented text.
It should be appreciated that the segmented data is sized in batch size
Inputting to the second bidirectional long-short time memory cyclic neural network, and adding the final layer of the generated forward hidden layer at the final moment +.>
And last layer last moment of backward hidden layer
Merging into text feature vector->
The second bidirectional LSTM is used for acquiring batch text features and complementing randomness of initialization of the double-diagram text features.
Further, the semantic graph construction module includes:
both the word and the document are regarded as nodes in the graph;
establishing two edges between nodes, wherein the first edge is the edge between a document node and a word node; the second type of edge refers to an edge between word nodes and word nodes; whether a connecting edge exists between the nodes is determined by the feature cosine similarity of the nodes, if the feature cosine similarity is larger than a set threshold, the connecting edge exists between the nodes, otherwise, the connecting edge does not exist;
weights are set for all the words and the edges between the words, and meanwhile weights are set for the edges between the document nodes and the word nodes, so that a constructed semantic graph is obtained; wherein the weight of the edge between the words is the ratio of the first value to the second value; the first numerical value refers to the number of times that two words have semantic relationships in all documents of the corpus; the second value refers to the number of times two words appear in all documents of the corpus; the weight of an edge between a document node and a word node is the word frequency-inverse document frequency.
Illustratively, the semantic graph construction module includes, in its working details:
firstly, inputting a preprocessed word segmentation result into a first bidirectional long-short-time memory cyclic neural network LSTM to perform short text classification pre-training of the names of the bid-in projects to obtain semantic features of each word; the name of the bidding project is short text, and long-distance dependency relation among text words is not provided, so that the first bidirectional long-short-time memory cyclic neural network is used for capturing context information among each word, and richer word characteristics are obtained.
And calculating the feature cosine similarity between word pairs, wherein the formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
and->
Word output for last layer of first bidirectional LSTM modelThe characteristics of i and the word j,
and->
The modulo, i.e. length, of the word i and word j features.
If the feature cosine similarity between word pairs exceeds a predefined threshold, it means that the two words have a semantic relationship in the current document.
All words in the bid term name corpus are used as word nodes in the semantic graph structure, and the edge weight calculation formula between the words is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is word pair +.>
Number of times of semantic relation in all documents in corpus,/for all documents in corpus>
Is word pair +.>
Number of occurrences in all documents.
And generating an adjacency matrix of the corresponding graph structure by the word and the setting mode of the edge weight between the word and the text.
Further, the sequence diagram construction module includes:
both the word and the document are regarded as nodes in the graph;
establishing two edges between nodes, wherein the first edge is the edge between a document node and a word node; the second type of edge refers to an edge between word nodes and word nodes; whether a connecting edge exists between the nodes is determined by an edge weight, if the edge weight is greater than a set threshold, the connecting edge exists between the nodes, and if the edge weight is less than or equal to the set threshold, the connecting edge does not exist between the nodes; the edge weight is calculated by adopting point-by-point mutual information;
setting weights for edges between word nodes and word nodes, and setting weights for edges between document nodes and word nodes to obtain a constructed sequence diagram; the weight of the word node and the edge between the word nodes is point-by-point mutual information; the weight of an edge between a document node and a word node is the word frequency-inverse document frequency.
Illustratively, the sequence diagram construction module includes:
all words in the corpus are used as word nodes in the sequence diagram;
the edge weight between words is calculated as PMI (PointwiseMutual Information, point-by-point mutual information), and the formula is as follows:
obtaining sequence context information for local co-occurrence between name words of the pre-processed bid term through a sliding window,
is word pair +.>
Probability of simultaneous occurrence in the same sliding window, < >>
Is to simultaneously contain the word->
And->
The number of sliding windows +.>
Is the total number of sliding windows in a single text. When the PMI value is positive, representing that the two words have high semantic relativity, adding an edge between the two word nodes, otherwise, not adding the edge. Wherein, because the short text length is shorter, the local co-occurrence of word pairs can be collected by using a sliding window with shorter length, and the weight of the word pairs in the short text is mined.
And setting an edge weight as a TF-IDF (term frequency-inverse document frequency) value between the text node and the word node of the sequence diagram, wherein the TF-IDF formula for setting the edge weight is as follows:
and generating an adjacency matrix of the corresponding graph structure by the word and the setting mode of the edge weight between the word and the text.
Further, the first feature initialization module includes:
and initializing an adjacent matrix of the semantic graph by adopting a single-heat coding mode to obtain a feature matrix of the semantic graph.
Further, the second feature initialization module includes:
and initializing an adjacent matrix of the sequence diagram by adopting a single-heat coding mode to obtain a characteristic matrix of the sequence diagram.
Further, the feature cache module includes:
and caching the text characteristics output by the second bidirectional long-short time memory cyclic neural network and the text serial numbers corresponding to the text characteristics into a memory bank.
Illustratively, text feature vectors
The text serial numbers corresponding to the features are cached in a repository, so that the features and the positions of the texts in the corpus can be clearly found out in the subsequent extraction.
It should be understood that, the simplified graph convolution network performs full-batch operation on all texts and words in the corpus, and does not consider the distinction between the training set and the test set, namely, transduction operation, but due to the memory limitation, the bidirectional LSTM can only select a part of texts at a time and cannot perform full-batch operation, so that the proposed repository can cache the text features and the text sequence numbers of the bidirectional LSTM for batch operation so as to find and replace the text features of the corresponding graph neural network, thereby ensuring that the bidirectional LSTM and the simplified graph convolution network can perform serial operation better and reduce the system memory occupation, and accelerating the training speed.
Further, the first mapping module includes:
extracting text features and corresponding text serial numbers from a memory bank; according to the text sequence number, mapping the text features into a semantic graph feature matrix to cover the original initialization single-hot vector corresponding to the text sequence number, and mapping the original semantic graph feature matrix
Conversion to mapped feature matrix +.>
。
Further, the second mapping module includes:
extracting text features and corresponding text serial numbers from a memory bank; according to the text serial number, mapping the text features into a sequence diagram feature matrix to cover the original initialization single-hot vector corresponding to the text serial number, thereby mapping the original sequence diagram feature matrix
Conversion to mapped feature matrix +.>
。
Further, the first graph neural network includes:
mapping the feature matrix
And->
And adjacency matrix->
And->
While being input into a first reduced graph convolution network (SGC, simplifyingGraphConvolutionalNetworks) for intra-graph propagation.
Further, the intra-graph propagation is a mapping of the feature matrix
And->
And adjacent matrix
And->
And performing simple graph convolution operation to obtain the in-graph high-order neighbor information of each node, wherein the formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
for simple graph the number of layers is laminated +.>
And->
For a trainable weight matrix, +.>
And->
For the final representation of text nodes and word nodes after propagation through the graph, < >>
To hide layer dimension, ++>
The formula of the adjacency matrix tensor after normalization operation for the adjacency matrix of the double-graph structure is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is a diagonal node degree matrix, ">
Is an identity matrix.
Further, the second graph neural network includes:
adjacency matrix for semantic graph
Adjacency matrix of sequence diagram->
And the output value of the first reduced graph convolutional network is input to the first reduced graph convolutional network simultaneouslyInter-graph propagation is performed in a second reduced graph convolutional network (SGC).
Further, the inter-graph propagation refers to:
combining the semantic graph and the sequence graph into a single graph for inter-graph propagation, wherein the formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
and->
For a trainable weight matrix, +.>
Is a matrix dot product operator.
Combining the single image adjacency matrix
Generated by propagation in the graph->
And->
Inputting the feature matrix into a second simplified graph convolution network, so that node information between semantic graphs and sequential graphs is subjected to inter-graph propagation exchange, heterogeneous information between different graphs is fused into one-generation information, and residual connection is added between a first simplified graph convolution network and a second simplified graph convolution network; the formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
and->
Is a trainable weight matrix, wherein,
and->
And fusing the final representation of the heterogeneous information between the semantic graph and the sequence graph for all the nodes through inter-graph propagation.
It should be understood that, since text nodes and word nodes exist in the semantic graph and the sequence graph, and two different types of edges of text-word and word-word exist in the semantic graph and the sequence graph, the semantic graph and the sequence graph have strong heterogeneity, a learnable edge weight strategy is adopted to coordinate weights between different nodes and different edges of the double graph, and residual connection is added, so that after heterogeneous information is exchanged between propagation nodes between graphs, original node information can be still kept in the graphs, on the other hand, generalization capability of a model can be enhanced, and gradient disappearance is avoided.
Further, the averaging layer and the classifying layer comprise:
after the inter-graph propagation is completed, the last layer of double-graph feature matrix of the second reduced graph convolutional network is subjected to a mean value pooling operation, and a fully-connected classification layer is connected to obtain the final representation of the node. The formula is as follows:
wherein, ": "merging the features of the last layer of double graph according to dimensions, mean is a Mean function, FC is a fully connected classification layer,
for the final representation of the node +.>
Is the number of categories for which the bid item names are classified.
Further, as shown in fig. 2, the training process of the trained short text classification model includes:
constructing a training set, wherein the training set is the name of a bid-inviting item with a known classification result;
and inputting the training set into the short text classification model, training the model, and stopping training when the total loss function value of the model is not reduced, or the training iteration number exceeds a set threshold value, so as to obtain the trained short text classification model.
Further, the word segmentation result is achieved through a first bidirectional long-short-time memory cyclic neural network by extracting semantic features of each word and context information among the words.
Further, the construction of the semantic graph based on the semantic features of all words and the context information between the words is realized through a semantic graph construction module.
Further, the adjacency matrix for obtaining the semantic graph further refers to an adjacency matrix corresponding to the obtained semantic graph according to the edge weights between the words and the documents in the semantic graph.
Further, initializing the adjacency matrix of the semantic graph to obtain the feature matrix of the semantic graph, which specifically comprises:
the semantic graph adjacency matrix is expressed as
Initializing a semantic graph adjacent matrix by using one-hot coding to obtain a feature matrix of the semantic graph>
The method comprises the steps of carrying out a first treatment on the surface of the n is the number of texts in the corpus, m is the number of words in the corpus, and d is the feature dimension of the node.
Further, the segmentation result is used for constructing a sequence chart, so that an adjacency matrix of the sequence chart is obtained, and the adjacency matrix is realized through a sequence chart construction module.
Further, initializing the adjacency matrix of the sequence diagram to obtain the feature matrix of the sequence diagram, which specifically comprises:
the sequence diagram adjacency matrix is expressed as
Obtaining a feature matrix of the sequence diagram by using the adjacency matrix of the single thermal encoding pair sequence diagram>
The method comprises the steps of carrying out a first treatment on the surface of the n is the number of texts in the corpus, m is the number of words in the corpus, and d is the feature dimension of the node.
Further, text features are extracted from the word segmentation result, and the word segmentation result is achieved through a second bidirectional long-short time memory cyclic neural network.
Further, the text features are cached in a repository, and are realized through a feature caching module.
Further, the text features are mapped into feature matrixes of the semantic graphs to obtain mapped feature matrixes of the semantic graphs, and the feature matrixes are realized through a first mapping module.
Further, the text feature is mapped into the feature matrix of the sequence chart to obtain a mapped sequence chart feature matrix, and the mapping is realized through a second mapping module.
Given the ambiguity of short text, building a dual graph structure provides it with powerful interpretability at the semantic and sequential levels, and secondly, intra-graph propagation within dual graphs and inter-graph propagation between dual graphs helps to overcome the inherent isomerism problem of isomerism graphs.
Acquiring text sequence numbers in a memory bank, and extracting final representation of nodes
And performing cross entropy loss operation on the corresponding nodes and the labels of the real samples in the training set.
It is worth mentioning that the information of the input algorithm may be, but is not limited to, notification content, bidding document content, technical terms and their interpretation documents, etc.
The automatic classification of the bid item names is beneficial to greatly reducing the manual processing cost, reducing the data error caused by subjectivity of people, improving the classification accuracy and ensuring the fair, public and fair bid tendering principle. For the bidding party, the problem of low bidding resource utilization rate is solved; for the bidding party, the bidding document is accurately positioned, so that too much time and cost on targets irrelevant to the bidding party are avoided, and the target bidding information collecting efficiency is improved.
Example two
The embodiment provides a labeling project name short text classification system based on a graph neural network;
a graph neural network based short text classification system for bid term names, comprising:
an acquisition module configured to: acquiring the names of bid-inviting items to be classified;
a preprocessing module configured to: preprocessing the names of the bid-inviting projects to be classified, and performing word segmentation on the preprocessed text;
a classification module configured to: inputting the word segmentation result into a trained short text classification model, and outputting a classification result;
wherein the trained short text classification model comprises:
extracting a feature matrix of the semantic graph from the segmentation result;
constructing a sequence diagram of the word segmentation result, and further obtaining an adjacent matrix of the sequence diagram; initializing an adjacent matrix of the sequence diagram to obtain a feature matrix of the sequence diagram;
extracting text features from the word segmentation result, and caching the text features into a memory bank;
mapping text features into feature matrixes of the semantic graphs and feature matrixes of the sequence graphs respectively to obtain mapped feature matrixes of the semantic graphs and mapped feature matrixes of the sequence graphs;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph, the mapped semantic graph feature matrix and the mapped sequence graph feature matrix, the intra-graph propagation of the semantic graph and the sequence graph is realized;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph and the intra-graph propagation result, inter-graph propagation of the semantic graph and the sequence graph is realized, and the inter-graph propagation result is classified to obtain a classification label.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. The method for classifying the short text of the bid-inviting project name based on the graph neural network is characterized by comprising the following steps: acquiring the names of bid-inviting items to be classified; preprocessing the names of the bid-inviting projects to be classified, and performing word segmentation on the preprocessed text; inputting the word segmentation result into a trained short text classification model, and outputting a classification result;
wherein the trained short text classification model is used for:
extracting a feature matrix of the semantic graph from the segmentation result;
constructing a sequence diagram of the word segmentation result, and further obtaining an adjacent matrix of the sequence diagram; initializing an adjacent matrix of the sequence diagram to obtain a feature matrix of the sequence diagram;
extracting text features from the word segmentation result, and caching the text features into a memory bank;
mapping text features into feature matrixes of the semantic graphs and feature matrixes of the sequence graphs respectively to obtain mapped feature matrixes of the semantic graphs and mapped feature matrixes of the sequence graphs;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph, the mapped semantic graph feature matrix and the mapped sequence graph feature matrix, the intra-graph propagation of the semantic graph and the sequence graph is realized;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph and the intra-graph propagation result, the inter-graph propagation of the semantic graph and the sequence graph is realized, and the inter-graph propagation result is classified to obtain a classification label;
the trained short text classification model has a network structure which specifically comprises: three parallel branches: a first branch, a second branch, and a third branch;
the first branch comprises: the system comprises a first bidirectional long-short-term memory cyclic neural network, a semantic graph construction module, a first feature initialization module and a first mapping module which are connected in sequence;
the second branch comprises: the sequence diagram construction module, the second characteristic initialization module and the second mapping module are sequentially connected;
the third branch comprises: the second bidirectional long and short-term memory cyclic neural network and the characteristic cache module are connected in sequence;
the output end of the characteristic cache module is respectively connected with the input end of the first mapping module and the input end of the second mapping module; the output end of the first mapping module and the output end of the second mapping module are connected with the input end of the first graph neural network;
the output end of the first graph neural network is connected with the input end of the second graph neural network; the output end of the first graph neural network is connected with the input end of the adder; the output end of the second graph neural network is connected with the input end of the adder, and the output end of the adder is connected with the input end of the average value pooling layer; the output end of the averaging layer is connected with the input end of the classifying layer.
2. The method for classifying short text of a bid term name based on a graph neural network according to claim 1, wherein the semantic graph-based adjacency matrix, the sequence graph-based adjacency matrix, the mapped semantic graph feature matrix and the mapped sequence graph feature matrix realize graph-in propagation of the semantic graph and the sequence graph, and specifically comprises the following steps:
inputting an adjacent matrix of the semantic graph and a mapped semantic graph feature matrix into a first graph neural network, and carrying out intra-graph propagation to obtain a first final representation after intra-graph propagation;
and inputting the adjacency matrix of the sequence diagram and the mapped sequence diagram feature matrix into a first diagram neural network to carry out intra-diagram propagation, so as to obtain a second final representation after intra-diagram propagation.
3. The method for classifying short text of a bid term name based on a graph neural network according to claim 1, wherein the semantic graph-based adjacency matrix, the sequence graph-based adjacency matrix and the intra-graph propagation result realize inter-graph propagation of the semantic graph and the sequence graph, and specifically comprises the following steps:
inputting the first final representation after the intra-graph propagation and the single-graph adjacency matrix after the combination into a second graph neural network, and carrying out inter-graph propagation to obtain a first intermediate value after the inter-graph propagation; the combined single-graph adjacency matrix refers to the weighted summation result of the adjacency matrix of the semantic graph and the adjacency matrix of the sequence graph;
inputting the second final representation after the intra-graph propagation and the single-graph adjacency matrix after the combination into a second graph neural network, and carrying out inter-graph propagation to obtain a second intermediate value after the inter-graph propagation;
summing the first intermediate value transmitted between the graphs and the first final representation transmitted in the graphs to obtain a first final representation of heterogeneous information between the fused semantic graph and the sequential graph;
and summing the second intermediate value after the inter-graph propagation with the second final representation after the intra-graph propagation to obtain a second final representation of the heterogeneous information between the fused semantic graph and the sequential graph.
4. The method for classifying short text of a bid term name based on a graph neural network according to claim 1, wherein the step of classifying the propagation result between graphs to obtain a classification label specifically comprises the following steps: and merging the first final representation of the heterogeneous information between the fusion semantic graph and the sequential graph with the second final representation of the heterogeneous information between the fusion semantic graph and the sequential graph according to dimensions, inputting the merged data into a mean value pooling layer for processing, inputting an output result of the mean value pooling layer into a classification layer, and outputting a classification label.
5. The method for classifying short text of a bid term name based on a graph neural network according to claim 1, wherein the semantic graph construction module comprises:
both the word and the document are regarded as nodes in the graph;
establishing two edges between nodes, wherein the first edge is the edge between a document node and a word node; the second type of edge refers to an edge between word nodes and word nodes; whether a connecting edge exists between the nodes is determined by the feature cosine similarity of the nodes, if the feature cosine similarity is larger than a set threshold, the connecting edge exists between the nodes, otherwise, the connecting edge does not exist;
weights are set for all the words and the edges between the words, and meanwhile weights are set for the edges between the document nodes and the word nodes, so that a constructed semantic graph is obtained; wherein the weight of the edge between the words is the ratio of the first value to the second value; the first numerical value refers to the number of times that two words have semantic relationships in all documents of the corpus; the second value refers to the number of times two words appear in all documents of the corpus; the weight of an edge between a document node and a word node is the word frequency-inverse document frequency.
6. The method for classifying short text of a bid term name based on a graph neural network according to claim 1, wherein the sequence diagram construction module comprises:
both the word and the document are regarded as nodes in the graph;
establishing two edges between nodes, wherein the first edge is the edge between a document node and a word node; the second type of edge refers to an edge between word nodes and word nodes; whether a connecting edge exists between the nodes is determined by an edge weight, if the edge weight is greater than a set threshold, the connecting edge exists between the nodes, and if the edge weight is less than or equal to the set threshold, the connecting edge does not exist between the nodes; the edge weight is calculated by adopting point-by-point mutual information;
setting weights for edges between word nodes and word nodes, and setting weights for edges between document nodes and word nodes to obtain a constructed sequence diagram; the weight of the word node and the edge between the word nodes is point-by-point mutual information; the weight of an edge between a document node and a word node is the word frequency-inverse document frequency.
7. The method for classifying short text of a bid term name based on a graph neural network according to claim 1, wherein the first feature initialization module comprises: and initializing an adjacent matrix of the semantic graph by adopting a single-heat coding mode to obtain a feature matrix of the semantic graph.
8. The method for classifying short text of a bid term name based on a graph neural network according to claim 1, wherein the feature buffer module comprises: caching text features output by the second bidirectional long-short time memory cyclic neural network and serial numbers corresponding to the text features into a memory bank;
extracting a feature matrix of the semantic graph from the word segmentation result, wherein the feature matrix specifically comprises the following steps:
extracting semantic features of each word and context information between the words, and constructing a semantic graph based on the semantic features of all the words and the context information between the words so as to obtain an adjacency matrix of the semantic graph; initializing the adjacent matrix of the semantic graph to obtain the feature matrix of the semantic graph.
9. The utility model provides a bid item name short text classification system based on graph neural network which characterized in that includes:
an acquisition module configured to: acquiring the names of bid-inviting items to be classified;
a preprocessing module configured to: preprocessing the names of the bid-inviting projects to be classified, and performing word segmentation on the preprocessed text;
a classification module configured to: inputting the word segmentation result into a trained short text classification model, and outputting a classification result;
wherein the trained short text classification model comprises:
extracting a feature matrix of the semantic graph from the segmentation result;
constructing a sequence diagram of the word segmentation result, and further obtaining an adjacent matrix of the sequence diagram; initializing an adjacent matrix of the sequence diagram to obtain a feature matrix of the sequence diagram;
extracting text features from the word segmentation result, and caching the text features into a memory bank;
mapping text features into feature matrixes of the semantic graphs and feature matrixes of the sequence graphs respectively to obtain mapped feature matrixes of the semantic graphs and mapped feature matrixes of the sequence graphs;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph, the mapped semantic graph feature matrix and the mapped sequence graph feature matrix, the intra-graph propagation of the semantic graph and the sequence graph is realized;
based on the adjacency matrix of the semantic graph, the adjacency matrix of the sequence graph and the intra-graph propagation result, the inter-graph propagation of the semantic graph and the sequence graph is realized, and the inter-graph propagation result is classified to obtain a classification label;
the trained short text classification model has a network structure which specifically comprises: three parallel branches: a first branch, a second branch, and a third branch;
the first branch comprises: the system comprises a first bidirectional long-short-term memory cyclic neural network, a semantic graph construction module, a first feature initialization module and a first mapping module which are connected in sequence;
the second branch comprises: the sequence diagram construction module, the second characteristic initialization module and the second mapping module are sequentially connected;
the third branch comprises: the second bidirectional long and short-term memory cyclic neural network and the characteristic cache module are connected in sequence;
the output end of the characteristic cache module is respectively connected with the input end of the first mapping module and the input end of the second mapping module; the output end of the first mapping module and the output end of the second mapping module are connected with the input end of the first graph neural network;
the output end of the first graph neural network is connected with the input end of the second graph neural network; the output end of the first graph neural network is connected with the input end of the adder; the output end of the second graph neural network is connected with the input end of the adder, and the output end of the adder is connected with the input end of the average value pooling layer; the output end of the averaging layer is connected with the input end of the classifying layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310132159.8A CN115858792B (en) | 2023-02-20 | 2023-02-20 | Short text classification method and system for bidding project names based on graphic neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310132159.8A CN115858792B (en) | 2023-02-20 | 2023-02-20 | Short text classification method and system for bidding project names based on graphic neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115858792A CN115858792A (en) | 2023-03-28 |
| CN115858792B true CN115858792B (en) | 2023-06-09 |
Family
ID=85658361
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310132159.8A Active CN115858792B (en) | 2023-02-20 | 2023-02-20 | Short text classification method and system for bidding project names based on graphic neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115858792B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114218389A (en) * | 2021-12-21 | 2022-03-22 | 一拓通信集团股份有限公司 | Long text classification method in chemical preparation field based on graph neural network |
| CN115510226A (en) * | 2022-09-02 | 2022-12-23 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Emotion classification method based on graph neural network |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2888375A1 (en) * | 2005-07-06 | 2007-01-12 | Thomson Licensing Sa | METHOD FOR OBTAINING A SOUND MAP FROM A PLURALITY OF SAME CARDS BASED ON DIFFERENT VISUAL SIZES |
| CN110111337A (en) * | 2019-04-16 | 2019-08-09 | 中山大学 | A kind of general human body analytical framework and its analytic method based on figure transfer learning |
| CN112765358B (en) * | 2021-02-23 | 2023-04-07 | 西安交通大学 | Taxpayer industry classification method based on noise label learning |
| US11983498B2 (en) * | 2021-03-18 | 2024-05-14 | Augmented Intelligence Technologies, Inc. | System and methods for language processing of document sequences using a neural network |
| CN114911933A (en) * | 2022-04-26 | 2022-08-16 | 济南大学 | False news detection method and system based on intra-graph and inter-graph joint information propagation |
-
2023
- 2023-02-20 CN CN202310132159.8A patent/CN115858792B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114218389A (en) * | 2021-12-21 | 2022-03-22 | 一拓通信集团股份有限公司 | Long text classification method in chemical preparation field based on graph neural network |
| CN115510226A (en) * | 2022-09-02 | 2022-12-23 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Emotion classification method based on graph neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115858792A (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | A review on graph neural network methods in financial applications | |
| Swathi et al. | An optimal deep learning-based LSTM for stock price prediction using twitter sentiment analysis | |
| Zhu et al. | Comparison of individual, ensemble and integrated ensemble machine learning methods to predict China’s SME credit risk in supply chain finance | |
| US20210264272A1 (en) | Training method and system of neural network model and prediction method and system | |
| CN104731962B (en) | Friend recommendation method and system based on similar corporations in a kind of social networks | |
| CN110751286A (en) | Training method and training system of neural network model | |
| Khan et al. | Gray method for multiple attribute decision making with incomplete weight information under the pythagorean fuzzy setting | |
| Qasem et al. | Extreme learning machine for credit risk analysis | |
| CN115858792B (en) | Short text classification method and system for bidding project names based on graphic neural network | |
| Sharma et al. | Analysis of big data | |
| Zhou et al. | Self-attention mechanism enhanced user interests modeling for personalized recommendation services in cyber-physical-social systems | |
| Liu et al. | A novel fuzzy neural network for unsupervised domain adaptation in heterogeneous scenarios | |
| Ma et al. | A novel business analytics approach and case study–fuzzy associative classifier based on information gain and rule-covering | |
| Wang et al. | Hierarchical multimodal fusion network with dynamic multi-task learning | |
| Jasim et al. | Analyzing Social Media Sentiment: Twitter as a Case Study | |
| CN114020901A (en) | Financial public opinion analysis method combining topic mining and emotion analysis | |
| Bao et al. | Hot news prediction method based on natural language processing technology and its application | |
| Parashivamurthy et al. | HDLNet: design and development of hybrid deep learning network for optimally recognising the handwritten Kannada characters | |
| PCD et al. | Advanced lightweight feature interaction in deep neural networks for improving the prediction in click through rate | |
| Kavitha et al. | Mal_CNN: An Enhancement for Malicious Image Classification Based on Neural Network | |
| CN115563289B (en) | Industry classification label generation method and device, electronic equipment and readable medium | |
| Agarwal et al. | Greedy search for active learning of OCR | |
| Alwidian et al. | An Online Machine Learning Approach to Sentiment Analysis in Social Media | |
| CN115687624A (en) | Text classification method and device, electronic equipment and medium | |
| Su et al. | Domain transfer convolutional attribute embedding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |