CN104915448B - A kind of entity based on level convolutional network and paragraph link method - Google Patents
A kind of entity based on level convolutional network and paragraph link method Download PDFInfo
- Publication number
- CN104915448B CN104915448B CN201510372795.3A CN201510372795A CN104915448B CN 104915448 B CN104915448 B CN 104915448B CN 201510372795 A CN201510372795 A CN 201510372795A CN 104915448 B CN104915448 B CN 104915448B
- Authority
- CN
- China
- Prior art keywords
- paragraph
- entity
- vectorization
- mrow
- sentence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 239000013598 vector Substances 0.000 claims abstract description 46
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 38
- 238000012549 training Methods 0.000 claims abstract description 15
- 238000000605 extraction Methods 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 33
- 239000011159 matrix material Substances 0.000 claims description 24
- 230000009466 transformation Effects 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 8
- 238000011478 gradient descent method Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 claims description 2
- 230000004927 fusion Effects 0.000 claims 1
- 238000012360 testing method Methods 0.000 abstract description 15
- 238000003062 neural network model Methods 0.000 abstract description 4
- 238000002474 experimental method Methods 0.000 description 6
- 230000004913 activation Effects 0.000 description 3
- 239000002585 base Substances 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 239000012458 free base Substances 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000007477 logistic regression Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000009411 base construction Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012733 comparative method Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
Abstract
A kind of entity based on level convolutional network and paragraph link method, including：Represent that changing into sentence vectorization represents by term vectorization using convolutional neural networks；Represent to again pass by convolutional neural networks using sentence vectorization and consider that the sentence order information obtains paragraph vectorization and represented；Sentence vectorization expression and paragraph vectorization represent to export by Softmax, carry out the training of the convolutional neural networks model as supervision message by existing entity；Meanwhile consider that the pair wise similarity informations between paragraph semantic vector feature and Entity Semantics vector characteristics further improve the training of convolutional neural networks model；A test description paragraph is given, carrying out Deep Semantics feature extraction using the neural network model trained obtains the vectorization expression of test paragraph, and being then based on this semantic expressiveness can be directly linked on target entity by Softmax outputs.
Description
Technical Field
The invention relates to the technical field of knowledge base construction, in particular to an entity and paragraph linking method based on a hierarchical convolutional network.
Background
Today, large-scale repositories in wide use are Freebase, WordNet, YAGO, and the like. They all work to build a global repository and allow machines to more conveniently access and obtain structured public information. At the same time, these knowledge bases provide Application Program Structures (APIs) to facilitate people to query richer information about related entities. For example, when we retrieve a city name "Washington d.c" in YAGO database, the results are returned as shown in table 1 below:
TABLE 1
It can be seen that the returned result information is some highly structured organization information. But these structured information do not fit into the actual context and semantic information that people understand an entity. Unlike the YAGO database, Freebase and WordNet return structured information and additionally return descriptive paragraphs related to the search entity, as shown in table 2 below:
TABLE 2
It can be seen that the descriptive paragraphs shown in table 2 are more useful for the user to understand the specific context and semantic information of the query entity words. However, the descriptive paragraph information of Freebase and WordNet is edited by human, which results in limitation of paragraph description on entities under big data and consumes a lot of time and manpower. Therefore, how to design an efficient entity and descriptive paragraph automatic linking method is an urgent task for constructing a knowledge base in the big data age.
As can be seen from the returned results in table 2, the descriptive contents do not necessarily include the query entity words, but only include some related words to describe the entities in a multi-aspect manner. Therefore, to solve this problem, the entity and paragraph linking method needs to be started from two aspects: 1. capturing subject matter information of text from a given descriptive paragraph; 2. important descriptive content related to the entity is found. Most of the conventional methods extract topic information of paragraphs based on topic model methods, such as dirichlet distribution (LDA) and Probabilistic Latent Semantic Analysis (PLSA). The general problems of the methods are that the extracted subject information is obtained based on word co-occurrence information of a document layer, is seriously influenced by high sparsity represented by short text characteristics in social media, and loses word sequence information in the text.
In recent years, with the rise of deep neural networks, some researchers try to learn deep implicit semantic feature representation of a descriptive paragraph by using a deep model and a word vectorization representation method to solve the problem of linking an entity with the paragraph. However, when solving the semantic feature extraction of the descriptive paragraphs, the existing depth model-based method simply treats the entire paragraph as a long sentence for processing or directly performs weighted averaging on a plurality of sentences to obtain a semantic vector. In fact, the sentence order in the paragraph also has semantic logical relationship.
On the other hand, it is also very important to capture descriptive cues in paragraphs that are closely related to entities. The descriptive section in the results returned from table 2 above, although not directly containing the query entity word "Washington d.c", contains many relevant words or phrases such as: "George Washington", "United States" and "Capital", etc. Thus, vectorizing the representation of the features of the entity facilitates the work of linking the entity with the descriptive paragraph.
Disclosure of Invention
In view of the above technical problems, a primary object of the present invention is to provide a method for linking entities and paragraphs based on a hierarchical convolutional network, so that entity words and descriptive paragraphs in the internet can be automatically linked without manual participation, which is helpful for building a semantic knowledge base under big data.
In order to achieve the above object, the present invention provides a method for linking an entity and a paragraph based on a hierarchical convolutional network, comprising the following steps:
converting the word vectorization representation into sentence vectorization representation by utilizing a convolutional neural network, wherein the convolutional network is favorable for extracting important clues of the query entity in the description paragraphs;
the sentence vectorization representation passes through the convolutional neural network again and paragraph vectorization representation is obtained by considering the sentence sequence information;
the sentence vectorization representation and the paragraph vectorization representation are output through Softmax, and training of the convolutional neural network model is carried out by means of existing entities serving as supervision information;
simultaneously considering pair-wise similarity information between the paragraph semantic vector features and the entity semantic vector features to further improve the training of the convolutional neural network model;
given a test description paragraph, deep semantic feature extraction is carried out by utilizing the trained neural network model to obtain vectorization representation of the test paragraph, and then the test description paragraph can be directly linked to a target entity through Softmax output based on the semantic representation.
The entity and paragraph linking method of the invention divides the feature learning problem in the link of the entity and the paragraph into four levels, which are respectively: the method comprises the steps that an original text paragraph is expressed through word vectorization to obtain a characteristic matrix layer; the sentence vectorization obtained through the convolutional neural network represents a feature layer; paragraph vectorization representation feature layers obtained through a convolutional neural network; and obtaining a vectorization representation characteristic layer of the entity words by using a word vector table look-up method. Through convolution feature network and word vector table lookup, the accuracy value ACC of the entity and paragraph linking method on two text data sets is obviously superior to other comparison methods, and compared with the best comparison method II, the accuracy value ACC of the entity and paragraph linking method on the two data sets is respectively improved by 12.4% and 16.76%.
Drawings
FIG. 1 is a flowchart of a method for linking entities and paragraphs based on a hierarchical convolutional network, which is an embodiment of the present invention;
FIG. 2 is a block diagram of a method for linking entities and paragraphs based on a hierarchical convolutional network, which is an embodiment of the present invention;
fig. 3 is a performance diagram of an entity and paragraph linking method based on a hierarchical convolutional network according to an embodiment of the present invention.
Detailed Description
In order that the objects, technical solutions and advantages of the present invention will become more apparent, the present invention will be further described in detail with reference to the accompanying drawings in conjunction with the following specific embodiments.
The invention discloses an entity and paragraph linking method based on a hierarchical convolutional network, which can automatically link entity words and descriptive paragraphs in the Internet without manual participation. Considering the order information of the sentences in the paragraphs, and convoluting the vectorized representation of the sentences again to obtain the vectorized representation of the paragraphs. And then, the entity features are used as supervision information to guide parameter learning of the convolutional neural network model, and simultaneously, pair-wise similarity information between the depth semantic features of the paragraphs and entity semantic vectorization representation is considered to improve learning of the convolutional neural network model. Given a new descriptive paragraph, the trained convolutional neural network model can be used to extract its deep semantic features, and the corresponding entity link is obtained based on the feature output.
More specifically, the method first converts through a word-vectorized representation into a sentence-vectorized representation using a convolutional neural network. And then, the sentence vectorization representation is utilized to pass through the convolutional neural network again, and the sentence order information is considered to obtain paragraph vectorization representation. And the sentence vectorization representation and the paragraph vectorization representation are output through Softmax, and the training of the convolutional neural network model is carried out by taking an existing entity as supervision information. Meanwhile, the training of the convolutional neural network model is further improved by considering the pair-wise similarity information between the paragraph semantic vector features and the entity semantic vector features. Giving a test description section, extracting deep semantic features by using a trained neural network model to obtain vectorization representation of the test section, and directly linking the test description section to a target entity through Softmax output based on the semantic representation.
The entity and paragraph linking method based on the hierarchical convolutional network as an embodiment of the present invention is described in detail below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for linking entities and paragraphs based on a hierarchical convolutional network according to an embodiment of the present invention.
Referring to fig. 1, in step S101, extracting vectorization representation features of each sentence in a paragraph to be processed through a convolutional neural network model and word vectorization representation;
according to an exemplary embodiment of the present invention, the step of extracting vectorization representation characteristics of each sentence in the paragraph to be processed through the convolutional neural network model and the word vectorization representation includes:
in step S1011, a sentence in the to-be-processed paragraph is given, a term quantization expression is obtained by using a table lookup method, and the sentence is characterized in a matrix form;
in step S1012, performing one-dimensional convolution on the sentence matrixing expression feature to obtain a feature matrix after convolution;
in step S1013, mean sampling is performed on the convolved feature matrix to compress the features, so as to obtain vectorized representation of the sentence.
According to an exemplary embodiment of the present invention, the step of obtaining a term quantization representation by using a table lookup method and characterizing the sentence in a matrix form comprises:
given a word2vec trained word vector setWhere | V | is the dictionary size and d is the dimension of the word vector. A sentence of length n in any paragraph can be represented as:
s＝(x1；x2；...；xn) (1)
wherein x isiIs the vectorization representation corresponding to the ith word found in the word vector set by using a table lookup method. Wherein, if the word xiNot in the trained word vector set, it is directly represented by random initialization in this exemplary embodiment of the invention.
In step S1012, the one-dimensional convolution is performed on the sentence matrixing expression feature, and the step of obtaining the convolved feature matrix includes:
here, use is made ofRepresenting h starting from the ith word in sentence ssA continuous word feature. Given a one-dimensional convolution kernelThen h issThe feature matrix after convolution of the features of the continuous words is as follows:
wherein, b(1)Is a bias term, f is an activation function,is hsCharacteristics of individual continuous wordsThe feature matrix after convolution. The feature matrix of the sentence is convolved as:
in step S1013, the step of performing mean value sampling on the convolved feature matrix to compress the features to obtain vectorized representation of the sentence includes:
in this exemplary embodiment of the present invention, the step of sampling with the mean value is:
to this end, each convolution kernelA d-dimensional feature vector is generatedIf k convolution kernels are used, a vectorized representation of the sentence is finally obtained through one convolution layerThe dimension of the sentence vectorized representation is d · k.
In step S102, learning a deep semantic feature of the paragraph by using a convolutional neural network structure and the sentence vectorization representation;
according to an exemplary embodiment of the present invention, the method for deep semantic feature learning of paragraphs includes:
in step S1021, using the sentence vector features in the paragraphs to characterize the paragraphs in a matrix form according to the word order of the sentences in the paragraphs;
in step S1022, performing one-dimensional convolution on the paragraph matrixing expression feature to obtain a feature matrix after convolution;
in step S1023, mean sampling is performed on the convolved feature matrix to compress the features and perform a linear transformation to obtain a vectorized representation of the paragraph.
According to an exemplary embodiment of the present invention, the step of characterizing the paragraphs in a matrix form according to the word order of the sentences in the paragraphs by using sentence vector features in the paragraphs comprises:
having obtained a vectorized representation of the l sentences of the paragraph, the paragraph can be represented as:
t＝(s1；s2；...；sl) (5)
in step S1022, the one-dimensional convolution is performed on the paragraph matrixing expression feature, and the step of obtaining the convolved feature matrix includes:
here, use is made ofRepresents h starting from the ith sentence in paragraph ttA continuous sentence characteristic. Given a one-dimensional convolution kernelThen h istThe convolution characteristics after convolution of the characteristics of the continuous sentences are as follows:
wherein, b(2)Is a bias term, f is an activation function,is htA characteristic of a continuous sentenceThe features after convolution. The features of the paragraph are convolved as:
in step S1023, the step of performing mean value sampling on the convolved feature matrix to compress the features and performing linear transformation once to obtain vectorized representation of the paragraph includes:
in this exemplary embodiment of the present invention, the step of sampling with the mean value is:
up to this point, through a convolution kernel W(2)Generating a d.k dimensional feature vectorIn order to facilitate the calculation of the similarity between the paragraph features and the entity features, if the uniformity of the vector dimensions needs to be ensured, the paragraph vector is subjected to linear transformation:
wherein,is a linear transformation matrix and the feature vector z is the final paragraph feature vector in an exemplary embodiment of the invention.
In step S103, the vectorized representation of the sentence and the vectorized representation of the paragraph are respectively subjected to Softmax output to fit the entity to which the paragraph belongs;
according to an exemplary embodiment of the present invention, the method of fitting the vectorized representation of the sentence and the paragraph to the entities to which the paragraph belongs, respectively, comprises the steps of:
in step S1031, performing linear transformation on the sentence vector and the paragraph vector to obtain output vectors, and performing regularization using Dropout technology;
at step S1032, calculating a link probability of the candidate entity using a Softmax function;
according to an exemplary embodiment of the present invention, the step of performing linear transformation on the sentence vector and the paragraph vector to obtain an output vector and performing regularization using Dropout technique includes:
the sentence vector feature s and the paragraph vector feature t are respectively subjected to linear change to obtain two output vectors:
ys＝W(4)·(sοr)+b(4)(10)
y＝W(5)·(zοr)+b(5)(11)
wherein,andis a weight matrix and m is the number of entities, symbols, in an exemplary embodiment of the invention. Represents a multiplication operation of matrix elements, andit is a bernoulli distribution obeying a certain probability p. Overfitting can be prevented using Dropout techniques, which can enhance the robustness of the neural network model.
At step S1032, the step of calculating the link probability of the candidate entity using the Softmax function includes:
calculating a probability value of each corresponding entity word using a Softmax activation function at both of the output layers of the sentence vector feature and the paragraph vector feature, respectively:
then in equation (12) and equation (13), psiAnd piRespectively representing probability values corresponding to the ith entity word.
In step S104, calculating pair-wise similarity information of the vectorized representation of the entity and the paragraph vectorized representation;
given a set of physical words E ═ E1，e2，...，emAnd initializing the entity word set by using word2vec, wherein the similarity between the entity word set E and the paragraph feature vector z is as follows:
sim(z，E)＝{z·e1，z·e2，...，z·em} (14)
wherein the operator z · e represents the similarity of the paragraph feature vector z and the corresponding entity word e.
In step S105, error back propagation training is carried out on the target entity word and paragraph feature vector through Softmax fitting and the pair-wise similarity information of the target entity word;
according to an exemplary embodiment of the present invention, the step of performing error back propagation on the trained convolutional neural network model by Softmax fitting of target entity words and pair-wise similarity information of the paragraph feature vectors and the target entity words comprises:
in step S1051, a target function is set according to the sentence feature and paragraph feature output and the fitting result of the Softmax to the target entity word in the training data set;
in step S1052, a target function is set according to the pair-wise similarity information between the paragraph feature and the target entity word;
in step S1053, a global objective constraint function is set;
in step S1054, parameters in the model are updated by a random gradient descent method;
according to an exemplary embodiment of the present invention, the step of setting an objective function according to sentence and paragraph feature output by using the Softmax to the fitting result of the target entity word in the training data set comprises:
using formulas (10) and (11) and formulas (12) and (13), respectively setting the target constraint functions of the sentence vectorization feature and the paragraph vectorization feature as follows:
wherein L issAn objective constraint function, L, for the sentence vectorized featuresp1An objective constraint function for the paragraph vectorization feature,set of all middle paragraphs in the corpusAll of the sentences in (a) are collected,is the correct and definite word to which the ith sentence belongsIs the positive idiom to which the ith paragraph belongs.
In step S1052, the step of setting a target function according to pair-wise similarity information between the paragraph feature and the target entity word includes:
in order to enhance the semantic expression ability of the paragraph and the entity, the similarity between the paragraph vectorization feature and the corresponding belonging entity word vectorization feature is enhanced by setting a target constraint function, and the similarity between the paragraph vectorization feature and the corresponding non-belonging entity word vectorization feature is weakened, wherein the target constraint function is as follows:
wherein e isrIs given the positive idiom to which the paragraph z belongs.
In step S1053, the step of setting the global objective constraint function is as follows:
L＝Ls+(1-α)·Lp1+α·Lp2(18)
α are weight harmonic coefficients used to balance the two constraints L of the paragraph vectorization featurep1And Lp2。
In step S1054, the step of updating the parameters in the model by using a stochastic gradient descent method includes:
all model training parameters in the set target constraint function are uniformly expressed as theta:
θ＝(x，W(1)，b(1)，W(2)，b(2)，α，W(3)，W(4)，b(4)，W(5)，b(5)，E) (19)
in an exemplary embodiment of the invention, the objective function is optimized using a random gradient descent method for error back propagation.
In step S106, deep semantic feature extraction is performed on the test descriptive paragraphs using the updated convolutional neural network model, and then linking is performed with corresponding entity words based on vectorized representation of the paragraphs.
According to an exemplary embodiment of the present invention, the step of performing deep semantic feature extraction on the test descriptive section by using the updated convolutional neural network model, and then linking with the corresponding entity word based on the vectorized representation of the section comprises:
in step S1061, a test paragraph text is given, and vectorization features S of sentences in the paragraph are calculated according to formulas (2), (3) and (4);
in step S1062, calculating a vectorization feature z of the paragraph by using formulas (6), (7), (8) and (9);
in step S1063, using the generated vectorized feature z of the paragraph, a linear transformation without Dropout and a Softmax function are used to output a matching probability of the corresponding entity word:
y＝W(5)·z+b(5)(20)
and the entity word with the highest matching probability is the entity word belonging to the test paragraph.
Fig. 2 is a schematic diagram of a framework of a method for linking entities and paragraphs based on a hierarchical convolutional network according to an embodiment of the present invention.
Referring to fig. 2, the entity and paragraph linking method based on the hierarchical convolutional network has four levels of feature vectorization representations, which are:
the first characteristic level is as follows: the method comprises the steps that an original text paragraph is expressed through word vectorization to obtain a feature matrix;
and (2) feature level two: sentence vectorization representation characteristics obtained through a convolutional neural network;
and (3) feature level three: paragraph vectorization representation characteristics obtained through a convolutional neural network;
feature level four: obtaining the vectorization expression characteristics of the entity words by using a word vector table look-up method;
the whole model training stage has three supervision information for guidance, which are respectively as follows:
monitoring information I: the vectorization representation characteristics of the sentence are subjected to linear change and Softmax output, and then the fitting information of the sentence is obtained;
and (5) monitoring information II: the vectorization of the paragraph represents the fitting information of the characteristic to the belonging entity word after linear change and Softmax output;
and (5) monitoring information III: the vectorization of the paragraph represents the Pair-wise similarity information of the entity words after the characteristics are linearly changed;
in order to accurately evaluate the link performance of the entity and the paragraph of the method, the method obtains the precision (ACC) of the method by comparing the consistency of the link results of the entity and the paragraph and the entity to which the paragraph really belongs. Given a descriptive paragraph sample x(i)The entity word linked by the method of the invention is e(i)And the paragraph is true the physical word isThe definition of accuracy is as follows:
wherein,is the number of descriptive paragraphs, δ (x, y) is an indicator function, 1 when x ≠ y, and 1 when x ≠ yThe number is 0.
Two open text data sets were used in the experiments of the present invention:
history: the data set contains 409 entities, 1704 paragraphs.
Literature: the data set contains 445 entities, 2247 paragraphs.
The present invention does not perform any processing (including word-kill and stem reduction operations) on these text data sets. On average, each paragraph contains 4-6 sentences, while each paragraph contains only 1 entity word. Specific statistics of the data set are shown in table 3:
TABLE 3
The following comparative methods were used in the experiments of the invention:
the first comparison method comprises the following steps: based on a bag-of-words model and a logistic regression method, the method directly adopts the logistic regression method on the bag-of-words model of the original text;
and a second comparison method comprises the following steps: the method adopts a traditional convolutional neural network model to simply consider an entity and paragraph link problem as a classification problem.
The parameters used in the experiments of the invention are set as shown in table 4:
TABLE 4
| Data set | ρ | hs | ht | d | k |
| History | 0.5 | 3 | 6 | 100 | 1 |
| Literature | 0.5 | 3 | 8 | 100 | 1 |
In Table 4, the parameter ρ is the specific gravity coefficient of Dropout used in model training, hsBounding size of convolution kernel, h, for sentence vectorized feature representationtThe frame mouth size of the convolution kernel when the paragraph vectorization feature is represented, d is the dimension of the word vector, and k is the number of the convolution kernels when the sentence vectorization feature is represented.
In the experiment of the present invention, the average precision (ACC) is obtained by performing the link method for all entities and paragraphs 50 times, and the final experiment result is shown in table 5:
TABLE 5
| Method of producing a composite material | History/precision value (%) | Literature/precision value (%) |
| Comparison method 1 | 65.10±0.01 | 61.17±0.05 |
| Comparison method two | 77.01±3.92 | 74.50±10.3 |
| The method of the invention | 89.41±1.05 | 91.26±0.50 |
Table 5 shows the evaluation results of the precision value (ACC) of the entity and paragraph linking method on two text data sets by the method of the present invention, the first comparison method and the second comparison method. Test results show that the performance of the method is obviously superior to that of other comparison methods. And compared with the best comparison method two, the method improves the precision value on two data sets by 12.4 percent and 16.76 percent respectively.
Meanwhile, the test of the invention verifies the influence of the size of the sliding word window of the convolution kernel on the performance of the entity and paragraph link precision value of the method of the invention when sentence characteristic representation is carried out, and the test result is shown in fig. 3. It can be seen that the performance of the method of the present invention is optimal on both data sets when the word window size is 3, whereas the performance of the precision value of the method of the present invention decreases when the word window size is greater than 3. Therefore, the sliding word window size of the sentence characteristic convolution kernel adopted in the experiment of the invention is 3.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A entity and paragraph linking method based on a hierarchical convolutional network comprises the following steps:
extracting vectorization representation characteristics of each sentence in the paragraph to be processed through a convolutional neural network model and word vectorization representation;
learning the depth semantic features of the paragraphs by using a convolutional neural network structure and sentence vectorization representation;
respectively outputting entities to which the fitted paragraphs belong by the vectorized representation of the sentence and the vectorized representation of the paragraphs through Softmax;
calculating pair-wise similarity information of the vectorized representation of the entity and the vectorized representation of the paragraph;
error back propagation is carried out by fitting target entity words and paragraph feature vectors with pair-wise similarity information of the target entity words through Softmax to train the convolutional neural network model;
and performing deep semantic feature extraction on the paragraph to be processed by using the updated convolutional neural network model, and then linking with a corresponding entity word based on vectorization representation of the paragraph.
2. The entity and paragraph linking method based on hierarchical convolutional network of claim 1, wherein the step of extracting vectorized representation features of each sentence in the paragraph to be processed through convolutional neural network model and word vectorization representation comprises:
giving a sentence in a paragraph to be processed, obtaining word vectorization representation by utilizing a table look-up method, and representing the sentence in a matrix form;
performing one-dimensional convolution on the sentence matrixing expression characteristic to obtain a feature matrix after convolution;
and performing mean value sampling on the convolved convolution characteristics to compress the characteristics to obtain vectorization expression of the sentence.
3. The hierarchical convolutional network-based entity-to-paragraph linking method of claim 1, wherein the step of learning the deep semantic features of the paragraphs using the convolutional neural network structure and the sentence vectorization representation comprises:
using sentence vector characteristics in the paragraphs to characterize the paragraphs into a matrix form according to the word order of the sentences in the paragraphs;
performing one-dimensional convolution on the paragraph matrixing expression characteristic to obtain a feature matrix after convolution;
and performing mean sampling on the convolved convolution characteristics to compress the characteristics and performing linear transformation once to obtain vectorization expression of the paragraph.
4. The method of claim 1, wherein the step of fitting the vectorized representation of the sentence and the vectorized representation of the paragraph to the entity to which the paragraph belongs via Softmax output respectively comprises:
performing linear transformation on the sentence vectors and the paragraph vectors respectively to obtain output vectors, and performing regularization by using a Dropout technology;
the probability of linkage of the candidate entity is calculated using the Softmax function.
5. The hierarchical convolutional network-based entity-paragraph linking method of claim 1, wherein the method of calculating pair-wise similarity information of the vectorized representation of the entity and the vectorized representation of the paragraph is as follows:
given a set of physical words E ═ E1，e2，...，emAnd initializing the entity word set by using word2vec, wherein the similarity between the entity word set E and the paragraph feature vector z is as follows:
sim(z，E)＝{z·e1，z·e2，...，z·em}.
wherein the operator z · e represents the similarity of the paragraph feature vector z and the corresponding entity word e.
6. The hierarchical convolutional network-based entity-paragraph linking method as claimed in claim 1, wherein the step of training the convolutional neural network model by error back propagation through Softmax fitting target entity words and pair-wise similarity information of the paragraph feature vectors and the target entity words comprises:
according to the sentence characteristic and paragraph characteristic output, setting a target function by utilizing the fitting result of the Softmax on the target entity words in the training data set;
setting a target function according to the pair-wise similarity information of the paragraph features and the target entity words;
setting a global target constraint function and carrying out unified fusion on the target functions;
and updating parameters in the convolutional neural network model by using a random gradient descent method.
7. The entity and paragraph linking method based on hierarchical convolutional network of claim 6, wherein the step of setting a target function according to the pair-wise similarity information between the paragraph feature and the target entity word comprises:
in order to enhance the semantic expression ability of the paragraph and the entity, the similarity between the paragraph vectorization feature and the corresponding belonging entity word vectorization feature is enhanced by setting a target constraint function, and the similarity between the paragraph vectorization feature and the corresponding non-belonging entity word vectorization feature is weakened, wherein the target constraint function is as follows:
<mrow> <msub> <mi>L</mi> <mrow> <mi>p</mi> <mn>2</mn> </mrow> </msub> <mo>=</mo> <munderover> <mo>&Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mrow> <mo>|</mo> <mi>c</mi> <mo>|</mo> </mrow> </munderover> <munder> <mo>&Sigma;</mo> <mrow> <msub> <mi>e</mi> <mi>j</mi> </msub> <mo>&Element;</mo> <mi>E</mi> <mrow> <mo>(</mo> <msub> <mi>e</mi> <mi>j</mi> </msub> <mo>&NotEqual;</mo> <msub> <mi>e</mi> <mi>r</mi> </msub> <mo>)</mo> </mrow> </mrow> </munder> <mi>m</mi> <mi>a</mi> <mi>x</mi> <mrow> <mo>(</mo> <mn>0</mn> <mo>,</mo> <mn>1</mn> <mo>-</mo> <mi>s</mi> <mi>i</mi> <mi>m</mi> <mo>(</mo> <mrow> <msup> <mi>z</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </msup> <mo>,</mo> <msubsup> <mi>e</mi> <mi>r</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </msubsup> </mrow> <mo>)</mo> <mo>+</mo> <mi>s</mi> <mi>i</mi> <mi>m</mi> <mo>(</mo> <mrow> <msup> <mi>z</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </msup> <mo>,</mo> <msubsup> <mi>e</mi> <mi>j</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </msubsup> </mrow> <mo>)</mo> <mo>)</mo> </mrow> <mo>;</mo> </mrow>
wherein e isrIs given the correct entity word to which the paragraph z belongs, er (i)The entity words are true for the paragraphs.
8. The entity and paragraph linking method based on hierarchical convolutional network of claim 6, wherein the step of setting global objective constraint function to uniformly fuse the objective functions comprises:
setting the global objective constraint function as follows:
L＝Ls+(1-α)·Lp1+α·Lp2；
wherein L issα is a weight harmonic coefficient for the target constraint function of sentence vectorization feature, which is used to balance the two constraints of the paragraph vectorization feature, i.e. the paragraph feature output utilizes Softmax to fit the constraint term L of the target entity word in the training data setp1And a pair-wise similarity constraint term L of paragraph features and the target entity wordsp2。
9. The entity and paragraph linking method based on hierarchical convolutional network of claim 1, wherein the step of performing deep semantic feature extraction on the paragraphs to be processed by using the updated convolutional neural network model and then linking with the corresponding entity words based on the vectorized representation of the paragraphs comprises:
giving a paragraph text to be processed, and calculating vectorization characteristics of sentences in the paragraph through a trained convolutional neural network model;
calculating vectorization characteristics of the paragraph through a trained convolutional neural network model;
and outputting the matching probability of the corresponding entity words by using the generated vectorization characteristics of the paragraphs and a Dropout-free linear transformation and a Softmax function.
10. The method of claim 1, wherein the convolutional neural network model uses a sentence-feature convolutional kernel with a sliding word window size of 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510372795.3A CN104915448B (en) | 2015-06-30 | 2015-06-30 | A kind of entity based on level convolutional network and paragraph link method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510372795.3A CN104915448B (en) | 2015-06-30 | 2015-06-30 | A kind of entity based on level convolutional network and paragraph link method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104915448A CN104915448A (en) | 2015-09-16 |
| CN104915448B true CN104915448B (en) | 2018-03-27 |
Family
ID=54084511
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510372795.3A Active CN104915448B (en) | 2015-06-30 | 2015-06-30 | A kind of entity based on level convolutional network and paragraph link method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104915448B (en) |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107220220A (en) | 2016-03-22 | 2017-09-29 | 索尼公司 | Electronic equipment and method for text-processing |
| CN106339718A (en) * | 2016-08-18 | 2017-01-18 | 苏州大学 | Classification method based on neural network and classification device thereof |
| CN106326985A (en) * | 2016-08-18 | 2017-01-11 | 北京旷视科技有限公司 | Neural network training method, neural network training device, data processing method and data processing device |
| CN106446526B (en) * | 2016-08-31 | 2019-11-15 | 北京千安哲信息技术有限公司 | Electronic health record entity relation extraction method and device |
| CN106844765B (en) * | 2017-02-22 | 2019-12-20 | 中国科学院自动化研究所 | Significant information detection method and device based on convolutional neural network |
| CN107144569A (en) * | 2017-04-27 | 2017-09-08 | 西安交通大学 | The fan blade surface defect diagnostic method split based on selective search |
| CN107168956B (en) * | 2017-05-26 | 2020-06-02 | 北京理工大学 | Chinese chapter structure analysis method and system based on pipeline |
| CN109426664A (en) * | 2017-08-30 | 2019-03-05 | 上海诺悦智能科技有限公司 | A kind of sentence similarity calculation method based on convolutional neural networks |
| CN107704563B (en) * | 2017-09-29 | 2021-05-18 | 广州多益网络股份有限公司 | Question recommendation method and system |
| CN108304552B (en) * | 2018-02-01 | 2021-01-08 | 浙江大学 | Named entity linking method based on knowledge base feature extraction |
| CN108764233B (en) * | 2018-05-08 | 2021-10-15 | 天津师范大学 | Scene character recognition method based on continuous convolution activation |
| CN109344244B (en) * | 2018-10-29 | 2019-11-08 | 山东大学 | A kind of the neural network relationship classification method and its realization system of fusion discrimination information |
| CN109697288B (en) * | 2018-12-25 | 2020-09-15 | 北京理工大学 | Instance alignment method based on deep learning |
| CN110147533B (en) | 2019-01-24 | 2023-08-29 | 腾讯科技（深圳）有限公司 | Encoding method, apparatus, device and storage medium |
| CN109992629B (en) * | 2019-02-28 | 2021-08-06 | 中国科学院计算技术研究所 | Neural network relation extraction method and system fusing entity type constraints |
| CN112328800A (en) * | 2019-08-05 | 2021-02-05 | 上海交通大学 | System and method for automatically generating programming specification question answers |
| CN110674317B (en) * | 2019-09-30 | 2022-04-12 | 北京邮电大学 | Entity linking method and device based on graph neural network |
| CN110717339B (en) | 2019-12-12 | 2020-06-30 | 北京百度网讯科技有限公司 | Semantic representation model processing method and device, electronic equipment and storage medium |
| CN111222314B (en) * | 2020-01-03 | 2021-12-21 | 北大方正集团有限公司 | Layout document comparison method, device, equipment and storage medium |
| CN113361261B (en) * | 2021-05-19 | 2022-09-09 | 重庆邮电大学 | Method and device for selecting legal case candidate paragraphs based on enhance matrix |
| CN115130435B (en) * | 2022-06-27 | 2023-08-11 | 北京百度网讯科技有限公司 | Document processing method, device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104317834A (en) * | 2014-10-10 | 2015-01-28 | 浙江大学 | Cross-media sorting method based on deep neural network |
| CN104462357A (en) * | 2014-12-08 | 2015-03-25 | 百度在线网络技术（北京）有限公司 | Method and device for realizing personalized search |
| CN104615767A (en) * | 2015-02-15 | 2015-05-13 | 百度在线网络技术（北京）有限公司 | Searching-ranking model training method and device and search processing method |
| CN104679863A (en) * | 2015-02-28 | 2015-06-03 | 武汉烽火众智数字技术有限责任公司 | Method and system for searching images by images based on deep learning |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130212049A1 (en) * | 2012-02-15 | 2013-08-15 | American Gnc Corporation | Machine Evolutionary Behavior by Embedded Collaborative Learning Engine (eCLE) |
-
2015
- 2015-06-30 CN CN201510372795.3A patent/CN104915448B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104317834A (en) * | 2014-10-10 | 2015-01-28 | 浙江大学 | Cross-media sorting method based on deep neural network |
| CN104462357A (en) * | 2014-12-08 | 2015-03-25 | 百度在线网络技术（北京）有限公司 | Method and device for realizing personalized search |
| CN104615767A (en) * | 2015-02-15 | 2015-05-13 | 百度在线网络技术（北京）有限公司 | Searching-ranking model training method and device and search processing method |
| CN104679863A (en) * | 2015-02-28 | 2015-06-03 | 武汉烽火众智数字技术有限责任公司 | Method and system for searching images by images based on deep learning |
Non-Patent Citations (3)
| Title |
|---|
| A Convolutional Neural Network for Modelling Sentences;N. Kalchbrenner;《Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics》;20140630;第655-665页 * |
| A Neural Network for Factoid Question Answering over Paragraphs;M Iyyer etal;《Conference on Empirical Methods in Natural Language Processing》;20141231;第633-644页 * |
| Convolutional neural networks for sentence classification;Y. Kim etal;《Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing》;20141231;第1746-1751页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104915448A (en) | 2015-09-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104915448B (en) | A kind of entity based on level convolutional network and paragraph link method | |
| CN112001185B (en) | Emotion classification method combining Chinese syntax and graph convolution neural network | |
| CN110287334B (en) | Method for constructing knowledge graph in school domain based on entity identification and attribute extraction model | |
| CN112001187B (en) | Emotion classification system based on Chinese syntax and graph convolution neural network | |
| CN108573411B (en) | Mixed recommendation method based on deep emotion analysis and multi-source recommendation view fusion of user comments | |
| CN105824922B (en) | A kind of sensibility classification method merging further feature and shallow-layer feature | |
| CN106372061B (en) | Short text similarity calculation method based on semantics | |
| CN112001186A (en) | Emotion classification method using graph convolution neural network and Chinese syntax | |
| Chang et al. | Research on detection methods based on Doc2vec abnormal comments | |
| CN110598005A (en) | Public safety event-oriented multi-source heterogeneous data knowledge graph construction method | |
| CN107180045A (en) | A kind of internet text contains the abstracting method of geographical entity relation | |
| CN105843799B (en) | A kind of academic paper label recommendation method based on multi-source heterogeneous information graph model | |
| CN103020167B (en) | A kind of computer Chinese file classification method | |
| CN114548321B (en) | Self-supervision public opinion comment viewpoint object classification method based on contrast learning | |
| CN111274790A (en) | Chapter-level event embedding method and device based on syntactic dependency graph | |
| CN113360582B (en) | Relation classification method and system based on BERT model fusion multi-entity information | |
| CN106446147A (en) | Emotion analysis method based on structuring features | |
| CN106445914B (en) | Construction method and construction device of microblog emotion classifier | |
| Sebti et al. | A new word sense similarity measure in WordNet | |
| CN114997288A (en) | Design resource association method | |
| CN107451116B (en) | Statistical analysis method for mobile application endogenous big data | |
| CN113051886B (en) | Test question duplicate checking method, device, storage medium and equipment | |
| CN108009187A (en) | A kind of short text Topics Crawling method for strengthening Text Representation | |
| CN114547303A (en) | Text multi-feature classification method and device based on Bert-LSTM | |
| CN109087223A (en) | A kind of educational resource model building method based on ontology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |