CN110750986B - Neural network word segmentation system and training method based on minimum information entropy - Google Patents
Neural network word segmentation system and training method based on minimum information entropy Download PDFInfo
- Publication number
- CN110750986B CN110750986B CN201810724646.2A CN201810724646A CN110750986B CN 110750986 B CN110750986 B CN 110750986B CN 201810724646 A CN201810724646 A CN 201810724646A CN 110750986 B CN110750986 B CN 110750986B
- Authority
- CN
- China
- Prior art keywords
- word
- neural network
- information entropy
- minimum information
- word segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the invention provides a neural network word segmentation system and a training method based on minimum information entropy. The system comprises: the system comprises a convolutional neural network, a two-way long-short-term memory neural network, a first word stock prediction layer and a minimum information entropy word stock prediction layer, wherein: the convolutional neural network is used for extracting the characteristic vector of the input text and outputting the characteristic vector to the two-way long-short-term memory neural network; the two-way long-short-term memory neural network is used for reading the context information of the feature vector and outputting the feature vector to the first word stock prediction layer and the minimum information entropy word stock prediction layer; the first word stock prediction layer is used for calculating and outputting labels of each word according to the first word stock; the minimum information entropy word stock prediction layer is used for calculating and outputting labels of each word according to the minimum information entropy word stock. According to the embodiment of the invention, the minimum information entropy word stock prediction layer is added in the neural network word segmentation system, so that the word segmentation system improves the recognition capability of the unregistered words, and further improves the word segmentation accuracy.
Description
Technical Field
The embodiment of the invention relates to the technical field of natural language processing, in particular to a neural network word segmentation system and a training method based on minimum information entropy.
Background
With the advent of deep learning, neural network word segmentation systems became a research hotspot. The current word segmentation method based on statistics has been adopted by various large companies, and a general neural network word segmentation system framework is shown in fig. 1, and referring to fig. 1, the word segmentation system comprises a CNN (Convolutional Neural Networks, convolutional neural network), a BLSTM (Bidirectional Long Short-term Memory, two-way long-short-term Memory neural network) and a prediction layer, wherein the BLSTM comprises a forward LSTM and a backward LSTM, and the prediction layer is a CRF (conditional random field ).
The word segmentation system based on the framework is used for inputting Chinese characters one by one, each word is converted into a multidimensional vector through a word2vector (word to vector) tool, and the word vector is input into a convolutional neural network in sentence units for feature extraction. The typical convolutional neural network outputs a feature map to a full-connection layer, which combines multiple groups of features after multi-layer convolutional pooling operation sequentially into a set of vectors, which are input into the BLSTM for learning. The BLSTM is stacked by two LSTM networks, one reads text from the forward direction and the other acquires text from the reverse direction, so that the context information of the text can be acquired simultaneously, the BLSTM outputs a result to a CRF (conditional random field layer), and the BLSTM layer calculates the output vector of the BLSTM layer through the conditional random field theory to obtain the label of each word, such as a single word S, and the beginning of one word is B. Thus, after labeling each word, we obtain word segmentation results.
However, the word segmentation method based on statistics is severely dependent on the corpus, and the segmentation of the words which do not exist in the corpus is almost not completed, so that the word segmentation result has low accuracy.
Disclosure of Invention
The embodiment of the invention provides a neural network word segmentation system and a training method based on minimum information entropy, which are used for solving the problems of low word segmentation result accuracy and the like in the prior art.
In one aspect, an embodiment of the present invention provides a neural network word segmentation system based on minimum information entropy, where the system includes: the system comprises a convolutional neural network, a two-way long-short-term memory neural network, a first word stock prediction layer and a minimum information entropy word stock prediction layer, wherein:
the convolutional neural network is used for extracting the characteristic vector of the input text and outputting the characteristic vector to the two-way long-short-term memory neural network;
the two-way long-short-term memory neural network is used for receiving the feature vector, reading the context information, removing redundancy, and outputting the context information to the first word stock prediction layer and the minimum information entropy word stock prediction layer;
the first word stock prediction layer is used for receiving the feature vector output by the two-way long-short term memory neural network and calculating and outputting the label of each word of the input text according to the first word stock;
the minimum information entropy word stock prediction layer is used for receiving the feature vector output by the two-way long-short term memory neural network and calculating and outputting the label of each word of the input text according to the minimum information entropy word stock.
On the other hand, the embodiment of the invention provides a training method of a neural network word segmentation system based on minimum information entropy, which comprises the following steps:
calculating a loss function L of the neural network word segmentation system based on the minimum information entropy total The calculation formula is as follows:
L total ＝L C +L C1 +L Fab
wherein L is C For the loss function corresponding to the first word stock, L C1 A loss function corresponding to the minimum information entropy word stock, L Fab The opposite number of the point information of the word segmentation result output through the first word stock;
minimizing a loss function L of the neural network word segmentation system based on minimum information entropy total And obtaining a converged neural network word segmentation system based on the minimum information entropy.
In another aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the steps of the training method of the neural network word segmentation system based on minimum information entropy as described above when the processor executes the program.
In another aspect, an embodiment of the present invention further provides a non-transitory computer readable storage medium, on which a computer program is stored, where the program when executed by a processor implements the steps of the training method of the neural network word segmentation system based on minimum information entropy as described above.
According to the embodiment of the invention, the minimum information entropy word stock prediction layer is added in the neural network word segmentation system, so that the word segmentation system has the minimum information entropy characteristic, the word segmentation with the minimum information entropy rule can be carried out on the input text, the recognition capability of the unregistered word is improved, and the word segmentation accuracy is further improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a conventional neural network word segmentation system;
fig. 2 is a schematic structural diagram of a neural network word segmentation system based on minimum information entropy according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a neural network word segmentation system based on minimum information entropy according to another embodiment of the present invention;
fig. 4 is a flowchart of a training method of a neural network word segmentation system based on minimum information entropy according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a method for constructing a minimum information entropy word stock according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 2 shows a schematic structural diagram of a neural network word segmentation system based on minimum information entropy according to an embodiment of the present invention.
As shown in fig. 2, the neural network word segmentation system based on minimum information entropy provided by the embodiment of the invention includes:
the method comprises a convolutional neural network 11, a two-way long-short term memory neural network 12, a first word stock prediction layer 13 and a minimum information entropy word stock prediction layer 14, wherein:
the convolutional neural network 11 is configured to extract a feature vector of an input text, and output the feature vector to the bidirectional long-short-term memory neural network;
the two-way long-short term memory neural network 12 is configured to receive the feature vector, read the context information and output the context information to the first word stock prediction layer and the minimum information entropy word stock prediction layer after redundancy removal;
the first word stock prediction layer 13 is configured to receive the feature vector output by the two-way long-short term memory neural network, and calculate and output a label of each word of the input text according to a first word stock;
the minimum information entropy lexicon prediction layer 14 is configured to receive the feature vector output by the two-way long-short term memory neural network, and calculate and output a label of each word of the input text according to the minimum information entropy lexicon.
The first word stock is an existing corpus, the first word stock prediction layer performs word segmentation according to the existing corpus, the minimum information entropy word stock is a corpus containing minimum information entropy features, and the minimum information entropy word stock prediction layer has the capability of predicting words which are not logged in the existing word stock according to the minimum information entropy word stock, for example, "I love Beijing" can segment labels of each word through the existing word stock, and "Beijing roast duck" can not accurately identify the labels of each word through the existing word stock, and can accurately identify the labels of each word through the minimum information entropy word stock.
According to the embodiment of the invention, the minimum information entropy word stock prediction layer is added in the neural network word segmentation system, so that the word segmentation system has the minimum information entropy characteristic, the word segmentation with the minimum information entropy rule can be carried out on the input text, the recognition capability of the unregistered word is improved, and the word segmentation accuracy is further improved.
Specifically, the minimum information entropy prediction layer comprises a full connection layer and a conditional random field layer, wherein the full connection layer is used for receiving the characteristic vector output by the two-way long-short-term memory neural network, classifying the characteristic vector and outputting the characteristic vector to the conditional random field layer; the conditional random field layer is used for calculating the feature vector output by the conditional random field according to the minimum information entropy word stock and outputting the label of each word of the input text.
The minimum information entropy prediction layer provided by the embodiment of the invention consists of a full connection layer and a conditional random field layer, and the network structure is consistent with the original word stock prediction layer.
Specifically, the label of the minimum information entropy word stock prediction layer for outputting a single word is S, the label of the first word of the multi-word is B, the label of the middle word of the multi-word is I, and the label of the last word of the multi-word is E.
Specifically, for example, the word "I am added as" S "tag, and the word" beijing roast duck "is labeled as" beijing I roast I duck E "by the minimum information entropy prediction layer.
Fig. 3 shows a schematic structural diagram of a neural network word segmentation system based on minimum information entropy according to an embodiment of the present invention.
Referring to fig. 3, a neural network word segmentation system with minimum information entropy provided by an embodiment of the present invention includes: CNN, forward LSTM, backward LSTM, original thesaurus prediction layer and minimum entropy thesaurus prediction layer, taking input text as "I love Beijing" as an example, the input text is marked as "SSBE" through the original thesaurus prediction layer and the minimum entropy thesaurus prediction layer.
The embodiment of the invention also provides a training method of the neural network word segmentation system based on the minimum information entropy.
Fig. 4 shows a flowchart of a training method of a neural network word segmentation system based on minimum information entropy according to an embodiment of the present invention.
As shown in fig. 4, the training method of the neural network word segmentation system based on the minimum information entropy provided by the embodiment of the invention specifically includes the following steps:
s21, calculating a loss function L of the neural network word segmentation system based on the minimum information entropy total The calculation formula is as follows:
L total ＝L C +L C1 +L Fab
wherein L is C For the loss function corresponding to the first word stock, L C1 A loss function corresponding to the minimum information entropy word stock, L Fab The opposite number of the point information of the word segmentation result output through the first word stock;
the embodiment of the invention modifies the loss function of the original neural network word segmentation system, wherein the modified loss function comprises the loss function corresponding to the original word stock prediction layer and the loss function corresponding to the added minimum information entropy prediction layer, and the added point information (cross entropy) is commonly used as the loss function of the neural network system.
S22, minimizing a loss function L of the neural network word segmentation system based on the minimum information entropy total And obtaining a converged neural network word segmentation system based on the minimum information entropy.
Specifically, according to the embodiment of the invention, the word segmentation system with the minimum loss function is obtained as an optimized system by adjusting each parameter in the word segmentation network.
According to the training method of the neural network word segmentation system based on the minimum information entropy, which is provided by the embodiment of the invention, the loss function of the neural network word segmentation system based on the minimum information entropy is calculated, and the optimized neural network word segmentation system is obtained by minimizing the loss function, so that the word segmentation system has the word segmentation capability through the minimum information entropy word stock.
Specifically, the loss function lc= - Σcorresponding to the first word stock i,x∈C logp(y|x；W,b)；
The loss function Lc corresponding to the minimum information entropy word stock 1 ＝-∑ i,x∈C1 logp(y|z；W,b)；
The inverse number of the point information
Wherein p (y|x; W, b) represents the probability of outputting the tag y given the input word vector x, the weight W and the bias b;
p (y|z; W, b) represents the probability of outputting the tag y given the input word vector z, the weight W and the bias b;
F ab the larger the value of (a) is, the smaller the entropy of information after word a and word b are combined together is, p ab Representing the probability of occurrence of the word a and the word b after combination.
Specifically, the network structures of the minimum information entropy prediction layer and the primitive library prediction layer are identical, but since the prediction targets are different, their internal weights are different. Thus, by improving the output layer, the network has the function of predicting the minimum information entropy word stock, and the word stock is obtained based on the minimum entropy principle, so that the vectors and weights obtained for the common part, namely the CNN part and the BLSTM part, also have the capability.
The aim of minimizing the loss function is to obtain a system with available convergence, and specifically, the embodiment of the invention adopts an Adam algorithm to timely adjust parameters such as weight W, bias b and the like, and continuously optimizes and trains a neural network word segmentation system to obtain a final word segmentation system with the capability of predicting the minimum information entropy word stock.
The embodiment of the invention also provides a method for constructing the minimum information entropy word stock.
Fig. 5 shows a flowchart of a method for constructing a minimum information entropy word stock according to an embodiment of the present invention.
As shown in fig. 5, the method for constructing the minimum information entropy word stock provided by the embodiment of the invention specifically includes the following steps:
s31, counting the probability p of the occurrence of k words after combination a Wherein k is greater than or equal to 2, and k is an integer;
s32, word segmentation is carried out in the corpus according to the probability of occurrence of the k words after combination, and a word segmentation result with the minimum entropy after combination of the k words is obtained;
and S33, marking the word segmentation result to obtain a minimum information entropy word stock.
The embodiment of the invention counts the occurrence probability of each word after combination, and performs word segmentation in a corpus according to the counted probability, for example, a plurality of words with the combination occurrence frequency smaller than a certain threshold value are separated. And (3) carrying out scanning statistics on the text to obtain a word segmentation result with minimum information entropy, marking the word segmentation result with minimum information entropy, for example, marking Beijing roast duck as Beijing I roast I duck E, adding the Beijing I roast I duck E into a minimum information entropy word stock, and continuously adding new tagged word segmentation to obtain the minimum information entropy word stock.
According to the method for constructing the word stock with the minimum information entropy, provided by the embodiment of the invention, the word segmentation result with the minimum information entropy is calculated, and the word segmentation result with the minimum entropy is marked, so that the word stock with the minimum information entropy can be obtained, the capability of identifying the unregistered words is realized, and the word segmentation accuracy is improved.
On the basis of the above embodiment, S32 specifically includes:
calculation ofF is made to a The maximum word segmentation result is the word segmentation result with the minimum entropy after combining k characters;
wherein p is a For the probability of occurrence of k words after combining, p i Is the probability of occurrence of a single word of the k words.
Specifically F a The larger the value of (2) indicates that the entropy of the information after combination is reduced, and thus, the embodiment of the invention is realized by enabling F to be a And (5) obtaining the word segmentation result with the minimum entropy by the maximum value of the (4).
On the basis of the above embodiment, S33 specifically includes:
the single word of the segmented word is marked as a label S, the first word of the segmented multi-word is marked as a label B, the middle word of the segmented multi-word is marked as a label I, and the last word of the segmented multi-word is marked as a label E.
The embodiment of the invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the method as shown in fig. 4 when executing the program.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
As shown in fig. 6, the electronic device provided by the embodiment of the present invention includes a memory 41, a processor 42, a bus 43, and a computer program stored on the memory 41 and executable on the processor 42. Wherein the memory 41 and the processor 42 communicate with each other via the bus 43.
The processor 42 is configured to invoke program instructions in the memory 41 to implement the method of fig. 4 when executing the program.
For example, the processor, when executing the program, implements the following method:
calculating a loss function of the neural network word segmentation system based on the minimum information entropy;
and minimizing the loss function of the neural network word segmentation system based on the minimum information entropy to obtain a converged neural network word segmentation system based on the minimum information entropy.
According to the electronic equipment provided by the embodiment of the invention, the optimized neural network word segmentation system is obtained by minimizing the loss function, so that the word segmentation system has the word segmentation capability through the minimum information entropy word stock.
Embodiments of the present invention also provide a non-transitory computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of fig. 4.
For example, the processor, when executing the program, implements the following method:
calculating a loss function of the neural network word segmentation system based on the minimum information entropy;
and minimizing the loss function of the neural network word segmentation system based on the minimum information entropy to obtain a converged neural network word segmentation system based on the minimum information entropy.
The non-transitory computer readable storage medium provided by the embodiment of the invention obtains the optimized neural network word segmentation system by minimizing the loss function, so that the word segmentation system has the word segmentation capability through the minimum information entropy word stock.
An embodiment of the present invention discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are capable of performing the methods provided by the above-described method embodiments, for example comprising:
calculating a loss function of the neural network word segmentation system based on the minimum information entropy;
and minimizing the loss function of the neural network word segmentation system based on the minimum information entropy to obtain a converged neural network word segmentation system based on the minimum information entropy.
Those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. A neural network word segmentation system based on minimum information entropy, the system comprising:
the system comprises a convolutional neural network, a two-way long-short-term memory neural network, a first word stock prediction layer and a minimum information entropy word stock prediction layer, wherein:
the convolutional neural network is used for extracting the characteristic vector of the input text and outputting the characteristic vector to the two-way long-short-term memory neural network;
the two-way long-short-term memory neural network is used for receiving the feature vector, reading the context information, removing redundancy and outputting the context information to the first word stock prediction layer and the minimum information entropy word stock prediction layer;
the first word stock prediction layer is used for receiving the feature vector output by the two-way long-short term memory neural network and calculating and outputting the label of each word of the input text according to the first word stock;
the minimum information entropy word stock prediction layer is used for receiving the feature vector output by the two-way long-short-term memory neural network, calculating and outputting the label of each word of the input text according to the minimum information entropy word stock;
the method for constructing the minimum information entropy word stock comprises the following steps of:
statistics of probability p of occurrence of k words after combination a Wherein k is greater than or equal to 2, and k is an integer;
word segmentation is carried out in the corpus according to the probability of occurrence of the k words after combination, and a word segmentation result with minimum entropy after combination of the k words is obtained;
marking the word segmentation result to obtain a minimum information entropy word stock;
the word segmentation result with the minimum entropy after combining the k words comprises the following steps:
calculation ofF is made to a The maximum word segmentation result is the word segmentation result with the minimum entropy after combining k characters;
wherein p is a For the probability of occurrence of k words after combining, p i Is the probability of occurrence of a single word of the k words.
2. The system of claim 1, wherein the minimum information entropy prediction layer comprises a fully connected layer and a conditional random field layer, the fully connected layer is used for receiving the feature vector output by the two-way long-short-term memory neural network, and outputting the feature vector to the conditional random field layer after classification; the conditional random field layer is used for calculating the feature vector output by the conditional random field according to the minimum information entropy word stock and outputting the label of each word of the input text.
3. The system of claim 1, wherein the minimum information entropy lexicon prediction layer is configured to output a single word with a label of S, a first word of the multiple word with a label of B, a middle word of the multiple word with a label of I, and a last word of the multiple word with a label of E.
4. A training method for a neural network word segmentation system based on minimum information entropy according to any one of claims 1 to 3, the method comprising:
calculating a loss function L of the neural network word segmentation system based on the minimum information entropy total The calculation formula is as follows:
L total ＝L C +L C1 +L Fab
wherein L is C For the loss function corresponding to the first word stock, L C1 A loss function corresponding to the minimum information entropy word stock, L Fab The opposite number of the point information of the word segmentation result output through the first word stock;
minimizing a loss function L of the neural network word segmentation system based on minimum information entropy total And obtaining a converged neural network word segmentation system based on the minimum information entropy.
5. The method of claim 4, wherein the step of determining the position of the first electrode is performed,
the loss function L corresponding to the first word stock C ＝-∑ i,x∈C logp(y|x；W,b)；
The loss function L corresponding to the minimum information entropy word stock C1 ＝-∑ i,x∈C1 logp(y|z；W,b)；
The inverse number of the point information
Wherein p (y|x; W, b) represents the probability of outputting the tag y given the input word vector x, the weight W and the bias b;
p (y|z; W, b) represents the probability of outputting the tag y given the input word vector z, the weight W and the bias b;
F ab the larger the value of (a) is, the smaller the entropy of information after word a and word b are combined together is, p ab Representing the probability of occurrence of the word a and the word b after combination.
6. The method of claim 4, wherein the marking the word segmentation result to obtain a minimum information entropy word stock comprises:
the single word of the segmented word is marked as a label S, the first word of the segmented multi-word is marked as a label B, the middle word of the segmented multi-word is marked as a label I, and the last word of the segmented multi-word is marked as a label E.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the training method of the neural network word segmentation system based on minimum information entropy as claimed in any one of claims 4 to 6 when the program is executed by the processor.
8. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the training method of the neural network word segmentation system based on minimum information entropy as claimed in any one of claims 4 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810724646.2A CN110750986B (en) | 2018-07-04 | 2018-07-04 | Neural network word segmentation system and training method based on minimum information entropy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810724646.2A CN110750986B (en) | 2018-07-04 | 2018-07-04 | Neural network word segmentation system and training method based on minimum information entropy |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110750986A CN110750986A (en) | 2020-02-04 |
| CN110750986B true CN110750986B (en) | 2023-10-10 |
Family
ID=69274634
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810724646.2A Active CN110750986B (en) | 2018-07-04 | 2018-07-04 | Neural network word segmentation system and training method based on minimum information entropy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110750986B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6505151B1 (en) * | 2000-03-15 | 2003-01-07 | Bridgewell Inc. | Method for dividing sentences into phrases using entropy calculations of word combinations based on adjacent words |
| CN102193929A (en) * | 2010-03-08 | 2011-09-21 | 阿里巴巴集团控股有限公司 | Method and equipment for determining word information entropy and searching by using word information entropy |
| CN105630890A (en) * | 2015-12-18 | 2016-06-01 | 北京中科汇联科技股份有限公司 | Neologism discovery method and system based on intelligent question-answering system session history |
| CN105740226A (en) * | 2016-01-15 | 2016-07-06 | 南京大学 | Method for implementing Chinese segmentation by using tree neural network and bilateral neural network |
| CN107145483A (en) * | 2017-04-24 | 2017-09-08 | 北京邮电大学 | A kind of adaptive Chinese word cutting method based on embedded expression |
| CN108038209A (en) * | 2017-12-18 | 2018-05-15 | 深圳前海微众银行股份有限公司 | Answer system of selection, device and computer-readable recording medium |
-
2018
- 2018-07-04 CN CN201810724646.2A patent/CN110750986B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6505151B1 (en) * | 2000-03-15 | 2003-01-07 | Bridgewell Inc. | Method for dividing sentences into phrases using entropy calculations of word combinations based on adjacent words |
| CN102193929A (en) * | 2010-03-08 | 2011-09-21 | 阿里巴巴集团控股有限公司 | Method and equipment for determining word information entropy and searching by using word information entropy |
| CN105630890A (en) * | 2015-12-18 | 2016-06-01 | 北京中科汇联科技股份有限公司 | Neologism discovery method and system based on intelligent question-answering system session history |
| CN105740226A (en) * | 2016-01-15 | 2016-07-06 | 南京大学 | Method for implementing Chinese segmentation by using tree neural network and bilateral neural network |
| CN107145483A (en) * | 2017-04-24 | 2017-09-08 | 北京邮电大学 | A kind of adaptive Chinese word cutting method based on embedded expression |
| CN108038209A (en) * | 2017-12-18 | 2018-05-15 | 深圳前海微众银行股份有限公司 | Answer system of selection, device and computer-readable recording medium |
Non-Patent Citations (4)
| Title |
|---|
| Hai Zhao等.Integrating unsupervised and supervised word segmentation: The role of goodness measures.Information Sciences.2011,第第181卷卷(第第181卷期),全文. * |
| 张华平 ; 商建云 ; .面向社会媒体的开放领域新词发现.中文信息学报.2017,第31卷(第03期),全文. * |
| 胡婕 ; 张俊驰 ; .双向循环网络中文分词模型.小型微型计算机系统.2017,(第03期),全文. * |
| 邓丽萍 ; 罗智勇 ; .基于半监督CRF的跨领域中文分词.中文信息学报.2017,第31卷(第04期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110750986A (en) | 2020-02-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109885692B (en) | Knowledge data storage method, apparatus, computer device and storage medium | |
| CN110969020B (en) | CNN and attention mechanism-based Chinese named entity identification method, system and medium | |
| US11232141B2 (en) | Method and device for processing an electronic document | |
| US20190377794A1 (en) | Method and apparatus for determining user intent | |
| CN110298035B (en) | Word vector definition method, device, equipment and storage medium based on artificial intelligence | |
| CN113313022B (en) | Training method of character recognition model and method for recognizing characters in image | |
| CN111985228B (en) | Text keyword extraction method, text keyword extraction device, computer equipment and storage medium | |
| CN110348012B (en) | Method, device, storage medium and electronic device for determining target character | |
| US20220245347A1 (en) | Entity recognition method, apparatus, electronic device and computer readable storage medium | |
| CN110751027B (en) | Pedestrian re-identification method based on deep multi-instance learning | |
| CN111583911B (en) | Speech recognition method, device, terminal and medium based on label smoothing | |
| CN114416979A (en) | Text query method, text query equipment and storage medium | |
| CN113657274A (en) | Table generation method and device, electronic equipment, storage medium and product | |
| CN113836992A (en) | Method for identifying label, method, device and equipment for training label identification model | |
| CN110298046B (en) | Translation model training method, text translation method and related device | |
| Nabil et al. | Cufe at semeval-2016 task 4: A gated recurrent model for sentiment classification | |
| CN110889287A (en) | Method and device for named entity recognition | |
| CN110750986B (en) | Neural network word segmentation system and training method based on minimum information entropy | |
| Jiang et al. | High precision deep learning-based tabular position detection | |
| CN113033205B (en) | Method, device, equipment and storage medium for entity linking | |
| CN115359296A (en) | Image recognition method and device, electronic equipment and storage medium | |
| US20210334647A1 (en) | Method, electronic device, and computer program product for determining output of neural network | |
| CN114882334A (en) | Method for generating pre-training model, model training method and device | |
| CN113962221A (en) | Text abstract extraction method and device, terminal equipment and storage medium | |
| CN110969016B (en) | Word segmentation processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |