CN111723547A - Text automatic summarization method based on pre-training language model - Google Patents
Text automatic summarization method based on pre-training language model Download PDFInfo
- Publication number
- CN111723547A CN111723547A CN202010449079.1A CN202010449079A CN111723547A CN 111723547 A CN111723547 A CN 111723547A CN 202010449079 A CN202010449079 A CN 202010449079A CN 111723547 A CN111723547 A CN 111723547A
- Authority
- CN
- China
- Prior art keywords
- vector
- representing
- word
- text
- language model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 28
- 238000000034 method Methods 0.000 title claims abstract description 27
- 239000013598 vector Substances 0.000 claims description 79
- 239000011159 matrix material Substances 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 10
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 4
- YTAHJIFKAKIKAV-XNMGPUDCSA-N [(1R)-3-morpholin-4-yl-1-phenylpropyl] N-[(3S)-2-oxo-5-phenyl-1,3-dihydro-1,4-benzodiazepin-3-yl]carbamate Chemical compound O=C1[C@H](N=C(C2=C(N1)C=CC=C2)C1=CC=CC=C1)NC(O[C@H](CCN1CCOCC1)C1=CC=CC=C1)=O YTAHJIFKAKIKAV-XNMGPUDCSA-N 0.000 claims description 3
- 230000017105 transposition Effects 0.000 claims description 2
- 238000003058 natural language processing Methods 0.000 abstract description 4
- 238000000605 extraction Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/126—Character encoding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/258—Heading extraction; Automatic titling; Numbering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Probability & Statistics with Applications (AREA)
- Machine Translation (AREA)
Abstract
The invention relates to a text automatic summarization method based on a pre-training language model, belonging to the technical field of natural language processing. The method comprises the following steps: encoding the source text information by using a pre-training language model BERT network; the digest is then automatically generated for the source text by the LSTM joint attention mechanism. In the automatic Chinese text summarization task, the generated Chinese summary achieves good readability, the generated summary is high in quality, meanwhile, the model training speed is high, and due to the fact that the pre-training language model is used as the encoder, even under the condition that training data are few, the summary with relatively high quality can be generated.
Description
Technical Field
The invention relates to a text automatic summarization method based on a pre-training language model, belonging to the technical field of natural language processing.
Background
With the continuous emergence of new media platforms, the information which people contact daily shows explosive growth, thereby bringing the trouble of information overload to people, and with the acceleration of life rhythm, people are overwhelmed to comb all the received information. By reading the abstract, people can improve the efficiency of understanding the original text and effectively reduce the time and energy for browsing information.
The text summarization technology commonly used at home and abroad at present can be mainly divided into an extraction type and a generation type.
The extraction method is to calculate the importance of a sentence according to the statistical characteristics of the text, such as word frequency, inverse document frequency, and the like, so as to select the most important sentence as the abstract of the original text, which can be regarded as a sentence classification task and becomes an abstract with a high score, otherwise, it becomes a non-abstract sentence. The development time of the extraction method is long, and the existing relatively mature automatic summarization technology still adopts the extraction method. The quality of the abstracts obtained by the method is lower, and the extracted sentences have higher redundancy, so that the quality and the effect of the generated abstracts are not satisfactory.
The generating method is that the computer 'understands' the text, and according to the way of writing the abstract by human beings, the syntactic and semantic analysis is carried out according to the information in the obtained text, thereby generating a new abstract sentence. With the continuous development of deep learning, the method for generating text summaries based on deep learning has greatly improved summary quality and fluency, and is also called as a mainstream research direction for text summary generation.
The Seq2Seq model is an end-to-end model, 2014 firstly proposes and applies to a machine translation task, and a good effect is obtained at that time. The text summarization task is also an end-to-end text generation task, so that researchers think that the Seq2Seq model is applied to the text summarization task, and then the generation-type summarization is carried out a series of researches based on the model idea, and the generation-type automatic text summarization technology is rapidly developed.
Pre-training the language model refers to applying the trained language model to other natural language processing tasks. Word Embedding (Word Embedding) appeared in 2003 is the earliest pre-training technology, does not need large-scale labeled data sets, and can learn semantic similarity between words without supervision. With the continuous development and improvement of pre-training language models, pre-training language models such as ELMO, GPT, BERT and the like appear, and the models have the common characteristic that good text representation can be obtained through a large amount of pre-training and strong information extraction capability in the models, and the results can be obtained with half the effort when the models are applied to downstream natural language processing tasks.
Disclosure of Invention
The invention provides a text automatic summarization method based on a pre-training language model, which aims to solve the problems that the existing text automatic summarization is provided, for example, a coding end cannot well learn information in a source text, so that a decoding end has the problems of repeated summarization or irrelevant semantics and the like.
The invention adopts the following technical scheme for solving the technical problems:
a text automatic summarization method based on a pre-training language model comprises the following steps:
(1) globally coding the context information of the text based on a BERT pre-training language model;
(2) decoding the coded output result based on an attention mechanism and an LSTM network to generate a text abstract;
(3) finally, calculating the hidden state vector, the context vector and the word vector of the previous abstract word to obtain the output state vector, and selecting the value with the maximum corresponding probability as the output of the step according to the probability corresponding to each possible output word.
In the step (1), the BERT pre-training language model is a superimposed self-attention model, and the self-attention model is coded as follows:
Ql＝Hl-1Wl Q，
wherein: wl QA matrix of query vectors representing the l-th layer,
a key vector matrix representing the l-th layer,
a value vector matrix representing the l-th layer; ql，Kl，VlRespectively representing a query vector, a key vector and a value vector of the l layer; dkVector dimensions representing Q and K; softmax denotes the softmax function, T denotes the transposition of the vector matrix, Hl-1Representing the result after the operation of the previous layer of the Transformer module; a. thelThe result after the self-attention model is finally shown, and the result H after the I layer coding is finally obtained after the full connection layer and the residual layerl。
The specific process of the step (2) is as follows: and calculating a context vector of a certain word vector through an attention mechanism, and calculating to obtain a next generated word to be predicted by combining the output of the encoder.
The method for calculating the context vector of a certain word vector is as follows:
wherein: hiRepresenting the i-th word vector output of the encoder, SjIndicating the hidden state of the decoder at step j, e (H)iSj) Is a function for calculating whether each word of the original text is related to the current decoder state, which is a feedforward neural network with a single hidden layer, exp () represents an exponential function with a natural constant e as the base, αi,jWeight of influence of output representing i position of encoder on j prediction result of decoder, cjRepresenting the hidden state after the attention mechanism for subsequent generation of a prediction of the word.
Hidden state vector S of j step of the decoderjThe calculation formula of (a) is as follows:
Sj＝LSTM(yj-1,cj-1,Sj-1)
where LSTM (·) denotes LSTM network operation, yj-1A word vector representing the previous generated word, cj-1Context vector, S, representing the previous word vectorj-1Representing the previous hidden state vector.
The output state vector in the step (3):
yj＝Wryj-1+Urcj+VrSj
wherein Wr，Ur，VrAre trainable weight matrix parameters.
The invention has the following beneficial effects:
the invention combines the pre-training language model BERT, the attention mechanism and the LSTM (long short term memory network) model, and effectively solves the defect that the traditional RNN (recurrent neural network) can not acquire long-distance information. Because the pre-training language model BERT is a bi-directional training language model and a network model using a self-attention mechanism, such as a Transformer, is adopted, the information of the context of the source text can be effectively acquired, and the defect of insufficient information acquisition in a long distance is overcome. Therefore, the hidden state obtained after the source text passes through the encoder actually contains rich context information, and more favorable for generating more accurate abstract in the subsequent decoding problem. Meanwhile, because the pre-training language model is trained by a large amount of text data, the effect which is more excellent than the prior effect can be obtained only by fine adjustment aiming at the text abstract task, and meanwhile, the training speed of the network model can be improved.
Drawings
Fig. 1 is a flowchart of a text automatic summarization method provided by the present invention.
Fig. 2 is a schematic diagram of an automatic text summarization model provided by the present invention.
FIG. 3 is a schematic diagram of the coding of the pre-training model BERT provided by the present invention.
Detailed Description
The invention is described in further detail below with reference to the accompanying drawings.
FIG. 1 is a flow chart of the text automatic summarization method based on a pre-training language model according to the present invention. The device is mainly divided into two parts:
(1) the source text is encoded using an encoder built based on a pre-trained language model.
(2) And decoding the coded vector by using a decoder constructed based on an LSTM network and combining an attention mechanism to generate the text abstract.
For a text content, a summary is automatically generated, which can be implemented by the method described in fig. 2. The specific process is as follows:
(1) the source text is encoded using a BERT pretrained network.
The processed source text information may be represented as H0＝[E1,…,EN]Wherein: e1A vector representation representing the first word (character) in the source text; eNA vector representation representing the nth word (character) in the source text. After the process of the BERT pre-training language model, that is, the computation of the Transformer model of the L layer, the coding result can be obtained:
Q＝Hl-1Wl Q，
Hland (3) representing the output result obtained after passing through the Transformer model of the previous layer.
Hl＝Transformerl(Hl-1),l∈[1,L]
For each input vector (input word vector), a query vector, a key vector, a value vector is created. Wl QThe query vector matrix representing the l-th layer, similarly,
a key vector matrix representing the l-th layer,
a vector matrix of values representing the l-th layer. Ql，Kl，VlRespectively representing the query vector, the key vector and the value vector of the l-th layer. dkRepresenting the vector dimensions of Q and K. softmax denotes the softmax function. T denotes transposing the vector matrix, Hl-1And (4) representing the result of the operation of the previous layer of the Transformer module. A. thelThe result after the self-attention model is finally shown, and the result H after the I layer coding is finally obtained after the full connection layer and the residual layerl。
(2) Computing encoder output using an attention mechanism
The vector matrix containing the context information of each word vector can be better obtained through an attention mechanism, and the decoding prediction process is better assisted in a decoding stage. The attention mechanism is calculated along with this decoding process.
(3) The unidirectional LSTM network acts as a decoder and incorporates an attention mechanism before predicting the word to be generated each time, since the attention mechanism can better extract the context information of all word vectors in the source text and thus can better assist in generating the summary.
Wherein HiRepresenting the i-th word vector output of the encoder, SjIndicating the hidden state of the decoder at step j, e (H)iSj) Is a function for calculating whether each word of the original text is related to the current decoder state, which is a feedforward neural network with a single hidden layer, exp () represents an exponential function with a natural constant e as the base, αi,jIndicating the ith position of the encoderThe influence weight of the output of (1) on the prediction result of the j step of the decoder, cjRepresenting the hidden state after the attention mechanism for subsequent generation of a prediction of the word. Wherein the hidden state vector S in the decoderjThe calculation formula of (2) is as follows:
Sj＝LSTM(yj-1,cj-1,Sj-1)
where LSTM (·) denotes LSTM network operation, yj-1A word vector representing the previous generated word, cj-1Context vector, S, representing the previous word vectorj-1Representing the previous hidden state vector, the initial hidden state vector S0Comprises the following steps:
S0＝tanh(WdhN+b)
wherein tanh (. cndot.) represents a hyperbolic tangent function, hNFor the result of the last coded output, WdA weight matrix representing trainable hidden states, b represents a hidden state bias vector.
Finally by the hidden state vector SjContext vector cjAnd the word vector y of the previous abstract wordj-1By performing the calculation, the output state vector can be obtained:
yj＝Wryj-1+Urcj+VrSj
wherein Wr，Ur，VrAre trainable weight matrix parameters.
And obtaining an output state vector, and selecting a value with the maximum corresponding probability as the output of the step according to the probability corresponding to each possible output word.
FIG. 3 shows a schematic diagram of the coding of the pre-training model BERT provided by the present invention. As shown, Trm represents an abbreviated form of the Transformer module. The BERT model can adopt a bidirectional language model to code the source text, and can more fully acquire the context information of the source text, thereby improving the quality of the final generated abstract.
Claims (6)
1. A text automatic summarization method based on a pre-training language model is characterized by comprising the following steps:
(1) globally coding the context information of the text based on a BERT pre-training language model;
(2) decoding the coded output result based on an attention mechanism and an LSTM network to generate a text abstract;
(3) finally, calculating the hidden state vector, the context vector and the word vector of the previous abstract word to obtain the output state vector, and selecting the value with the maximum corresponding probability as the output of the step according to the probability corresponding to each possible output word.
2. The method for automatically abstracting text based on pre-trained language model as claimed in claim 1, wherein the BERT pre-trained language model in step (1) is a superimposed self-attention model encoded as:
Ql＝Hl-1Wl Q，Kl＝Hl-1Wl K，Vl＝Hl-1Wl V，
wherein: wl QQuery vector matrix, W, representing the l-th layerl KA key vector matrix, W, representing the l-th layerl VA value vector matrix representing the l-th layer; ql，Kl，VlRespectively representing a query vector, a key vector and a value vector of the l layer; dkVector dimensions representing Q and K; softmax denotes the softmax function, T denotes the transposition of the vector matrix, Hl-1Representing the result after the operation of the previous layer of the Transformer module; a. thelThe result after the self-attention model is finally shown, and the result H after the I layer coding is finally obtained after the full connection layer and the residual layerl。
3. The method for automatically abstracting text based on pre-trained language model as claimed in claim 1, wherein the specific process of step (2) is as follows: and calculating a context vector of a certain word vector through an attention mechanism, and calculating to obtain a next generated word to be predicted by combining the output of the encoder.
4. The method of claim 3, wherein the method of calculating the context vector of a word vector comprises:
wherein: hiRepresenting the i-th word vector output of the encoder, SjIndicating the hidden state of the decoder at step j, e (H)iSj) Is a function for calculating whether each word of the original text is related to the current decoder state, which is a feedforward neural network with a single hidden layer, exp () represents an exponential function with a natural constant e as the base, αi,jWeight of influence of output representing i position of encoder on j prediction result of decoder, cjRepresenting the hidden state after the attention mechanism for subsequent generation of a prediction of the word.
5. The method according to claim 4, wherein the hidden state vector S of the decoder at the j-th step is a hidden state vector SjThe calculation formula of (a) is as follows:
Sj＝LSTM(yj-1,cj-1,Sj-1)
where LSTM (·) denotes LSTM network operation, yj-1A word vector representing the previous generated word, cj-1Context vector, S, representing the previous word vectorj-1Before showingA hidden state vector.
6. The method for automatically abstracting text based on pre-trained language model as claimed in claim 5, wherein the state vector outputted in step (3):
yj＝Wryj-1+Urcj+VrSj
wherein Wr，Ur，VrAre trainable weight matrix parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010449079.1A CN111723547A (en) | 2020-05-25 | 2020-05-25 | Text automatic summarization method based on pre-training language model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010449079.1A CN111723547A (en) | 2020-05-25 | 2020-05-25 | Text automatic summarization method based on pre-training language model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111723547A true CN111723547A (en) | 2020-09-29 |
Family
ID=72564980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010449079.1A Pending CN111723547A (en) | 2020-05-25 | 2020-05-25 | Text automatic summarization method based on pre-training language model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111723547A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112395841A (en) * | 2020-11-18 | 2021-02-23 | 福州大学 | BERT-based method for automatically filling blank text |
| CN112417139A (en) * | 2020-11-19 | 2021-02-26 | 深圳大学 | Abstract generation method based on pre-training language model |
| CN112507081A (en) * | 2020-12-16 | 2021-03-16 | 平安科技（深圳）有限公司 | Similar sentence matching method and device, computer equipment and storage medium |
| CN112559730A (en) * | 2020-12-08 | 2021-03-26 | 北京京航计算通讯研究所 | Text abstract automatic generation method and system based on global feature extraction |
| CN112559702A (en) * | 2020-11-10 | 2021-03-26 | 西安理工大学 | Transformer-based natural language problem generation method in civil construction information field |
| CN113128214A (en) * | 2021-03-17 | 2021-07-16 | 重庆邮电大学 | Text abstract generation method based on BERT pre-training model |
| CN113157855A (en) * | 2021-02-22 | 2021-07-23 | 福州大学 | Text summarization method and system fusing semantic and context information |
| CN113159168A (en) * | 2021-04-19 | 2021-07-23 | 清华大学 | Pre-training model accelerated reasoning method and system based on redundant word deletion |
| CN113407711A (en) * | 2021-06-17 | 2021-09-17 | 成都崇瑚信息技术有限公司 | Gibbs limited text abstract generation method by using pre-training model |
| CN115081437A (en) * | 2022-07-20 | 2022-09-20 | 中国电子科技集团公司第三十研究所 | Machine-generated text detection method and system based on linguistic feature contrast learning |
| CN115687031A (en) * | 2022-11-15 | 2023-02-03 | 北京优特捷信息技术有限公司 | Method, device, equipment and medium for generating alarm description text |
| WO2023061107A1 (en) * | 2021-10-13 | 2023-04-20 | 北京有竹居网络技术有限公司 | Language translation method and apparatus based on layer prediction, and device and medium |
| CN116562275A (en) * | 2023-06-09 | 2023-08-08 | 创意信息技术股份有限公司 | Automatic text summarization method combined with entity attribute diagram |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018233647A1 (en) * | 2017-06-22 | 2018-12-27 | 腾讯科技（深圳）有限公司 | Abstract generation method, device and computer device and storage medium |
| CN110737769A (en) * | 2019-10-21 | 2020-01-31 | 南京信息工程大学 | pre-training text abstract generation method based on neural topic memory |
-
2020
- 2020-05-25 CN CN202010449079.1A patent/CN111723547A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018233647A1 (en) * | 2017-06-22 | 2018-12-27 | 腾讯科技（深圳）有限公司 | Abstract generation method, device and computer device and storage medium |
| CN110737769A (en) * | 2019-10-21 | 2020-01-31 | 南京信息工程大学 | pre-training text abstract generation method based on neural topic memory |
Non-Patent Citations (1)
| Title |
|---|
| 唐子惠 编著: "《医学人工智能导论》", 30 April 2020, 上海科学技术出版社 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112559702A (en) * | 2020-11-10 | 2021-03-26 | 西安理工大学 | Transformer-based natural language problem generation method in civil construction information field |
| CN112559702B (en) * | 2020-11-10 | 2022-09-30 | 西安理工大学 | Method for generating natural language problem in civil construction information field based on Transformer |
| CN112395841B (en) * | 2020-11-18 | 2022-05-13 | 福州大学 | BERT-based method for automatically filling blank text |
| CN112395841A (en) * | 2020-11-18 | 2021-02-23 | 福州大学 | BERT-based method for automatically filling blank text |
| CN112417139A (en) * | 2020-11-19 | 2021-02-26 | 深圳大学 | Abstract generation method based on pre-training language model |
| CN112417139B (en) * | 2020-11-19 | 2023-07-25 | 深圳大学 | Abstract generation method based on pre-training language model |
| WO2022104967A1 (en) * | 2020-11-19 | 2022-05-27 | 深圳大学 | Pre-training language model-based summarization generation method |
| CN112559730A (en) * | 2020-12-08 | 2021-03-26 | 北京京航计算通讯研究所 | Text abstract automatic generation method and system based on global feature extraction |
| CN112507081A (en) * | 2020-12-16 | 2021-03-16 | 平安科技（深圳）有限公司 | Similar sentence matching method and device, computer equipment and storage medium |
| CN113157855A (en) * | 2021-02-22 | 2021-07-23 | 福州大学 | Text summarization method and system fusing semantic and context information |
| CN113128214A (en) * | 2021-03-17 | 2021-07-16 | 重庆邮电大学 | Text abstract generation method based on BERT pre-training model |
| CN113128214B (en) * | 2021-03-17 | 2022-05-06 | 重庆邮电大学 | Text abstract generation method based on BERT pre-training model |
| CN113159168B (en) * | 2021-04-19 | 2022-09-02 | 清华大学 | Pre-training model accelerated reasoning method and system based on redundant word deletion |
| CN113159168A (en) * | 2021-04-19 | 2021-07-23 | 清华大学 | Pre-training model accelerated reasoning method and system based on redundant word deletion |
| CN113407711A (en) * | 2021-06-17 | 2021-09-17 | 成都崇瑚信息技术有限公司 | Gibbs limited text abstract generation method by using pre-training model |
| CN113407711B (en) * | 2021-06-17 | 2023-04-07 | 成都崇瑚信息技术有限公司 | Gibbs limited text abstract generation method by using pre-training model |
| WO2023061107A1 (en) * | 2021-10-13 | 2023-04-20 | 北京有竹居网络技术有限公司 | Language translation method and apparatus based on layer prediction, and device and medium |
| CN115081437A (en) * | 2022-07-20 | 2022-09-20 | 中国电子科技集团公司第三十研究所 | Machine-generated text detection method and system based on linguistic feature contrast learning |
| CN115687031A (en) * | 2022-11-15 | 2023-02-03 | 北京优特捷信息技术有限公司 | Method, device, equipment and medium for generating alarm description text |
| CN116562275A (en) * | 2023-06-09 | 2023-08-08 | 创意信息技术股份有限公司 | Automatic text summarization method combined with entity attribute diagram |
| CN116562275B (en) * | 2023-06-09 | 2023-09-15 | 创意信息技术股份有限公司 | Automatic text summarization method combined with entity attribute diagram |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111723547A (en) | Text automatic summarization method based on pre-training language model | |
| CN111897949B (en) | Guided text abstract generation method based on Transformer | |
| CN109635124B (en) | Remote supervision relation extraction method combined with background knowledge | |
| CN109918666B (en) | Chinese punctuation mark adding method based on neural network | |
| CN111241816B (en) | Automatic news headline generation method | |
| CN109992775B (en) | Text abstract generation method based on high-level semantics | |
| CN110929030A (en) | Text abstract and emotion classification combined training method | |
| CN111143563A (en) | Text classification method based on integration of BERT, LSTM and CNN | |
| CN110737769A (en) | pre-training text abstract generation method based on neural topic memory | |
| CN109992669B (en) | Keyword question-answering method based on language model and reinforcement learning | |
| CN111858932A (en) | Multiple-feature Chinese and English emotion classification method and system based on Transformer | |
| CN111125333B (en) | Generation type knowledge question-answering method based on expression learning and multi-layer covering mechanism | |
| CN112232053A (en) | Text similarity calculation system, method and storage medium based on multi-keyword pair matching | |
| CN111966797B (en) | Method for machine reading and understanding by using word vector introduced with semantic information | |
| CN116414962A (en) | Question-answer matching method based on attention mechanism | |
| CN115238691A (en) | Knowledge fusion based embedded multi-intention recognition and slot filling model | |
| CN115169349A (en) | Chinese electronic resume named entity recognition method based on ALBERT | |
| CN114757184A (en) | Method and system for realizing knowledge question answering in aviation field | |
| CN112818124A (en) | Entity relationship extraction method based on attention neural network | |
| CN116521857A (en) | Method and device for abstracting multi-text answer abstract of question driven abstraction based on graphic enhancement | |
| CN116663501A (en) | Chinese variant text conversion method based on multi-modal sharing weight | |
| Jiang et al. | A hierarchical bidirectional LSTM sequence model for extractive text summarization in electric power systems | |
| CN115759102A (en) | Chinese poetry wine culture named entity recognition method | |
| Luo et al. | Learning document embeddings with crossword prediction | |
| CN114611487B (en) | Unsupervised Thai dependency syntax analysis method based on dynamic word embedding alignment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200929 |
|
| RJ01 | Rejection of invention patent application after publication |