CN111723547A - Text automatic summarization method based on pre-training language model - Google Patents

Text automatic summarization method based on pre-training language model Download PDF

Info

Publication number
CN111723547A
CN111723547A CN202010449079.1A CN202010449079A CN111723547A CN 111723547 A CN111723547 A CN 111723547A CN 202010449079 A CN202010449079 A CN 202010449079A CN 111723547 A CN111723547 A CN 111723547A
Authority
CN
China
Prior art keywords
vector
representing
word
text
language model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010449079.1A
Other languages
Chinese (zh)
Inventor
王宇
师岩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hohai University HHU
Original Assignee
Hohai University HHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hohai University HHU filed Critical Hohai University HHU
Priority to CN202010449079.1A priority Critical patent/CN111723547A/en
Publication of CN111723547A publication Critical patent/CN111723547A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/12Use of codes for handling textual entities
    • G06F40/126Character encoding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/258Heading extraction; Automatic titling; Numbering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/30Semantic analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/044Recurrent networks, e.g. Hopfield networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/047Probabilistic or stochastic networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/049Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Machine Translation (AREA)

Abstract

The invention relates to a text automatic summarization method based on a pre-training language model, belonging to the technical field of natural language processing. The method comprises the following steps: encoding the source text information by using a pre-training language model BERT network; the digest is then automatically generated for the source text by the LSTM joint attention mechanism. In the automatic Chinese text summarization task, the generated Chinese summary achieves good readability, the generated summary is high in quality, meanwhile, the model training speed is high, and due to the fact that the pre-training language model is used as the encoder, even under the condition that training data are few, the summary with relatively high quality can be generated.

Description

Text automatic summarization method based on pre-training language model
Technical Field
The invention relates to a text automatic summarization method based on a pre-training language model, belonging to the technical field of natural language processing.
Background
With the continuous emergence of new media platforms, the information which people contact daily shows explosive growth, thereby bringing the trouble of information overload to people, and with the acceleration of life rhythm, people are overwhelmed to comb all the received information. By reading the abstract, people can improve the efficiency of understanding the original text and effectively reduce the time and energy for browsing information.
The text summarization technology commonly used at home and abroad at present can be mainly divided into an extraction type and a generation type.
The extraction method is to calculate the importance of a sentence according to the statistical characteristics of the text, such as word frequency, inverse document frequency, and the like, so as to select the most important sentence as the abstract of the original text, which can be regarded as a sentence classification task and becomes an abstract with a high score, otherwise, it becomes a non-abstract sentence. The development time of the extraction method is long, and the existing relatively mature automatic summarization technology still adopts the extraction method. The quality of the abstracts obtained by the method is lower, and the extracted sentences have higher redundancy, so that the quality and the effect of the generated abstracts are not satisfactory.
The generating method is that the computer 'understands' the text, and according to the way of writing the abstract by human beings, the syntactic and semantic analysis is carried out according to the information in the obtained text, thereby generating a new abstract sentence. With the continuous development of deep learning, the method for generating text summaries based on deep learning has greatly improved summary quality and fluency, and is also called as a mainstream research direction for text summary generation.
The Seq2Seq model is an end-to-end model, 2014 firstly proposes and applies to a machine translation task, and a good effect is obtained at that time. The text summarization task is also an end-to-end text generation task, so that researchers think that the Seq2Seq model is applied to the text summarization task, and then the generation-type summarization is carried out a series of researches based on the model idea, and the generation-type automatic text summarization technology is rapidly developed.
Pre-training the language model refers to applying the trained language model to other natural language processing tasks. Word Embedding (Word Embedding) appeared in 2003 is the earliest pre-training technology, does not need large-scale labeled data sets, and can learn semantic similarity between words without supervision. With the continuous development and improvement of pre-training language models, pre-training language models such as ELMO, GPT, BERT and the like appear, and the models have the common characteristic that good text representation can be obtained through a large amount of pre-training and strong information extraction capability in the models, and the results can be obtained with half the effort when the models are applied to downstream natural language processing tasks.
Disclosure of Invention
The invention provides a text automatic summarization method based on a pre-training language model, which aims to solve the problems that the existing text automatic summarization is provided, for example, a coding end cannot well learn information in a source text, so that a decoding end has the problems of repeated summarization or irrelevant semantics and the like.
The invention adopts the following technical scheme for solving the technical problems:
a text automatic summarization method based on a pre-training language model comprises the following steps:
(1) globally coding the context information of the text based on a BERT pre-training language model;
(2) decoding the coded output result based on an attention mechanism and an LSTM network to generate a text abstract;
(3) finally, calculating the hidden state vector, the context vector and the word vector of the previous abstract word to obtain the output state vector, and selecting the value with the maximum corresponding probability as the output of the step according to the probability corresponding to each possible output word.
In the step (1), the BERT pre-training language model is a superimposed self-attention model, and the self-attention model is coded as follows:
Ql=Hl-1Wl Q
Figure BDA0002506866590000031
Figure BDA0002506866590000032
wherein: wl QA matrix of query vectors representing the l-th layer,
Figure BDA0002506866590000033
a key vector matrix representing the l-th layer,
Figure BDA0002506866590000034
a value vector matrix representing the l-th layer; ql,Kl,VlRespectively representing a query vector, a key vector and a value vector of the l layer; dkVector dimensions representing Q and K; softmax denotes the softmax function, T denotes the transposition of the vector matrix, Hl-1Representing the result after the operation of the previous layer of the Transformer module; a. thelThe result after the self-attention model is finally shown, and the result H after the I layer coding is finally obtained after the full connection layer and the residual layerl
The specific process of the step (2) is as follows: and calculating a context vector of a certain word vector through an attention mechanism, and calculating to obtain a next generated word to be predicted by combining the output of the encoder.
The method for calculating the context vector of a certain word vector is as follows:
Figure BDA0002506866590000041
Figure BDA0002506866590000042
wherein: hiRepresenting the i-th word vector output of the encoder, SjIndicating the hidden state of the decoder at step j, e (H)iSj) Is a function for calculating whether each word of the original text is related to the current decoder state, which is a feedforward neural network with a single hidden layer, exp () represents an exponential function with a natural constant e as the base, αi,jWeight of influence of output representing i position of encoder on j prediction result of decoder, cjRepresenting the hidden state after the attention mechanism for subsequent generation of a prediction of the word.
Hidden state vector S of j step of the decoderjThe calculation formula of (a) is as follows:
Sj=LSTM(yj-1,cj-1,Sj-1)
where LSTM (·) denotes LSTM network operation, yj-1A word vector representing the previous generated word, cj-1Context vector, S, representing the previous word vectorj-1Representing the previous hidden state vector.
The output state vector in the step (3):
yj=Wryj-1+Urcj+VrSj
wherein Wr,Ur,VrAre trainable weight matrix parameters.
The invention has the following beneficial effects:
the invention combines the pre-training language model BERT, the attention mechanism and the LSTM (long short term memory network) model, and effectively solves the defect that the traditional RNN (recurrent neural network) can not acquire long-distance information. Because the pre-training language model BERT is a bi-directional training language model and a network model using a self-attention mechanism, such as a Transformer, is adopted, the information of the context of the source text can be effectively acquired, and the defect of insufficient information acquisition in a long distance is overcome. Therefore, the hidden state obtained after the source text passes through the encoder actually contains rich context information, and more favorable for generating more accurate abstract in the subsequent decoding problem. Meanwhile, because the pre-training language model is trained by a large amount of text data, the effect which is more excellent than the prior effect can be obtained only by fine adjustment aiming at the text abstract task, and meanwhile, the training speed of the network model can be improved.
Drawings
Fig. 1 is a flowchart of a text automatic summarization method provided by the present invention.
Fig. 2 is a schematic diagram of an automatic text summarization model provided by the present invention.
FIG. 3 is a schematic diagram of the coding of the pre-training model BERT provided by the present invention.
Detailed Description
The invention is described in further detail below with reference to the accompanying drawings.
FIG. 1 is a flow chart of the text automatic summarization method based on a pre-training language model according to the present invention. The device is mainly divided into two parts:
(1) the source text is encoded using an encoder built based on a pre-trained language model.
(2) And decoding the coded vector by using a decoder constructed based on an LSTM network and combining an attention mechanism to generate the text abstract.
For a text content, a summary is automatically generated, which can be implemented by the method described in fig. 2. The specific process is as follows:
(1) the source text is encoded using a BERT pretrained network.
The processed source text information may be represented as H0=[E1,…,EN]Wherein: e1A vector representation representing the first word (character) in the source text; eNA vector representation representing the nth word (character) in the source text. After the process of the BERT pre-training language model, that is, the computation of the Transformer model of the L layer, the coding result can be obtained:
Q=Hl-1Wl Q
Figure BDA0002506866590000061
Figure BDA0002506866590000062
Hland (3) representing the output result obtained after passing through the Transformer model of the previous layer.
Hl=Transformerl(Hl-1),l∈[1,L]
For each input vector (input word vector), a query vector, a key vector, a value vector is created. Wl QThe query vector matrix representing the l-th layer, similarly,
Figure BDA0002506866590000063
a key vector matrix representing the l-th layer,
Figure BDA0002506866590000064
a vector matrix of values representing the l-th layer. Ql,Kl,VlRespectively representing the query vector, the key vector and the value vector of the l-th layer. dkRepresenting the vector dimensions of Q and K. softmax denotes the softmax function. T denotes transposing the vector matrix, Hl-1And (4) representing the result of the operation of the previous layer of the Transformer module. A. thelThe result after the self-attention model is finally shown, and the result H after the I layer coding is finally obtained after the full connection layer and the residual layerl
(2) Computing encoder output using an attention mechanism
The vector matrix containing the context information of each word vector can be better obtained through an attention mechanism, and the decoding prediction process is better assisted in a decoding stage. The attention mechanism is calculated along with this decoding process.
(3) The unidirectional LSTM network acts as a decoder and incorporates an attention mechanism before predicting the word to be generated each time, since the attention mechanism can better extract the context information of all word vectors in the source text and thus can better assist in generating the summary.
Figure BDA0002506866590000071
Figure BDA0002506866590000072
Wherein HiRepresenting the i-th word vector output of the encoder, SjIndicating the hidden state of the decoder at step j, e (H)iSj) Is a function for calculating whether each word of the original text is related to the current decoder state, which is a feedforward neural network with a single hidden layer, exp () represents an exponential function with a natural constant e as the base, αi,jIndicating the ith position of the encoderThe influence weight of the output of (1) on the prediction result of the j step of the decoder, cjRepresenting the hidden state after the attention mechanism for subsequent generation of a prediction of the word. Wherein the hidden state vector S in the decoderjThe calculation formula of (2) is as follows:
Sj=LSTM(yj-1,cj-1,Sj-1)
where LSTM (·) denotes LSTM network operation, yj-1A word vector representing the previous generated word, cj-1Context vector, S, representing the previous word vectorj-1Representing the previous hidden state vector, the initial hidden state vector S0Comprises the following steps:
S0=tanh(WdhN+b)
wherein tanh (. cndot.) represents a hyperbolic tangent function, hNFor the result of the last coded output, WdA weight matrix representing trainable hidden states, b represents a hidden state bias vector.
Finally by the hidden state vector SjContext vector cjAnd the word vector y of the previous abstract wordj-1By performing the calculation, the output state vector can be obtained:
yj=Wryj-1+Urcj+VrSj
wherein Wr,Ur,VrAre trainable weight matrix parameters.
And obtaining an output state vector, and selecting a value with the maximum corresponding probability as the output of the step according to the probability corresponding to each possible output word.
FIG. 3 shows a schematic diagram of the coding of the pre-training model BERT provided by the present invention. As shown, Trm represents an abbreviated form of the Transformer module. The BERT model can adopt a bidirectional language model to code the source text, and can more fully acquire the context information of the source text, thereby improving the quality of the final generated abstract.

Claims (6)

1. A text automatic summarization method based on a pre-training language model is characterized by comprising the following steps:
(1) globally coding the context information of the text based on a BERT pre-training language model;
(2) decoding the coded output result based on an attention mechanism and an LSTM network to generate a text abstract;
(3) finally, calculating the hidden state vector, the context vector and the word vector of the previous abstract word to obtain the output state vector, and selecting the value with the maximum corresponding probability as the output of the step according to the probability corresponding to each possible output word.
2. The method for automatically abstracting text based on pre-trained language model as claimed in claim 1, wherein the BERT pre-trained language model in step (1) is a superimposed self-attention model encoded as:
Ql=Hl-1Wl Q,Kl=Hl-1Wl K,Vl=Hl-1Wl V
Figure FDA0002506866580000011
wherein: wl QQuery vector matrix, W, representing the l-th layerl KA key vector matrix, W, representing the l-th layerl VA value vector matrix representing the l-th layer; ql,Kl,VlRespectively representing a query vector, a key vector and a value vector of the l layer; dkVector dimensions representing Q and K; softmax denotes the softmax function, T denotes the transposition of the vector matrix, Hl-1Representing the result after the operation of the previous layer of the Transformer module; a. thelThe result after the self-attention model is finally shown, and the result H after the I layer coding is finally obtained after the full connection layer and the residual layerl
3. The method for automatically abstracting text based on pre-trained language model as claimed in claim 1, wherein the specific process of step (2) is as follows: and calculating a context vector of a certain word vector through an attention mechanism, and calculating to obtain a next generated word to be predicted by combining the output of the encoder.
4. The method of claim 3, wherein the method of calculating the context vector of a word vector comprises:
Figure FDA0002506866580000021
Figure FDA0002506866580000022
wherein: hiRepresenting the i-th word vector output of the encoder, SjIndicating the hidden state of the decoder at step j, e (H)iSj) Is a function for calculating whether each word of the original text is related to the current decoder state, which is a feedforward neural network with a single hidden layer, exp () represents an exponential function with a natural constant e as the base, αi,jWeight of influence of output representing i position of encoder on j prediction result of decoder, cjRepresenting the hidden state after the attention mechanism for subsequent generation of a prediction of the word.
5. The method according to claim 4, wherein the hidden state vector S of the decoder at the j-th step is a hidden state vector SjThe calculation formula of (a) is as follows:
Sj=LSTM(yj-1,cj-1,Sj-1)
where LSTM (·) denotes LSTM network operation, yj-1A word vector representing the previous generated word, cj-1Context vector, S, representing the previous word vectorj-1Before showingA hidden state vector.
6. The method for automatically abstracting text based on pre-trained language model as claimed in claim 5, wherein the state vector outputted in step (3):
yj=Wryj-1+Urcj+VrSj
wherein Wr,Ur,VrAre trainable weight matrix parameters.
CN202010449079.1A 2020-05-25 2020-05-25 Text automatic summarization method based on pre-training language model Pending CN111723547A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010449079.1A CN111723547A (en) 2020-05-25 2020-05-25 Text automatic summarization method based on pre-training language model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010449079.1A CN111723547A (en) 2020-05-25 2020-05-25 Text automatic summarization method based on pre-training language model

Publications (1)

Publication Number Publication Date
CN111723547A true CN111723547A (en) 2020-09-29

Family

ID=72564980

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010449079.1A Pending CN111723547A (en) 2020-05-25 2020-05-25 Text automatic summarization method based on pre-training language model

Country Status (1)

Country Link
CN (1) CN111723547A (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112395841A (en) * 2020-11-18 2021-02-23 福州大学 BERT-based method for automatically filling blank text
CN112417139A (en) * 2020-11-19 2021-02-26 深圳大学 Abstract generation method based on pre-training language model
CN112507081A (en) * 2020-12-16 2021-03-16 平安科技(深圳)有限公司 Similar sentence matching method and device, computer equipment and storage medium
CN112559702A (en) * 2020-11-10 2021-03-26 西安理工大学 Transformer-based natural language problem generation method in civil construction information field
CN112559730A (en) * 2020-12-08 2021-03-26 北京京航计算通讯研究所 Text abstract automatic generation method and system based on global feature extraction
CN113128214A (en) * 2021-03-17 2021-07-16 重庆邮电大学 Text abstract generation method based on BERT pre-training model
CN113159168A (en) * 2021-04-19 2021-07-23 清华大学 Pre-training model accelerated reasoning method and system based on redundant word deletion
CN113157855A (en) * 2021-02-22 2021-07-23 福州大学 Text summarization method and system fusing semantic and context information
CN113407711A (en) * 2021-06-17 2021-09-17 成都崇瑚信息技术有限公司 Gibbs limited text abstract generation method by using pre-training model
CN115081437A (en) * 2022-07-20 2022-09-20 中国电子科技集团公司第三十研究所 Machine-generated text detection method and system based on linguistic feature contrast learning
CN115687031A (en) * 2022-11-15 2023-02-03 北京优特捷信息技术有限公司 Method, device, equipment and medium for generating alarm description text
WO2023061107A1 (en) * 2021-10-13 2023-04-20 北京有竹居网络技术有限公司 Language translation method and apparatus based on layer prediction, and device and medium
CN116562275A (en) * 2023-06-09 2023-08-08 创意信息技术股份有限公司 Automatic text summarization method combined with entity attribute diagram

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018233647A1 (en) * 2017-06-22 2018-12-27 腾讯科技(深圳)有限公司 Abstract generation method, device and computer device and storage medium
CN110737769A (en) * 2019-10-21 2020-01-31 南京信息工程大学 pre-training text abstract generation method based on neural topic memory

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018233647A1 (en) * 2017-06-22 2018-12-27 腾讯科技(深圳)有限公司 Abstract generation method, device and computer device and storage medium
CN110737769A (en) * 2019-10-21 2020-01-31 南京信息工程大学 pre-training text abstract generation method based on neural topic memory

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
唐子惠 编著: "《医学人工智能导论》", 30 April 2020, 上海科学技术出版社 *

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112559702A (en) * 2020-11-10 2021-03-26 西安理工大学 Transformer-based natural language problem generation method in civil construction information field
CN112559702B (en) * 2020-11-10 2022-09-30 西安理工大学 Method for generating natural language problem in civil construction information field based on Transformer
CN112395841B (en) * 2020-11-18 2022-05-13 福州大学 BERT-based method for automatically filling blank text
CN112395841A (en) * 2020-11-18 2021-02-23 福州大学 BERT-based method for automatically filling blank text
CN112417139A (en) * 2020-11-19 2021-02-26 深圳大学 Abstract generation method based on pre-training language model
CN112417139B (en) * 2020-11-19 2023-07-25 深圳大学 Abstract generation method based on pre-training language model
WO2022104967A1 (en) * 2020-11-19 2022-05-27 深圳大学 Pre-training language model-based summarization generation method
CN112559730A (en) * 2020-12-08 2021-03-26 北京京航计算通讯研究所 Text abstract automatic generation method and system based on global feature extraction
CN112507081A (en) * 2020-12-16 2021-03-16 平安科技(深圳)有限公司 Similar sentence matching method and device, computer equipment and storage medium
CN113157855A (en) * 2021-02-22 2021-07-23 福州大学 Text summarization method and system fusing semantic and context information
CN113128214A (en) * 2021-03-17 2021-07-16 重庆邮电大学 Text abstract generation method based on BERT pre-training model
CN113128214B (en) * 2021-03-17 2022-05-06 重庆邮电大学 Text abstract generation method based on BERT pre-training model
CN113159168B (en) * 2021-04-19 2022-09-02 清华大学 Pre-training model accelerated reasoning method and system based on redundant word deletion
CN113159168A (en) * 2021-04-19 2021-07-23 清华大学 Pre-training model accelerated reasoning method and system based on redundant word deletion
CN113407711A (en) * 2021-06-17 2021-09-17 成都崇瑚信息技术有限公司 Gibbs limited text abstract generation method by using pre-training model
CN113407711B (en) * 2021-06-17 2023-04-07 成都崇瑚信息技术有限公司 Gibbs limited text abstract generation method by using pre-training model
WO2023061107A1 (en) * 2021-10-13 2023-04-20 北京有竹居网络技术有限公司 Language translation method and apparatus based on layer prediction, and device and medium
CN115081437A (en) * 2022-07-20 2022-09-20 中国电子科技集团公司第三十研究所 Machine-generated text detection method and system based on linguistic feature contrast learning
CN115687031A (en) * 2022-11-15 2023-02-03 北京优特捷信息技术有限公司 Method, device, equipment and medium for generating alarm description text
CN116562275A (en) * 2023-06-09 2023-08-08 创意信息技术股份有限公司 Automatic text summarization method combined with entity attribute diagram
CN116562275B (en) * 2023-06-09 2023-09-15 创意信息技术股份有限公司 Automatic text summarization method combined with entity attribute diagram

Similar Documents

Publication Publication Date Title
CN111723547A (en) Text automatic summarization method based on pre-training language model
CN111897949B (en) Guided text abstract generation method based on Transformer
CN109635124B (en) Remote supervision relation extraction method combined with background knowledge
CN109918666B (en) Chinese punctuation mark adding method based on neural network
CN112100351A (en) Method and equipment for constructing intelligent question-answering system through question generation data set
CN111241816B (en) Automatic news headline generation method
CN109992775B (en) Text abstract generation method based on high-level semantics
CN110929030A (en) Text abstract and emotion classification combined training method
CN110737769A (en) pre-training text abstract generation method based on neural topic memory
CN111143563A (en) Text classification method based on integration of BERT, LSTM and CNN
CN109992669B (en) Keyword question-answering method based on language model and reinforcement learning
CN111858932A (en) Multiple-feature Chinese and English emotion classification method and system based on Transformer
CN111125333B (en) Generation type knowledge question-answering method based on expression learning and multi-layer covering mechanism
CN112232053A (en) Text similarity calculation system, method and storage medium based on multi-keyword pair matching
CN111966797B (en) Method for machine reading and understanding by using word vector introduced with semantic information
CN116414962A (en) Question-answer matching method based on attention mechanism
CN115238691A (en) Knowledge fusion based embedded multi-intention recognition and slot filling model
CN115169349A (en) Chinese electronic resume named entity recognition method based on ALBERT
CN114757184A (en) Method and system for realizing knowledge question answering in aviation field
CN112818124A (en) Entity relationship extraction method based on attention neural network
CN116521857A (en) Method and device for abstracting multi-text answer abstract of question driven abstraction based on graphic enhancement
CN116663501A (en) Chinese variant text conversion method based on multi-modal sharing weight
Jiang et al. A hierarchical bidirectional LSTM sequence model for extractive text summarization in electric power systems
Luo et al. Learning document embeddings with crossword prediction
Zhang Automatic generation of Chinese abstract based on vocabulary and LSTM neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200929

RJ01 Rejection of invention patent application after publication