WO2018233647A1 - 摘要生成方法、装置、计算机设备以及存储介质 - Google Patents

摘要生成方法、装置、计算机设备以及存储介质 Download PDF

Info

Publication number
WO2018233647A1
WO2018233647A1 PCT/CN2018/092078 CN2018092078W WO2018233647A1 WO 2018233647 A1 WO2018233647 A1 WO 2018233647A1 CN 2018092078 W CN2018092078 W CN 2018092078W WO 2018233647 A1 WO2018233647 A1 WO 2018233647A1
Authority
WO
WIPO (PCT)
Prior art keywords
sentence
document
coding vector
word
vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/092078
Other languages
English (en)
French (fr)
Inventor
孔行
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Publication of WO2018233647A1 publication Critical patent/WO2018233647A1/zh
Priority to US16/545,904 priority Critical patent/US11409960B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/34—Browsing; Visualisation therefor
    • G06F16/345—Summarisation for human users
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Definitions

  • the embodiments of the present application relate to the field of natural language processing, and in particular, to a digest generating method, apparatus, computer device, and storage medium.
  • Automatic Text Summarization is used to refine a document to produce a concise, fluid summary that contains the subject matter of the article. Automatic text summarization is a major problem in the field of natural language processing.
  • the related art provides an automatic text summarization technology based on an abstract model for extracting words from each sentence in a document, and then recombining the extracted words into one Sentences to form a summary.
  • An embodiment of the present application provides a method for generating a digest, including:
  • a target word is output according to the m candidate sentences by a generative model, and a digest is generated according to the target word.
  • An embodiment of the present application provides a digest generating apparatus, where the apparatus includes: a memory, a processor; wherein the memory stores computer readable instructions, and the processor executes the computer readable instructions in the storing, Used for:
  • the document including at least one sentence, each sentence including at least one word;
  • a target word is output according to the m candidate sentences by a generative model, and a digest is generated according to the target word.
  • the processor further executes the computer readable instructions for:
  • the processor further executes the computer readable instructions for:
  • the processor further executes the computer readable instructions for: inputting a sentence-level coding vector of n sentences into a second bidirectional cyclic neural network in a forward propagation direction to obtain a second forward coding a vector, where n is the number of sentences in the document;
  • the processor further executes the computer readable instructions for:
  • the dictionary includes the frequency of occurrence in the document is higher than the pre- Conditional words
  • the target word is extracted from the target sentence in the m candidate sentences as the i-th word in the digest.
  • the processor further executes the computer readable instructions for:
  • the implicit state of the decoding instant is used to represent a partial input sequence that the decoder has processed at the current decoding time;
  • the processor further executes the computer readable instructions for:
  • the target word is determined from the words of the target sentence by a second maximum likelihood estimation function.
  • An embodiment of the present application provides a computer device, where the device includes a processor and a memory, where the memory stores at least one instruction, at least one program, a code set, or a set of instructions, the at least one instruction, the at least one segment
  • the program, the code set or the instruction set is loaded by the processor and executes the digest generating method described in the embodiment of the present application.
  • the embodiment of the present application provides a computer readable storage medium, where the storage medium stores at least one instruction, at least one program, a code set, or a set of instructions, the at least one instruction, the at least one program, and the code.
  • the set or instruction set is loaded by the processor and executes the digest generating method described in the embodiment of the present application.
  • FIG. 1 is a flowchart of a digest generating method provided by an exemplary embodiment of the present application
  • 3a-3c are flowcharts of a digest generating method provided by another exemplary embodiment of the present application.
  • FIG. 4 is a flowchart of a digest generating method provided by another exemplary embodiment of the present application.
  • FIG. 5 is a flowchart of a digest generating method provided by another exemplary embodiment of the present application.
  • FIG. 6 is a block diagram of a digest generating apparatus provided by an exemplary embodiment of the present application.
  • FIG. 7 is a block diagram of a computer device provided by an exemplary embodiment of the present application.
  • FIG. 1 is a flowchart of a digest generating method provided by an exemplary embodiment of the present application.
  • the digest generation method is performed by a computer device, including:
  • step 101 a document D is acquired, which includes at least one sentence.
  • step 102 m candidate sentences are extracted from the document D by a decimation model.
  • the depletion model includes an Attention based model.
  • the attention mechanism based model is used to calculate the probability value of each sentence in the document D, the sentence whose probability value is greater than the preset threshold is extracted as a candidate sentence, and the sentence whose probability is less than the preset threshold is discarded.
  • the decimation model is used to extract m candidate sentences suitable as digests from the document D.
  • step 103 the target word is output from the m candidate sentences by the generative model, and the digest is generated according to the target word.
  • the generative model includes a model based on a Pointer-Generator Networks, where the generator model includes an encoder and a decoder, wherein the encoder corresponds to an encoding moment, and the decoder corresponds to a decoding moment.
  • the output probability of each decoding time is calculated by the generated model, and the output probability is compared with a preset threshold. If the output probability is greater than a preset threshold, the output probability is marked as 1 and corresponding to the document D.
  • a word is generated in the dictionary for output; if the output probability is less than the preset threshold, the output probability is marked as 0, and the target word is extracted from the m candidate sentences for output.
  • the dictionary consists of the most frequently occurring words in document D, s being a positive integer.
  • step 104 a digest is generated based on the target words.
  • the m candidate sentences suitable for the abstract are extracted from the document D by using the extractive model, thereby reducing the length of the text sequence to be processed by the generated model, and then selecting candidates according to the generated model.
  • the sentence generates or extracts the target word, and synthesizes the digest of the document D according to the target word, thereby improving the readability and the amount of information of the finally generated summary.
  • the decimation model is a model based on an attention mechanism that encodes each sentence in the document D through a bidirectional cyclic neural network to obtain a sentence-level coding vector for each sentence, and then according to each The sentence-level coding vector of the sentence extracts m candidate sentences suitable as digests from the document D.
  • step 102 can be implemented as steps 102a to 102d instead, as shown in FIG. 2a. :
  • step 102a a sentence-level coding vector for each sentence in document D is calculated.
  • the sentence-level coding vector may be calculated by a bidirectional cyclic neural network or a one-way cyclic neural network, which is not limited in this embodiment of the present application.
  • this step may include three steps, as shown in FIG. 2b:
  • the words in each sentence in the document D are input into the first bidirectional cyclic neural network in a backward propagation direction to obtain a first backward coding vector;
  • the sentence-level coding vector of the sentence is obtained according to the first forward coding vector and the first backward coding vector.
  • a single sentence ⁇ w 1 , w 2 , w 3 ... w t ⁇ , w i is the i-th word in the sentence, and the first bidirectional cyclic neural network is an RNN;
  • Sh f i is the coding result of the input of RNN in the forward propagation direction of the i-th word in the sentence
  • Sh f i-1 is the coding result after the input of the RNN in the forward propagation direction of the i-th word in the sentence.
  • w i is the i-th word in the sentence
  • the word-level coding result of the i-1th word in the sentence and the ith word are input into the first bidirectional cyclic neural network to obtain the word-level coding result of the i-th word, wherein the t-th word is input after the RNN
  • the result of the encoding is the first forward coding vector of the sentence, and t is the number of words in the sentence.
  • each word in the sentence is input to the RNN in a backward propagation direction in a backward-to-front order, and is calculated by the following formula:
  • Sh b i is the coding result of inputting RNN in the backward propagation direction of the i-th word in the sentence
  • Sh b i+1 is the coding result of inputting RNN in the backward propagation direction of the i+1th word in the sentence
  • w i For the i-th word in the sentence, RNN is the first bidirectional cyclic neural network
  • the word-level coding result of the i+1th word in the sentence and the ith word are input into the first bidirectional cyclic neural network to obtain the word-level coding result of the i-th word, wherein the code after the first word is input into the RNN
  • the result is the first backward coding vector of the sentence.
  • step 11 and step 12 need to be performed for each sentence in document D.
  • step 13 the calculation is performed using the following calculation formula:
  • Sh f it is the first forward coding vector obtained by inputting the RNN in the forward propagation direction of the t-th sentence (the last one) in the i-th sentence
  • t is the number of words in the sentence
  • Sh b i1 is The first backward coding vector obtained after the first word in the i-th sentence is input to the RNN in the backward propagation direction
  • S i is the sentence-level coding vector of the i-th sentence.
  • step 102b the document level coding vector of the document D is calculated from the sentence level coding vector of each sentence.
  • the document-level coding vector may be calculated by a bidirectional cyclic neural network or a one-way cyclic neural network, which is not limited in this application.
  • this step may include three steps, as shown in FIG. 2c:
  • the sentence-level coding vector of the n sentences is input into the second bidirectional cyclic neural network along the forward propagation direction to obtain a second forward coding vector, where n is the number of sentences in the document;
  • the sentence-level coding vector of the n sentences is input into the second bidirectional cyclic neural network in a backward propagation direction to obtain a second backward coding vector;
  • the second bidirectional cyclic neural network is an RNN, and the calculation is performed by using the following formula:
  • h f i is the coding result after the input of the RNN in the forward propagation direction of the i-th sentence in the document D
  • h f i-1 is the code after the input of the RNN in the forward propagation direction of the i-1th sentence in the document D
  • S i is the sentence-level coding vector of the ith sentence.
  • the second bidirectional cyclic neural network is an RNN, and the calculation is performed by using the following formula:
  • h b i RNN(h b i+1 ,S i )
  • h b i is the encoding result of the input of RNN in the backward propagation direction of the ith sentence in document D
  • h b i+1 is the encoding of the i+1th sentence in document D after inputting RNN in the forward propagation direction.
  • S i is the sentence-level coding vector of the ith sentence.
  • step 23 the following calculation formula is used for calculation:
  • h f i is the second forward coding vector of the ith sentence
  • h b i is the second backward coding vector of the ith sentence
  • h i refers to each sentence in the document D.
  • step 102c the probability p of each sentence in the document is calculated from the code vector of the sentence level and the code vector of the document level.
  • the depletion model includes a model based on an attention mechanism.
  • the attention mechanism based model is used to calculate the probability value of each sentence based on the sentence level coding vector of each sentence and the document level coding vector.
  • S i is the sentence-level coding vector of the ith sentence in document D
  • h i is the document-level coding vector of the document D
  • a is a preset feedforward neural network
  • ⁇ is a preset first nonlinearity
  • p(S i 1
  • D) is denoted as 1 when the probability of the i-th sentence is greater than a preset threshold, and extracts the i-th sentence; when the probability of the i-th sentence is less than a preset threshold Mark it as 0 and discard the ith sentence.
  • step 102d the sentence in the document D whose probability p is greater than the preset threshold is extracted as m candidate sentences.
  • m candidate sentences suitable as digests are extracted from the document D by using a model based on the attention mechanism, thereby discarding useless sentences in the document D that are not noticed, thereby reducing input to
  • the noise in the text sequence of the generative model improves the amount of information generated or extracted by the generative model, ensuring that the generated digest meets the expected result, thereby improving the readability of the resulting digest.
  • the generative model is a model based on a pointer generation network, the generation model including an encoder and a decoder, the decoder includes a pointer generation network, and the generator model is calculated by sentence-level coding vectors of m candidate sentences.
  • i is the context vector of the decoding time, and obtains the implicit state of the i-th decoding time, and outputs the i-th word in the digest according to the implicit state of the i-th decoding time.
  • step 103 can be implemented as follows as step 310 to step 312, as shown in FIG. 3a:
  • step 310 the sentence-level coding vector of the m candidate sentences is input as an input sequence into the generator model, and the output probability of the i-th decoding time is calculated.
  • the calculating the output probability of the ith decoding moment includes the following steps, as shown in FIG. 3b:
  • the default value can be all initialized to 0.
  • step 31 the following calculation formula is used for calculation:
  • ⁇ ij is the weight value of the decoder at the i-th decoding time
  • h j is the implicit state of the encoder at the j-th encoding time
  • h′ i-1 represents the implicit state of the i-th decoding time
  • T represents the input length of the input sequence
  • a represents the preset feedforward neural network.
  • the implicit state of the decoder at the i-1th decoding instant is a partial input sequence that the decoder has processed at the i-1th decoding instant
  • the implicit state of the encoder at the jth encoding instant is that the encoder is at the jth A partial input sequence that has been processed at the decoding instant.
  • the weight value at the i-th decoding time is calculated by the weight value of the decoder at the i-th decoding time and the implicit state of the encoder at the j-th encoding time to obtain the output probability of the i-th decoding time.
  • step 32 the calculation is performed using the following formula:
  • h' i RNN(h' i-1 ,c i ,y i-1 )
  • h' i is the implicit state of the decoder at the ith decoding instant
  • h' i-1 is the implicit state of the decoder at the i-1th decoding instant
  • c i is the context vector
  • y i-1 For the i-1th word in the abstract, the RNN is a preset pointer generation network.
  • step 33 the following calculation formula is used for calculation:
  • h' i is the implicit state of the decoder at the ith decoding instant
  • c i is the context vector
  • y i-1 is the i-1th word in the digest
  • ⁇ represents the preset second nonlinear function
  • step 311 when the output probability is greater than the preset threshold, the output probability is labeled as 1, and a word is generated from the dictionary corresponding to the document D as the i-th word of the digest.
  • the dictionary consists of the most frequently occurring words in document D, s being a positive integer.
  • step 312 when the output probability is less than the preset threshold, the output probability is marked as 0, and the target word is extracted from the target sentence in the m candidate sentences as the i-th word of the digest.
  • the way in which the target sentence and the target word are extracted includes extraction through a model based on the attention mechanism.
  • the step includes the following steps, as shown in FIG. 3c:
  • P ij is the optimal solution for the first maximum likelihood estimation function of the m candidate sentences at the ith decoding time, and the calculation formula of e ij is as shown in step 31.
  • h' i-1 represents the implicit state of the i-1th decoding time
  • a represents the preset feedforward neural network
  • sh jk represents the coding of the kth word in the target sentence in the first bidirectional cyclic neural network.
  • the first maximum likelihood estimation function and the second maximum likelihood estimation function are functions determined based on the attention mechanism.
  • m candidate sentences suitable as abstracts are extracted from the document D by using the extractive model, and then the target words are generated or extracted according to the candidate sentences by using the generated model, and the generated model is used in the generated model.
  • the feedforward neural network is used to weight the implicit state of each coding moment, so that the context vector output by the encoder is more suitable as the background when the target word is selected at the current coding moment.
  • the first maximum likelihood estimation function and the second maximum likelihood estimation function are used to select a more reasonable target word in the decoding process, and the generated model is generated or extracted.
  • the accuracy ensures that the generated summary meets the expected results, which in turn improves the readability and amount of information of the resulting summary.
  • a document D is obtained, which includes 6 sentences, and the third sentence includes at least 4 words w1, w2, w3, w4.
  • step 302 to step 303 the words in each sentence in the document D are input into the first bidirectional cyclic neural network in the forward propagation direction to obtain a first forward coding vector;
  • the words in each sentence are input into the first bidirectional cyclic neural network in the backward propagation direction to obtain a first backward coding vector.
  • step 304 sentence-level encoded vectors s1 through s6 are obtained from the first forward encoding vector and the first backward encoding vector.
  • step 305 to step 307 the sentence-level coding vector of the six sentences is input into the second bidirectional cyclic neural network along the forward propagation direction to obtain a second forward coding vector; the sentence-level coding vector of the six sentences is backward The propagation direction is input to the second bidirectional cyclic neural network to obtain a second backward coding vector; and the document level coding vectors h1 to h6 of the document D are obtained according to the second forward coding vector and the second backward coding vector.
  • step 308 the sentence-level coding vector and the document-level coding vector are input into a probability p-based model to calculate the probability p of each sentence in the document.
  • the probability p of the second sentence, the fourth sentence, and the fifth sentence is greater than a preset threshold, and is marked as 1; the probability p of the first sentence, the third sentence, and the sixth sentence is less than a preset threshold, and is marked as 0.
  • step 309 the sentence in the document D whose probability p is greater than the preset threshold is extracted as a candidate sentence, that is, the second sentence, the fourth sentence, and the fifth sentence are extracted as candidate sentences h'2, h'4, and h'5.
  • step 310 the sentence-level coding vector of the candidate sentence is input as an input sequence into the generator model, and the output probability of the i-th decoding time is calculated.
  • the output probability of h'2 at the i-th decoding time is greater than a preset threshold, and is marked as 1, and the output probability of h'4 at the i+kth decoding time is less than a preset threshold and is marked.
  • the output probability of 0, h'5 at the i+1th decoding time is greater than a preset threshold, and is labeled as 1, wherein the i-th decoding time, the i+kth decoding time, and the i+l decoding are The time is three different decoding times, and k and l are integers.
  • step 311 when the output probability is greater than the preset threshold, a word is generated as a digest word from the dictionary corresponding to the document D, that is, w1 and w3 are words generated from the dictionary,
  • step 312 when the output probability is less than the preset threshold, the target word is extracted from the target sentence in the candidate sentence as the digest word, that is, w2 is the word extracted from the second sentence of the target sentence.
  • FIG. 6 is a block diagram of a digest generating apparatus provided by an exemplary embodiment of the present application.
  • the device can be implemented as all or part of a computer device by software, hardware or a combination of both.
  • the device includes an acquisition module 52, an extraction module 54, and an output module 56.
  • An obtaining module 52 configured to acquire a document D, the document D includes at least one sentence, each sentence including at least one word;
  • the extracting module 54 is configured to extract m candidate sentences from the document D by using a decimation model
  • the output module 56 is configured to output a target word according to the m candidate sentences by using a generative model, and generate a digest according to the target word.
  • the extracting module includes:
  • a calculating unit configured to calculate a sentence-level coding vector of each sentence in the document D
  • the calculating unit is further configured to calculate a document level coding vector of the document D according to the sentence level coding vector of each sentence;
  • the calculating unit is further configured to calculate, according to the sentence-level coding vector and the document-level coding vector, a probability P of each sentence in the document, where the probability P in the document D is greater than a preset threshold
  • the sentence is extracted as the m candidate sentences, and the probability P of each of the sentences is 1 or 0.
  • the computing unit includes:
  • a first input subunit configured to input the words in the sentence into the first bidirectional cyclic neural network in a forward propagation direction to obtain a first forward coding vector
  • the first input subunit is further configured to input the words in the sentence into the first bidirectional cyclic neural network in a backward propagation direction to obtain a first backward coding vector;
  • the first input subunit is further configured to obtain a sentence level coding vector of the sentence according to the first forward coding vector and the first backward coding vector.
  • the input subunit is further configured to input a sentence-level coding vector of n sentences into the second bidirectional cyclic neural network in a forward propagation direction to obtain a second forward coding vector, where n is The number of sentences in the document;
  • the first input subunit is further configured to input the sentence level coding vector of the n sentences into the second bidirectional cyclic neural network in a backward propagation direction to obtain a second backward coding vector;
  • the first input subunit is further configured to obtain a document level coding vector of the document D according to the second forward coding vector and the second backward coding vector.
  • the output module includes:
  • An input submodule configured to input a sentence-level coding vector of the m candidate sentences as an input sequence into the generator model, and calculate an output probability of the i-th decoding time, where i is a positive integer;
  • Generating a sub-module further configured to: when the output probability is greater than the preset threshold, generate a word from a dictionary corresponding to the document D as an i-th word in the digest; a word in the document D that has a frequency higher than a preset condition;
  • the extracting submodule is further configured to: when the output probability is less than the preset threshold, extract a target word from the target sentence in the m candidate sentences as the i th word in the digest.
  • the input sub-module includes:
  • a calculating subunit configured to calculate, by the encoder, a context vector c i of the i th decoding time according to the sentence level coding vector of the m candidate sentences;
  • a second input subunit configured to input an implicit state of the i-1th decoding time, the context vector c i, and an i-1th word in the digest into the decoder, to calculate an i th An implicit state of the decoding instant, the implicit state of the decoding instant being used to represent a partial input sequence that the decoder has processed at the current decoding instant;
  • the calculating subunit is further configured to calculate an output probability of the i th decoding time according to an iith word in the digest, the context vector c i, and an implicit state of the i th decoding time
  • i is a positive integer
  • the i-1th word takes a default value
  • the calculation subunit is further configured to calculate the context vector c i according to the following formula:
  • h j represents the implicit state of the jth coding instant
  • T represents the length of the input sequence
  • h' i-1 represents the implicit state of the i-1th decoding instant
  • a represents the preset feedforward neural A network, wherein the implicit state of the encoding instant is used to represent a partial input sequence that the encoder has processed at the current encoding instant.
  • the output module further includes:
  • a sentence sub-module configured to determine a target sentence in the i-th decoding time from the m candidate sentences by using a first maximum likelihood estimation function
  • a word sub-module for determining a target word from words in the target sentence by a second maximum likelihood estimation function.
  • FIG. 7 is a block diagram of a computer device provided by an exemplary embodiment of the present application.
  • the computer device can be a cell phone, a tablet, a laptop portable computer, a desktop computer and a server, etc., the computer device comprising: a processor 62 and a memory 64, optionally including a display screen 66.
  • Processor 62 can be at least one of a single core processor, a multi-core processor, an embedded chip, and a processor with computing power.
  • the memory 64 stores executable instructions of the processor.
  • the computer 64 is a computer readable storage medium, where the storage medium stores at least one instruction, at least one program, a code set or a set of instructions, the at least one instruction, the at least one program, the The code set or instruction set is loaded and executed by the processor 62 to implement any one of the digest generation methods as shown in FIGS. 1 through 5.
  • display screen 66 is the content for the computer device to display the content of document D and the automatic text summary generated from the content of document D.
  • the embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores at least one instruction, at least one program, a code set, or an instruction.
  • the set, the at least one instruction, the at least one program, the code set, or the instruction set are loaded and executed by the processor to implement any one of the digest generation methods as shown in FIGS. 1 through 5, optionally
  • the computer readable storage medium includes a high speed access memory, a nonvolatile memory.
  • a person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium.
  • the storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Algebra (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种摘要生成方法、装置及计算机设备，属于自然语言处理领域。所述方法包括：获取文档，所述文档包括至少一个句子（101）；通过抽取式模型从所述文档中抽取出m个候选句子（102）；通过生成式模型根据所述m个候选句子输出目标词语（103）；根据所述目标词语生成摘要（104）。

Description

摘要生成方法、装置、计算机设备以及存储介质
本申请要求于2017年06月22日提交中国专利局、申请号为201710481703.4、名称为“摘要生成方法、装置及计算机设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及自然语言处理领域，特别涉及一种摘要生成方法、装置、计算机设备以及存储介质。
背景
自动文本摘要(Automatic Text Summarization)用于将一篇文档进行提炼，生成简洁、流畅且包含文章主旨的摘要。自动文本摘要是自然语言处理领域的一大难题。
相关技术中提供了一种基于生成式(Abstractive)模型的自动文本摘要技术，该生成式模型用于从一篇文档中的每个句子中提取出词语，然后将提取出的词语重新组合为一个个句子，从而形成摘要。
技术内容
本申请实施例提供了一种摘要生成方法所述方法包括：
获取文档，所述文档包括至少一个句子；
通过抽取式模型从所述文档中抽取出m个候选句子；
通过生成式模型根据所述m个候选句子输出目标词语，根据所述目标词语生成摘要。
本申请实施例提供了一种摘要生成装置，所述装置包括：存储器、 处理器；其中，所述存储器中存储有计算机可读指令，所述处理器执行所述存储中的计算机可读指令，用于：
获取文档，所述文档包括至少一个句子，每个句子包括至少一个词语；
通过抽取式模型从所述文档中抽取出m个候选句子；
通过生成式模型根据所述m个候选句子输出目标词语，根据所述目标词语生成摘要。
在一些实施例中，所述处理器进一步执行所述计算机可读指令，用于：
计算所述文档中每个句子的句子级编码向量；
根据所述每个句子的句子级编码向量计算所述文档的文档级编码向量；
根据所述句子级编码向量和所述文档级编码向量计算所述文档中的每个句子的概率P，将所述文档中所述概率P大于预设阈值的句子抽取为所述m个候选句子，每个所述句子的概率P的取值为1或0。
在一些实施例中，所述处理器进一步执行所述计算机可读指令，用于：
将所述句子中的词语沿前向传播方向输入第一双向循环神经网络，得到第一前向编码向量；
将所述句子中的词语沿后向传播方向输入所述第一双向循环神经网络，得到第一后向编码向量；
根据所述第一前向编码向量和所述第一后向编码向量，得到所述句子的句子级编码向量。
在一些实施例中，所述处理器进一步执行所述计算机可读指令，用于：将n个句子的句子级编码向量沿前向传播方向输入第二双向循 环神经网络，得到第二前向编码向量，其中，n为所述文档中的句子的个数；
将所述n个句子的句子级编码向量沿后向传播方向输入所述第二双向循环神经网络，得到第二后向编码向量，其中，n为所述文档中的句子的个数；
根据所述第二前向编码向量和所述第二后向编码向量，得到所述文档的文档级编码向量。
在一些实施例中，所述处理器进一步执行所述计算机可读指令，用于：
将所述m个候选句子的句子级编码向量作为输入序列输入所述生成式模型，计算第i个解码时刻的输出概率，i为正整数；
当所述输出概率大于所述预设阈值时，从所述文档对应的词典中生成一个词语，作为所述摘要中的第i个词语；所述词典中包括所述文档中出现频率高于预设条件的词语；
当所述输出概率小于所述预设阈值时，从所述m个候选句子中的目标句子中抽取出目标词语，作为所述摘要中的第i个词语。
在一些实施例中，所述处理器进一步执行所述计算机可读指令，用于：
通过所述编码器根据所述m个候选句子的句子级编码向量计算第i个解码时刻的上下文向量c i；
将第i-1个解码时刻的隐含状态、所述上下文向量c i和所述摘要中的第i-1个词语输入所述解码器，计算得到第i个解码时刻的隐含状态，所述解码时刻的隐含状态用于表示所述解码器在当前解码时刻下已经处理的部分输入序列；
根据所述摘要中的第i-1个词语、所述上下文向量c i和所述第i 个解码时刻的隐含状态，计算第i个解码时刻的输出概率，其中，i为正整数，且当i-1＝0时，所述第i-1个解码时刻的隐含状态取默认值，所述第i-1个词语取默认值。
在一些实施例中，所述处理器进一步执行所述计算机可读指令，用于：：
通过第一最大似然估计函数从所述m个候选句子中确定出所述第i个解码时刻下的目标句子；
通过第二最大似然估计函数从所述目标句子的词语中确定出目标词语。
本申请实施例提供了一种计算机设备，所述设备包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行本申请实施例所述的摘要生成方法。
本申请实施例提供了一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行本申请实施例所述的摘要生成方法。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请一个示例性实施例提供的摘要生成方法的流程图；
图2a至图2c是本申请另一个示例性实施例提供的摘要生成方法 的流程图；
图3a至图3c是本申请另一个示例性实施例提供的摘要生成方法的流程图；
图4是本申请另一个示例性实施例提供的摘要生成方法的流程图；
图5是本申请另一个示例性实施例提供的摘要生成方法的流程图；
图6是本申请一个示例性实施例提供的摘要生成装置的框图；
图7是本申请一个示例性实施例提供的计算机设备的框图。
实施方式
下面为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
图1是本申请一个示例性实施例提供的摘要生成方法的流程图。该摘要生成方法由计算机设备执行，包括：
在步骤101中，获取文档D，该文档D中包括至少一个句子。
在步骤102中，通过抽取式模型从文档D中抽取出m个候选句子。
可选地，该抽取式模型包括基于注意力机制(Attention)的模型。
基于注意力机制的模型用于计算文档D中的每个句子的概率值，将概率值大于预设阈值的句子抽取为候选句子，将概率小于预设阈值的句子抛弃。其中，抽取式模型用于从文档D中抽取出适合作为摘要的m个候选句子。
在步骤103中，通过生成式模型根据m个候选句子输出目标词语，根据目标词语生成摘要。
可选地，该生成式模型包括基于指针生成网络(Pointer-Generator Networks)的模型，生成式模型中包括编码器与解码器，其中，编码 器对应有编码时刻，解码器对应有解码时刻。
通过该生成式模型对每个解码时刻的输出概率进行计算，将该输出概率与预设阈值进行比较，若输出概率大于预设阈值，则将该输出概率标注为1，并从文档D对应的词典中生成一个词语进行输出；若输出概率小于预设阈值，则将该输出概率标注为0，并从m个候选句子中抽取目标词语进行输出。可选地，该词典由文档D中出现频率最高的s个词语组成，s为正整数。
在步骤104中，根据目标词语生成摘要。
综上所述，本申请实施例中先通过抽取式模型从文档D中抽取出适合作为摘要的m个候选句子，从而减少生成式模型所需要处理的文本序列长度，再通过生成式模型根据候选句子生成或提取出目标词语，根据目标词语合成文档D的摘要，提高了最终生成的摘要的可读性和信息量。
在一些实施例中，抽取式模型是基于注意力机制的模型，抽取式模型通过双向循环神经网络对文档D中的每个句子进行编码，得到每个句子的句子级编码向量，再根据每个句子的句子级编码向量从文档D中抽取m个适合作为摘要的候选句子，基于图1所示的摘要生成方法的流程图，步骤102可替代实现为步骤102a至步骤102d，如图2a所示：
在步骤102a中，计算文档D中每个句子的句子级编码向量。
可选地，该句子级编码向量可由双向循环神经网络或单向循环神经网络计算得到，本申请实施例对此不加以限定。
以每个句子的句子级编码向量由双向循环神经网络计算为例，本步骤可以包括三个步骤，如图2b所示：
11，将文档D中的每个句子中的词语沿前向传播方向输入第一双 向循环神经网络得到第一前向编码向量；
12，将文档D中的每个句子中的词语沿后向传播方向输入第一双向循环神经网络得到第一后向编码向量；
13，根据第一前向编码向量和第一后向编码向量得到句子的句子级编码向量。
可选地，针对步骤11，设单个句子＝{w 1、w 2、w 3…w t}，w i是表示句子中的第i个词语，第一双向循环神经网络为RNN；
将句子中每个词语按照由前到后的顺序沿前向传播方向输入RNN，采用如下计算公式计算：
Sh f i＝RNN(Sh f i-1，w i)
其中，Sh f i为句子中第i个词语沿前向传播方向输入RNN后的编码结果，Sh f i-1为句子中第i-1个词语沿前向传播方向输入RNN后的编码结果，w i为句子中第i个的词语，RNN为第一双向循环神经网络，其中，i为正整数，当i-1＝0时，第i-1个词语取空值或默认值；
也即，将句子中第i-1个词语的词语级编码结果与第i个词语输入第一双向循环神经网络得到第i个词语的词语级编码结果，其中，第t个词语输入RNN后的编码结果为该句子的第一前向编码向量，t为该句子中的词语个数。
可选地，针对步骤12，将句子中每个词语按照由后到前的顺序沿后向传播方向输入RNN，采用如下计算公式计算：
Sh b i＝RNN(Sh b i+1，w i)
其中，Sh b i为句子中第i个词语沿后向传播方向输入RNN的编码结果，Sh b i+1为句子中第i+1个词语沿后向传播方向输入RNN的编码结果，w i为句子中第i个的词语，RNN为第一双向循环神经网络；
也即，将句子中第i+1个词语的词语级编码结果与第i个词语输 入第一双向循环神经网络得到第i个词语的词语级编码结果，其中第1个词语输入RNN后的编码结果为该句子的第一后向编码向量。
可选地，需要对文档D中的每个句子执行步骤11和步骤12。
可选地，针对步骤13，采用如下计算公式进行计算：
S i＝[Sh f it，Sh b i1]
其中，Sh f it为第i个句子中第t个词语(最后一个)沿前向传播方向输入RNN后得到的第一前向编码向量，t为该句子中的词语个数，Sh b i1为第i个句子中第一个词语沿后向传播方向输入RNN后得到的第一后向编码向量，S i为第i个句子的句子级编码向量。
在步骤102b中，根据每个句子的句子级编码向量计算文档D的文档级编码向量。
可选地，该文档级编码向量可由双向循环神经网络或单向循环神经网络计算得到，本申请对此不加以限定。
以该文档级编码向量由双向循环神经网络计算为例，本步骤可以包括三个步骤，如图2c所示：
21，将n个句子的句子级编码向量沿前向传播方向输入第二双向循环神经网络，得到第二前向编码向量，其中，n为所述文档中的句子的个数；
22，将n个句子的句子级编码向量沿后向传播方向输入第二双向循环神经网络，得到第二后向编码向量；
23，根据第二前向编码向量和第二后向编码向量得到文档D的文档级编码向量。
可选地，针对步骤21，第二双向循环神经网络为RNN，采用如下计算公式进行计算：
h f i＝RNN(h f i-1，S i)
其中，h f i为文档D中第i个句子沿前向传播方向输入RNN后的编码结果，h f i-1为文档D中第i-1个句子沿前向传播方向输入RNN后的编码结果，S i为第i个句子的句子级编码向量。
可选地，针对步骤22，第二双向循环神经网络为RNN，采用如下计算公式进行计算：
h b i＝RNN(h b i+1，S i)
其中，h b i为文档D中第i个句子沿后向传播方向输入RNN后的编码结果，h b i+1为文档D中第i+1个句子沿前向传播方向输入RNN后的编码结果，S i为第i个句子的句子级编码向量。
可选地，针对步骤23，采用如下计算公式进行计算：
h i＝[h f i，h b i]
其中，h f i为第i个句子的第二前向编码向量，h b i为第i个句子的第二后向编码向量，h i泛指文档D中的各个句子。
在步骤102c中，根据句子级的编码向量和文档级的编码向量计算文档中的每个句子的概率p。
可选地，该抽取式模型包括基于注意力机制的模型。
该基于注意力机制的模型用于根据每个句子的句子级编码向量以及文档级编码向量计算每个句子的概率值。
计算公式如下：
P(S i＝1|D)＝σ(a(S i，h i))
其中，S i为文档D中第i个句子的句子级编码向量，h i为所述文档D的文档级编码向量，a为预设的前馈神经网络，σ为预设的第一非线性函数，p(S i＝1|D)代表在第i个句子的概率大于预设阈值时被标 注为1，并抽取该第i个句子；在第i个句子的概率小于预设阈值时被标注为0并抛弃该第i个句子。
在步骤102d中，将文档D中概率p大于预设阈值的句子抽取为m个候选句子。
综上所述，本申请实施例中通过基于注意力机制的模型从文档D中抽取出适合作为摘要的m个候选句子，从而将文档D中不被注意的无用句子进行丢弃，从而减少输入至生成式模型的文本序列中的噪声，提高生成式模型生成或提取出目标词语的信息量，保证了生成的摘要符合预期结果，进而提高最终生成的摘要的可读性。
在一些实施例中，生成式模型是基于指针生成网络的模型，该生成模型包括编码器及解码器，解码器中包括指针生成网络，生成式模型通过m个候选句子的句子级编码向量计算第i个解码时刻的上下文向量，并得到该第i个解码时刻的隐含状态，根据第i个解码时刻的隐含状态来输出摘要中第i个词语。具体地，基于图1所示的摘要生成方法的流程图，步骤103可替代实现为如下步骤310至步骤312，如图3a所示：
在步骤310中，将m个候选句子的句子级编码向量作为输入序列输入生成式模型，计算第i个解码时刻的输出概率。
其中，计算第i个解码时刻的输出概率包括如下步骤，如图3b所示：
31，通过编码器根据m个候选句子的句子级编码向量计算第i个解码时刻的上下文向量c i；
32，将第i-1个解码时刻的隐含状态、上下文向量c i和摘要中的第i-1个词语输入指针生成网络，计算得到第i个解码时刻的隐含状态，其中，i为正整数，当i-1＝0时，第i-1个解码时刻的隐含状态取 默认值。默认值可以是全部初始化为0。
33，根据摘要中的第i-1个词语、上下文向量和第i个解码时刻的隐含状态，计算第i个解码时刻的输出概率。
可选地，针对步骤31，采用如下计算公式进行计算：
e ij＝a(h' i-1,h j)
Figure PCTCN2018092078-appb-000001
Figure PCTCN2018092078-appb-000002
其中，α ij为解码器在第i个解码时刻的权重值，h j表示编码器在第j个编码时刻的隐含状态，h’ i-1表示第i-1个解码时刻的隐含状态，T表示输入序列的输入长度，a表示预设的前馈神经网络。解码器在第i-1个解码时刻的隐含状态是解码器在第i-1个解码时刻已经处理的部分输入序列；编码器在第j个编码时刻的隐含状态是编码器在第j个解码时刻已经处理的部分输入序列。
将解码器在第i-1个解码时刻的隐含状态以及编码器在第j个编码时刻的隐含状态输入预设的前馈神经网络a得到一个输出值，通过该输出值计算得到解码器在第i个解码时刻的权重值，通过解码器在第i个解码时刻的权重值与编码器在第j个编码时刻的隐含状态计算得到第i个解码时刻的输出概率。
可选地，针对步骤32，采用如下计算公式进行计算：
h’ i＝RNN(h’ i-1,c i,y i-1)
其中，h’ i为解码器在第i个解码时刻的隐含状态，h’ i-1为解码器在第i-1个解码时刻的隐含状态、c i为上下文向量，y i-1为摘要中的第i-1个词语，RNN为预设的指针生成网络。
可选的，针对步骤33，采用如下计算公式进行计算：
p(G＝1|D)＝σ(h' i,c i,y i-1)
其中，h’ i为解码器在第i个解码时刻的隐含状态、c i为上下文向量，y i-1为摘要中的第i-1个词语，σ表示预设的第二非线性函数。
在步骤311中，当输出概率大于预设阈值时，将该输出概率标注为1，并从文档D对应的词典中生成一个词语作为摘要的第i个词语。
可选地，该词典由文档D中出现频率最高的s个词语组成，s为正整数。
在步骤312中，当输出概率小于预设阈值时，将该输出概率标注为0，并从m个候选句子中的目标句子中抽取出目标词语作为摘要的第i个词语。
抽取目标句子和目标词语的方式包括通过基于注意力机制的模型进行抽取。可选地，本步骤包括如下步骤，如图3c所示：
41，通过第一最大似然估计函数从m个候选句子中确定第i个解码时刻下的目标句子。
计算公式如下：
p ij＝argmax(e ij)
其中，P ij是m个候选句子在第i个解码时刻下针对第一最大似然估计函数的最优解，e ij的计算公式如步骤31所示。
42，通过第二最大似然估计函数从目标句子中的词语中确定出目标词语。
计算公式如下：
p ijk＝argmax(e ijk)
其中，P ij是目标句子中的各个词语在第i个解码时刻下针对第二最大似然估计函数的最优解，ei jk的计算公式如下所示：
e ijk＝a(h’ i-1,sh jk)
其中，h’ i-1代表第i-1个解码时刻的隐含状态，a代表预设的前馈神经网络，sh jk代表目标句子中第k个词语在第一双向循环神经网络中的编码结果。
可选地，第一最大似然估计函数和第二最大似然估计函数是基于注意力机制所确定的函数。
综上所述，本申请实施例中先通过抽取式模型从文档D中抽取出适合作为摘要的m个候选句子，再通过生成式模型根据候选句子生成或提取出目标词语，该生成式模型中采用了前馈神经网络给每个编码时刻的隐含状态进行加权，使得编码器输出的上下文向量更适合作为当前编码时刻时选择目标词语时的背景。
本申请实施例中还通过第一最大似然估计函数和第二最大似然估计函数，在解码过程中使用注意力机制选择出更为合理的目标词语，提高生成式模型生成或提取出目标词语的准确性，保证了生成的摘要符合预期结果，进而提高最终生成的摘要的可读性和信息量。
在一个具体的例子中，结合图4和图5对本申请实施例提供的摘要生成方法进行说明：
在步骤301中，获取文档D，该文档D中包括6个句子，第三个句子中包括至少4个词语w1、w2、w3、w4。
在步骤302至步骤303中，如图4指针所示，将文档D中的每个句子中的词语沿前向传播方向输入第一双向循环神经网络得到第一前向编码向量；将文档D中的每个句子中的词语沿后向传播方向输入第一双向循环神经网络得到第一后向编码向量。
在步骤304中，根据第一前向编码向量和第一后向编码向量得到 句子级编码向量s1至s6。
在步骤305至步骤307中，将6个句子的句子级编码向量沿前向传播方向输入第二双向循环神经网络，得到第二前向编码向量；将6个句子的句子级编码向量沿后向传播方向输入第二双向循环神经网络，得到第二后向编码向量；根据第二前向编码向量和第二后向编码向量得到文档D的文档级编码向量h1至h6。
在步骤308中，将句子级编码向量和文档级编码向量输入基于注意力机制的模型计算文档中的每个句子的概率p。
其中，第二句、第四句以及第五句的概率p大于预设阈值，被标注为1；第一句、第三句以及第六句的概率p小于预设阈值，被标注为0。
在步骤309中，将文档D中概率p大于预设阈值的句子抽取为候选句子，即抽取第二句、第四句以及第五句为候选句子h’2、h’4以及h’5。
在步骤310中，将候选句子的句子级编码向量作为输入序列输入生成式模型，计算第i个解码时刻的输出概率。
如图4可知，h’2在第i个解码时刻的输出概率大于预设阈值，并被标注为1，h’4在第i+k个解码时刻的输出概率小于预设阈值，并被标注为0，h’5在第i+l个解码时刻的输出概率大于预设阈值，并被标注为1，其中，第i个解码时刻、第i+k个解码时刻以及第i+l个解码时刻为三个不同的解码时刻，且k、l均为整数。
在步骤311中，当输出概率大于预设阈值时，从文档D对应的词典中生成一个词语作为摘要的词语，即w1与w3为从词典中生成的词语，
在步骤312中，当输出概率小于预设阈值时，从候选句子中的目 标句子中抽取出目标词语作为摘要的词语，即w2为从目标句子第二句中抽取出的词语。
图6是本申请一个示例性实施例提供的摘要生成装置的框图。该装置可以通过软件、硬件或两者的结合实现成为计算机设备的全部或一部分。所述装置包括：获取模块52、抽取模块54以及输出模块56。
获取模块52，用于获取文档D，所述文档D包括至少一个句子，每个句子包括至少一个词语；
抽取模块54，用于通过抽取式模型从所述文档D中抽取出m个候选句子；
输出模块56，用于通过生成式模型根据所述m个候选句子输出目标词语，根据所述目标词语生成摘要。
在一些实施例中，所述抽取模块，包括：
计算单元，用于计算所述文档D中每个句子的句子级编码向量；
所述计算单元，还用于根据所述每个句子的句子级编码向量计算所述文档D的文档级编码向量；
所述计算单元，还用于根据所述句子级编码向量和所述文档级编码向量计算所述文档中的每个句子的概率P，将所述文档D中所述概率P大于预设阈值的句子抽取为所述m个候选句子，每个所述句子的概率P的取值为1或0。
在一些实施例中，所述计算单元，包括：
第一输入子单元，用于将所述句子中的词语沿前向传播方向输入第一双向循环神经网络，得到第一前向编码向量；
所述第一输入子单元，还用于将所述句子中的词语沿后向传播方向输入所述第一双向循环神经网络，得到第一后向编码向量；
所述第一输入子单元，还用于根据所述第一前向编码向量和所述 第一后向编码向量，得到所述句子的句子级编码向量。
在一些实施例中，所述输入子单元，还用于将n个句子的句子级编码向量沿前向传播方向输入第二双向循环神经网络，得到第二前向编码向量，其中，n为所述文档中的句子的个数；
所述第一输入子单元，还用于将所述n个句子的句子级编码向量沿后向传播方向输入所述第二双向循环神经网络，得到第二后向编码向量；
所述第一输入子单元，还用于根据所述第二前向编码向量和所述第二后向编码向量，得到所述文档D的文档级编码向量。
在一些实施例中，所述输出模块，包括：
输入子模块，用于将所述m个候选句子的句子级编码向量作为输入序列输入所述生成式模型，计算第i个解码时刻的输出概率，i为正整数；
生成子模块，还用于当所述输出概率大于所述预设阈值时，从所述文档D对应的词典中生成一个词语，作为所述摘要中的第i个词语；所述词典中包括所述文档D中出现频率高于预设条件的词语；
抽取子模块，还用于当所述输出概率小于所述预设阈值时，从所述m个候选句子中的目标句子中抽取出目标词语，作为所述摘要中的第i个词语。
在一些实施例中，所述输入子模块，包括：
计算子单元，用于通过所述编码器根据所述m个候选句子的句子级编码向量计算第i个解码时刻的上下文向量c i；
第二输入子单元，还用于将第i-1个解码时刻的隐含状态、所述上下文向量c i和所述摘要中的第i-1个词语输入所述解码器，计算得到第i个解码时刻的隐含状态，所述解码时刻的隐含状态用于表示所 述解码器在当前解码时刻下已经处理的部分输入序列；
所述计算子单元，还用于根据所述摘要中的第i-1个词语、所述上下文向量c i和所述第i个解码时刻的隐含状态，计算第i个解码时刻的输出概率，其中，i为正整数，且当i-1＝0时，所述第i-1个解码时刻的隐含状态取默认值，所述第i-1个词语取默认值。
在一些实施例中，所述计算子单元，还用于按照如下公式计算所述上下文向量c i：
Figure PCTCN2018092078-appb-000003
Figure PCTCN2018092078-appb-000004
e ij＝a(h’ i-1,h j)
其中，h j代表第j个编码时刻的隐含状态，T代表所述输入序列的长度，h’ i-1代表第i-1个解码时刻的隐含状态，a代表预设的前馈神经网络，其中，所述编码时刻的隐含状态用于表示所述编码器在当前编码时刻下已经处理的部分输入序列。
在一些实施例中，所述输出模块，还包括：
句子子模块，用于通过第一最大似然估计函数从所述m个候选句子中确定出所述第i个解码时刻下的目标句子；
词语子模块，用于通过第二最大似然估计函数从所述目标句子的词语中确定出目标词语。
图7是本申请一个示例性实施例提供的计算机设备的框图。该计算机设备可以是手机、平板电脑、膝上型便携计算机、台式计算机和服务器等等，该计算机设备包括：处理器62和存储器64，可选地，该计算机设备还包括显示屏66。
处理器62可以是单核处理器、多核处理器、嵌入式芯片以及具有计算能力的处理器中的至少一种。
存储器64存储有处理器的可执行指令。示意性的，存储器64中一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器62加载并执行以实现如图1至图5所示的任意一种摘要生成方法。
可选地，显示屏66是用于计算机设备显示文档D的内容以及根据文档D的内容生成的自动文本摘要的内容。
可选地，本申请实施例还提供了一种计算机可读存储介质，该存储介质中一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如图1至图5所示的任意一种摘要生成方法，可选地，该计算机可读存储介质包括高速存取存储器、非易失性存储器。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅为本申请的较佳实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (11)

  1. 一种摘要生成方法，由计算机设备执行，所述方法包括：
    获取文档，所述文档包括至少一个句子；
    通过抽取式模型从所述文档中抽取出m个候选句子；
    通过生成式模型根据所述m个候选句子输出目标词语，根据所述目标词语生成摘要。
  2. 根据权利要求1所述的方法，其中，所述通过抽取式模型从所述文档中抽取出m个候选句子，包括：
    计算所述文档中每个句子的句子级编码向量；
    根据所述每个句子的句子级编码向量计算所述文档的文档级编码向量；
    根据所述句子级编码向量和所述文档级编码向量计算所述文档中的每个句子的概率P，将所述文档中所述概率P大于预设阈值的句子抽取为所述m个候选句子。
  3. 根据权利要求2所述的方法，其中，所述抽取式模型包括：第一双向循环神经网络；
    所述计算所述文档中每个句子的句子级编码向量，包括：
    将所述句子中的词语沿前向传播方向输入所述第一双向循环神经网络，得到第一前向编码向量；
    将所述句子中的词语沿后向传播方向输入所述第一双向循环神经网络，得到第一后向编码向量；
    根据所述第一前向编码向量和所述第一后向编码向量，得到所述句子的句子级编码向量。
  4. 根据权利要求2所述的方法，其中，所述抽取式模型包括：第二双向循环神经网络；
    所述根据所述每个句子的句子级编码向量计算所述文档的文档级编码向量，包括：
    将n个句子的句子级编码向量沿前向传播方向输入所述第二双向循环神经网络，得到第二前向编码向量，其中，n为所述文档中的句子的个数；
    将所述n个句子的句子级编码向量沿后向传播方向输入所述第二双向循环神经网络，得到第二后向编码向量；
    根据所述第二前向编码向量和所述第二后向编码向量，得到所述文档的文档级编码向量。
  5. 根据权利要求2所述的方法，其中，所述抽取式模型包括基于注意力机制的模型；
    所述基于注意力机制的模型为：
    p(S i＝1|D)＝σ(a(S i,h i))；
    其中，D为所述文档；S i为所述文档D中第i个句子的句子级编码向量，所述h i为所述文档D的文档级编码向量，a为预设的前馈神经网络，σ为预设的第一非线性函数，p(S i＝1|D)代表在第i个句子的概率大于所述预设阈值时将所述第i个句子标注为1，并抽取所述第i个句子，i为正整数。
  6. 根据权利要求2至5任一所述的方法，其中，所述通过生成式模型根据所述m个候选句子输出目标词语，根据所述目标词语生成摘要，包括：
    将所述m个候选句子的句子级编码向量作为输入序列输入所述生成式模型，计算第i个解码时刻的输出概率，i为正整数；
    当所述输出概率大于所述预设阈值时，从所述文档对应的词典中生成一个词语，作为所述摘要中的第i个词语；所述词典中包括所述 文档中出现频率高于预设条件的词语；
    当所述输出概率小于所述预设阈值时，从所述m个候选句子中的目标句子中抽取出目标词语，作为所述摘要中的第i个词语。
  7. 根据权利要求6所述的方法，其中，所述生成式模型包括编码器和解码器；
    所述将所述m个候选句子的句子级编码向量作为输入序列输入所述生成式模型，计算第i个解码时刻的输出概率，包括：
    通过所述编码器根据所述m个候选句子的句子级编码向量计算第i个解码时刻的上下文向量c i；
    将第i-1个解码时刻的隐含状态、所述上下文向量c i和所述摘要中的第i-1个词语输入所述解码器，计算得到第i个解码时刻的隐含状态，所述第i个解码时刻的隐含状态用于表示所述解码器在所述第i个解码时刻下已经处理的部分输入序列；
    根据所述摘要中的第i-1个词语、所述上下文向量c i和所述第i个解码时刻的隐含状态，计算第i个解码时刻的输出概率。
  8. 根据权利要求6所述的方法，其中，所述生成式模型还包括：第一最大似然估计函数和第二最大似然估计函数；
    所述从所述m个候选句子中的目标句子中抽取出目标词语，作为所述摘要中的第i个词语，包括：
    通过所述第一最大似然估计函数从所述m个候选句子中确定出所述第i个解码时刻下的目标句子；
    通过所述第二最大似然估计函数从所述目标句子的词语中确定出目标词语。
  9. 一种摘要生成装置，所述装置包括：存储器、处理器；其中，所述存储器中存储有计算机可读指令，所述处理器执行所述存储中的 计算机可读指令，用于：
    获取文档，所述文档包括至少一个句子，每个句子包括至少一个词语；
    通过抽取式模型从所述文档中抽取出m个候选句子；
    通过生成式模型根据所述m个候选句子输出目标词语，根据所述目标词语生成摘要。
  10. 一种计算机设备，所述设备包括处理器和存储器，所述存储器中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如如权利要求1至8任一所述的摘要生成方法。
  11. 一种计算机可读存储介质，所述存储介质中存储有至少一条指令、至少一段程序、代码集或指令集，所述至少一条指令、所述至少一段程序、所述代码集或指令集由所述处理器加载并执行以实现如权利要求1至8任一所述的摘要生成方法。
PCT/CN2018/092078 2017-06-22 2018-06-21 摘要生成方法、装置、计算机设备以及存储介质 Ceased WO2018233647A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/545,904 US11409960B2 (en) 2017-06-22 2019-08-20 Summary generation method, apparatus, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710481703.4A CN108280112B (zh) 2017-06-22 2017-06-22 摘要生成方法、装置及计算机设备
CN201710481703.4 2017-06-22

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/545,904 Continuation US11409960B2 (en) 2017-06-22 2019-08-20 Summary generation method, apparatus, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2018233647A1 true WO2018233647A1 (zh) 2018-12-27

Family

ID=62799363

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/092078 Ceased WO2018233647A1 (zh) 2017-06-22 2018-06-21 摘要生成方法、装置、计算机设备以及存储介质

Country Status (3)

Country Link
US (1) US11409960B2 (zh)
CN (1) CN108280112B (zh)
WO (1) WO2018233647A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110457661A (zh) * 2019-08-16 2019-11-15 腾讯科技（深圳）有限公司 自然语言生成方法、装置、设备及存储介质
CN110489541A (zh) * 2019-07-26 2019-11-22 昆明理工大学 基于案件要素及BiGRU的涉案舆情新闻文本摘要方法
CN111723547A (zh) * 2020-05-25 2020-09-29 河海大学 一种基于预训练语言模型的文本自动摘要方法
US10902191B1 (en) 2019-08-05 2021-01-26 International Business Machines Corporation Natural language processing techniques for generating a document summary
CN113761895A (zh) * 2021-02-04 2021-12-07 北京沃东天骏信息技术有限公司 文本摘要的生成方法、装置、电子设备及存储介质

Families Citing this family (51)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108280112B (zh) 2017-06-22 2021-05-28 腾讯科技（深圳）有限公司 摘要生成方法、装置及计算机设备
US10909157B2 (en) * 2018-05-22 2021-02-02 Salesforce.Com, Inc. Abstraction of text summarization
CN109325109B (zh) * 2018-08-27 2021-11-19 中国人民解放军国防科技大学 基于注意力编码器的抽取式新闻摘要生成装置
CN109214003B (zh) * 2018-08-29 2019-08-09 陕西师范大学 基于多层注意力机制的循环神经网络生成标题的方法
CN110990578A (zh) * 2018-09-30 2020-04-10 北京奇虎科技有限公司 一种改写模型的构建方法及装置
CN109522411B (zh) * 2018-11-12 2022-10-28 南京德磐信息科技有限公司 一种基于神经网络的写作辅助方法
CN109284367B (zh) * 2018-11-30 2021-05-18 北京字节跳动网络技术有限公司 用于处理文本的方法和装置
CN109726281A (zh) * 2018-12-12 2019-05-07 Tcl集团股份有限公司 一种文本摘要生成方法、智能终端及存储介质
CN109657054B (zh) * 2018-12-13 2021-02-02 北京百度网讯科技有限公司 摘要生成方法、装置、服务器及存储介质
CN110147533B (zh) * 2019-01-24 2023-08-29 腾讯科技（深圳）有限公司 编码方法、装置、设备及存储介质
CN109885683B (zh) * 2019-01-29 2022-12-02 桂林远望智能通信科技有限公司 一种基于K-means模型和神经网络模型的生成文本摘要的方法
CN110019768B (zh) * 2019-03-28 2021-09-21 北京寓乐世界教育科技有限公司 生成文本摘要的方法及装置
CN110119444B (zh) * 2019-04-23 2023-06-30 中电科大数据研究院有限公司 一种抽取式与生成式相结合的公文摘要生成模型
CN110189173B (zh) * 2019-05-28 2021-11-30 北京百度网讯科技有限公司 广告生成方法及装置
CN110377808A (zh) * 2019-06-14 2019-10-25 北京达佳互联信息技术有限公司 文档处理方法、装置、电子设备及存储介质
CN110209803B (zh) * 2019-06-18 2023-11-14 腾讯科技（深圳）有限公司 故事生成方法、装置、计算机设备及存储介质
CN110298436B (zh) * 2019-06-28 2023-05-09 乐山金蜜工业卫士服务股份有限公司 基于指针生成网络的数据到文本的生成模型
CN110532463B (zh) * 2019-08-06 2024-11-26 北京三快在线科技有限公司 推荐理由生成装置及方法、存储介质以及电子设备
WO2021038827A1 (ja) * 2019-08-30 2021-03-04 富士通株式会社 情報処理方法、情報処理プログラムおよび情報処理装置
CN110909152B (zh) * 2019-10-21 2021-07-09 昆明理工大学 一种融合主题信息的司法舆情文本摘要方法
CN110888976B (zh) * 2019-11-14 2023-06-20 北京香侬慧语科技有限责任公司 一种文本摘要生成方法和装置
CN111026861B (zh) * 2019-12-10 2023-07-04 腾讯科技（深圳）有限公司 文本摘要的生成方法、训练方法、装置、设备及介质
CN113011160A (zh) * 2019-12-19 2021-06-22 中国移动通信有限公司研究院 文本摘要生成方法、装置、设备及存储介质
CN111177366B (zh) * 2019-12-30 2023-06-27 北京航空航天大学 一种基于查询机制的抽取式文档摘要自动生成方法、装置及系统
KR102816242B1 (ko) * 2020-01-03 2025-06-05 삼성전자주식회사 디스플레이 장치 및 그 제어 방법
CN111858911B (zh) * 2020-04-21 2024-11-05 北京嘀嘀无限科技发展有限公司 工单描述信息的生成方法、装置、电子设备及存储介质
CN113626581B (zh) * 2020-05-07 2024-09-20 北京沃东天骏信息技术有限公司 摘要生成方法及装置、计算机可读存储介质、电子设备
CN111581374A (zh) * 2020-05-09 2020-08-25 联想(北京)有限公司 文本的摘要获取方法、装置及电子设备
US11294945B2 (en) * 2020-05-19 2022-04-05 International Business Machines Corporation Unsupervised text summarization with reinforcement learning
US11593556B2 (en) * 2020-05-26 2023-02-28 Mastercard International Incorporated Methods and systems for generating domain-specific text summarizations
CN111831783B (zh) * 2020-07-07 2023-12-08 北京北大软件工程股份有限公司 一种篇章级关系抽取方法
CN111966820B (zh) * 2020-07-21 2023-03-24 西北工业大学 一种生成式摘要模型构建、提取生成式摘要方法及系统
CN111857728B (zh) * 2020-07-22 2021-08-31 中山大学 一种代码摘要生成方法和装置
CN112069309B (zh) * 2020-09-14 2024-03-15 腾讯科技（深圳）有限公司 信息获取方法、装置、计算机设备及存储介质
CN112417139B (zh) * 2020-11-19 2023-07-25 深圳大学 一种基于预训练语言模型的摘要生成方法
CN112541348B (zh) * 2020-12-10 2024-12-06 平安科技（深圳）有限公司 长文本生成方法、装置、设备及存储介质
CN112560501B (zh) * 2020-12-25 2022-02-25 北京百度网讯科技有限公司 语义特征的生成方法、模型训练方法、装置、设备及介质
CN112732898B (zh) * 2020-12-30 2025-03-28 平安科技（深圳）有限公司 文献摘要生成方法、装置、计算机设备及存储介质
CN113779277B (zh) * 2021-01-22 2024-12-10 北京沃东天骏信息技术有限公司 用于生成文本的方法和装置
CN113761177A (zh) * 2021-02-19 2021-12-07 北京沃东天骏信息技术有限公司 物品的摘要生成方法、装置和设备
CN113282742B (zh) * 2021-04-30 2022-08-12 合肥讯飞数码科技有限公司 摘要获取方法以及电子设备、存储装置
CN113688231B (zh) * 2021-08-02 2024-11-08 北京小米移动软件有限公司 一种答案文本的摘要提取方法及装置、电子设备及介质
CN113935328B (zh) * 2021-10-20 2024-11-05 平安国际智慧城市科技股份有限公司 文本摘要生成方法、装置、电子设备及存储介质
KR20230062064A (ko) * 2021-10-29 2023-05-09 삼성에스디에스 주식회사 문서 요약을 위한 방법 및 장치
CN114386390B (zh) * 2021-11-25 2022-12-06 马上消费金融股份有限公司 一种数据处理方法、装置、计算机设备及存储介质
CN114168727B (zh) * 2021-12-06 2024-07-12 哈尔滨工业大学 面向金融领域的文档级事件主体对抽取的方法、存储介质及设备
CN115062596A (zh) * 2022-06-07 2022-09-16 南京信息工程大学 一种气象专报的生成方法、装置、电子设备及存储介质
CN115270728A (zh) * 2022-06-20 2022-11-01 阿里巴巴达摩院(杭州)科技有限公司 会议记录处理方法、装置、设备及存储介质
CN115757763A (zh) * 2022-12-27 2023-03-07 北京百度网讯科技有限公司 事件摘要生成方法、摘要生成模型训练方法、装置及设备
US20250348527A1 (en) * 2024-05-07 2025-11-13 Bank Of America Corporation System and method for creating a controllable output summary from text
CN120336574B (zh) * 2025-06-18 2025-08-19 中国矿业大学 一种基于多级语义协同匹配的跨模态遥感图文检索方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101751425A (zh) * 2008-12-10 2010-06-23 北京大学 文档集摘要获取方法及装置
CN106502985A (zh) * 2016-10-20 2017-03-15 清华大学 一种用于生成标题的神经网络建模方法及装置

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7607083B2 (en) * 2000-12-12 2009-10-20 Nec Corporation Test summarization using relevance measures and latent semantic analysis
US8108398B2 (en) * 2007-06-29 2012-01-31 Microsoft Corporation Auto-summary generator and filter
US9020808B2 (en) * 2013-02-11 2015-04-28 Appsense Limited Document summarization using noun and sentence ranking
CN103136359B (zh) * 2013-03-07 2016-01-20 宁波成电泰克电子信息技术发展有限公司 单文档摘要生成方法
JP2015225412A (ja) * 2014-05-26 2015-12-14 日本電信電話株式会社 文書要約装置、方法、及びプログラム
CN105488021B (zh) * 2014-09-15 2018-09-28 华为技术有限公司 一种生成多文档摘要的方法和装置
CN104503958B (zh) * 2014-11-19 2017-09-26 百度在线网络技术（北京）有限公司 文档摘要的生成方法及装置
US20160267165A1 (en) * 2015-03-14 2016-09-15 Hui Wang Automated Key Words (Phrases) Discovery In Document Stacks And Its Application To Document Classification, Aggregation, and Summarization
CN105930314B (zh) * 2016-04-14 2019-02-05 清华大学 基于编码-解码深度神经网络的文本摘要生成系统及方法
CN106126620A (zh) * 2016-06-22 2016-11-16 北京鼎泰智源科技有限公司 基于机器学习的中文自动文摘方法
CN108280112B (zh) 2017-06-22 2021-05-28 腾讯科技（深圳）有限公司 摘要生成方法、装置及计算机设备

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101751425A (zh) * 2008-12-10 2010-06-23 北京大学 文档集摘要获取方法及装置
CN106502985A (zh) * 2016-10-20 2017-03-15 清华大学 一种用于生成标题的神经网络建模方法及装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LIN, JING: "On Extractive Summarization via Hybrid Neural Networks", JOURNAL OF HUAIHUA UNIVERSITY, vol. 36, no. 5, 31 May 2017 (2017-05-31), pages 71 - 74 *
RUSH, A.M. ET AL.: "A Neural Attention Model for Sentence Summarization", PROCEEDINGS OF THE 2015 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING, 21 September 2015 (2015-09-21), pages 379 - 389, XP055546567 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110489541A (zh) * 2019-07-26 2019-11-22 昆明理工大学 基于案件要素及BiGRU的涉案舆情新闻文本摘要方法
US10902191B1 (en) 2019-08-05 2021-01-26 International Business Machines Corporation Natural language processing techniques for generating a document summary
CN110457661A (zh) * 2019-08-16 2019-11-15 腾讯科技（深圳）有限公司 自然语言生成方法、装置、设备及存储介质
CN110457661B (zh) * 2019-08-16 2023-06-20 腾讯科技（深圳）有限公司 自然语言生成方法、装置、设备及存储介质
CN111723547A (zh) * 2020-05-25 2020-09-29 河海大学 一种基于预训练语言模型的文本自动摘要方法
CN113761895A (zh) * 2021-02-04 2021-12-07 北京沃东天骏信息技术有限公司 文本摘要的生成方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
US11409960B2 (en) 2022-08-09
CN108280112A (zh) 2018-07-13
CN108280112B (zh) 2021-05-28
US20190370338A1 (en) 2019-12-05

Similar Documents

Publication Publication Date Title
WO2018233647A1 (zh) 摘要生成方法、装置、计算机设备以及存储介质
US10831997B2 (en) Intent classification method and system
JP7799861B2 (ja) 対照キャプションニューラルネットワーク
CN112860919B (zh) 基于生成模型的数据标注方法、装置、设备及存储介质
WO2020244065A1 (zh) 基于人工智能的字向量定义方法、装置、设备及存储介质
US20200302127A1 (en) Machine translation method, device, and computer-readable storage medium
CN110019793A (zh) 一种文本语义编码方法及装置
CN114861889B (zh) 深度学习模型的训练方法、目标对象检测方法和装置
CN111666758A (zh) 中文分词方法、训练设备以及计算机可读存储介质
CN108132932B (zh) 带有复制机制的神经机器翻译方法
US20200356556A1 (en) Assertion-based question answering
CN113762459B (zh) 一种模型训练方法、文本生成方法、装置、介质及设备
US20210150154A1 (en) Chapter-level text translation method and device
CN114912450B (zh) 信息生成方法与装置、训练方法、电子设备和存储介质
WO2022228127A1 (zh) 要素文本处理方法、装置、电子设备和存储介质
Li et al. Teaching small language models to reason for knowledge-intensive multi-hop question answering
WO2021139344A1 (zh) 基于人工智能的文本生成方法、装置、计算机设备和介质
CN114756691A (zh) 结构图生成方法、模型的训练方法、图谱生成方法及装置
CN107766498B (zh) 用于生成信息的方法和装置
CN112926334B (zh) 确定词表示向量的方法、装置及电子设备
US11514248B2 (en) Non-transitory computer readable recording medium, semantic vector generation method, and semantic vector generation device
Pham et al. Classification of pediatric dental diseases from panoramic radiographs using natural language transformer and deep learning models
WO2024087297A1 (zh) 文本情感分析方法、装置、电子设备及存储介质
CN112883711B (zh) 摘要生成的方法、装置及电子设备
Hsu et al. Let’s fuse step by step: A generative fusion decoding algorithm with LLMs for robust and instruction-aware ASR and OCR

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18820693

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18820693

Country of ref document: EP

Kind code of ref document: A1