CN112364643A - Method, apparatus, electronic device, and medium for generating natural language text - Google Patents
Method, apparatus, electronic device, and medium for generating natural language text Download PDFInfo
- Publication number
- CN112364643A CN112364643A CN201911423510.9A CN201911423510A CN112364643A CN 112364643 A CN112364643 A CN 112364643A CN 201911423510 A CN201911423510 A CN 201911423510A CN 112364643 A CN112364643 A CN 112364643A
- Authority
- CN
- China
- Prior art keywords
- subject
- words
- processing
- generating
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 239000013598 vector Substances 0.000 claims abstract description 60
- 238000012545 processing Methods 0.000 claims abstract description 56
- 230000007246 mechanism Effects 0.000 claims abstract description 24
- 230000015654 memory Effects 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 12
- 238000010586 diagram Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 235000012054 meals Nutrition 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000003205 fragrance Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000004804 winding Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Machine Translation (AREA)
Abstract
The present disclosure provides a method for generating a natural language text, including obtaining a plurality of subject terms, processing the plurality of subject terms based on an attention mechanism to obtain a reference subject term vector at a current time, processing terms in a generated sentence based on the attention mechanism to obtain a reference sentence vector, and generating a term at the current time based on the reference subject term vector and the reference sentence vector. The present disclosure also provides an apparatus for generating a natural language text, an electronic device, and a computer-readable storage medium.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, an electronic device, and a medium for generating a natural language text.
Background
Along with the continuous development of scientific technology, the demand of automatically generating texts is more and more, and at present, many scenes have been applied with the automatic text generation technology, such as automatically generating poems, automatically generating advertisement patterns, automatically generating lyrics and the like. The existing text generation technology based on the neural network requires a user to give a set of keywords, generate a sentence by sequentially utilizing each keyword, and combine the generated sentences in sequence to obtain a generated text. However, the inventor finds that the technology is very sensitive to the sequence of the subject words selected by the user, the input sequence of the same group of subject words is different, the quality of the generated result is greatly different, and the user experience is influenced.
Disclosure of Invention
In view of the above, the present disclosure provides a method, an apparatus, an electronic device, and a medium for generating a natural language text.
One aspect of the present disclosure provides a method for generating a natural language text, including obtaining a plurality of subject words, processing the plurality of subject words based on an attention mechanism to obtain a reference subject word vector at a current time, processing words in a generated sentence based on the attention mechanism to obtain a reference sentence vector, and generating the words at the current time based on the reference subject word vector and the reference sentence vector.
Optionally, the method further includes determining whether the number of generated sentences reaches a preset number, and stopping the process of generating the words when the number of generated sentences reaches the preset number.
Optionally, the preset number is smaller than the number of the subject words.
Optionally, the processing the plurality of subject words based on the attention mechanism to obtain the reference subject word vector at the current time comprises vectorizing the plurality of subject words, determining a plurality of weight values based on the trained parameters and the hidden vectors, and processing the vectorized subject words based on the weight values to obtain the reference subject word vector.
Another aspect of the disclosure provides an apparatus for generating natural language text, comprising an obtaining module, a first processing module, a second processing module, and a generating module. The obtaining module is used for obtaining a plurality of subject terms. And the first processing module is used for processing the plurality of subject terms based on the attention mechanism to obtain a reference subject term vector at the current moment. And the second processing module is used for processing the words in the generated sentences based on the attention mechanism to obtain the reference sentence vectors. And the generating module is used for generating words at the current moment based on the reference subject word vector and the reference statement vector.
Optionally, the apparatus further includes a determining module and a stopping module. And the judging module is used for judging whether the generated statement quantity reaches a preset quantity. And the stopping module is used for stopping the process of generating the words when the generated sentence quantity reaches the preset quantity.
Optionally, the preset number is smaller than the number of the subject words.
Optionally, the first processing module includes a vectorization sub-module, a determination sub-module, and a processing sub-module. And the vectorization submodule is used for vectorizing the plurality of subject terms. A determination submodule to determine a plurality of weight values based on the trained parameters and the hidden vectors. And the processing submodule is used for processing the vectorized subject term based on the weight value to obtain a reference subject term vector.
Another aspect of the disclosure provides an electronic device comprising at least one processor and at least one memory storing one or more computer-readable instructions, wherein the one or more computer-readable instructions, when executed by the at least one processor, cause the processor to perform the method as described above.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions for implementing the method as described above when executed.
Another aspect of the disclosure provides a computer program comprising computer executable instructions for implementing the method as described above when executed.
The method of the embodiment of the disclosure at least partially solves the defect that the quality of the generated text depends on the sequence of the input subject words by introducing an attention mechanism to the subject words and processing a plurality of subject words to obtain the reference subject word vector at the current moment.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:
fig. 1 schematically shows an application scenario of a method for generating natural language text according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow diagram of a method for generating natural language text in accordance with an embodiment of the present disclosure;
FIG. 3 schematically shows a schematic diagram of a method for generating natural language text according to an embodiment of the present disclosure;
FIG. 4 schematically illustrates a flow diagram of a method for generating natural language text according to another embodiment of the present disclosure;
FIG. 5 schematically illustrates a schematic diagram of an apparatus for generating natural language text according to an embodiment of the present disclosure; and
FIG. 6 schematically illustrates a schematic diagram of a computer system suitable for implementing an apparatus for generating natural language text according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase "a or B" should be understood to include the possibility of "a" or "B", or "a and B".
An embodiment of the present disclosure provides a method for generating a natural language text, including obtaining a plurality of subject words, processing the plurality of subject words based on an attention mechanism to obtain a reference subject word vector at a current time, processing words in a generated sentence based on the attention mechanism to obtain a reference sentence vector, and generating the words at the current time based on the reference subject word vector and the reference sentence vector.
Fig. 1 schematically shows an application scenario of a method for generating natural language text according to an embodiment of the present disclosure.
As shown in FIG. 1, an example of a default blessing phrase given a set of keywords is shown, which a user may then use to generate a default blessing phrase.
Fig. 2 schematically shows a flow diagram of a method for generating natural language text according to an embodiment of the present disclosure.
As shown in fig. 2, the method includes operations S210 to S230.
In operation S210, a plurality of topic words are obtained. For example, as shown in fig. 1, "except for sunset, meal, gorgeous, wealth, dream, good luck".
In operation S220, the plurality of subject words are processed based on the attention mechanism to obtain a reference subject word vector at the current time.
According to an embodiment of the present disclosure, the processing the plurality of subject words based on the attention mechanism to obtain the reference subject word vector at the current time includes vectorizing the plurality of subject words, determining a plurality of weight values based on the trained parameters and the hidden vectors, and processing the vectorized subject words based on the weight values to obtain the reference subject word vector.
In operation S230, words in the generated sentence are processed based on the attention mechanism to obtain a reference sentence vector.
In operation S240, a word at the current time is generated based on the reference subject word vector and the reference sentence vector.
The method of generating natural language text of an embodiment of the present disclosure is further described below in conjunction with the embodiment of fig. 3.
Fig. 3 schematically shows a schematic diagram of a method for generating natural language text according to an embodiment of the present disclosure.
As shown in fig. 3, a process for generating text based on a given set of subject terms is illustrated.
According to the embodiment of the present disclosure, for example, "fireworks", "gorgeous", "people", "smile" have been generated in the first sentence "In the case where the input of the encoder at the time of generating the second sentence includes T determined based on the plurality of subject wordskeyword(i.e., vector representation of subject term, alternatively referred to as reference subject term vector) and T determined based on "fireworks", "gorgeous", "people", "smile" output from a previous sentencesentence(can be vector representation of the first N sentences, N is a natural number or called a reference sentence vector), and the decoder sequentially outputs 'meal', 'fragrance', 'happiness', 'winding'. Both encoder and decoder implementations may use, but are not limited to, various timing encoders (e.g., RNN recurrent neural networks, LSTM long term memory networks, GRU gated recurrent unit-based neural networks). The decoder can generate each word of the current sentence in turn according to the hidden layer semantic information generated by the current encoder.
According to embodiments of the present disclosure, x may be usedjA vector representation representing the jth word in the first N words entered, assuming the first N words entered contain a total of k words, may define:
wherein, TstDenotes T at time Tsentence，atjIndicating that decoding has reached time t corresponding to xjWeight of atjThe calculation formula of (2) is as follows:
wherein
Wa，UaIs three matrices, ht-1And optimizing the hidden layer vector at the t-1 moment in the training process.
The above are sureDefinite TstThe process of (1) is a process of processing words in the generated sentence based on the attention mechanism to obtain a reference sentence vector in operation S230.
On the other hand, it is also necessary to determine the reference subject word vector T at the current timekeywordSimilarly, T is first definedktIs T at time Tkeyword。
The method of the disclosed embodiment, after a user gives a set of subject words, does not give only one word of the subject words every time a sentence is generated, but rather, all the subject words are all fed into the input of the encoder. To get rid of the dependency on the subject word order, the subject word encoding does not go through any temporal encoder (RNN, LSTM, GRU, etc.), but instead generates the hidden layer vector h in parallel as input.
In decoding, the subject word vector T is referred toktWill also vary with each decoding step by the attention mechanism. Herein is defined as wjA vector expression representing the jth subject word, then:
wherein, btjIndicating that the decoding has reached the time t corresponding to wjWeight of (a), btjThe calculation formula of (2) is as follows:
wherein the content of the first and second substances,
Wb，Ubare three matrices that are optimized during the training process.
Obtain TktAnd TstThen we can decode with the decoder, the formula is as followsThe following:
P(yt|yt-1，Tkt，Tst)＝softmax(g(ht)) (7)
wherein g denotes a linear transformation function, ytIndicating the output at the t-th instant in decoding.
Due to Tkt，TstAre all variables, therefore htThe update formula of (c) can be expressed as:
ht＝f(ht-1，yt-1，Tkt，Tst) (8)
where f represents an activation function, for example, tanh, relu, etc. may be used.
Through the steps, the attention mechanism for the subject term and the first N sentences is realized. Therefore, when the text is generated, the sequence of inputting the subject words is not relied on, and the length of the generated text is not limited by the number of the subject words input by the user.
The method of the embodiment of the disclosure at least partially solves the defect that the quality of the generated text depends on the sequence of the input subject words by processing a plurality of subject words to obtain a reference subject word vector at the current moment by introducing an attention mechanism to the subject words without encoding the subject words by any time sequence encoder. In addition, when each sentence is generated, all the subject words input by the user are used as input, and the length of the generated text is not limited by the number of the input subject words.
Fig. 4 schematically illustrates a flow diagram of a method for generating natural language text according to another embodiment of the present disclosure.
As shown in fig. 4, the method further includes operation S410 and operation S420 on the basis of the foregoing embodiment.
After generating the word at the current time by performing operation S240, operation S410 may be performed to determine whether the number of sentences that have been generated reaches a preset number. The generated sentence number refers to the number of sentences which have already been generated completely, and if the current sentence is not complete, the current sentence is not counted as a sentence. If it is determined that the number of sentences that have been generated reaches the preset number, operation S420 is performed, the process of generating words is stopped, and the generation of the natural language text is completed. If the generated sentences are determined not to reach the preset number, otherwise, returning to execute the operations S220-S240, and continuing to generate words to form the sentences.
The number of the sentences generated by the method of the embodiment of the disclosure does not depend on the number of the given subject terms, but the number of the sentences can be customized, and the sentences with any number of the sentences can be generated.
According to an embodiment of the present disclosure, the preset number may be smaller than the number of the subject words. For example, to obtain a seventh-to-speak poem, a user may provide the machine with more than five subject words. Because each sentence generated in the method of the embodiment of the disclosure does not have a strong correlation with a certain predetermined word, the user does not need to be concerned with which four words are input into the machine in which order, and the user can more freely expect the subject word to be used for generating poetry, thereby reducing the burden of the user for determining the subject word.
Based on the same inventive concept, the disclosed embodiment also provides an apparatus for generating a natural language text, and the apparatus for generating a natural language text of the disclosed embodiment is described below with reference to fig. 5.
Fig. 5 schematically shows a block diagram of an apparatus 500 for generating natural language text according to an embodiment of the present disclosure.
As shown in fig. 5, the apparatus 500 for generating a natural language text includes an obtaining module 510, a first processing module 520, a second processing module 530, and a generating module 540. The apparatus 500 may perform the various methods described above with reference to fig. 2.
The obtaining module 510, for example, performs operation S210 described above with reference to fig. 2, for obtaining a plurality of subject words.
The first processing module 520, for example, performs the operation S220 described above with reference to fig. 2, for processing the plurality of subject words based on the attention mechanism to obtain a reference subject word vector at the current time.
The second processing module 530, for example, performs operation S230 described above with reference to fig. 2, for processing the words in the generated sentence based on the attention mechanism to obtain the reference sentence vector.
The generating module 540, for example, performs operation S240 described above with reference to fig. 2, to generate the word at the current time based on the reference subject word vector and the reference sentence vector.
According to an embodiment of the present disclosure, the apparatus 500 may further include a determining module and a stopping module. And the judging module is used for judging whether the generated statement quantity reaches a preset quantity. And the stopping module is used for stopping the process of generating the words when the generated sentence quantity reaches the preset quantity.
According to an embodiment of the present disclosure, the preset number may be smaller than the number of the subject words.
According to an embodiment of the present disclosure, the first processing module 520 may include a vectorization sub-module, a determination sub-module, and a processing sub-module. And the vectorization submodule is used for vectorizing the plurality of subject terms. A determination submodule to determine a plurality of weight values based on the trained parameters and the hidden vectors. And the processing submodule is used for processing the vectorized subject term based on the weight value to obtain a reference subject term vector.
Any number of modules, sub-modules, units, sub-units, or at least part of the functionality of any number thereof according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in any other reasonable manner of hardware or firmware by integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware implementations. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.
For example, any plurality of the obtaining module 510, the first processing module 520, the second processing module 530, the generating module 540, the judging module, the stopping module, the vectorizing sub-module, the determining sub-module, and the processing sub-module may be combined and implemented in one module, or any one of the modules may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the obtaining module 510, the first processing module 520, the second processing module 530, the generating module 540, the judging module, the stopping module, the vectoring sub-module, the determining sub-module and the processing sub-module may be at least partially implemented as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented by any one of three implementations of software, hardware and firmware, or a suitable combination of any of them. Alternatively, at least one of the obtaining module 510, the first processing module 520, the second processing module 530, the generating module 540, the judging module, the stopping module, the vectoring sub-module, the determining sub-module and the processing sub-module may be at least partially implemented as a computer program module which, when executed, may perform a corresponding function.
FIG. 6 schematically illustrates a block diagram of a computer system suitable for implementing the methods and apparatus for generating natural language text according to an embodiment of the present disclosure. The computer system illustrated in FIG. 6 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the disclosure. The computer system shown in fig. 6 may be implemented as a server cluster including at least one processor (e.g., processor 601) and at least one memory (e.g., storage 608).
As shown in fig. 6, a computer system 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. Processor 601 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 601 may also include onboard memory for caching purposes. Processor 601 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the disclosure.
In the RAM 603, various programs and data necessary for the operation of the system 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 602 and/or RAM 603. It is to be noted that the programs may also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, system 600 may also include an input/output (I/O) interface 605, input/output (I/O) interface 605 also connected to bus 604. The system 600 may also include one or more of the following components connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program, when executed by the processor 601, performs the above-described functions defined in the system of the embodiments of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
The present disclosure also provides a computer-readable medium, which may be embodied in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer readable medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, a computer readable medium may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical fiber cable, radio frequency signals, etc., or any suitable combination of the foregoing.
For example, according to an embodiment of the present disclosure, a computer-readable medium may include the ROM 602 and/or the RAM 603 and/or one or more memories other than the ROM 602 and the RAM 603 described above.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.
Claims (10)
1. A method for generating natural language text, comprising:
obtaining a plurality of subject terms;
processing the plurality of subject terms based on an attention mechanism to obtain a reference subject term vector at the current moment;
processing words in the generated sentence based on the attention mechanism to obtain a reference sentence vector;
and generating words at the current moment based on the reference subject word vector and the reference statement vector.
2. The method of claim 1, further comprising:
judging whether the number of generated sentences reaches a preset number;
and when the generated sentence quantity reaches a preset quantity, stopping the process of generating the words.
3. The method of claim 2, wherein the preset number is less than the number of subject words.
4. The method of claim 1, wherein the processing the plurality of subject words based on the attention mechanism to obtain a reference subject word vector for a current time comprises:
vectorizing the plurality of subject words;
determining a plurality of weight values based on the trained parameters and the hidden vectors;
and processing the vectorized subject term based on the weight value to obtain a reference subject term vector.
5. An apparatus for generating natural language text, comprising:
the acquisition module is used for acquiring a plurality of subject terms;
the first processing module is used for processing the plurality of subject terms based on an attention mechanism to obtain a reference subject term vector at the current moment;
the second processing module is used for processing words in the generated sentences based on the attention mechanism to obtain reference sentence vectors;
and the generating module is used for generating words at the current moment based on the reference subject word vector and the reference statement vector.
6. The apparatus of claim 5, further comprising:
the judging module is used for judging whether the generated statement quantity reaches a preset quantity;
and the stopping module is used for stopping the process of generating the words when the generated sentence quantity reaches the preset quantity.
7. The apparatus of claim 6, wherein the preset number is less than the number of subject words.
8. The apparatus of claim 5, wherein the first processing module comprises:
the vectorization submodule is used for vectorizing the plurality of subject terms;
a determination submodule for determining a plurality of weight values based on the trained parameters and the hidden vectors;
and the processing submodule is used for processing the vectorized subject term based on the weight value to obtain a reference subject term vector.
9. An electronic device, comprising:
a processor; and
a memory having computer readable instructions stored thereon which, when executed by the processor, cause the processor to perform the method of any of claims 1 to 4.
10. A computer readable storage medium having computer readable instructions stored thereon which, when executed by a processor, cause the processor to perform the method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911423510.9A CN112364643B (en) | 2019-12-31 | 2019-12-31 | Method, apparatus, electronic device and medium for generating natural language text |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911423510.9A CN112364643B (en) | 2019-12-31 | 2019-12-31 | Method, apparatus, electronic device and medium for generating natural language text |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112364643A true CN112364643A (en) | 2021-02-12 |
| CN112364643B CN112364643B (en) | 2024-04-16 |
Family
ID=74516848
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911423510.9A Active CN112364643B (en) | 2019-12-31 | 2019-12-31 | Method, apparatus, electronic device and medium for generating natural language text |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112364643B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105955964A (en) * | 2016-06-13 | 2016-09-21 | 北京百度网讯科技有限公司 | Method and apparatus for automatically generating poem |
| CN108062388A (en) * | 2017-12-15 | 2018-05-22 | 北京百度网讯科技有限公司 | Interactive reply generation method and device |
| AU2018101514A4 (en) * | 2018-10-11 | 2018-11-15 | Chi, Henan Mr | An automatic text-generating program for Chinese Hip-hop lyrics |
| CN109740145A (en) * | 2018-11-30 | 2019-05-10 | 平安科技（深圳）有限公司 | Lyrics intelligent generation method, device, storage medium and computer equipment |
| CN110209801A (en) * | 2019-05-15 | 2019-09-06 | 华南理工大学 | A kind of text snippet automatic generation method based on from attention network |
-
2019
- 2019-12-31 CN CN201911423510.9A patent/CN112364643B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105955964A (en) * | 2016-06-13 | 2016-09-21 | 北京百度网讯科技有限公司 | Method and apparatus for automatically generating poem |
| CN108062388A (en) * | 2017-12-15 | 2018-05-22 | 北京百度网讯科技有限公司 | Interactive reply generation method and device |
| AU2018101514A4 (en) * | 2018-10-11 | 2018-11-15 | Chi, Henan Mr | An automatic text-generating program for Chinese Hip-hop lyrics |
| CN109740145A (en) * | 2018-11-30 | 2019-05-10 | 平安科技（深圳）有限公司 | Lyrics intelligent generation method, device, storage medium and computer equipment |
| CN110209801A (en) * | 2019-05-15 | 2019-09-06 | 华南理工大学 | A kind of text snippet automatic generation method based on from attention network |
Non-Patent Citations (1)
| Title |
|---|
| ZHE WANG等: "Chinese Poetry Generation with Planning Based Neural Network", ARXIV, pages 1 - 11 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112364643B (en) | 2024-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110192210B (en) | Construction and processing of computational graphs for dynamically structured machine learning models | |
| KR102208989B1 (en) | Device placement optimization through reinforcement learning | |
| CN108630190B (en) | Method and apparatus for generating speech synthesis model | |
| KR102565275B1 (en) | Translating method and apparatus based on parallel processing | |
| US10922611B2 (en) | Neural network optimizer search | |
| CN111699497B (en) | Fast decoding of sequence models using discrete latent variables | |
| US10978042B2 (en) | Method and apparatus for generating speech synthesis model | |
| WO2022022421A1 (en) | Language representation model system, pre-training method and apparatus, device and medium | |
| CN108090218B (en) | Dialog system generation method and device based on deep reinforcement learning | |
| US20200364617A1 (en) | Training machine learning models using teacher annealing | |
| CN110851573A (en) | Statement processing method and system and electronic equipment | |
| US9507852B2 (en) | Techniques for discriminative dependency parsing | |
| CN115362497A (en) | Sequence-to-sequence speech recognition with delay threshold | |
| US11182566B2 (en) | Processing text sequences using neural networks | |
| CN110288974B (en) | Emotion recognition method and device based on voice | |
| CN115908640A (en) | Method and device for generating image, readable medium and electronic equipment | |
| CN112182281B (en) | Audio recommendation method, device and storage medium | |
| CN103870563B (en) | It is determined that the method and apparatus of the theme distribution of given text | |
| CN113591472A (en) | Lyric generation method, lyric generation model training method and device and electronic equipment | |
| CN109657046B (en) | Content analysis processing method and device, electronic equipment and storage medium | |
| CN112364643B (en) | Method, apparatus, electronic device and medium for generating natural language text | |
| CA3156968A1 (en) | Depth-parallel training of neural networks | |
| CN115641879B (en) | Music tag acquisition model training method, music tag acquisition method and device | |
| US20230196105A1 (en) | Generating labeled training data using a pre-trained language model neural network | |
| CN116522958A (en) | Session sample generation method, model training method, emotion recognition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |