CN115982342B - Integration formulation method and system based on achievement conversion standard - Google Patents
Integration formulation method and system based on achievement conversion standard Download PDFInfo
- Publication number
- CN115982342B CN115982342B CN202211686284.5A CN202211686284A CN115982342B CN 115982342 B CN115982342 B CN 115982342B CN 202211686284 A CN202211686284 A CN 202211686284A CN 115982342 B CN115982342 B CN 115982342B
- Authority
- CN
- China
- Prior art keywords
- chapter
- text
- network
- probability
- abstract
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application provides an integration formulation method and system based on a result conversion standard, which constructs a data set of the long text content-chapter abstract title and chapter structure sequence of each chapter in the standard by collecting the published standard text; the method comprises the steps of combining keyword information to obtain a summary type short text meeting the keyword requirement, and logically reconstructing the generated plurality of summary type short texts by utilizing a chapter logic structure generation sub-network in a network model to obtain a standard text chapter structure sequence with clear logic; optimizing a loss function of the network to realize convergence of a network model; and inputting a plurality of text data to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations, and effectively enabling the generated abstract text and chapter structure sequence to be more attached to a theme by utilizing artificial intelligence, so that redundancy of the data is better reduced.
Description
Technical Field
The application relates to the technical field of deep learning application and distributed big data, in particular to an integration formulation method and system based on a result conversion standard.
Background
The integration and formulation of the result conversion standard refers to the large data industrial application of the structured data of the standard text, wherein the published text data of various information results are collected and recorded through data collection means such as web crawlers, and then the text analysis and the text generation are carried out through an artificial intelligence technology. Standard text is a nationwide unified technical requirement, and is a foundation stone for customizing other standards. Through handling the standard text, the participating enterprises can have the regular speaking right, guide the development direction of the same industry and even cause the repositioning of the industry; meanwhile, the method can help enterprises to preempt market opportunities and promote the development of the enterprises. The standard text is preferably capable of forming a preliminary draft first, according to the strict requirements of standard text writing. However, it is inevitable that a lot of investigation materials, a lot of market research, and a lot of industry experience are required in the draft forming process. More importantly, standard text requires strict organization of chapters, which require clear logical relationships to each other. These make the formation of standard text drafts very difficult. Therefore, it is desirable to be able to assist in the generation of standard text drafts by introducing keywords and related document generation methods.
The combination of artificial intelligence and various industries is a necessary trend for realizing the development of the intelligent direction, and has important significance for promoting the development of the industry towards the intelligent aspect. The foremost in the artificial intelligence field is to design a corresponding deep learning network model aiming at different industrial tasks. With the improvement of computer power, the difficulty of network training is greatly reduced, and the network prediction precision is also continuously improved. The deep learning network has the basic characteristics of strong model fitting capability, large information quantity and high precision, and can meet different requirements in different industries.
For standard text draft formation, a key issue is how to screen vast, e.g., smoke, material for off-hook text, which would require keyword guidance. On the other hand, after the related abstract text is obtained, the logical relations among different abstract texts are carded, and a reasonable standard text chapter structure is constructed, so that a corresponding network is required to be designed, and the sequence of the different abstract texts is ensured. Aiming at the two problems, a corresponding reasonable deep learning network framework is designed, the network is trained by utilizing the processing capacity of a computer, a standard text chapter structure generation model can be obtained, and further the standard text chapter structure with clear logic can be obtained from a complex text through the corresponding network.
Disclosure of Invention
In order to solve the defects in the prior art, the application provides an integration formulation method based on a result conversion standard, which designs a related model by using a deep learning network frame, further can obtain a standard text chapter structure generation model, generates a standard text chapter structure with clear logic, and lays a real foundation for subsequent standard text writing.
The first objective of the present application is to provide an integrated formulation method based on achievement transformation criteria.
A second object of the present application is to provide an integrated formulation system based on achievement transformation criteria.
A third object of the present application is to provide a computer device.
A fourth object of the present application is to provide a storage medium.
The first object of the present application can be achieved by adopting the following technical scheme:
an integration formulation method based on a result conversion standard, the method comprises the following steps:
acquiring a data set, acquiring a published standard text, and constructing a data set of the long text content, the chapter abstract title and the chapter structure sequence of each chapter in the standard text as a training data set;
inputting a pre-training language model as a generation model by combining the keyword information to obtain a abstract text generation sub-network, and generating abstract short texts meeting the keyword requirements by using the abstract text generation sub-network;
learning a chapter structure sequence by using another pre-training language model, taking the chapter structure sequence as a chapter logic structure generation sub-network, and logically reconstructing the generated multiple abstract short texts by using the chapter logic structure generation sub-network to obtain a standard text chapter structure sequence with clear logic;
combining the abstract text generation sub-network and the chapter logic structure generation sub-network, optimizing the loss functions of the abstract text generation sub-network and the chapter logic structure generation sub-network according to the generated abstract short text, standard text chapter structure sequence and chapter abstract title and chapter structure sequence in the training data set corresponding to the long text content, and obtaining a trained network model;
and inputting a plurality of text data to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations.
Further, the trained network model is composed of a summary text generation sub-network and a chapter logic structure generation sub-network; the abstract text generation sub-network outputs the generated short texts in the form of a plurality of predicted words, wherein one predicted word is a character string, the abstract text generation sub-network respectively outputs the predicted probability of each predicted word in the generation process as a word prediction probability and counts the true occurrence probability of the predicted word in the training data set as a word true probability; the chapter logic structure generating sub-network carries out logic reconstruction on the plurality of predicted words, namely the chapter logic structure generating sub-network predicts the number of chapters needed by the plurality of predicted words, takes the sequence of one chapter in each chapter as a node according to the sequence, predicts the probability that each predicted word correspondingly appears in each node as chapter prediction probability, and counts the probability that the predicted word actually appears in each node pair in the training data set as chapter actual probability; thus, the trained network model outputs the plurality of predicted words.
Further, after the trained network model is obtained, when new data not belonging to the training data set is added, the new data can be used for fine tuning the trained network model, specifically:
according to the new data added, the trained network model is optimized again, so that the trained network model outputs a plurality of predicted words, and the word prediction probability of the predicted words is recorded as P short The true probability of the words is recorded as P GTshort The chapter predictive probability is denoted as P relate The chapter true probability is noted as P GTrelate The loss function is set as:
where ji and li, the superscript, denote the sequence numbers that go through: j represents the number of predicted words, wherein the serial number is ji; li denotes the number of chapters, wherein the number is li; traversing the word prediction probability, the word true probability, the chapter prediction probability and the chapter true probability of each predicted word corresponding to each chapter in the added new data;
and (3) fine tuning is performed by optimizing the loss function through an optimization algorithm based on gradient descent, so that convergence of a network model is realized.
Further, preferably, after the trained network model is obtained, when new data not belonging to the training data set is added, the method for performing fine tuning is not used, and a prediction search plane may be further constructed according to the word prediction probability, the word true probability, the chapter prediction probability and the chapter true probability of the prediction word, and then the prediction search is performed on the prediction search plane, and then each prediction word is output and transmitted to the client or stored in the database, specifically:
optimizing the trained network model again according to the added new data, enabling the trained network model to output a plurality of predicted words, obtaining word prediction probability, word true probability, chapter prediction probability and chapter true probability of the predicted words,
the word prediction probability section true probability corresponding to the prediction word vector is set that the trained network model outputs j prediction words, each prediction word corresponds to L nodes and has L section prediction probabilities and section true probabilities, the L section prediction probabilities and section true probabilities of the L nodes corresponding to one prediction word are used as a probability comparison array, the L elements are contained in the probability comparison array, each element is a binary array, each binary array is the section prediction probability and section true probability of one node in the L nodes corresponding to the prediction word corresponding to the probability comparison array, the probability comparison array corresponding to the j prediction words is used as j rows respectively, L binary arrays are arranged in each row, wherein each row keeps a corresponding relation with a corresponding predicted word, word prediction probability and word true probability corresponding to the predicted word of each row and the corresponding predicted word of each corresponding predicted word keep a corresponding relation, a matrix of j rows and L columns is formed by probability contrast arrays corresponding to j predicted words and is called as a basic probability word plane and is recorded as Bis, wherein the sequence number of the row is ji, the sequence number of the column is li, the word is P (ji) s, the word true probability corresponding to the predicted word of the ji row is P (ji) gts, and the chapter prediction probability in the binary array of the ji row is P (ji, li) rel and chapter true probability P (ji, li) gtrel, so that a predicted search plane is constructed: the prediction search plane is also a matrix of j rows and L columns, the serial numbers of the rows are kept consistent, ji is also used, the serial numbers of the columns are kept consistent, li is also used, the prediction search word plane is recorded as Vis, the value of the element of the ji row and li column in the prediction search word plane is recorded as Vis (ji, li), the calculation of Vis (ji, li) needs to find the element of the ji row and li column in the basic probability word plane as Bis (ji, li) first, then each element adjacent to the Bis (ji, li) in the basic probability word plane is obtained, the arithmetic average value of the word prediction probability corresponding to the prediction word vector of each adjacent element is used as a prediction probability average value P (ji) savg, the arithmetic average value of the true probability of the word is used as a true probability average value P (ji) gtsavg, the average value of the chapter prediction probability of each adjacent element is used as P (ji, li) is calculated, and the arithmetic average value of the chapter prediction probability of each adjacent element is used as a true probability average value P (ji, li) is calculated as a chapter (ji, li):
Vis(ji,li)＝sqrt[|sin(π*P(ji)savg)*cos(π*P(ji)gtsavg)|*|sin(π*P(ji,li)relavg)*cos(π*P(ji,li)gtrelavg)|],
thus, vis (ji, li) is called the predicted search value of the element of the ji row, li column in the predicted search term plane; then, performing predictive search on a predictive search plane Vis; and transmitting each prediction word in the output set vset to a client or storing the prediction words in the output set vset to a database as the output of the network model.
The method for generating the standard text chapter structure frame with clear logical relationship can comprise the following steps:
a plurality of long texts and keywords to be processed are obtained by using a manual interaction method;
and inputting the long texts and the keywords into the trained network model to generate a standard text chapter structure frame with clear logical relations.
The method for acquiring the training data set may further include:
acquiring a current standard text document, and classifying according to keywords, standard text chapter abstract titles and standard text chapter contents (long texts);
inputting the standard text chapter outline in the standard text document database into an existing relation analysis tool to obtain standard text chapter sequence expression;
the standard text chapter content (long text), keywords, and the standard text chapter sequence are represented as image pairs of a training dataset for model training.
The second object of the application can be achieved by adopting the following technical scheme:
an integrated formulation system based on achievement transformation criteria, wherein the integrated formulation system based on achievement transformation criteria implements steps in the integrated formulation method based on achievement transformation criteria, and the integrated formulation system based on achievement transformation criteria may include:
the acquisition module is used for acquiring a data set, acquiring a published standard text, and constructing a data set of the long text content, the chapter abstract title and the chapter structure sequence of each chapter in the standard text as a training data set;
the abstract text generation module is used for inputting a pre-training language model as a generation model in combination with the keyword information to obtain an abstract text generation sub-network, and generating abstract short texts meeting the keyword requirements by using the abstract text generation sub-network;
the chapter logic structure generating module is used for learning a chapter structure sequence by using another pre-training language model to be used as a chapter logic structure generating sub-network, and logically reconstructing the generated plurality of abstract short texts by using the chapter logic structure generating sub-network to obtain a standard text chapter structure sequence with clear logic;
the optimizing module is used for combining the abstract text generation sub-network and the chapter logic structure generation sub-network, optimizing the loss function of the abstract text generation sub-network and the chapter logic structure generation sub-network according to the generated abstract short text, standard text chapter structure sequence and chapter abstract title and chapter structure sequence in the training data set corresponding to the long text content, and obtaining a trained network model;
and the generating module is used for inputting a plurality of text data to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations.
The third object of the present application can be achieved by adopting the following technical scheme:
a computer device includes a processor and a memory for storing a program executable by the processor, where the processor executes the program stored in the memory to implement the specific method in the generating module in the integrated formulation system based on the achievement transformation criteria.
The fourth object of the present application can be achieved by adopting the following technical scheme:
a storage medium storing a program, which when executed by a processor, implements a specific method in a generation module in an integrated formulation system based on a result conversion criterion as described above.
The undefined variables according to the application may preferably be input as manually defined or preset thresholds or specific values during the implementation.
Compared with the prior art, the application has the following beneficial effects:
1. the application applies deep learning technology and attention mechanism to the construction of a network model to generate a standard text chapter structure frame with clear logical relations.
2. According to the application, the characteristic processing is carried out on the information characterization of each node in the network model by utilizing artificial intelligence, so that the generated abstract text and chapter structure sequence are more attached to the subject, and the redundancy of data is reduced.
3. The application can obtain the standard text chapter structure frame with clear logic relationship by using the constructed network model, and lays a solid foundation for the writing application of the subsequent standard text.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of an integrated formulation method based on a result conversion standard according to an embodiment of the present application.
Fig. 2 is a block diagram of an integrated formulation system based on a result conversion standard according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments, and all other embodiments obtained by those skilled in the art without making any inventive effort based on the embodiments of the present application are within the scope of protection of the present application. It should be understood that the detailed description is intended to illustrate the application, and is not intended to limit the application.
This embodiment is based on the Pytorch framework and the Pycharm development environment. The Pythach framework is a development framework based on python language, can conveniently and quickly build a reasonable deep learning network, and has good cross-platform interaction capability; interfaces for numerous encapsulation functions and various graph node processing functions in the deep learning architecture can be provided, including related graph node processing functions; meanwhile, the model can be trained and verified by using the GPU, and the calculation efficiency is improved.
The Pycharm development environment (IDE) under the Windows platform or the Linux platform is one of the first choice for the design and development of the deep learning network at present. Pycharm provides new templates, design tools, and test and debug tools for clients, while being able to provide an interface for clients to directly invoke remote servers.
The training of the network model can be performed on a high-performance GPU, and the specific training parameter design can be as follows:
using Adam optimizer, its parameters were set to 0.9/0.999; the epoch required for the network model training was set to 100, and the learning rate initial value was set to 0.0001. In order to ensure that the model reaches an optimal value as much as possible in the training process and avoid reaching a suboptimal value, the learning rate is continuously reduced along with the increase of epoch by utilizing a cosine annealing strategy; depending on the training data set and GPU memory size, the batch may be set larger when the training data set sample is large and the GPU memory is large, and vice versa.
By optimizing the loss function, the convergence of the network model is realized, which can be specifically:
the result of the loss function is calculated, a random gradient descent method is adopted to carry out backward propagation calculation, and network parameters of a network model are optimized;
after the epoch of the network model reaches the preset epoch, the network model is converged, so that a trained network model is obtained.
In summary, the main process of the integrated formulation method based on the achievement transformation criteria disclosed in the present embodiment includes three stages of network model establishment, network model training and network model application, wherein:
establishing a network model: generating a sub-network by utilizing the abstract text in the network model in combination with the keyword information to obtain an abstract short text meeting the keyword requirement; generating a sub-network by using a chapter logic structure in the network model to logically reconstruct the generated short text with a plurality of abstract types, thereby obtaining a standard text chapter structure sequence with clear logic;
training of a network model: the cloud server trains the network model, and adjusts parameters of the network model through optimizing the loss function until the network model converges.
Application of network model: firstly, acquiring a plurality of text data and keywords by using a manual interaction method; and inputting a plurality of text data to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations.
Those skilled in the art will appreciate that all or part of the steps in a method implementing the above embodiments may be implemented by a program to instruct related hardware, and the corresponding program may be stored in a computer readable storage medium.
It should be noted that although the method operations of the above embodiments are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in that particular order or that all illustrated operations be performed in order to achieve desirable results. Rather, the depicted steps may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.
As shown in fig. 1, the present embodiment provides an integrated formulation method based on a result conversion standard, which mainly includes the following steps:
acquiring a data set, acquiring a published standard text, and constructing a data set of the long text content, the chapter abstract title and the chapter structure sequence of each chapter in the standard text as a training data set;
inputting a pre-training language model as a generation model by combining the keyword information to obtain a abstract text generation sub-network, and generating abstract short texts meeting the keyword requirements by using the abstract text generation sub-network;
learning a chapter structure sequence by using another pre-training language model, taking the chapter structure sequence as a chapter logic structure generation sub-network, and logically reconstructing the generated multiple abstract short texts by using the chapter logic structure generation sub-network to obtain a standard text chapter structure sequence with clear logic;
combining the abstract text generation sub-network and the chapter logic structure generation sub-network, optimizing the loss functions of the abstract text generation sub-network and the chapter logic structure generation sub-network according to the generated abstract short text, standard text chapter structure sequence and chapter abstract title and chapter structure sequence in the training data set corresponding to the long text content, and obtaining a trained network model;
and inputting a plurality of text data to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations.
The pre-training language model can be a BERT model or other language models based on Attention mechanisms (Attention) or open sources based on a transform, the BERT model can be used for automatically generating texts and outputting probability statistics and predictions of token (i.e. predicted words), and the cross-entropy is used as a target optimization function of a loss function in the open source language model based on the transform, so that the prediction probability of token and the true probability of occurrence of statistics in a training data set can be output before entering a neural network layer where the cross-entropy is located.
Further, the trained network model is composed of a summary text generation sub-network and a chapter logic structure generation sub-network; the abstract text generation sub-network outputs the generated short texts in the form of a plurality of predicted words, wherein one predicted word is a character string, the abstract text generation sub-network respectively outputs the predicted probability of each predicted word in the generation process as a word prediction probability and counts the true occurrence probability of the predicted word in the training data set as a word true probability; the chapter logic structure generating sub-network carries out logic reconstruction on the plurality of predicted words, namely the chapter logic structure generating sub-network predicts the number of chapters needed by the plurality of predicted words, takes the sequence of one chapter in each chapter as a node according to the sequence, predicts the probability that each predicted word correspondingly appears in each node as chapter prediction probability, and counts the probability that the predicted word actually appears in each node pair in the training data set as chapter actual probability; thus, the trained network model outputs the plurality of predicted words.
Further, after the trained network model is obtained, when new data not belonging to the training data set is added, the new data can be used for fine tuning the trained network model, specifically:
according to the new data added, the trained network model is optimized again, so that the trained network model outputs a plurality of predicted words, and the word prediction probability of the predicted words is recorded as P short The true probability of the words is recorded as P GTshort The chapter predictive probability is denoted as P relate The chapter true probability is noted as P GTrelate The loss function is set as:
where ji and li, the superscript, denote the sequence numbers that go through: j represents the number of predicted words, wherein the serial number is ji; li denotes the number of chapters, wherein the number is li; traversing the word prediction probability, the word true probability, the chapter prediction probability and the chapter true probability of each predicted word corresponding to each chapter in the added new data;
and (3) fine tuning is performed by optimizing the loss function through an optimization algorithm based on gradient descent, so that convergence of a network model is realized.
Further, preferably, in an actual industrial application scenario, the network model is operated in a distributed big data cluster to access the network model at a high frequency through a high concurrency large number of data interfaces, in this case, the convergence of the network model is extremely high in time cost by optimizing the loss function, often compensating for the time cost of the network model by sacrificing a large number of computing resources, for a high access amount web page application and a mobile phone client application, such a way is that the high access amount web page application and the mobile phone client application are difficult to follow in a long term, or, when a new batch of data is recorded in the data, the existing network model needs to be trimmed, but notably that the trimming of the network model is excessively frequently performed when the number of text data of part of the types increases in a previous period and then decreases), the resource waste is extremely high, a method is needed that can optimize the word prediction probability and the true probability corresponding to the prediction vector of the prediction word through a long model training process, the node corresponds to the calculation method of the true chapter probability to cope with the high concurrency prediction probability is not needed, or when the search word is further trained on the search plane prediction probability, the search model is further built based on the search plane prediction probability, and the search model is further needed, the prediction probability is further saved, and the search plane prediction probability is further found based on the search probability, or the search plane prediction probability is further needed, and the prediction probability is found, and the true prediction probability is based on the real prediction is based on the search word, and the prediction is based on the prediction word, and the prediction is on the prediction is and on the prediction:
according to the new data added, the trained network model is optimized again, the trained network model outputs a plurality of predicted words, word prediction probability, word true probability, section prediction probability and section true probability of the predicted words are obtained, the word prediction probability section true probability corresponding to the predicted word vector is set to the trained network model outputs j predicted words, each predicted word corresponds to L nodes and has L section prediction probabilities and section true probabilities, L section prediction probabilities and section true probabilities of one predicted word corresponding to L nodes are used as a probability comparison array, L elements are contained in one probability comparison array and each element is a binary array, each binary array is the chapter prediction probability and chapter true probability of one node in L nodes corresponding to the prediction words corresponding to the probability comparison array, the probability comparison array corresponding to j prediction words is respectively used as j rows, L binary arrays are arranged in each row, wherein each row keeps a corresponding relation with the corresponding prediction words, each row keeps a corresponding relation with the word prediction probability and word true probability corresponding to the prediction words of the corresponding prediction words, the matrix of the j probability comparison array corresponding to the j prediction words forming a j row and L columns is called a basic probability word plane and is recorded as Bis, wherein the sequence number of the row is ji, ji epsilon [1, j ] is arranged, the sequence number of the column is li, li epsilon [1, L ] is arranged, the corresponding word prediction probability of the prediction words corresponding to the ji row is recorded as P (ji) s, the true probability is recorded as P (ji) gts, the chapter prediction probability in the binary array of the ji row li is recorded as P (ji), li) rel and chapter true probabilities P (ji, li) gtrel, thereby constructing a predictive search plane: the prediction search plane is also a matrix of j rows and L columns, the serial numbers of the rows are kept consistent, ji is also used, the serial numbers of the columns are kept consistent, li is also used, the prediction search word plane is recorded as Vis, the value of the element of the ji row and li column in the prediction search word plane is recorded as Vis (ji, li), the calculation of Vis (ji, li) needs to find the element of the ji row and li column in the basic probability word plane as Bis (ji, li) first, then each element of the Bis (ji, li) adjacent to the basic probability word plane is obtained, the arithmetic average value of the word prediction probability corresponding to the prediction word vector of each element adjacent to the base probability word is used as a prediction probability average value P (ji) savg, the arithmetic average value of the true probability of each word is used as a true probability P (ji) tsavg, the average value of the chapter prediction probability of each element adjacent to the base probability is recorded as Bis (ji, li) and the arithmetic average value of the chapter prediction probability of each element adjacent to the adjacent element to the base probability is recorded as a chapter (ji, li) is calculated as a mathematical average value of v: :
vis (ji, li) =sqrt [ |sin (pi P (ji) savg) |cos (pi P (ji) gtsavg) |sin (pi P (ji, li) relavg) |cos (pi P (ji, li) gtrelavg) | ] wherein sqrt () is a function of open square root, sin is a sine function, cos is a cosine function, pi represents a circumference ratio, |represents an absolute value, and is also called Vis (ji, li) as a predicted search value of an element of a ji row li column in the predicted search term plane; (the prediction search plane is constructed because the fluctuation of the prediction words on the prediction probability and the true probability of the chapters corresponding to the respective nodes is difficult to be accurately expressed on the basic probability word plane, in the prior art, only the optimization of the gradient in the training or fine tuning process of the metering model is needed, the implementation example is to jump out of the limitation of the training and fine tuning process, the optimization of the gradient in the training or fine tuning process of the existing metering model is not needed, the same technical purpose can not be realized, the method for constructing the prediction search plane effectively obtains the fluctuation of the prediction words on the prediction probability and the true probability of the chapters corresponding to the respective nodes by calculating the prediction search value corresponding to the respective positions by calculating the probability of each dimension of each element adjacent to the basic probability word plane and simultaneously using the extraction function of the trigonometric function, and particularly, for model parameters of hundreds of millions of classes, the method saves the calculation resource and time cost of the model parameters;
then, a predictive search is performed on a predictive search plane Vis, specifically:
s501, creating a set vset for collecting predicted words, reserving the sequence of adding elements in the set vset, and setting the initial value of the set vset as an empty set;
s502, selecting an element with the largest predictive search value in a row with the row number of 1 in the Vis (namely, in a first row) as a starting element, and putting a predictive word corresponding to the row with the row number of 1 in the Vis into a set vset, wherein the row is marked by the set vset;
s503, selecting a column in which the initial element is located in the Vis, acquiring an element which is in the column and has the smallest absolute value of the numerical value difference between the initial element and the initial element on the predicted search value, except for the initial element, as a current target element, putting a predicted word corresponding to a row in which the current target element is located into a set vset, and marking the row by the set vset; wherein the target element can be updated;
s504, selecting a column in which a current target element is located, selecting an element with the smallest absolute value of the difference of numerical values on a predicted search value from elements in the column, which are not marked by the set vset, as the current target element, putting a predicted word corresponding to the updated current target element in the set vset, and marking the updated current target element in the set vset; judging whether all rows in the set Vis are marked by the set vset, if so, turning to S505, and if not, turning to S506;
s505, outputting a set vset;
s506, repeating S504 until all rows in the set Vis are marked by the set vset and the re-output set vset is satisfied; and taking each predicted word in the output set vset as the output of the network model (so that the sequence of each predicted word can be further optimized, and generating text in a summary or abstract mode can be obtained quickly and efficiently without the need of model training or fine tuning, so that the fine tuning of the network model is avoided too frequently), and transmitting the generated text to a client or storing the generated text in a database, wherein the obtained output of the network model is the standard text chapter structure sequence with clear logic.
Inputting a plurality of long texts to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations, wherein the method specifically comprises the following steps:
a plurality of long texts and keywords to be processed are obtained by using a manual interaction method;
and inputting the long texts and the keywords into the trained network model to generate a standard text chapter structure frame with clear logical relations.
Wherein, obtaining the training data set may specifically include:
acquiring a current standard text document, and classifying according to keywords, standard text chapter abstract titles and standard text chapter contents (long texts);
inputting the standard text chapter outline in the standard text document database into an existing relation analysis tool to obtain standard text chapter sequence expression;
the standard text chapter content (long text), keywords, and the standard text chapter sequence are represented as image pairs of a training dataset for model training. An integrated formulation system based on achievement transformation criteria, as shown in fig. 2, implements steps in the integrated formulation method based on achievement transformation criteria, where the integrated formulation system based on achievement transformation criteria may include:
the acquisition module is used for acquiring a data set, acquiring a published standard text, and constructing a data set of the long text content, the chapter abstract title and the chapter structure sequence of each chapter in the standard text as a training data set;
the abstract text generation module is used for inputting a pre-training language model as a generation model in combination with the keyword information to obtain an abstract text generation sub-network, and generating abstract short texts meeting the keyword requirements by using the abstract text generation sub-network;
the chapter logic structure generating module is used for learning a chapter structure sequence by using another pre-training language model to be used as a chapter logic structure generating sub-network, and logically reconstructing the generated plurality of abstract short texts by using the chapter logic structure generating sub-network to obtain a standard text chapter structure sequence with clear logic;
the optimizing module is used for combining the abstract text generation sub-network and the chapter logic structure generation sub-network, optimizing the loss function of the abstract text generation sub-network and the chapter logic structure generation sub-network according to the generated abstract short text, standard text chapter structure sequence and chapter abstract title and chapter structure sequence in the training data set corresponding to the long text content, and obtaining a trained network model;
and the generating module is used for inputting a plurality of text data to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations. A computer device includes a processor and a memory for storing a program executable by the processor, where the processor executes the program stored in the memory to implement the specific method in the generating module in the integrated formulation system based on the achievement transformation criteria.
A storage medium storing a program, which when executed by a processor, implements a specific method in a generation module in an integrated formulation system based on a result conversion criterion as described above.
The undefined variables according to the application may preferably be input as manually defined or preset thresholds or specific values during the implementation.
It should be noted that the computer readable storage medium of the present embodiment may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, a system or device thereof, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The application provides an integration formulation method and system based on a result conversion standard, which constructs a data set of the long text content-chapter abstract title and chapter structure sequence of each chapter in the standard by collecting the published standard text; the method comprises the steps of combining keyword information to obtain a summary type short text meeting the keyword requirement, and logically reconstructing the generated plurality of summary type short texts by utilizing a chapter logic structure generation sub-network in a network model to obtain a standard text chapter structure sequence with clear logic; optimizing a loss function of the network to realize convergence of a network model; and inputting a plurality of text data to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations, so that the generated abstract text and chapter structure sequence can be more attached to a theme effectively, and the redundancy of the data is better reduced.
The above-mentioned embodiments are only preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can make equivalent substitutions or modifications according to the technical solution and the inventive concept of the present application within the scope of the present application disclosed in the present application patent, and all those skilled in the art belong to the protection scope of the present application.
Claims (5)
1. An integration formulation method based on a result conversion standard is characterized by comprising the following steps:
acquiring a data set, acquiring a published standard text, and constructing a data set of the long text content, the chapter abstract title and the chapter structure sequence of each chapter in the standard text as a training data set;
inputting a pre-training language model as a generation model by combining the keyword information to obtain a abstract text generation sub-network, and generating abstract short texts meeting the keyword requirements by using the abstract text generation sub-network;
learning a chapter structure sequence by using another pre-training language model, taking the chapter structure sequence as a chapter logic structure generation sub-network, and logically reconstructing the generated multiple abstract short texts by using the chapter logic structure generation sub-network to obtain a standard text chapter structure sequence with clear logic;
combining the abstract text generation sub-network and the chapter logic structure generation sub-network, optimizing the loss functions of the abstract text generation sub-network and the chapter logic structure generation sub-network according to the generated abstract short text, standard text chapter structure sequence and chapter abstract title and chapter structure sequence in the training data set corresponding to the long text content, and obtaining a trained network model;
inputting a plurality of text data to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations;
the trained network model consists of a summary text generation sub-network and a chapter logic structure generation sub-network; the abstract text generation sub-network outputs the generated short texts in the form of a plurality of predicted words, wherein one predicted word is a character string, the abstract text generation sub-network respectively outputs the predicted probability of each predicted word in the generation process as a word prediction probability and counts the true occurrence probability of the predicted word in the training data set as a word true probability; the chapter logic structure generating sub-network carries out logic reconstruction on the plurality of predicted words, namely the chapter logic structure generating sub-network predicts the number of chapters needed by the plurality of predicted words, takes the sequence of one chapter in each chapter as a node according to the sequence, predicts the probability that each predicted word correspondingly appears in each node as chapter prediction probability, and counts the probability that the predicted word actually appears in each node pair in the training data set as chapter actual probability; thereby, the trained network model outputs the plurality of predicted words;
after the trained network model is obtained, when new data which does not belong to a training data set is added, a predictive search plane is constructed according to the word predictive probability, the word true probability, the chapter predictive probability and the chapter true probability of the predictive words, predictive search is carried out on the predictive search plane, and then each predictive word is output and transmitted to a client or stored in a database.
2. The integrated formulation method based on achievement transformation criteria of claim 1, wherein after obtaining the trained network model, when new data not belonging to a training data set is added, fine tuning the trained network model using the new data is specifically:
according to the new data added, the trained network model is optimized again, so that the trained network model outputs a plurality of predicted words, and the word prediction probability of the predicted words is recorded as P short The true probability of the words is recorded as P GTshort The chapter predictive probability is denoted as P relate The chapter true probability is noted as P GTrelate The loss function is set as:
where ji and li, the superscript, denote the sequence numbers that go through: j represents the number of predicted words, wherein the serial number is ji; li denotes the number of chapters, wherein the number is li; traversing the word prediction probability, the word true probability, the chapter prediction probability and the chapter true probability of each predicted word corresponding to each chapter in the added new data;
and (3) fine tuning is performed by optimizing the loss function through an optimization algorithm based on gradient descent, so that convergence of a network model is realized.
3. An achievement-conversion-criteria-based integrated formulation system implementing the steps of an achievement-conversion-criteria-based integrated formulation method as claimed in any one of claims 1 to 2, the achievement-conversion-criteria-based integrated formulation system comprising:
the acquisition module is used for acquiring a data set, acquiring a published standard text, and constructing a data set of the long text content, the chapter abstract title and the chapter structure sequence of each chapter in the standard text as a training data set;
the abstract text generation module is used for inputting a pre-training language model as a generation model in combination with the keyword information to obtain an abstract text generation sub-network, and generating abstract short texts meeting the keyword requirements by using the abstract text generation sub-network;
the chapter logic structure generating module is used for learning a chapter structure sequence by using another pre-training language model to be used as a chapter logic structure generating sub-network, and logically reconstructing the generated plurality of abstract short texts by using the chapter logic structure generating sub-network to obtain a standard text chapter structure sequence with clear logic;
the optimizing module is used for combining the abstract text generation sub-network and the chapter logic structure generation sub-network, optimizing the loss function of the abstract text generation sub-network and the chapter logic structure generation sub-network according to the generated abstract short text, standard text chapter structure sequence and chapter abstract title and chapter structure sequence in the training data set corresponding to the long text content, and obtaining a trained network model;
and the generating module is used for inputting a plurality of text data to be tested and keywords into the trained network model to generate a standard text chapter structure frame with clear logic relations.
4. A computer device, wherein an integrated formulation system based on outcome conversion criteria as claimed in claim 3 is run in the computer device, the computer device comprising a processor and a memory for storing a program executable by the processor, the processor implementing the method in the generation module when executing the program stored in the memory.
5. A storage medium storing a program which, when executed by a processor, implements the method in the generation module of claim 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211686284.5A CN115982342B (en) | 2022-12-27 | 2022-12-27 | Integration formulation method and system based on achievement conversion standard |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211686284.5A CN115982342B (en) | 2022-12-27 | 2022-12-27 | Integration formulation method and system based on achievement conversion standard |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115982342A CN115982342A (en) | 2023-04-18 |
| CN115982342B true CN115982342B (en) | 2023-08-25 |
Family
ID=85957546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211686284.5A Active CN115982342B (en) | 2022-12-27 | 2022-12-27 | Integration formulation method and system based on achievement conversion standard |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115982342B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460132A (en) * | 2020-03-10 | 2020-07-28 | 哈尔滨工业大学 | Generation type conference abstract method based on graph convolution neural network |
| US10885436B1 (en) * | 2020-05-07 | 2021-01-05 | Google Llc | Training text summarization neural networks with an extracted segments prediction objective |
| CN113723078A (en) * | 2021-09-07 | 2021-11-30 | 杭州叙简科技股份有限公司 | Text logic information structuring method and device and electronic equipment |
| CN114691858A (en) * | 2022-03-15 | 2022-07-01 | 电子科技大学 | Improved UNILM abstract generation method |
-
2022
- 2022-12-27 CN CN202211686284.5A patent/CN115982342B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460132A (en) * | 2020-03-10 | 2020-07-28 | 哈尔滨工业大学 | Generation type conference abstract method based on graph convolution neural network |
| US10885436B1 (en) * | 2020-05-07 | 2021-01-05 | Google Llc | Training text summarization neural networks with an extracted segments prediction objective |
| CN113723078A (en) * | 2021-09-07 | 2021-11-30 | 杭州叙简科技股份有限公司 | Text logic information structuring method and device and electronic equipment |
| CN114691858A (en) * | 2022-03-15 | 2022-07-01 | 电子科技大学 | Improved UNILM abstract generation method |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于注意力机制的中文短文本关键词提取模型;杨丹浩;吴岳辛;范春晓;;计算机科学(01);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115982342A (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Scornet | Tuning parameters in random forests | |
| Fang | Managing data lakes in big data era: What's a data lake and why has it became popular in data management ecosystem | |
| Liu et al. | Generative artificial intelligence and its applications in materials science: Current situation and future perspectives | |
| CN110597992A (en) | Semantic reasoning method and device based on knowledge graph and electronic equipment | |
| CN109871542B (en) | Text knowledge extraction method, device, equipment and storage medium | |
| US20220100800A1 (en) | Automatic knowledge graph construction | |
| WO2021061233A1 (en) | Inter-document attention mechanism | |
| Binnig et al. | Towards interactive curation & automatic tuning of ml pipelines | |
| CN114341795A (en) | Conversation design robot for system design | |
| CN114239536B (en) | Event extraction method, system and computer readable storage medium | |
| Jiang et al. | AutoML-Emo: Automatic knowledge selection using congruent effect for emotion identification in conversations | |
| CN115982342B (en) | Integration formulation method and system based on achievement conversion standard | |
| CN109656952B (en) | Query processing method and device and electronic equipment | |
| CN112015890A (en) | Movie scenario abstract generation method and device | |
| CN116860221A (en) | Intelligent recommendation method and system for low-code data model | |
| Jiang et al. | Development of an ontology-based semantic retrieval method for construction project risk management | |
| CN111930959B (en) | Method and device for generating text by map knowledge | |
| Yin et al. | Data Visualization Analysis Based on Explainable Artificial Intelligence: A Survey | |
| CN111242519B (en) | User characteristic data generation method and device and electronic equipment | |
| CN114118058A (en) | Emotion analysis system and method based on fusion of syntactic characteristics and attention mechanism | |
| CN113901789A (en) | Gate-controlled hole convolution and graph convolution based aspect-level emotion analysis method and system | |
| Ivaschenko et al. | Semantic analysis implementation in engineering enterprise content management systems | |
| CN111160048A (en) | Translation engine optimization system and method based on cluster evolution | |
| CN116737964B (en) | Artificial intelligence brain system | |
| CN114860805A (en) | Data mining analysis system and analysis method for Internet of things equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |