CN114881040B - Method and device for processing semantic information of paragraphs and storage medium - Google Patents
Method and device for processing semantic information of paragraphs and storage medium Download PDFInfo
- Publication number
- CN114881040B CN114881040B CN202210517950.6A CN202210517950A CN114881040B CN 114881040 B CN114881040 B CN 114881040B CN 202210517950 A CN202210517950 A CN 202210517950A CN 114881040 B CN114881040 B CN 114881040B
- Authority
- CN
- China
- Prior art keywords
- semantic
- information
- logic
- span
- cutting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Machine Translation (AREA)
Abstract
The invention relates to a semantic information processing method, a semantic information processing device and a semantic information processing storage medium for paragraphs, and belongs to the technical field of semantic information processing; the method comprises the following steps: importing an original paragraph, a semantic logic cutting model and a semantic understanding and expressing model; inputting the original paragraph into a semantic logic cutting model for logic cutting to obtain logic cutting span information of the original paragraph; inputting the logic cutting span information of the original paragraph into a semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the original paragraph; adding semantic information of the logic cutting span information into the original paragraph to obtain a complete paragraph; and updating and iterating the complete paragraph according to the semantic logic cutting model and the semantic understanding and expressing model to obtain final semantic information. The invention can obtain accurate and comprehensive paragraph semantic information, and improves the accuracy of paragraph semantic understanding and expression by paying attention to the logical boundary of the paragraph context.
Description
Technical Field
The present invention relates to the technical field of semantic information processing, and in particular, to a method and an apparatus for processing semantic information of paragraphs, and a storage medium.
Background
Semantic understanding and expression are a hot research problem in natural language processing. Semantic understanding is a method for converting a text into structured data which can be read by a computer, enabling the computer to understand the semantics, intention and the like of the text in a specific scene, and extracting information from the semantics, intention and the like so as to complete corresponding downstream tasks. Semantic expressions are the task of mapping text to machine-interpretable meaning representations. The framework based on semantic understanding and expression of different scales can be used for reading and expressing languages with different degrees of precision, and the establishment of the method research for semantic understanding and expression of dynamic scenes is an effective way for constructing the elastic accurate and comprehensive understanding and expression of semantics. In the initial stage, a traditional language model n-gram statistical model and a semantic-based combinability principle are respectively adopted for semantic understanding and expression. The N-gram statistical model has the problems that only the first N-1 words can be calculated, long-term dependence is lacked, model parameters grow along with the increase of N, and the like. Semantic expression models based on the principle of composition rely heavily on handmade grammars, dictionaries and features.
Secondly, representing the semantics by adopting a neural network-based parser in the aspect of semantic representation. For example, researchers have proposed data reconstruction methods that input a priori knowledge into the RNN model to improve the performance of the model. In 2018, researchers designed structure-aware neural architectures that split semantic representations into two phases. In 2019, researchers use the graph neural network architecture to merge related entities and their relationship information during the parsing process. However, such methods have poor generalization capability due to the lack of capture of combinatorial structures in utterances.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a method, an apparatus and a storage medium for processing semantic information of paragraphs, which are directed to the deficiencies of the prior art.
The technical scheme for solving the technical problems is as follows: a semantic information processing method of paragraphs includes the following steps:
importing an original paragraph, a semantic logic cutting model and a semantic understanding and expressing model;
inputting the original paragraph into the semantic logic cutting model for logic cutting to obtain logic cutting span information of the original paragraph;
inputting the logic cutting span information of the original paragraph into the semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the original paragraph;
adding semantic information of the logic cutting span information into the original paragraph to obtain a complete paragraph;
and updating and iterating the complete paragraph according to the semantic logic cutting model and the semantic understanding and expressing model to obtain final semantic information.
The invention has the beneficial effects that: the method comprises the steps of performing logic cutting on an original paragraph to obtain logic cutting span information, processing semantics through a semantic understanding and expression model, adding the obtained semantic information into the original paragraph to form a more perfect complete paragraph, performing updating iteration on the complete paragraph through the semantic logic cutting model and the semantic understanding and expression model to obtain semantic information accurately and comprehensively expressing the semantics of the paragraph, and improving the accuracy of semantic understanding and expression through paying attention to a logic boundary of a sentence context.
Another technical solution of the present invention for solving the above technical problems is as follows: a semantic information processing apparatus of a paragraph, comprising:
the import module is used for importing an original paragraph, a semantic logic cutting model and a semantic understanding and expressing model;
the logic cutting module is used for inputting the original paragraph into the semantic logic cutting model for logic cutting to obtain logic cutting span information of the original paragraph;
the semantic processing module is used for inputting the logic cutting span information of the original paragraph into the semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the original paragraph;
adding the semantic information of the logic cutting span information into the original paragraph to obtain a complete paragraph;
and updating and iterating the complete paragraph according to the semantic logic cutting model and the semantic understanding and expressing model to obtain final semantic information.
Another technical solution of the present invention for solving the above technical problems is as follows: a semantic information processing apparatus of a paragraph, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the semantic information processing method of the paragraph as described above being implemented when the computer program is executed by the processor.
Another technical solution of the present invention for solving the above technical problems is as follows: a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, implements the semantic information processing method of the above-described paragraphs.
Drawings
FIG. 1 is a flow chart illustrating a semantic information processing method according to an embodiment of the present invention;
FIG. 2 is a block diagram of functional modules of a semantic information processing apparatus according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a logic slicing method according to an embodiment of the present invention;
fig. 4 is a flowchart illustrating an update iteration method according to an embodiment of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
Example 1:
as shown in fig. 1, a semantic information processing method for paragraphs includes the following steps:
s1: importing an original paragraph, a semantic logic cutting model and a semantic understanding and expressing model;
s2: inputting the original paragraph into the semantic logic cutting model for logic cutting to obtain logic cutting span information of the original paragraph;
s3: inputting the logic cutting span information of the original paragraph into the semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the original paragraph;
s4: adding semantic information of the logic cutting span information into the original paragraph to obtain a complete paragraph;
s5: and updating and iterating the complete paragraph according to the semantic logic cutting model and the semantic understanding and expressing model to obtain final semantic information.
In the above embodiment, the original paragraph is logically cut to obtain the logically cut span information, the semantics are processed through the semantic understanding and expression model, the obtained semantic information is added to the original paragraph to form a more complete paragraph, the complete paragraph is updated and iterated through the semantic logical cutting model and the semantic understanding and expression model, the semantic information for accurately and comprehensively expressing the paragraph semantics can be obtained, and the semantic understanding and expression accuracy is improved by paying attention to the logical boundary of the sentence context.
Preferably, before performing the logical cut, the method further includes a step of training the semantic logical cut model, specifically:
importing the training paragraphs into the semantic understanding and expression model for decoding to obtain semantic information, selecting a plurality of key semantic information in the semantic information according to an attention mechanism, calculating conditional probabilities of the key semantic information through a probability calculation expression, obtaining logic cutting span contents corresponding to the key semantic information through the conditional probabilities, and taking the logic cutting span contents as training labels respectively;
inputting all training labels into the semantic logic cutting model, and training the semantic logic cutting model after the training labels are input by a pseudo-supervised training method;
the probability calculation expression is as follows:
L＝LOG logic (x)，
p(i|x)＝softmax i (LW LI )，
wherein L represents the hidden layer information encoded by the encoder, LOG logic Represents a neural network based paragraph logic segmentation model, p (i | θ |) (i) ) Representing input information logicConditional probability of the starting position of the boundary, p (j | θ |) (j) ) Representing conditional probability of the position of the end of the logical boundary of the input information, ln () representing the ln function, W LI And W LJ A weight matrix representing the starting and ending positions of the logical boundary line, respectively, i represents the starting position of the logical boundary line for the input information, j represents the ending position of the logical boundary line for the input information, | - | represents an absolute value function, θ i And theta j Respectively, representing the result of multiplying the input information by a weight matrix of the logical boundary line start and end positions. l. the k 、l j 、w j And w k The components of the weight matrix representing the encoder-encoded hidden layer information and the logical boundary start and end positions, respectively.
It should be understood that the semantic information of the content is obtained by inputting the original corpus into the semantic understanding and the encoding and decoding of the expression model Bi-GRU, then the key information in the semantic information is selected by using an attention mechanism, the key semantic information and the content obtained by decoding at the previous moment are input into a decoder, and the semantic information in the initial logic cutting span is expressed and understood reasonably by calculating the conditional probability of the content. And taking the logic cutting span content as a logic span label of a training set, and training a semantic logic cutting model by using the training set with the label. The pseudo-supervised training method does not need to manually make and correct labels in a large amount of data, and can improve efficiency and reduce cost.
Firstly, inputting an original paragraph into a GRU encoder of the semantic logic cutting model to obtain hidden layer information. Secondly, predicting the position of the original paragraph logic segmentation by using a conditional probability model to further obtain the initial and ending logic segmentation positions of the original paragraph, wherein the calculation expression is as follows:
p(i|x)＝softmax i (UW I )，
p(j|x)＝softmax j (UW J )，
wherein U is a hidden layer signal encoded by an encoderInformation, GRU seg In order to gate the recurrent neural network,
is a real number field
In an m × n dimensional space. p (i | x) is the conditional probability of the beginning position of the logical boundary of the input information. p (j | x) is the conditional probability of the location of the end of the logical boundary of the input information. softmax i (.) is the softmax function. W I And W J The weight matrices for the starting and ending positions of the logical boundary, respectively. i is the start position of the logical boundary of the input information. j is the location of the end of the logical boundary of the input information.
In the embodiment, the semantic logic cutting model can be trained, manual production and correction of labels in a large amount of data are not needed through the pseudo-supervised training method, the efficiency can be improved, and the cost can be reduced, so that an accurate statement context logic boundary can be obtained through the semantic logic cutting model.
Preferably, the original paragraph is input into the semantic logic cutting model for logic cutting, so as to obtain the logic cutting span information of the original paragraph, specifically:
inputting the original paragraph into the semantic logic cutting model, calculating the semantic distribution probability of the content to be cut in the whole original paragraph through the semantic logic cutting model, obtaining the position which can most express the complete semantic of the content to be cut according to the semantic distribution probability, setting the position as a logic cutting point, setting the distance from the initial position of the original paragraph to the logic cutting point as a logic cutting span, and expressing the information in the logic cutting span through a logic cutting expression, wherein the logic cutting expression is as follows:
wherein x is (k+1) Representing logical cut span information after k +1 iterations，
Represents the information in the logical cut span(s),
representing the logic cutting span information after k iterations, i and j respectively representing the initial position and the end position of the logic span, and x k Representing the logical cut span information after k iterations,
representing a semantic representation in a logical cut span,
representing the understanding and expression of semantic representations in a logical cut span,
a mapping relationship is represented.
The computational expression processing is illustrated below, for example:
as shown in figure 3 of the drawings,
iteration 1:
the folded flowers are sent to customers, and the vacant people feel unhappy. The people who remember and search out are always familiar, and when people are unfamiliar, the people are not suitable, i.e. the people are not suitable for your pinkish red wine green, or the people are not suitable for my firework dust. Even if the distance is the same, the people are far away and only happy and love; not only a sentence likes and loves you, it is enough. I get it graduallyIt is believed that most of these are rare and spurious, and the experience tells each one to be inappropriate.
Iteration 1:
the flowers are sent to guests and the vacant people are afraid of the flowers. The memory searched out is always familiar, and the like.
Iteration 1: x is the number of (k+1) : $ Create $, we are not fit, i are not fit for your pinkish red wine green, or you are not fit for my fireworks dust. Even if the distance is the same, the people are far away and only have love; not only a sentence likes and loves you, it is enough. I believe gradually that most of these are rare and spurious, and the experience tells each one to be inappropriate.
In the embodiment, the method can be used for accurately obtaining the logic cutting span information of the original paragraph, and the accuracy of semantic analysis is improved by paying attention to the logic boundary of the paragraph context.
Preferably, the step of inputting the logic cutting span information of the original paragraph into the semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the original paragraph includes:
inputting the logic cutting span information into the semantic understanding and expression model, and performing semantic processing on the logic cutting span information through encoding and decoding in the semantic understanding and expression model to obtain semantic information of the logic cutting span information, wherein the encoding and decoding comprises the following steps:
dec＝Attn-Bi-GRU dec (enc)，
wherein enc represents hidden layer information coded by an encoder in the semantic understanding and expression model, enc = Bi-GRU enc (s)，Bi-GRU dec Representing a bi-directional gated recurrent neural network, s representing input logical cut span information, dec tableIndicating decoder decoding information, attn-Bi-GRU dec (enc) denotes a bi-directional gated recurrent neural network that contains an attention mechanism for picking key information in the encoder output content.
It should be understood that the information in the logic cutting span is input into the semantic information of the content obtained by encoding and decoding of a semantic understanding and expression model (Bi-GRU), then the key information in the semantic information is selected by using an attention mechanism, the key semantic information and the content obtained by decoding at the previous moment are input into a decoder, and the conditional probability of the content in the span is calculated to express and understand the semantic information in the logic cutting span. In order to solve the long-distance dependence problem, the Bi-GRU network is selected as the semantic understanding and expression model, so that the network is adopted for modeling, and the network has the advantages of capability of capturing context semantic information of sentences, less parameters and easiness in convergence. The Bi-GRU Network can also solve the problem of gradient explosion and disappearance of a Recurrent Neural Network (RNN for short). The understanding and expression model consists of an encoder of the Bi-GRU network and a decoder of the Bi-GRU network including attention mechanism.
In the above embodiment, the semantic information can be analyzed through the semantic understanding and expression model, and the key information in the content can be output.
Preferably, the adding the semantic information of the logical cut span information to the original paragraph to obtain complete paragraph information specifically includes:
and adding the semantic information of the logic cutting span information to the cutting position in the original paragraph to obtain a complete paragraph, wherein the complete paragraph comprises a part of the original paragraph and the semantic information of the logic cutting span information.
In the above embodiment, the semantic information of the logical segmentation span information is added to the segmentation position in the original paragraph to form a more complete paragraph, which is convenient for more accurately analyzing the semantic information of the complete paragraph in the following process.
Preferably, the complete paragraph is updated and iterated according to the semantic logic cutting model and the semantic understanding and expression model to obtain final semantic information, which specifically includes:
inputting the complete paragraph into the semantic logic cutting model for logic cutting to obtain logic cutting span information of the complete paragraph;
inputting the logic cutting span information of the complete paragraph into the semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the complete paragraph;
updating and iterating the logic cutting span information and the semantic information of the complete paragraph by updating an iteration expression to obtain final semantic information, wherein the updating iteration expression is as follows:
wherein, x represents the inputted information,
representing information in a logical cut span, C representing an inclusion relation,
representing semantic representations in a logical cut span, C representing containment relationships,
representing the conditional probability of logically cutting information in a span under the conditions of the original input information,
represents a conditional probability of the semantic representation after the logical cut under the condition that the input information and the logical cut span information exist, p (y | x) represents a conditional probability of the semantic representation after the logical cut under the condition that the input information exists,
representing conditional probabilities of semantic representations in a logical cut span given the input information and the logical cut span information,
representing conditional probabilities of semantic representations after logical cuts under conditions of semantic representations in existing input information, logical cut span information, and logical cut spans, p (reducedldic _ y | reducedlogic _ x) representing conditional probabilities of semantic representations after logical cuts that are reduced under conditions of existing reduced input information, reducedlogic _ x representing existing reduced input information, reducedlogic _ y representing after reduced logical cuts,
representing the understanding and expression of semantic representations in a logical cut span,
a mapping relationship is represented.
It should be understood that the condition of the original input information refers to the condition of the original paragraph that was originally imported.
As shown in fig. 4, the following exemplifies the computational expression processing procedure:
iteration 1: the flowers are sent to guests and the vacant people are afraid of the flowers. The people who remember and search out are always familiar, and when people are unfamiliar, the people are not suitable, i.e. the people are not suitable for your pinkish red wine green, or the people are not suitable for my firework dust. Even if the distance is the same, the people are far away and only happy and love; not only a sentence likes and loves you, it is enough. I believe gradually that most of these words are really rare and fake, and the experience tells each one of them to be inappropriate.
Iteration 1: the flowers are sent to guests and the vacant people are afraid of the flowers. The memory searched out is always familiar, and the like.
Iteration 1: $ discount, we are not fit, i are not fit for your pinkish green, or you are not fit for my firework dust. Even if the distance is the same, the people are far away and only happy and love; not only a sentence likes and loves you, it is enough. I believe gradually that most of these words are really rare and fake, and the experience tells each one of them to be inappropriate.
Iteration 2: we are not fit, i.e. i do not fit your reddish green wine, or you are not fit my firework dust.
Iteration 2: $ fold flower $, $ us $. Even if the distance is the same, the people are far away and only happy and love; not only a sentence likes and loves you, it is enough. I believe gradually that most of these are rare and spurious, and the experience tells each one to be inappropriate.
Iteration 3: we: even the same distance, remote is only happy, love; not only a sentence likes and loves you, it is enough.
Iteration 3: $ fold flower $, $ us $. We gradually believe that most of these words are rare and spurious, and the experience tells each one to be inappropriate.
Iteration 4: i believe gradually that most of these words are really rare and fake, and the experience tells each one of them to be inappropriate.
Iteration 4: $ fold flower, $ us $. $ We $ $ even $ I get $ gradually.
In the above embodiment, the final semantic information is obtained by performing iterative processing on the complete paragraph update and gradually understanding the whole part of the text with the logical boundary of the paragraph context.
Example 2:
as shown in fig. 2, a semantic information processing apparatus of a paragraph includes:
the import module is used for importing an original paragraph, a semantic logic cutting model and a semantic understanding and expressing model;
the logic cutting module is used for inputting the original paragraph into the semantic logic cutting model for logic cutting to obtain logic cutting span information of the original paragraph;
the semantic processing module is used for inputting the logic cutting span information of the original paragraph into the semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the original paragraph;
adding semantic information of the logic cutting span information into the original paragraph to obtain a complete paragraph;
and updating and iterating the complete paragraph according to the semantic logic cutting model and the semantic understanding and expressing model to obtain final semantic information.
Preferably, the system further comprises a training module, configured to train the semantic logic cutting model before logic cutting, specifically:
importing the training paragraphs into the semantic understanding and expression model for decoding to obtain semantic information, selecting a plurality of key semantic information in the semantic information according to an attention mechanism, calculating conditional probabilities of the key semantic information through a probability calculation expression, obtaining logic cutting span contents corresponding to the key semantic information through the conditional probabilities, and taking the logic cutting span contents as training labels respectively;
inputting all training labels into the semantic logic cutting model, and training the semantic logic cutting model after the training labels are input by a pseudo-supervised training method;
the probability calculation expression is as follows:
L＝LOG logic (x)，
p(i|x)＝softmax i (LW LI )，
where L represents the encoder-encoded hidden layer information, LOG logic Represents a neural network based paragraph logic segmentation model, p (i | θ |) (i) ) Conditional probability, p (j | θ), representing the starting position of the logical boundary of the input information (j) ) Conditional probability representing the position of the end of the logical boundary of the input information, ln (.) representing the ln function, W LI And W LJ Weight matrices representing the start and end positions of the logic boundary line, respectively, i represents the start position of the logic boundary line of the input information, j represents the end position of the logic boundary line of the input information, | represents an absolute value function, | theta i And theta j Which represent the result of multiplying the input information by a weight matrix of the logical boundary start and end positions, respectively. l k 、l j 、w j And w k The components of the weight matrix representing the encoder-encoded hidden layer information and the logical boundary start and end positions, respectively.
Example 3:
a semantic information processing apparatus of a paragraph, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program, when executed by the processor, implementing the semantic information processing method of the paragraph as described above.
Example 4:
a computer-readable storage medium storing a computer program which, when executed by a processor, implements a semantic information processing method as described above.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, a division of a unit is only a logical division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (7)
1. A semantic information processing method of paragraphs is characterized by comprising the following steps:
importing an original paragraph, a semantic logic cutting model and a semantic understanding and expressing model;
inputting the original paragraph into the semantic logic cutting model for logic cutting to obtain logic cutting span information of the original paragraph;
inputting the logic cutting span information of the original paragraph into the semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the original paragraph;
adding semantic information of the logic cutting span information into the original paragraph to obtain a complete paragraph;
updating and iterating the complete paragraph according to the semantic logic cutting model and the semantic understanding and expressing model to obtain final semantic information;
before the logic cutting, the method also comprises the step of training the semantic logic cutting model, and specifically comprises the following steps:
importing the training paragraphs into the semantic understanding and expression model for decoding to obtain semantic information, selecting a plurality of key semantic information in the semantic information according to an attention mechanism, calculating conditional probabilities of the key semantic information through a probability calculation expression, obtaining logic cutting span contents corresponding to the key semantic information through the conditional probabilities, and taking the logic cutting span contents as training labels respectively;
inputting all training labels into the semantic logic cutting model, and training the semantic logic cutting model after the training labels are input by a pseudo-supervised training method;
the probability calculation expression is as follows:
L＝LOG logic (x)，
p(i|x)＝softmax i (LW LI )，
wherein L represents the hidden layer information encoded by the encoder, LOG logic Represents a neural network based paragraph logic segmentation model, p (i | θ) (i) ) Conditional probability, p (j | θ), representing the starting position of the logical boundary of the input information (j) ) Conditional probability representing the position of the end of the logical boundary of the input information, ln (.) representing the ln function, W LI And W LJ A weight matrix representing the starting and ending positions of the logical boundary line, respectively, i represents the starting position of the logical boundary line for the input information, j represents the ending position of the logical boundary line for the input information, | - | represents an absolute value function, θ i And theta j The result of multiplying the input information by a weight matrix representing the logical boundary start and end positions, respectively,/ k 、l j 、w j And w k Components of a weight matrix representing encoder-encoded hidden layer information and logical boundary start and end positions, respectively;
inputting the original paragraph into the semantic logic cutting model for logic cutting to obtain logic cutting span information of the original paragraph, specifically:
inputting the original paragraph into the semantic logic cutting model, calculating the semantic distribution probability of the content to be cut in the whole original paragraph through the semantic logic cutting model, obtaining the position which can most express the complete semantic of the content to be cut according to the semantic distribution probability, setting the position as a logic cutting point, setting the distance from the initial position of the original paragraph to the logic cutting point as a logic cutting span, and expressing the information in the logic cutting span through a logic cutting expression, wherein the logic cutting expression is as follows:
wherein x is (k+1) Representing the logical cut span information after k +1 iterations,
represents the information in the logical cut span,
representing the logic cutting span information after k iterations, i and j respectively representing the initial position and the end position of the logic span, and x k Representing the logical cut span information after k iterations,
representing a semantic representation in a logical cut span,
representing the understanding and expression of semantic representations in a logical cut span,
a mapping relationship is represented.
2. The semantic information processing method according to claim 1, wherein the semantic information obtained by inputting the logical segmentation span information of the original paragraph into the semantic understanding and expression model for semantic processing is specifically:
inputting the logic cutting span information into the semantic understanding and expression model, and performing semantic processing on the logic cutting span information through encoding and decoding in the semantic understanding and expression model to obtain semantic information of the logic cutting span information, wherein the encoding and decoding comprises the following steps:
dec＝Attn-Bi-GRU dec (enc),
wherein enc represents hidden layer information coded by an encoder in the semantic understanding and expression model, enc = Bi-GRU enc (s)，Bi-GRU dec Representing a Bi-directional gated recurrent neural network, s representing input logical cut span information, dec representing decoder decoding information, attn-Bi-GRU dec (enc) denotes a bi-directional gated recurrent neural network that contains an attention mechanism for picking key information in the encoder output content.
3. The semantic information processing method according to claim 1, wherein the semantic information of the logical cut span information is added to the original paragraph to obtain a complete paragraph, specifically:
and adding the semantic information of the logic cutting span information to the cutting position in the original paragraph to obtain a complete paragraph, wherein the complete paragraph comprises a part of the original paragraph and the semantic information of the logic cutting span information.
4. The semantic information processing method according to claim 3, wherein the complete paragraph is updated and iterated according to the semantic logic cutting model and the semantic understanding and expressing model to obtain final semantic information, specifically:
inputting the complete paragraph into the semantic logic cutting model for logic cutting to obtain logic cutting span information of the complete paragraph;
inputting the logic cutting span information of the complete paragraph into the semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the complete paragraph;
updating and iterating the logic cutting span information and the semantic information of the complete paragraph by updating an iteration expression to obtain final semantic information, wherein the updating iteration expression is as follows:
wherein, x represents the information that is input,
representing information in a logical cut span, C representing an inclusion relation,
representing a semantic representation in a logical cut span,
representing the conditional probability of logically cutting information in a span under the conditions of the original input information,
representing a conditional probability of the semantic representation after logical cutting under the condition of the existing input information and the logical cut span information, p (y | x) representing a conditional probability of the semantic representation after logical cutting under the condition of the existing input information,
language for representing logical cut span under existing input information and logical cut span informationThe conditional probability of the meaning is indicated,
represents a conditional probability of the semantic representation after logical cutting under conditions of the semantic representation in the existing input information, the logical cutting span information, and the logical cutting span, p (reducedloic _ y | reducedlogic _ x) represents a conditional probability of the semantic representation after logical cutting reduced under conditions of the existing reduced input information, reducedlogic _ x represents the existing reduced input information, reducedlogic _ y represents after logical cutting reduced,
representing the understanding and expression of semantic representations in a logical cut span,
a mapping relationship is represented.
5. A semantic information processing apparatus of a paragraph, characterized by comprising:
the import module is used for importing an original paragraph, a semantic logic cutting model and a semantic understanding and expressing model;
the logic cutting module is used for inputting the original paragraph into the semantic logic cutting model for logic cutting to obtain logic cutting span information of the original paragraph;
the semantic processing module is used for inputting the logic cutting span information of the original paragraph into the semantic understanding and expression model for semantic processing to obtain the semantic information of the logic cutting span information of the original paragraph;
adding the semantic information of the logic cutting span information into the original paragraph to obtain a complete paragraph;
updating and iterating the complete paragraph according to the semantic logic cutting model and the semantic understanding and expressing model to obtain final semantic information;
the system further comprises a training module, which is used for training the semantic logic cutting model before logic cutting, and specifically comprises:
before logic cutting, importing a training paragraph into the semantic understanding and expression model for decoding to obtain semantic information, selecting a plurality of key semantic information in the semantic information according to an attention mechanism, calculating conditional probabilities of the key semantic information through a probability calculation expression, obtaining logic cutting span contents corresponding to the key semantic information through the conditional probabilities, and taking the logic cutting span contents as training labels respectively;
inputting all training labels into the semantic logic cutting model, and training the semantic logic cutting model after the training labels are input by a pseudo-supervised training method;
the probability calculation expression is as follows:
L＝LOG logic (x)，
p(i|x)＝softmax i (LW LI )，
where L represents the encoder-encoded hidden layer information, LOG logic Represents a neural network based paragraph logic segmentation model, p (i | θ) (i) ) Conditional probability, p (j | θ), representing the starting position of the logical boundary of the input information (j) ) Conditional probability representing the position of the end of the logical boundary of the input information, ln (.) representing the ln function, W LI And W LJ Weight matrices representing the start and end positions of the logic boundary line, respectively, i represents the start position of the logic boundary line of the input information, j represents the end position of the logic boundary line of the input information, | represents an absolute value function, | theta i And theta j The result of multiplying the input information by a weight matrix representing the logical boundary start and end positions, respectively,/ k 、l j 、w j And w k Components of a weight matrix representing encoder-encoded hidden layer information and logical boundary start and end positions, respectively;
inputting the original paragraph into the semantic logic cutting model for logic cutting to obtain the logic cutting span information of the original paragraph, which specifically includes:
inputting the original paragraph into the semantic logic cutting model, calculating the semantic distribution probability of the content to be segmented in the whole original paragraph through the semantic logic cutting model, obtaining the position which can most express the complete semantic of the content to be segmented according to the semantic distribution probability, setting the position as a logic cutting point, setting the distance from the initial position of the original paragraph to the logic cutting point as a logic cutting span, and expressing the information in the logic cutting span through a logic cutting expression, wherein the logic cutting expression is as follows:
wherein x is (k+1) Represents the logical cutting span information after k +1 iterations,
represents the information in the logical cut span,
representing the logic cutting span information after k iterations, i and j respectively representing the initial position and the end position of the logic span, and x k Representing the logical cut span information after k iterations,
representing a semantic representation in a logical cut span,
representing the understanding and expression of semantic representations in logical cut spans,
a mapping relationship is represented.
6. A semantic information processing apparatus of a paragraph, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the semantic information processing method of the paragraph according to any one of claims 1 to 4 is implemented.
7. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, implements the semantic information processing method of the paragraphs according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210517950.6A CN114881040B (en) | 2022-05-12 | 2022-05-12 | Method and device for processing semantic information of paragraphs and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210517950.6A CN114881040B (en) | 2022-05-12 | 2022-05-12 | Method and device for processing semantic information of paragraphs and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114881040A CN114881040A (en) | 2022-08-09 |
| CN114881040B true CN114881040B (en) | 2022-12-06 |
Family
ID=82675545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210517950.6A Active CN114881040B (en) | 2022-05-12 | 2022-05-12 | Method and device for processing semantic information of paragraphs and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114881040B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111639486A (en) * | 2020-04-30 | 2020-09-08 | 深圳壹账通智能科技有限公司 | Paragraph searching method and device, electronic equipment and storage medium |
| CN111914067A (en) * | 2020-08-19 | 2020-11-10 | 苏州思必驰信息科技有限公司 | Chinese text matching method and system |
| WO2020228376A1 (en) * | 2019-05-16 | 2020-11-19 | 华为技术有限公司 | Text processing method and model training method and apparatus |
| WO2021082786A1 (en) * | 2019-10-30 | 2021-05-06 | 腾讯科技（深圳）有限公司 | Semantic understanding model training method and apparatus, and electronic device and storage medium |
| CN113204611A (en) * | 2021-04-06 | 2021-08-03 | 北京百度网讯科技有限公司 | Method for establishing reading understanding model, reading understanding method and corresponding device |
| CN113536801A (en) * | 2020-04-16 | 2021-10-22 | 北京金山数字娱乐科技有限公司 | Reading understanding model training method and device and reading understanding method and device |
| CN113553848A (en) * | 2021-07-19 | 2021-10-26 | 北京奇艺世纪科技有限公司 | Long text classification method, system, electronic equipment and computer readable storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10255273B2 (en) * | 2017-06-15 | 2019-04-09 | Microsoft Technology Licensing, Llc | Method and system for ranking and summarizing natural language passages |
| CN109933773B (en) * | 2017-12-15 | 2023-05-26 | 上海擎语信息科技有限公司 | Multiple semantic statement analysis system and method |
| US20210034676A1 (en) * | 2019-07-30 | 2021-02-04 | International Business Machines Corporation | Semantic relationship search against corpus |
| CN113536804B (en) * | 2021-06-29 | 2022-05-03 | 北京理工大学 | Natural language feature extraction method based on keyword enhancement GRU and Kronecker |
-
2022
- 2022-05-12 CN CN202210517950.6A patent/CN114881040B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020228376A1 (en) * | 2019-05-16 | 2020-11-19 | 华为技术有限公司 | Text processing method and model training method and apparatus |
| WO2021082786A1 (en) * | 2019-10-30 | 2021-05-06 | 腾讯科技（深圳）有限公司 | Semantic understanding model training method and apparatus, and electronic device and storage medium |
| CN113536801A (en) * | 2020-04-16 | 2021-10-22 | 北京金山数字娱乐科技有限公司 | Reading understanding model training method and device and reading understanding method and device |
| CN111639486A (en) * | 2020-04-30 | 2020-09-08 | 深圳壹账通智能科技有限公司 | Paragraph searching method and device, electronic equipment and storage medium |
| CN111914067A (en) * | 2020-08-19 | 2020-11-10 | 苏州思必驰信息科技有限公司 | Chinese text matching method and system |
| CN113204611A (en) * | 2021-04-06 | 2021-08-03 | 北京百度网讯科技有限公司 | Method for establishing reading understanding model, reading understanding method and corresponding device |
| CN113553848A (en) * | 2021-07-19 | 2021-10-26 | 北京奇艺世纪科技有限公司 | Long text classification method, system, electronic equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114881040A (en) | 2022-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109960724B (en) | Text summarization method based on TF-IDF | |
| CN109753660B (en) | LSTM-based winning bid web page named entity extraction method | |
| US6188976B1 (en) | Apparatus and method for building domain-specific language models | |
| CN110688862A (en) | Mongolian-Chinese inter-translation method based on transfer learning | |
| CN103116578A (en) | Translation method integrating syntactic tree and statistical machine translation technology and translation device | |
| CN108319734A (en) | A kind of product feature structure tree method for auto constructing based on linear combiner | |
| CN110362797B (en) | Research report generation method and related equipment | |
| CN111090736A (en) | Question-answering model training method, question-answering method, device and computer storage medium | |
| CN111930895B (en) | MRC-based document data retrieval method, device, equipment and storage medium | |
| CN110808032A (en) | Voice recognition method and device, computer equipment and storage medium | |
| CN112036184A (en) | Entity identification method, device, computer device and storage medium based on BilSTM network model and CRF model | |
| CN111984782B (en) | Tibetan text abstract generation method and system | |
| CN115759119B (en) | Financial text emotion analysis method, system, medium and equipment | |
| CN111813923A (en) | Text summarization method, electronic device and storage medium | |
| CN109977430B (en) | Text translation method, device and equipment | |
| CN115033706A (en) | Method for automatically complementing and updating knowledge graph | |
| CN114881040B (en) | Method and device for processing semantic information of paragraphs and storage medium | |
| WO2023115770A1 (en) | Translation method and related device therefor | |
| CN111428509B (en) | Latin letter-based Uygur language processing method and system | |
| CN115994544A (en) | Parallel corpus screening method, parallel corpus screening device, and readable storage medium | |
| CN115168544A (en) | Information extraction method, electronic device and storage medium | |
| CN114997190A (en) | Machine translation method, device, computer equipment and storage medium | |
| CN111488423B (en) | Index data-based natural language processing method and system | |
| CN114328894A (en) | Document processing method, document processing device, electronic equipment and medium | |
| CN112256970B (en) | News text pushing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |