TWI662425B - A method of automatically generating semantic similar sentence samples - Google Patents

Abstract

The invention provides a method for automatically generating semantically similar sentence samples, which belongs to the field of language processing technology. The method includes: obtaining sentence samples and performing word segmentation processing; using a word vector model to obtain a set of similar words close to the semantics of each word; respectively Select a similar word from the set and replace the word to form a sample of semantically similar sentences; use a language model to generate possible values that represent the possibility of semantics for each sample of semantically similar sentences, and compare the possible values from high to low according to the possible values Sort all semantically similar sentence samples; select and retain the first N semantically similar sentence samples for subsequent processing steps based on the retained semantically similar sentence samples. The above technical solution has the beneficial effect that it can automatically generate a large number of sentence samples with similar semantics without the need for a large number of subsequent sentence sets, which saves a lot of manual work.

Description

Method for automatically generating semantically similar sentence samples

The invention relates to the technical field of natural language processing, in particular to a method for automatically generating semantically similar sentence samples.

In the prior art, in the process of natural language processing, many processing tasks require a large number of sentences or sentence sets with similar semantics. These semantically similar sentences or sentence sets need to be written manually, so it will consume a lot of manpower and time.

With the development of automation technology, more and more semantically similar sentences can be written in an automated way. At present, there are mainly several ways to obtain semantically similar sentence sets in the following ways:

(1) A large number of semantically similar sentences are obtained in a retrieval mode. The so-called search method means to find a set of sentences with similar semantics through a certain search formula among a large number of candidate sentences. The premise of the application of this method is that it needs a large number of candidate sentence sets. In the process of using search to find and generate semantically similar sentences, the performance requirements of the semantic similarity search module are very high, that is, the semantic similarity search module. The performance of the group determines the accuracy of the semantically similar sentences obtained by the retrieval method.

(2) Use sequence to sequence to obtain a large number of semantically similar sentences. This method is currently very active in the field of academic research, but many sentences generated by this method in practical applications are not reasonable, and their performance is not very good, so it lacks certain practicability.

According to the above-mentioned problems in the prior art, a technical solution of a method for automatically generating semantically similar sentence samples is now provided, which aims to efficiently and automatically generate a large number of semantically similar sentence samples, saving a lot of manual work.

The above technical solution specifically includes: a method for automatically generating a sample of semantically similar sentences, which is applicable to the process of natural language processing; among which, a word vector model for training and forming semantically similar words is pre-trained and used to judge the generated semantically similar words; The language model of semantic possibility of semantically similar sentence samples further includes: step S1, obtaining externally input sentence samples; step S2, performing word segmentation processing on the sentence samples to decompose the sentence samples into a plurality of sequentially arranged words Combination; step S3, using a word vector model, to obtain a set of similar words with similar semantics to each word included in the sentence sample; step S4, selecting a similar word from the set corresponding to each word and replacing it Words to form a semantically similar sentence sample associated with the sentence sample; Step S5, determine whether there are similar words that have not been selected in the set: If so, return to Step S4; Step S6, use the language model for each semantics separately Similar sentence samples generate possible values that represent semantic possibilities, and Energy descending sentences semantically similar for all samples sorting; a subsequent processing step step S7, to select and retain the first N samples sentences semantically similar to similar samples semantic sentence reserved.

Preferably, the method for automatically generating semantically similar sentence samples, wherein the types of sentence samples include: sentence types, sentence samples of sentence types include a plurality of words arranged in sequence; sentence types, sentence samples of sentence types Step S1 includes multiple words and multiple part-of-speech tags arranged sequentially, or sentence-type sentence samples include multiple part-of-speech tags arranged sequentially. Step S1 specifically includes: Step S11, obtaining an externally input sentence sample; Step S12, determine the type of the sentence sample: if the sentence sample is a sentence type, go to step S13; if the sentence sample is a sentence type, go directly to step S2; step S13, replace each part of speech tag in the sentence sample with A high-frequency word corresponding to the part-of-speech tag to form a complete sentence sample, and then proceeds to step S2.

Preferably, the method for automatically generating semantically similar sentence samples, wherein a preset word segmentation method is used to pre-train and form a word vector model; then in step S2, a sentence segmentation method is used to perform word segmentation processing on the sentence samples.

Preferably, the method for automatically generating a sample of semantically similar sentences, wherein, in step S4, the similar words selected and used for replacement have the same part of speech as the replaced words.

Preferably, the method for automatically generating a sample of semantically similar sentences, wherein, in step S6, the possible value of each sample of semantically similar sentences is a semantic value used to indicate the possibility that each sample of semantically similar sentences is established as a complete sentence. Academic scoring.

Preferably, the method for automatically generating semantically similar sentence samples, wherein the types of semantically similar sentence samples include: sentence type, the semantically similar sentence samples of sentence type include a plurality of words arranged in sequence; sentence type, sentence style The semantically similar sentence samples of the type include multiple words and word class tags arranged sequentially, or the sentence sample of the sentence type includes multiple word class tags arranged sequentially. Then step S7 specifically includes: step S71, Select and retain the first N semantically similar sentence samples; Step S72, determine whether it is necessary to output semantically similar sentence samples of the sentence type: if yes, go to step S73; if not, go to step S74; step S73, sample the semantically similar sentences The words included in the words are replaced with corresponding part-of-speech tags to form a complete sample of semantically similar sentences, followed by subsequent processing steps; step S74, the subsequent processing steps are performed according to the retained samples of semantically similar sentences.

The above technical solution has the beneficial effect of providing a method for automatically generating sentence samples with similar semantics, which can automatically generate a large number of sentence samples with similar semantics without the need for a large number of subsequent sentence sets, saving a lot of manual work.

In the following, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that, in the case of no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

The present invention is further described below with reference to the accompanying drawings and specific embodiments, but not as a limitation of the present invention.

Based on the above problems in the prior art, a method for automatically generating semantically similar sentence samples is now provided, and the method is applicable to the process of natural language processing.

In the above method, a word vector model for processing and obtaining semantically similar words and a language model for judging the semantic possibility of the generated semantically similar sentence samples are formed and trained in advance.

The above method is specifically shown in FIG. 1 and includes: Step S1, obtaining an externally input sentence sample; Step S2, performing a word segmentation process on the sentence sample to decompose the sentence sample into a combination including a plurality of sequentially arranged words; Step S3 , Using the word vector model, to obtain a set of similar words with similar semantics to each word included in the sentence sample; step S4, selecting a similar word from the set corresponding to each word and replacing the word to form Step S5, determine whether there are similar words that have not been selected in the set: If so, go back to Step S4; Step S6, use the language model to generate each semantically similar sentence sample separately It is used to represent the possible values of semantic possibility, and all samples of semantically similar sentences are sorted according to the possible values from high to low. Step S7, selecting and retaining the first N samples of semantically similar sentences to perform according to the retained samples of semantically similar sentences. Follow-up processing steps.

In this embodiment, the above word vector model can be formed by using some tools that represent words as real-valued vectors, such as Word2vec. This tool can use the idea of deep learning to simplify the processing of text content into a K-dimensional vector space through training. Vector operations, and the similarity in vector space can be used to represent the semantic similarity of text. The above word vector refers to modeling a language model using a neural network, and simultaneously obtaining a word representation in a vector space. By processing a word using the word vector, a similar word of the word can be obtained based on the similarity between the words.

In this embodiment, the training samples trained to form the above word vector model may be a large amount of text data, and these text data sources may be text data on different forums, and need to undergo word segmentation processing before input.

After passing through the above word vector model, the output should be a low-dimensional real number vector used to represent words, and each word in the training corpus should correspond to a low-dimensional real number vector.

The above real number vectors can usually be expressed as [0.792, −0.177, −0.107, 0.109, −0.542, ...] or similar forms, and the dimensions are more common in 50 and 100 dimensions. The distance between the word and the vector of words can be measured by the most traditional Euclidean distance, or by the angle of cos. For vectors expressed in this way, the distance between "Mike" and "Microphone" will be much smaller than "Mike" and "Weather". E.g. The similarity can be calculated by calculating the angle of cos to obtain the similar words of the specified word. In the process of calculating the similarity between other words and the specified word, the higher similarity is the similar word.

In this embodiment, the language model may be a model used to calculate a sentence completion probability of a sentence, for example, expressed as P (W1, W2, ... Wk). Using a language model, you can determine which word sequence is more likely to be a sentence, or given a number of words, you can predict the next most likely word. In short, the language model is used to determine whether a word sequence composed of several words conforms to the habit of speaking, that is, the possibility that the word sequence is a sentence. In a preferred embodiment of the present invention, the language model may be implemented by using an n-gram model.

Specifically, in the process of training a language model, each text sentence that is input to the model is processed by word segmentation, and the output can be the probability of collocation and combination of words in each text sentence.

In this embodiment, in step S1 above, the sentence samples obtained from the external input may be manually input or may be obtained by connecting to an external sentence sample database. The obtained sentence samples can be purely random sentence samples, and only need to follow the most basic semantic rules, such as conforming to the necessary conditions for semantically forming a sentence, and being a smooth sentence.

In this embodiment, in step S2 described above, word segmentation processing is performed on each sentence sample separately, so a sentence sample can be decomposed into a combination including a plurality of words arranged in order. For example, for a sentence sample "I want to listen to Jay Chou's blue and white porcelain", after the word segmentation is formed "I + to + listen + Jay Chou + + blue and white porcelain", which should be followed in the next steps should have specific meanings Words such as the noun "Jay Chou" and the noun "blue and white porcelain". Further, each word in the above sentence sample has a corresponding part-of-speech tag. For example, the part-of-speech tag of "Jay Chou" is "singer" (which may be represented by "singer" during computer processing). The part-of-speech tag is "song" (which may be represented by "song" in computer processing) and so on. In this embodiment, the part-of-speech tag may also be referred to as a tag of the word.

In this embodiment, after the word segmentation processing is performed on the sentence samples, a word vector model processing is performed according to each word to obtain a corresponding set of similar words. Specifically, the so-called similar words refer to words with similar semantic meanings consistent with the part-of-speech of the word. For example, for "Jay Chou", the label is "singer", and the similar words corresponding to the label are processed according to the word vector model. There may be "Wang Lihong", "Tao Yun", "Chen Yixun", and "Na Ying", etc. According to the word vector model, the set of similar words can be processed and output. Correspondingly, if Jay Chou's label is "male singer" (may be expressed as "male-Singer" during computer processing), the similar words corresponding to the label may have "Wang Lihong" and "Tao Yan" And "Eason Chan". In other words, the labels corresponding to different words determine the set of similar words for that word.

In this embodiment, in step S4 above, a similar word is selected from the set corresponding to each word and the word is replaced to form a semantically similar sentence sample associated with the sentence sample. For example, there may be a word corresponding to a sentence sample, that is, a sentence sample is formed by a sequence of a word, and for each word, there is a set of similar words, and each set has b semantically closest words to the word. Similar words, a sentence sample may correspond to ba semantically similar sentence samples, that is, a set of semantically similar sentence samples exists for one sentence sample, and multiple sets of semantically similar sentence samples may exist for multiple sentence samples. Automatically generate a large number of semantically similar sentence samples.

In this embodiment, the above step S5 is a cyclic selection of the similar word set, that is, the operations implemented in the above steps S4-S5 to generate a large number of semantically similar sentence samples for a batch of input sentence samples.

In this embodiment, when generating semantically similar sentence samples, some semantically similar sentence samples may not be semantically smooth due to the stacking of simple similar words, and thus cannot be used as a normal sentence sample for subsequent processing. Therefore, in the above step S6, after generating the semantically similar sentence samples, the above-trained and generated language model needs to be used to analyze the semantic possibility of each semantically similar sentence sample. Finally, the semantically similar sentence sample can be generated for For the possible value representing the semantic possibility of the sentence, the possible value can be used to represent the semantic rationality of the sentence. Then, the semantically similar sentence samples are ranked from highest to lowest according to the possible value. Specifically, for a given sentence S = W1, W2, ..., Wk, where S is used to mark the sentence, and Wk (k = 1,2,3 ...) is used to represent the kth word in the sentence.

Then the possible values of the above sentence can be expressed as: P (S) = P (W1, W2,…, Wk) ~ P (W1) P (W2 | W1)… P (Wk | W1, W2,…, Wk-1 ), The probabilities such as "P (W1)" and "P (W2 | W1)" in the above formula are formed by the above language model training. Therefore, the possible value P (S) of each sentence S can be obtained through the language model processing, and the possible value can also be regarded as the semantic score of the sentence.

Finally, in the above step S7, the first N semantically similar sentence samples are selected and retained, and then subsequent processing steps are performed on the retained semantically similar sentence samples, and other unreserved semantically similar sentence samples are discarded. The above N can be a natural number, and its value can be freely set by the user according to the actual situation.

Specifically, for step S7 described above, in a preferred embodiment of the present invention, the first N semantically similar sentence samples may be retained for each input sentence sample. In another embodiment of the present invention, only the first N may be retained for all formed semantically similar sentence samples. The selected object range can be set by the user as needed.

In a preferred embodiment of the present invention, the types of the input sentence samples include: sentence type, the sentence samples of the sentence type include a plurality of words arranged in order; sentence type, the sentence samples of the sentence type include according to A plurality of words and part-of-speech tags arranged in order, or only a plurality of parts-of-speech tags arranged in order; then the above step S1 is specifically shown in FIG. 2 and includes: Step S11, obtaining an externally input sentence sample; Step S12, judging a sentence Type of sample: If the sentence sample is a sentence type, go to step S13; if the sentence sample is a sentence type, go directly to step S2; Step S13, replace each part of speech tag in the sentence sample with a A high-frequency word is formed to form a complete sentence sample, and then the process proceeds to step S2.

Specifically, in this embodiment, the types of the sentence samples may include a sentence type and a sentence pattern type.

The so-called sentence type refers to a sentence that includes multiple words arranged in order. For example, "I want to listen to Jay Chou's blue and white porcelain" is a sentence.

The so-called sentence type refers to a sentence that includes multiple words and part-of-speech tags arranged in order, or only a sentence that includes multiple part-of-speech tags arranged in order. For example, "I want to listen to 'song' of a singer" is a sentence Style, where "singer" and "song" are part of speech tags.

Further, as long as a part-of-speech tag appears in a sentence sample, the sentence sample is a sentence sample of a sentence type. For example, "I want to listen to Jay Chou's" song "is a sample sentence type.

Then, in this embodiment, the sentence sample can be entered into the foregoing step S2 for subsequent operations without any processing.

For the sentence pattern sample, the part-of-speech tags need to be replaced with words corresponding to the tags to form a complete sentence, and then sent to the above step S2 for subsequent processing.

Specifically, in the above step S13, the part-of-speech tag in the sentence sample judged as a sentence type is replaced with a high-frequency word in the tag to form a complete sentence sample. The so-called high-frequency words refer to the words that appear frequently under a part-of-speech tag and are used frequently from statistical data. These high-frequency words can be used to replace the corresponding part-of-speech tags in sentence-type sentence samples. A reasonable and complete sample of the sentence.

In a preferred embodiment of the present invention, a preset word segmentation method is used to pre-train and form a word vector model;

Then, in step S2 above, a preset word segmentation method is used to perform word segmentation processing on the sentence samples.

Specifically, in this embodiment, the same word segmentation method as that used to form the above word vector model is used to perform word segmentation processing on a sentence sample, which can reduce words other than the set in subsequent processing steps, thereby helping to improve the final processing effect. .

In a preferred embodiment of the present invention, the above-mentioned preset word segmentation method may adopt a processing method of word segmentation based on forward maximum matching logic of a large dictionary: taking m characters in a sentence to be segmented from left to right as Match field, m is the word length of the longest word in the large dictionary; find the large dictionary to match, if the match is successful, segment the successfully matched field as a word; if the match is unsuccessful, remove the last word of the matched field, The remaining string is used as a new matching field, and matching is performed again, and the above process is repeated until all words are segmented.

In another preferred embodiment of the present invention, the above-mentioned preset word segmentation method may adopt a method for processing word segmentation based on a reverse maximum matching logic of a large dictionary, specifically: taking m m sentences to be segmented from right to left The character is used as the matching field, and m is the word length of the longest word in the large dictionary; the large dictionary is searched for matching. If the match is successful, the successfully matched field is split as a word; if the match is not successful, the first word of the matched field Remove the remaining string as a new matching field, and perform matching again. Repeat the above process until all words are segmented.

In another preferred embodiment of the present invention, the preset word segmentation method may also use a two-dimensional maximum matching logic based on a large dictionary for word segmentation processing, that is, the combination of the above forward maximum matching logic and the reverse maximum matching logic for word segmentation. Method of processing. Specifically: if the results of the forward maximum match and the reverse maximum match are the same, take any one of the results and output; if the results of the forward maximum match and the reverse maximum match are different, first select the result with fewer words after segmentation; If the number of words is the same, the result of the reverse maximum match is selected.

The so-called "big dictionary" in the above embodiment refers to a dictionary database containing a large number of words formed after collection.

In other embodiments of the present invention, other word segmentation methods may also be applied to the present invention, without affecting the protection scope of the present invention.

In a preferred embodiment of the present invention, in the above step S4, the similar words selected and used for replacement have the same part-of-speech between the replaced words, such as the same nouns or verbs, so the replacement operation can be guaranteed. Accuracy to avoid unreasonable sentence logic after replacement.

In a preferred embodiment of the present invention, the types of the semantically similar sentence samples include: a sentence type, and the semantically similar sentence samples of the sentence type include a plurality of words arranged in order; a sentence type, a sentence type semantically similar sentence The sample includes multiple words and part-of-speech tags arranged in order, or only multiple part-of-speech tags arranged in order; as shown in FIG. 3, the above step S7 specifically includes: Step S71, selecting and retaining the first N semantically similar Sentence sample; Step S72, determine whether it is necessary to output a sentence type semantically similar sentence sample: if yes, go to step S73; if not, go to step S74; step S73, replace the words included in the semantically similar sentence sample with corresponding ones Part-of-speech tags to form a complete sample of semantically similar sentences, followed by subsequent processing steps; step S74, subsequent processing steps are performed according to the retained sample of semantically similar sentences.

Specifically, similar to the above, the above-mentioned semantically similar sentence samples also include a sentence type and a sentence type. Then, in this embodiment, the user can set the final semantically similar sentence samples as sentence type or sentence type. If the user sets the final semantically similar sentence samples as sentence type, the output is directly filtered by the language model. Semanticly similar sentence samples and subsequent processing steps.

If the user sets the final semantically similar sentence sample as the sentence type, the words included in the semantically similar sentence sample need to be replaced with corresponding part-of-speech tags to form a complete sentence type semantically similar sentence sample. Follow-up processing steps.

In a preferred embodiment of the present invention, the subsequent processing steps mentioned above may include the development of a semantic open platform based on an automatically generated large number of semantically similar sentence samples, or calculation of semantic similarity.

Specifically, in a preferred embodiment of the present invention, the function of the semantic open platform is to open the semantic interface to other developers to help developers complete the development of specific projects. When a user enters a sentence or a sentence pattern, the above method can automatically generate a large number of similar sentences or sentence patterns, thereby increasing the ability of semantic generalization, enhancing the ability of semantic understanding, and reducing a large number of manual operations, saving time and improving effectiveness.

Correspondingly, in the preferred embodiment of the present invention, in the process of calculating the semantic similarity, a large number of sentences or sentence patterns with similar semantics need to be used. Then, the method described above can be used to generate a large number of sentences for semantic similarity. Sentence samples for degree calculation training process.

In a preferred embodiment of the present invention, in the above step S7, a set including the retained semantically similar sentence samples may be finally output for subsequent processing.

The above are only preferred embodiments of the present invention, and therefore do not limit the implementation and protection scope of the present invention. For those skilled in the art, they should be able to realize that equivalent substitutions and Obvious changes should be included in the protection scope of the present invention.

S1‧‧‧Step S1

S2‧‧‧Step S2

S3‧‧‧Step S3

S4‧‧‧Step S4

S5‧‧‧Step S5

S6‧‧‧Step S6

S7‧‧‧Step S7

S11‧‧‧Step S11

S12‧‧‧Step S12

S13‧‧‧Step S13

S71‧‧‧Step S71

S72‧‧‧Step S72

S73‧‧‧Step S73

S74‧‧‧Step S74

FIG. 1 is a schematic diagram of an overall process of a method for automatically generating semantically similar sentence samples in a preferred embodiment of the present invention; FIG. 2 is a preferred embodiment of the present invention based on FIG. 1 to obtain external input Figure 3 is a schematic flow chart of processing and processing of a sample of a sentence; Figure 3 is a preferred embodiment of the present invention, on the basis of Figure 1, select and retain samples of sentences with similar semantics while processing the output of samples of sentences with similar semantics schematic diagram.

Claims (7)

A method for automatically generating a sample of semantically similar sentences, which is applicable to the process of natural language processing; in which, a word vector model for processing and obtaining semantically similar words is pre-trained and formed, and a method for judging the generated sample of the semantically similar sentences is used. The language model of semantic possibility further includes: step S1, obtaining a sentence sample of external input; step S2, performing word segmentation processing on the sentence sample to decompose the sentence sample into a combination including a plurality of sequentially arranged words; step S3. Use the word vector model to obtain a set of similar words that are semantically similar to each of the words included in the sentence sample. Step S4, select one of the similar words from the set corresponding to each of the words. Word and replace the word to form a semantically similar sentence sample associated with the sentence sample; step S5, determine whether there are any similar words in the set that have not yet been selected: if so, return to step S4; step S6, Using the language model, for each semantically similar sentence sample, a The energy value, and sort all the semantically similar sentence samples according to the possible value from high to low; step S7, select and retain the first N semantically similar sentence samples to perform subsequent processing steps according to the retained semantically similar sentence samples . The method for automatically generating a semantically similar sentence sample according to claim 1, wherein the types of the sentence samples include: a sentence type, and the sentence sample of the sentence type includes a plurality of the words arranged in sequence; a sentence type, The sentence sample of the sentence type includes a plurality of the words and a part of speech tag of the word in sequence, or the sentence sample of the sentence type includes a plurality of the word tags of the sentence in order; this step; S1 specifically includes: Step S11, obtaining an externally input sentence sample; Step S12, judging the type of the sentence sample: If the sentence sample is the sentence type, go to step S13; If the sentence sample is the sentence type, then Go directly to step S2; Step S13, replace each part-of-speech tag in the sentence sample with a high-frequency word corresponding to the part-of-speech tag to form a complete sample of the sentence, and then turn to step S2. The method for automatically generating a sample of semantically similar sentences as described in claim 1, wherein a preset word segmentation method is used to pre-train and form the word vector model; then in step S2, the preset word segmentation method is adopted to the sentence. The sample is processed for word segmentation. The method for automatically generating a sample of semantically similar sentences as described in claim 1, wherein, in step S4, the similar word selected and used for replacement has the same part of speech as the replaced word. The method for automatically generating a semantically similar sentence sample as described in claim 1, wherein, in step S6, the possible value of each semantically similar sentence sample is used to indicate that each semantically similar sentence sample is taken as a complete sentence Semantic scoring of the possibility of establishment. The method for automatically generating a semantically similar sentence sample according to claim 1, wherein the types of the semantically similar sentence sample include: a sentence type, and the semantically similar sentence sample of the sentence type includes a plurality of the words arranged in order; Sentence type, the semantically similar sentence sample of the sentence type includes a plurality of the words and a part of the part of speech tag of the word sequentially, or the sentence sample of the sentence type includes a plurality of the words arranged sequentially The part-of-speech tag; then step S7 specifically includes: step S71, selecting and retaining the first N samples of the semantically similar sentence; step S72, determining whether to output the sample of the semantically similar sentence of the sentence type: if yes, go to step S73 ; If not, go to step S74; step S73, replace the words included in the semantically similar sentence sample with the corresponding part-of-speech tags to form a complete sample of the semantically similar sentence, and then perform subsequent processing steps; step S74, according to The retained semantically similar sentence samples are subjected to subsequent processing steps. The method for automatically generating samples of semantically similar sentences as described in claim 1, wherein in step S7, after selecting and retaining the first N samples of semantically similar sentences, outputting a sample set including the retained samples of semantically similar sentences, For subsequent processing steps.