CN113177414A - Semantic feature processing method and device and storage medium - Google Patents

Semantic feature processing method and device and storage medium Download PDF

Info

Publication number
CN113177414A
CN113177414A CN202110458796.5A CN202110458796A CN113177414A CN 113177414 A CN113177414 A CN 113177414A CN 202110458796 A CN202110458796 A CN 202110458796A CN 113177414 A CN113177414 A CN 113177414A
Authority
CN
China
Prior art keywords
word
level
sub
words
vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110458796.5A
Other languages
Chinese (zh)
Inventor
蔡晓东
郑淑婷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guilin University of Electronic Technology
Original Assignee
Guilin University of Electronic Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guilin University of Electronic Technology filed Critical Guilin University of Electronic Technology
Priority to CN202110458796.5A priority Critical patent/CN113177414A/en
Publication of CN113177414A publication Critical patent/CN113177414A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/216—Parsing using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Machine Translation (AREA)

Abstract

The invention relates to a semantic feature processing method, a semantic feature processing device and a storage medium, wherein the method comprises the following steps: importing a statement to be processed; respectively taking a plurality of words in the sentence to be processed as processing objects, and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word; respectively taking a plurality of words in the sentence to be processed as processing objects, extracting sub-word level characteristics of each word, and obtaining sub-word level vectors corresponding to each word; and respectively taking the word level embedded vector and the sub-word level vector corresponding to the same word as the input of a gating dynamic selection mechanism, and obtaining the word semantic features corresponding to the same word through the gating dynamic selection mechanism. The invention can process the sentence to be processed at word level and sub-word level, namely, between words and characters, to obtain word level embedded vectors and sub-word level vectors, can well process and balance low-frequency vectors and unknown words, and can be used as a gating dynamic selection mechanism for input, thereby improving the accuracy of semantic features.

Description

Semantic feature processing method and device and storage medium
Technical Field
The invention relates to the field of voice data processing, in particular to a semantic feature processing method, a semantic feature processing device and a storage medium.
Background
Because of the diversity of words and syntactic structures, the semantics of low-frequency vectors and unknown words cannot be accurately represented by the words, namely word levels, so that the semantic feature representation task becomes an important challenge, and the unknown words refer to words which do not appear in a text vocabulary library. The conventional methods such as embedding and the like have the problem of dimension disaster caused by sparse representation of word vector features, and the obtained semantic features are inaccurate due to the fact that dependence on a long distance is difficult to learn.
Disclosure of Invention
The invention aims to solve the technical problem of the prior art and provides a semantic feature processing method, a semantic feature processing device and a semantic feature processing storage medium.
The technical scheme for solving the technical problems is as follows: a semantic feature processing method comprises the following steps:
importing a sentence to be processed, wherein the sentence to be processed comprises a plurality of words, and the words are English words;
respectively taking a plurality of words in the sentence to be processed as processing objects, and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word;
respectively taking a plurality of words in the sentence to be processed as processing objects, extracting sub-word level characteristics of each word, and obtaining sub-word level vectors corresponding to each word;
and respectively taking the word level embedded vector and the sub-word level vector corresponding to the same word as the input of a gating dynamic selection mechanism, and obtaining the word semantic features corresponding to the same word through the gating dynamic selection mechanism.
Another technical solution of the present invention for solving the above technical problems is as follows: a semantic feature processing system comprising:
the system comprises an importing module, a processing module and a processing module, wherein the importing module is used for importing a sentence to be processed, the sentence to be processed comprises a plurality of words, and the words are English words;
the processing module is used for respectively taking a plurality of words in the sentence to be processed as processing objects, and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word;
respectively taking a plurality of words in the sentence to be processed as processing objects, extracting sub-word level characteristics of each word, and obtaining sub-word level vectors corresponding to each word;
and the output module is used for respectively taking the word level embedded vector and the sub-word level vector corresponding to the same word as the input of a gating dynamic selection mechanism, and obtaining the semantic features of the word corresponding to the same word through the gating dynamic selection mechanism.
Another technical solution of the present invention for solving the above technical problems is as follows: a semantic feature processing system comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the semantic feature processing method as described above being implemented when the computer program is executed by the processor.
Another technical solution of the present invention for solving the above technical problems is as follows: a computer-readable storage medium, storing a computer program which, when executed by a processor, implements a semantic feature processing method as described above.
The invention has the beneficial effects that: the invention divides the sentence into word level and sub-word level forms, can process the sentence to be processed together with word level and sub-word level, namely word and sub-word, because the granularity of sub-word level is between word and character, solve the problem of data sparsity brought by word vector feature representation, can better process the semantics of low frequency vector and unknown word through sub-word level vector, and input the word level embedded vector and sub-word level vector as a gating dynamic selection mechanism, thus improving the accuracy of semantic feature.
Drawings
Fig. 1 is a flowchart of a semantic feature processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of data flow provided by an embodiment of the present invention;
fig. 3 is a structural diagram of a semantic feature processing system according to an embodiment of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
Example 1
As shown in fig. 1, a semantic feature processing method includes the following steps:
s1: importing a sentence to be processed, wherein the sentence to be processed comprises a plurality of words, and the words are English words;
s2: respectively taking a plurality of words in the sentence to be processed as processing objects, and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word;
s3: respectively taking a plurality of words in the sentence to be processed as processing objects, extracting sub-word level characteristics of each word, and obtaining sub-word level vectors corresponding to each word;
s4: and respectively taking the word level embedded vector and the sub-word level vector corresponding to the same word as the input of a gating dynamic selection mechanism, and obtaining the word semantic features corresponding to the same word through the gating dynamic selection mechanism.
It should be understood that the method is applicable to words in english sentences.
In the embodiment, the sentences are divided into word-level and sub-word-level forms, the sentences to be processed can be processed together in word level and sub-word level, namely words and sub-words, the problem of data sparsity caused by word vector feature representation is solved because the granularity of the sub-word level is between words and characters, the semantics of low-frequency vectors and unknown words can be better processed through the sub-word-level vectors, the word-level embedded vectors and the sub-word-level vectors are input as a gating dynamic selection mechanism, and the accuracy of semantic features is improved.
The granularity of character-level embedding serving as a solution of low-frequency vectors and unknown words is too fine, and the granularity of subwords (subwords) is between words and characters, so that the low-frequency vectors and the unknown words can be better processed.
Because the granularity of the character-level embedding between the low-frequency vector and the unknown word is too fine, and the granularity of the Subword is between words and characters, the problems of the low-frequency vector and the unknown word can be well balanced.
Specifically, in the word-level dimension: the process of respectively using a plurality of words in the sentence to be processed as processing objects to perform word-level vectorization representation on each word and obtaining the word-level embedded vector corresponding to each word comprises the following steps:
and performing word-level vectorization representation on each word through a GloVe word embedding tool to obtain a word-level embedding vector corresponding to each word in the to-be-processed sentence, wherein the dimension of the GloVe word embedding tool is 300 dimensions.
It is understood that Global Vectors for Word Representation, which is a Global Word frequency statistics-based Word Representation tool, can represent a Word as a vector of real numbers, and these Vectors capture some semantic features, such as similarity, between words.
The GloVe word embedding tool needs to be trained in advance and then used, and the GloVe word embedding tool is trained in 300 dimensions.
In the embodiment, the sentences to be processed can be processed at word level and sub-word level, namely words and characters, to obtain word level embedded vectors and sub-word level vectors, so that the problems of low-frequency vectors and unknown words can be well balanced and input as a gating dynamic selection mechanism, and the accuracy of semantic features is improved.
Specifically, in the subword dimension: the process of respectively taking a plurality of words in the sentence to be processed as processing objects, extracting the sub-word level features of each word and obtaining the sub-word level vector corresponding to each word comprises the following steps:
dividing each word into a plurality of sub-words according to a Skin-gram method, wherein the plurality of sub-words of the same word are a sub-word group, the sub-words are in a combination form of n-gram characters, and n is the number of the characters;
mapping sub-words of each word to sub-word level vectors of the same dimension through a bidirectional LSTM network, wherein the sub-word level vectors are represented as
Figure BDA0003041585720000051
Wherein,
Figure BDA0003041585720000052
s is a sub-word which is,
Figure BDA0003041585720000053
is the forward hidden layer vector of the jth sub-phrase,
Figure BDA0003041585720000054
Figure BDA0003041585720000055
is the backward hidden layer vector of the jth sub-phrase,
Figure BDA0003041585720000056
is the number, w, of forward hidden layer vectors or backward hidden layer vectors corresponding to the sub-word groupsAnd bsAre all sub-word level parameter matrices. H1、H2… … are forward or backward hidden layer vectors of subwords.
For example, n is set to 3, and taking the word W as sunshine for example, W can be expressed as: w { < su, sun, uns, nsh, shi, hie, ine, ne >. Wherein, "< su" is a group, and "ne >" is a group, and also includes several sub-words of sun, uns, nsh, shi, hie, ine. Taking the word W as hats for example, W can be expressed as: < ha, hat, ats, ts > the 4 sub-words.
That is, one word W may be composed of a plurality of subwords.
In the above embodiment, the subword in each word can be mapped into a subword level vector of the same dimension through the bidirectional LSTM network, so that each word is mapped in the same dimension, and the subword level vector is used as another part of the input of the gating dynamic selection mechanism, thereby significantly improving the accuracy of semantic feature representation.
Specifically, the process of respectively inputting the word-level embedded vector and the sub-word-level vector corresponding to each word as a gated dynamic selection mechanism, and respectively obtaining the semantic features of the words corresponding to each word through the gated dynamic selection mechanism includes:
determining a gating dynamics selection coefficient g from a primary patternjSaid gated dynamic selection coefficient gjThe method is used for carrying out weight distribution on a word-level embedded vector and a sub-word-level vector of the same word, and the first formula is
Figure BDA0003041585720000061
Figure BDA0003041585720000062
For word-level embedding of vectors, wTB is a word-level parameter matrix, and sigma is a sigmoid activation function;
using word level embedded vector and sub-word level vector of the same word as input of gate control dynamic selection mechanism, and utilizing the gate control dynamic selection mechanism and the gate control dynamic selection coefficient gjObtaining word semantic features which are
Figure BDA0003041585720000063
Wherein,
Figure BDA0003041585720000064
is a sub-word level vector, and is,
Figure BDA0003041585720000065
vector is embedded for word level, and element product operation is performed for vector.
It should be understood that the sigmoid activation function is a Linear function Linear.
In the above embodiment, a more accurate semantic representation of the words is obtained by calculating the weight coefficients of the word-level embedding vectors and the sub-word-level vectors of the words.
In the embodiment, the gating dynamic selection coefficient is calculated to determine the weight coefficients of the word-level embedded vector and the sub-word-level vector, the word-level embedded vector and the sub-word-level vector are used as characteristics to be input into the gating dynamic selection mechanism, and the final word semantic feature representation is obtained by using the gating dynamic selection mechanism. The invention obtains the semantic representation of the words by calculating the weight coefficient of the words, and can simultaneously calculate in parallel, thereby improving the timeliness.
As shown in fig. 2, the following describes the data processing flow by taking a certain word as an example, for example, the word "hats":
data flow to line 1: performing word-level vectorization representation on hats to obtain word-level embedded vectors corresponding to all words
Figure BDA0003041585720000066
Data flow to line 2: dividing hats into separate parts according to Skin-gram method<ha，hat，ats，ts>These 4 sub-words.
In the data flow line 1, the word level embedded vector is obtained
Figure BDA0003041585720000067
Then, the vector is embedded through the word level
Figure BDA0003041585720000068
Figure BDA0003041585720000069
Determining a gating dynamics selection coefficient g by sigma and a primary equationj。
In the data flow to line 2, will<ha，hat，ats，ts>As sub-words, inputting each sub-word into the forward hidden layer of the bidirectional LSTM network to obtain the vector of the forward hidden layer
Figure BDA00030415857200000610
And inputting each sub-word into a backward hidden layer of the bidirectional LSTM network to obtain a backward hidden layer vector
Figure BDA00030415857200000611
Will be provided with
Figure BDA00030415857200000612
And
Figure BDA00030415857200000613
substitution into
Figure BDA0003041585720000071
To obtain sub-word level vector
Figure BDA0003041585720000072
Finally, by
Figure BDA0003041585720000073
Outputting semantic character v of word corresponding to 'hats'j。
It should be appreciated that data flow to line 1 and data flow to line 2 can be performed simultaneously, with parallel computation, improving timeliness. Or according to the sequence, and the sequence can be unfixed.
Example 2
As shown in fig. 2, a semantic feature processing system includes:
the system comprises an importing module, a processing module and a processing module, wherein the importing module is used for importing a sentence to be processed, the sentence to be processed comprises a plurality of words, and the words are English words;
the processing module is used for respectively taking a plurality of words in the sentence to be processed as processing objects, and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word;
respectively taking a plurality of words in the sentence to be processed as processing objects, extracting sub-word level characteristics of each word, and obtaining sub-word level vectors corresponding to each word;
and the output module is used for respectively inputting the word level embedded vector and the sub-word level vector corresponding to the same word as a gating dynamic selection mechanism, and obtaining the semantic features of the word corresponding to the same word through the gating dynamic selection mechanism.
In the embodiment, the sentence to be processed can be processed at word level and sub-word level, namely, words and characters are processed together to obtain word level embedded vectors and sub-word level vectors, the problems of low-frequency vectors and unknown words can be well balanced and used as input of a gating dynamic selection mechanism, and the accuracy of semantic features is improved.
Specifically, in the processing module, the process of respectively taking a plurality of words in the sentence to be processed as processing objects and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word includes:
and performing word-level vectorization representation on each word through a GloVe word embedding tool to obtain a word-level embedding vector corresponding to each word in the to-be-processed sentence, wherein the dimension of the GloVe word embedding tool is 300 dimensions.
Specifically, in the processing module, the process of extracting the sub-word level features of each word by using the plurality of words in the sentence to be processed as processing objects respectively to obtain the sub-word level vector corresponding to each word includes:
dividing each word into a plurality of sub-words according to a Skin-gram method, wherein the plurality of sub-words of the same word are a sub-word group, the sub-words are in a combination form of n-gram characters, and n is the number of the characters;
mapping sub-words of each word to sub-word level vectors of the same dimension through a bidirectional LSTM network, wherein the sub-word level vectors are represented as
Figure BDA0003041585720000081
Wherein,
Figure BDA0003041585720000082
s is a sub-word which is,
Figure BDA0003041585720000083
is the forward hidden layer vector of the jth sub-phrase,
Figure BDA0003041585720000084
Figure BDA0003041585720000085
is the backward hidden layer vector of the jth sub-phrase,
Figure BDA0003041585720000086
is the number, w, of forward hidden layer vectors or backward hidden layer vectors corresponding to the sub-word groupsAnd bsAre all sub-word level parameter matrices.
Specifically, in the input module, the word-level embedded vector and the sub-word-level vector corresponding to each word are respectively input as a gated dynamic selection mechanism, and the process of respectively obtaining the semantic features of the words corresponding to each word through the gated dynamic selection mechanism includes:
determining a gating dynamics selection coefficient g from a primary patternjSaid gated dynamic selection coefficient gjThe method is used for carrying out weight distribution on a word-level embedded vector and a sub-word-level vector of the same word, and the first formula is
Figure BDA0003041585720000087
gjCoefficients are dynamically selected for the jth gate, w is the word level,
Figure BDA00030415857200000811
for word-level embedding of vectors, wTB is a word-level parameter matrix, and sigma is a sigmoid activation function;
using word level embedded vector and sub-word level vector of the same word as input of gate control dynamic selection mechanism, and utilizing the gate control dynamic selection mechanism and the gate control dynamic selection coefficient gjObtaining word semantic features which are
Figure BDA0003041585720000088
Wherein,
Figure BDA0003041585720000089
is a sub-word level vector, and is,
Figure BDA00030415857200000810
vector is embedded for word level, and element product operation is performed for vector.
Example 3
A semantic feature processing system comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the semantic feature processing method as described above being implemented when the computer program is executed by the processor.
Example 4
A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out a semantic feature processing method as set forth above.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The technical solutions provided by the embodiments of the present invention are described in detail above, and the principles and embodiments of the present invention are explained in this patent by applying specific examples, and the descriptions of the embodiments above are only used to help understanding the principles of the embodiments of the present invention; the present invention is not limited to the above preferred embodiments, and any modifications, equivalent replacements, improvements, etc. within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A semantic feature processing method is characterized by comprising the following steps:
importing a sentence to be processed, wherein the sentence to be processed comprises a plurality of words, and the words are English words;
respectively taking a plurality of words in the sentence to be processed as processing objects, and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word;
respectively taking a plurality of words in the sentence to be processed as processing objects, extracting sub-word level characteristics of each word, and obtaining sub-word level vectors corresponding to each word;
and respectively taking the word level embedded vector and the sub-word level vector corresponding to the same word as the input of a gating dynamic selection mechanism, and obtaining the word semantic features corresponding to the same word through the gating dynamic selection mechanism.
2. The semantic feature processing method according to claim 1, wherein the process of respectively taking a plurality of words in the sentence to be processed as processing objects and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word comprises:
and performing word-level vectorization representation on each word through a GloVe word embedding tool to obtain a word-level embedding vector corresponding to each word in the to-be-processed sentence, wherein the dimension of the GloVe word embedding tool is 300 dimensions.
3. The semantic feature processing method according to claim 1, wherein the process of extracting sub-word level features of each word by using a plurality of words in the sentence to be processed as processing objects, and obtaining sub-word level vectors corresponding to the words comprises:
dividing each word into a plurality of sub-words according to a Skin-gram method, wherein the plurality of sub-words of the same word are a sub-word group, the sub-words are in a combination form of n-gram characters, and n is the number of the characters;
mapping sub-words of each word to sub-word level vectors of the same dimension through a bidirectional LSTM network, wherein the sub-word level vectors are represented as
Figure FDA0003041585710000011
Wherein,
Figure FDA0003041585710000012
s is a sub-word which is,
Figure FDA0003041585710000021
is the forward hidden layer vector of the jth sub-phrase,
Figure FDA0003041585710000022
Figure FDA0003041585710000023
is the backward hidden layer vector of the jth sub-phrase, l is the number of the forward hidden layer vector or the backward hidden layer vector corresponding to the sub-phrase, wsAnd bsAre all sub-word level parameter matrices.
4. The semantic feature processing method according to claim 3, wherein the word-level embedded vector and the sub-word-level vector corresponding to each word are respectively input as a gated dynamic selection mechanism, and the process of obtaining semantic features of words corresponding to the same word by the gated dynamic selection mechanism comprises:
determining a gating dynamic selection coefficient according to a first formula, wherein the gating dynamic selection coefficient is used for carrying out weight distribution on word-level embedded vectors and sub-word-level vectors of the same word, and the first formula is
Figure FDA0003041585710000024
gjCoefficients are dynamically selected for the jth gate, w is the word level,
Figure FDA0003041585710000025
for word-level embedding of vectors, wTB is a word-level parameter matrix, and sigma is a sigmoid activation function;
using word level embedded vector and sub-word level vector of the same word as input of gate control dynamic selection mechanism, and utilizing the gate control dynamic selection mechanism and the gate control dynamic selection coefficient gjObtaining word semantic features which are
Figure FDA0003041585710000026
Wherein,
Figure FDA0003041585710000027
is a sub-word level vector, and is,
Figure FDA0003041585710000028
the vector is embedded for the word-level,as an element product operation is performed on the vector.
5. A semantic feature processing system, comprising:
the system comprises an importing module, a processing module and a processing module, wherein the importing module is used for importing a sentence to be processed, the sentence to be processed comprises a plurality of words, and the words are English words;
the processing module is used for respectively taking a plurality of words in the sentence to be processed as processing objects, and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word;
respectively taking a plurality of words in the sentence to be processed as processing objects, extracting sub-word level characteristics of each word, and obtaining sub-word level vectors corresponding to each word;
and the output module is used for respectively taking the word level embedded vector and the sub-word level vector corresponding to the same word as the input of a gating dynamic selection mechanism, and obtaining the semantic features of the word corresponding to the same word through the gating dynamic selection mechanism.
6. The semantic feature processing system according to claim 5, wherein in the processing module, the process of respectively taking a plurality of words in the sentence to be processed as processing objects and performing word-level vectorization representation on each word to obtain a word-level embedded vector corresponding to each word comprises:
and performing word-level vectorization representation on each word through a GloVe word embedding tool to obtain a word-level embedding vector corresponding to each word in the to-be-processed sentence, wherein the dimension of the GloVe word embedding tool is 300 dimensions.
7. The semantic feature processing system according to claim 5, wherein in the processing module, taking a plurality of words in the sentence to be processed as processing objects, respectively, and extracting sub-word level features of each word to obtain a sub-word level vector corresponding to each word comprises:
dividing each word into a plurality of sub-words according to a Skin-gram method, wherein the plurality of sub-words of the same word are a sub-word group, the sub-words are in a combination form of n-gram characters, and n is the number of the characters;
mapping sub-words of each word to sub-word level vectors of the same dimension through a bidirectional LSTM network, wherein the sub-word level vectors are represented as
Figure FDA0003041585710000031
Wherein,
Figure FDA0003041585710000032
s is a sub-word which is,
Figure FDA0003041585710000033
is the forward hidden layer vector of the jth sub-phrase,
Figure FDA0003041585710000034
Figure FDA0003041585710000035
is the backward hidden layer vector of the jth sub-phrase, l is the number of the forward hidden layer vector or the backward hidden layer vector corresponding to the sub-phrase, wsAnd bsAre all sub-word level parameter matrices.
8. The semantic feature processing system according to claim 7, wherein the input module inputs the word-level embedding vector and the sub-word-level vector corresponding to each word as a gated dynamic selection mechanism, and the process of obtaining the semantic features of the words corresponding to each word by the gated dynamic selection mechanism comprises:
determining a gating dynamics selection coefficient g from a primary patternjSaid gated dynamic selection coefficient gjThe method is used for carrying out weight distribution on a word-level embedded vector and a sub-word-level vector of the same word, and the first formula is
Figure FDA0003041585710000036
gjIs as followsj gated dynamic selection coefficients, w being the word level,
Figure FDA0003041585710000037
for word-level embedding of vectors, wTB is a word-level parameter matrix, and sigma is a sigmoid activation function;
using word level embedded vector and sub-word level vector of the same word as input of gate control dynamic selection mechanism, and utilizing the gate control dynamic selection mechanism and the gate control dynamic selection coefficient gjObtaining word semantic features which are
Figure FDA0003041585710000041
Wherein,
Figure FDA0003041585710000042
is a sub-word level vector, and is,
Figure FDA0003041585710000043
vector is embedded for word level, and element product operation is performed for vector.
9. A semantic feature processing system comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that when the computer program is executed by the processor, the semantic feature processing method according to any one of claims 1 to 4 is implemented.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the semantic feature processing method according to any one of claims 1 to 4.
CN202110458796.5A 2021-04-27 2021-04-27 Semantic feature processing method and device and storage medium Pending CN113177414A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110458796.5A CN113177414A (en) 2021-04-27 2021-04-27 Semantic feature processing method and device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110458796.5A CN113177414A (en) 2021-04-27 2021-04-27 Semantic feature processing method and device and storage medium

Publications (1)

Publication Number Publication Date
CN113177414A true CN113177414A (en) 2021-07-27

Family

ID=76926415

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110458796.5A Pending CN113177414A (en) 2021-04-27 2021-04-27 Semantic feature processing method and device and storage medium

Country Status (1)

Country Link
CN (1) CN113177414A (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200050667A1 (en) * 2018-08-09 2020-02-13 CloudMinds Technology, Inc. Intent Classification Method and System
CN111274826A (en) * 2020-01-19 2020-06-12 南京新一代人工智能研究院有限公司 Semantic information fusion-based low-frequency word translation method
CN111444700A (en) * 2020-04-02 2020-07-24 山东山大鸥玛软件股份有限公司 Text similarity measurement method based on semantic document expression

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200050667A1 (en) * 2018-08-09 2020-02-13 CloudMinds Technology, Inc. Intent Classification Method and System
CN111274826A (en) * 2020-01-19 2020-06-12 南京新一代人工智能研究院有限公司 Semantic information fusion-based low-frequency word translation method
CN111444700A (en) * 2020-04-02 2020-07-24 山东山大鸥玛软件股份有限公司 Text similarity measurement method based on semantic document expression

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JORGE A. BALAZS 等: "Gating Mechanisms for Combining Character and Word-level Word Representations: An Empirical Study", 《PROCEEDINGS OF THE 2019 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: STUDENT RESEARCH WORKSHOP》 *
PIOTR BOJANOWSKI 等: "EnrichingWord Vectors with Subword Information", 《TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS》 *

Similar Documents

Publication Publication Date Title
CN108595706B (en) Document semantic representation method based on topic word similarity, and text classification method and device
WO2020062770A1 (en) Method and apparatus for constructing domain dictionary, and device and storage medium
CN110019471B (en) Generating text from structured data
Singer et al. The MITLL NIST LRE 2011 language recognition system
JP5377889B2 (en) Language processing apparatus and program
US20140156575A1 (en) Method and Apparatus of Processing Data Using Deep Belief Networks Employing Low-Rank Matrix Factorization
JP6815899B2 (en) Output statement generator, output statement generator and output statement generator
JP6461308B2 (en) Speech recognition device and rescoring device
CN112711948A (en) Named entity recognition method and device for Chinese sentences
CN105229676A (en) The learning device of the learning method of deep-neural-network and learning device and category independently sub-network
Kann et al. One-shot neural cross-lingual transfer for paradigm completion
JP2004355483A (en) Morpheme analysis device, morpheme analysis method and morpheme analysis program
Zhang et al. Deep autoencoding topic model with scalable hybrid Bayesian inference
US20220164600A1 (en) Unsupervised document representation learning via contrastive augmentation
JP6291443B2 (en) Connection relationship estimation apparatus, method, and program
KR102024845B1 (en) Device and method for analyzing speech act
CN112183117B (en) Translation evaluation method and device, storage medium and electronic equipment
WO2014073206A1 (en) Information-processing device and information-processing method
CN112185361B (en) Voice recognition model training method and device, electronic equipment and storage medium
Morishita et al. Improving neural machine translation by incorporating hierarchical subword features
CN112669845A (en) Method and device for correcting voice recognition result, electronic equipment and storage medium
CN114995903B (en) Class label identification method and device based on pre-training language model
Chordia PunKtuator: A multilingual punctuation restoration system for spoken and written text
Jurgovsky et al. Evaluating memory efficiency and robustness of word embeddings
CN109446334A (en) A kind of method that realizing English Text Classification and relevant device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210727

RJ01 Rejection of invention patent application after publication