CN108170667A - Term vector processing method, device and equipment - Google Patents
Term vector processing method, device and equipment Download PDFInfo
- Publication number
- CN108170667A CN108170667A CN201711235849.7A CN201711235849A CN108170667A CN 108170667 A CN108170667 A CN 108170667A CN 201711235849 A CN201711235849 A CN 201711235849A CN 108170667 A CN108170667 A CN 108170667A
- Authority
- CN
- China
- Prior art keywords
- word
- vector
- term vector
- neural networks
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Machine Translation (AREA)
Abstract
This specification embodiment discloses term vector processing method, device and equipment.The method includes：Obtain each word segmented to language material, establish the term vector of each word, according to the term vector of each word, and the term vector of up and down cliction of each word in language material, convolutional neural networks are trained, according to the convolutional neural networks after the term vector of each word and training, the training result of the term vector of each word is obtained.
Description
Technical field
This specification is related to computer software technical field more particularly to term vector processing method, device and equipment.
Background technology
The solution of natural language processing of today, mostly using the framework based on neural network, and in this framework
Next important basic technology is exactly term vector.Term vector is the vector that word is mapped to a fixed dimension, the vector table
The semantic information of the word is levied.
In the prior art, the algorithm for being commonly used in generation term vector such as includes：The term vector algorithm of Google,
Deep neural network algorithm of Microsoft etc..
Based on the prior art, a kind of more accurately term vector scheme is needed.
Invention content
This specification embodiment provides term vector processing method, device and equipment, to solve following technical problem：It needs
It will a kind of more accurately term vector scheme.
In order to solve the above technical problems, what this specification embodiment was realized in：
A kind of term vector processing method that this specification embodiment provides, including：
Obtain each word segmented to language material；
Establish the term vector of each word；
According to the term vector of the cliction up and down of the term vector of each word and each word in the language material, to volume
Product neural network is trained；
According to the convolutional neural networks after the term vector of each word and training, the term vector of each word is obtained
Training result.
A kind of term vector processing unit that this specification embodiment provides, including：
Acquisition module obtains each word segmented to language material；
Module is established, establishes the term vector of each word；
Training module, according to the word of the cliction up and down of the term vector of each word and each word in the language material
Vector is trained convolutional neural networks；
Processing module according to the convolutional neural networks after the term vector of each word and training, obtains each word
Term vector training result.
Another term vector processing method that this specification embodiment provides, including：
Step 1, the vocabulary being made up of each word segmented to language material is established, each word is not included in institute's predicate
Occurrence number is less than the word of setting number in material；Jump procedure 2；
Step 2, the total quantity of each word is determined, identical word is only counted once；Jump procedure 3；
Step 3, the different 1-hot term vector that dimension is the quantity is established respectively for each word；Jump procedure 4；
Step 4, the language material after traversal participle performs step 5 to the current word traversed, is performed if completion is traversed
Step 6, otherwise continue to traverse；
Step 5, centered on current word, more k words is respectively slid to both sides and establish window, current word will be removed in window
Word in addition carries out convolution as upper and lower cliction, and by the convolutional layer of the term vector input convolutional neural networks of all clictions up and down
It calculates, the pond layer that convolutional calculation result inputs the convolutional neural networks carries out pondization calculating, obtains primary vector；It will be current
The full articulamentum that the term vector of word and the negative sample word selected in the language material inputs the convolutional neural networks is counted
It calculates, respectively obtains secondary vector and third vector；According to the primary vector, the secondary vector, the third vector and
The loss function specified updates the parameter of the convolutional neural networks；
The convolutional calculation is carried out according to equation below：
The pondization calculating is carried out according to equation below：
Or
The loss function includes：
Wherein, xiRepresent the term vector of about i-th cliction, xi:i+θ-1It represents the word of about i-th~i+ θ -1 clictions
The vector that vector splicing obtains, yiRepresent i-th of the element of vector obtained by the convolutional calculation, ω represents convolutional layer
Weight parameter, ζ represent the offset parameter of convolutional layer, and σ represents excitation function, and max represents maximizing function, and average is represented
It averages function, c (j) represents j-th of element of the primary vector that pondization obtains after calculating, and t represents upper and lower cliction
Quantity, c represent the primary vector, and w represents the secondary vector, w'mRepresent the corresponding third of m-th of negative sample word to
Amount, ω represent the weight parameter of convolutional layer, and ζ represents the offset parameter of convolutional layer,Represent the weight parameter of full articulamentum, τ is represented
The offset parameter of full articulamentum, γ represent hyper parameter, and s represents similarity calculation function, and λ represents the quantity of negative sample word；
Step 6, by the term vector of each word respectively input training after the convolutional neural networks full articulamentum into
Row calculates, and obtains corresponding term vector training result.
A kind of term vector processing equipment that this specification embodiment provides, including：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be performed by least one processor, and described instruction is by described at least one
A processor performs, so that at least one processor can：
Language material is segmented to obtain each word；
Establish the term vector of each word；
According to the term vector of the cliction up and down of the term vector of each word and each word in the language material, to volume
Product neural network is trained；
According to the convolutional neural networks after the term vector of each word and training, the term vector of each word is obtained
Training result.
Above-mentioned at least one technical solution that this specification embodiment uses can reach following advantageous effect：Convolutional Neural
Network can be calculated by convolutional calculation and pondization, and the context entirety semantic information of word is portrayed, in extraction more
Hereafter semantic information, and then more accurate term vector training result can be obtained, therefore, can partially or entirely it solve
Above-mentioned technical problem.
Description of the drawings
In order to illustrate more clearly of this specification embodiment or technical solution of the prior art, below will to embodiment or
Attached drawing needed to be used in the description of the prior art is briefly described, it should be apparent that, the accompanying drawings in the following description is only
Some embodiments described in this specification, for those of ordinary skill in the art, in not making the creative labor property
Under the premise of, it can also be obtained according to these attached drawings other attached drawings.
Fig. 1 is a kind of overall architecture schematic diagram that the scheme of this specification is related under a kind of practical application scene；
Fig. 2 is the flow diagram of a kind of term vector processing method that this specification embodiment provides；
Fig. 3 is a kind of structure diagram of convolutional neural networks under the practical application scene that this specification embodiment provides；
Fig. 4 is the flow diagram of another term vector processing method that this specification embodiment provides；
Fig. 5 is the structure diagram of a kind of term vector processing unit corresponding to Fig. 2 that this specification embodiment provides.
Specific embodiment
This specification embodiment provides term vector processing method, device and equipment.
In order to which those skilled in the art is made to more fully understand the technical solution in this specification, below in conjunction with this explanation
Attached drawing in book embodiment is clearly and completely described the technical solution in this specification embodiment, it is clear that described
Embodiment be merely a part but not all of the embodiments of the present application.Based on this specification embodiment, this field
Those of ordinary skill's all other embodiments obtained without creative efforts, should all belong to the application
The range of protection.
Fig. 1 is a kind of overall architecture schematic diagram that the scheme of this specification is related under a kind of practical application scene.This is whole
In body framework, four parts are related generally to：The term vector of up and down cliction of the term vector and word of word, word in language material in language material,
Convolutional neural networks training server.The action that preceding three parts are related to can be held by corresponding software and/or hardware function
Row, for example, can also be performed by convolutional neural networks training server.
Word and its term vector of upper and lower cliction are for training convolutional neural networks, and then with the convolutional neural networks after trained
Term vector is made inferences again, by network training process and term vector reasoning process, realizes term vector training, the reasoning results
As term vector training result.
The scheme of this specification is suitable for the term vector of English words, is also applied for any languages such as Chinese, Japanese and German
Term vector.For ease of description, following embodiment says the scheme of this specification mainly for the scene of English words
It is bright.
Fig. 2 is the flow diagram of a kind of term vector processing method that this specification embodiment provides.Slave device angle and
Speech, the executive agent of the flow such as include following at least one equipment：Personal computer, large and medium-sized computer, computer collection
Group, mobile phone, tablet computer, intelligent wearable device, vehicle device etc..
Flow in Fig. 2 may comprise steps of：
S202：Obtain each word segmented to language material.
In this specification embodiment, each word can be specifically：At least occurred in primary word extremely in language material
Small part word.For the ease of subsequent processing, each word can be stored in vocabulary, need to read word from vocabulary when using
.
It should be noted that if the number occurred in language material in view of certain word is very little, when subsequent processing, changes accordingly
Generation number is also few, and training result confidence level is relatively low, therefore, can screen out this word, it is made to be not included in each word.
In this case, each word is specifically：At least occurred the part word in primary word in language material.
S204：Establish the term vector of each word.
In this specification embodiment, the term vector established can be the term vector of initialization, need by after training
It can preferably reflect the meaning of a word.
In order to ensure the effect of scheme, when establishing term vector, some restrictive conditions are might have.For example, generally it is not
Different words establish identical term vector；For another example, the element value in term vector cannot generally be all 0；Etc..
In this specification embodiment, there are many modes of establishing term vector, for example, establish solely hot (1-hot) term vector,
Or term vector etc. is established at random.
If in addition, having been based on other language materials before, the corresponding term vector of certain words was trained, then had been further based on
Language material in Fig. 2 trains the term vector of these words, can no longer re-establish the term vector of these words, but based in Fig. 2
Language material and training result before, then be trained.
S206：According to the term vector of the cliction up and down of the term vector of each word and each word in the language material,
Convolutional neural networks are trained.
In this specification embodiment, the convolutional layer of convolutional neural networks is used to extract the information of local neuron, convolution
The pond layer of neural network is used to integrate each local message of convolutional layer and then obtain global information.Specific to the field of this specification
Scape, local message can refer to the whole semantic of partial context word, and the entirety that global information can refer to all cliction up and down is semantic.
S208：According to the convolutional neural networks after the term vector of each word and training, the word of each word is obtained
The training result of vector.
Can be that convolutional neural networks determine rational parameter so that convolutional Neural net by training convolutional neural networks
Network can relatively accurately portray the semanteme of the whole semantic and corresponding current word of upper and lower cliction.The parameter is such as wrapped
Include weight parameter and offset parameter etc..
Term vector is made inferences using the full articulamentum of the convolutional neural networks after training, term vector training can be obtained
As a result.
By the method for Fig. 2, convolutional neural networks can be calculated by convolutional calculation and pondization, whole to the context of word
Semantic information is portrayed, and extracts more context semantic informations, and then can obtain more accurate term vector training knot
Fruit.
Method based on Fig. 2, this specification embodiment additionally provide some specific embodiments and extension of this method
Scheme is illustrated below.
In this specification embodiment, for establishing 1-hot term vectors.For step S204, it is described establish it is described each
The term vector of word can specifically include：
Determine the total quantity of each word (identical word is only counted once)；It is described total that respectively described each word, which establishes dimension,
The term vector of quantity, wherein, the term vector of each word is different, in the term vector there are one element be 1, remaining element
It is 0.
For example, each word is numbered one by one, number since 0, add one successively, it is assumed that the total quantity of each word is Nc, then
The number of the last one word is Nc-1.Respectively each word establishes a dimension as Nc1-hot term vectors, specifically, it is assumed that certain word
Number be 256, for its foundation term vector in the 256th element can be 1, then remaining element be 0.
In this specification embodiment, when being trained to convolutional neural networks, target be so that current word with up and down
After the convolutional neural networks reasoning after training, similarity can relatively heighten the term vector of cliction.
Further, upper and lower cliction is considered as positive sample word, it, can also be current according to certain rule selection as control
One or more negative sample word of word also assists in training, is so conducive to train Fast Convergent and obtains more accurate instruction
Practice result.In this case, the target can also be included so that the term vector of current word and negative sample word is after training
Convolutional neural networks reasoning after, similarity opposite can be lower.Negative sample word can such as randomly choose in language material to be obtained,
Or selection obtains, etc. in cliction non-up and down.This specification does not limit the concrete mode for calculating similarity, than
Such as, can similarity be calculated based on vectorial included angle cosine operation, can similarity be calculated based on vectorial quadratic sum operation, etc.
Deng.
According to the analysis of the preceding paragraph, for step S206, the term vector and each word according to each word exists
The term vector of cliction up and down in the language material, is trained convolutional neural networks.Can specifically it include：
According to cliction up and down in the language material of the term vector of each word and each word and the word of negative sample word
Vector is trained convolutional neural networks.
In this specification embodiment, the training process of convolutional neural networks can be that iteration carries out, fairly simple
A kind of mode is that the language material after participle is traversed, and the word often traversed in above-mentioned each word carries out an iteration, directly
It is finished to traversal, can be considered as and the language material has been utilized to train convolutional neural networks.
Specifically, the cliction up and down according to the term vector and each word of each word in the language material and
The term vector of negative sample word, is trained convolutional neural networks, can include：
The language material after participle is traversed, performs that (it is an iteration to perform content to the current word that traverses
Process)：
Determine one or more of the described language material of current word after participle cliction and negative sample word up and down；It will be current
The convolutional layer of the term vector input convolutional neural networks of the cliction up and down of word carries out convolutional calculation；Convolutional calculation result is inputted into institute
The pond layer for stating convolutional neural networks carries out pondization calculating, obtains primary vector；The term vector of current word is inputted into the convolution
The full articulamentum of neural network is calculated, and obtains secondary vector and the term vector of the negative sample word of current word is inputted institute
The full articulamentum for stating convolutional neural networks is calculated, and obtains third vector；According to the primary vector, the secondary vector,
The third vector and the loss function specified update the parameter of the convolutional neural networks.
More intuitively, it is illustrated with reference to Fig. 3.Fig. 3 is one kind under the practical application scene that this specification embodiment provides
The structure diagram of convolutional neural networks.
The convolutional neural networks of Fig. 3 are mainly including convolutional layer, pond layer, full articulamentum and Softmax layers.In training
During convolutional neural networks, the vector of upper and lower cliction is handled by convolutional layer and pond layer, whole to extract cliction up and down
The word sense information of body, and the term vector of current word and its negative sample word can then be handled by full articulamentum.Separately below in detail
It describes in detail bright.
In this specification embodiment, it is assumed that determine cliction up and down using sliding window, the center of sliding window is time
The current word gone through, other words in sliding window in addition to current word are upper and lower cliction.By the term vector of whole clictions up and down
Convolutional layer is inputted, and then convolutional calculation can be carried out according to equation below：
Wherein, xiRepresent the term vector of about i-th cliction (it is assumed herein that xiIt is column vector), xi:i+θ-1Represent i-th~
The vector that the term vector of about i+ θ -1 clictions splices, yiRepresent vector (the convolution meter obtained by the convolutional calculation
Calculate result) i-th of element, ω represent convolutional layer weight parameter, ζ represent convolutional layer offset parameter, σ represent excitation letter
Number, for example, Sigmoid functions, then
Further, after obtaining convolutional calculation result, pond layer can be inputted and carry out pondization calculating, specifically may be used most
Bigization pondization calculates or average pondization calculates etc..
It is calculated, such as using the following formula according to pondization is maximized：
It is calculated according to average pondization, such as using the following formula：
Wherein, max represents maximizing function, and average represents function of averaging, and c (j) represents that pondization obtains after calculating
J-th of element of the primary vector arrived, t represent the quantity of cliction up and down.
Fig. 3 also schematically illustrates 6 of some current word " liquid ", the current word in certain language material in the language material
A cliction " as ", " the ", " vegan ", " gelatin ", " substitute ", " absorbs " and the current word up and down exist
Two negative sample words " year ", " make " in the language material.Assume that established 1-hot term vectors are N in Fig. 3cDimension, θ=
3, represent the length of convolution window, then the dimension of vector spliced during convolutional calculation is θ Nc=3NcDimension.
For current word, term vector can input full articulamentum, for example be calculated according to the following formula：
Wherein, w represents the secondary vector that full articulamentum exports after handling the term vector of current word,Expression connects entirely
The weight parameter of layer is connect, q represents the term vector of current word, and τ represents the offset parameter of full articulamentum.
Similarly, for each negative sample word, term vector can input full articulamentum respectively, with reference to the mode of current word
It is handled, obtains third vector, the corresponding third vector of m-th of negative sample word is expressed as w'm。
Further, it is described according to the primary vector, the secondary vector, third vector and the damage specified
Function is lost, updates the parameter of the convolutional neural networks, for example can include：Calculate the secondary vector and the primary vector
The first similarity and the third vector and the primary vector the second similarity；According to first similarity, institute
The second similarity and the loss function specified are stated, updates the parameter of the convolutional neural networks.
A kind of loss function is enumerated as example.The loss function such as can be：
Wherein, c represents the primary vector, and w represents the secondary vector, w'mRepresent the corresponding institute of m-th of negative sample word
Third vector is stated, ω represents the weight parameter of convolutional layer, and ζ represents the offset parameter of convolutional layer,Represent the weight ginseng of full articulamentum
Number, τ represent the offset parameter of full articulamentum, and γ represents hyper parameter, and s represents similarity calculation function, and λ represents the number of negative sample word
Amount.
In practical applications, if using negative sample word, then can correspondingly removing calculating the in the loss function used
One vector and the item of the similarity of third vector.
In this specification embodiment, convolutional neural networks training after, term vector can be made inferences, obtain word to
Measure training result.Specifically, for step S208, the term vector according to each word and the convolutional Neural after training
Network obtains the training result of the term vector of each word, can specifically include：
The full articulamentum of the convolutional neural networks term vector of each word inputted respectively after training calculates,
The vector exported after being calculated, as corresponding term vector training result.
Based on same thinking, this specification embodiment provides another term vector processing method, is the word in Fig. 2
A kind of illustrative specific embodiment of vector processing method.Fig. 4 is the flow diagram of the another kind term vector processing method.
Flow in Fig. 4 may comprise steps of：
Step 1, the vocabulary being made up of each word segmented to language material is established, each word is not included in institute's predicate
Occurrence number is less than the word of setting number in material；Jump procedure 2；
Step 2, the total quantity of each word is determined, identical word is only counted once；Jump procedure 3；
Step 3, the different 1-hot term vector that dimension is the quantity is established respectively for each word；Jump procedure 4；
Step 4, the language material after traversal participle performs step 5 to the current word traversed, is performed if completion is traversed
Step 6, otherwise continue to traverse；
Step 5, centered on current word, more k words is respectively slid to both sides and establish window, current word will be removed in window
Word in addition as upper and lower cliction, and will it is all up and down clictions term vector input convolutional neural networks convolutional layer, rolled up
Product calculates, and the pond layer that convolutional calculation result inputs the convolutional neural networks carries out pondization calculating, obtains primary vector；It ought
The full articulamentum that the term vector of preceding word and the negative sample word selected in the language material inputs the convolutional neural networks carries out
It calculates, respectively obtains secondary vector and third vector；It is vectorial according to the primary vector, the secondary vector, the third, with
And the loss function specified, update the parameters of the convolutional neural networks；
The convolutional calculation is carried out according to equation below：
The pondization calculating is carried out according to equation below：
Or
The loss function includes：
Wherein, xiRepresent the term vector of about i-th cliction, xi:i+θ-1It represents the word of about i-th~i+ θ -1 clictions
The vector that vector splicing obtains, yiRepresent i-th of the element of vector obtained by the convolutional calculation, ω represents convolutional layer
Weight parameter, ζ represent the offset parameter of convolutional layer, and σ represents excitation function, and max represents maximizing function, and average is represented
It averages function, c (j) represents j-th of element of the primary vector that pondization obtains after calculating, and t represents upper and lower cliction
Quantity, c represent the primary vector, and w represents the secondary vector, w'mRepresent the corresponding third of m-th of negative sample word to
Amount, ω represent the weight parameter of convolutional layer, and ζ represents the offset parameter of convolutional layer,Represent the weight parameter of full articulamentum, τ is represented
The offset parameter of full articulamentum, γ represent hyper parameter, and s represents similarity calculation function, and λ represents the quantity of negative sample word；
Step 6, by the term vector of each word respectively input training after the convolutional neural networks full articulamentum into
Row calculates, and obtains corresponding term vector training result.
Each step can be performed by identical or different module in the another kind term vector processing method, this specification pair
This is simultaneously not specifically limited.
The term vector processing method provided above for this specification embodiment, based on same thinking, this specification is implemented
Example additionally provides corresponding device, as shown in Figure 5.
Fig. 5 is the structure diagram of a kind of term vector processing unit corresponding to Fig. 2 that this specification embodiment provides, should
Device can be located at the executive agent of flow in Fig. 2, including：
Acquisition module 501 obtains each word segmented to language material；
Module 502 is established, establishes the term vector of each word；
Training module 503, according to the cliction up and down of the term vector of each word and each word in the language material
Term vector is trained convolutional neural networks；
Processing module 504 according to the convolutional neural networks after the term vector of each word and training, obtains described each
The training result of the term vector of word.
Optionally, the term vector established module 502 and establish each word, specifically includes：
The total quantity established module 502 and determine each word；
Respectively described each word establishes the term vector that dimension is the total quantity, wherein, the term vector of each word is mutually not
It is identical, in the term vector there are one element be 1, remaining element be 0.
Optionally, the training module 503 according to the term vector and each word of each word in the language material
The term vector of upper and lower cliction, is trained convolutional neural networks, specifically includes：
Context of the training module 503 according to the term vector and each word of each word in the language material
The term vector of word and negative sample word, is trained convolutional neural networks.
Optionally, the training module 503 according to the term vector and each word of each word in the language material
Upper and lower cliction and the term vector of negative sample word, are trained convolutional neural networks, specifically include：
The training module 503 traverses the language material after participle, and the current word traversed is performed：
Determine one or more of the described language material of current word after participle cliction and negative sample word up and down；
The convolutional layer of the term vector input convolutional neural networks of the cliction up and down of current word is subjected to convolutional calculation；
The pond layer that convolutional calculation result is inputted to the convolutional neural networks carries out pondization calculating, obtains primary vector；
The full articulamentum that the term vector of current word is inputted to the convolutional neural networks calculates, and obtains secondary vector,
And calculate the full articulamentum of the term vector input convolutional neural networks of the negative sample word of current word, obtain third
Vector；
According to the primary vector, the secondary vector, third vector and the loss function specified, institute is updated
State the parameter of convolutional neural networks.
Optionally, the training module 503 carries out convolutional calculation, specifically includes：
The training module 503 carries out convolutional calculation according to equation below：
Wherein, xiRepresent the term vector of about i-th cliction, xi:i+θ-1It represents the word of about i-th~i+ θ -1 clictions
The vector that vector splicing obtains, yiRepresent i-th of the element of vector obtained by the convolutional calculation, ω represents convolutional layer
Weight parameter, ζ represent the offset parameter of convolutional layer, and σ represents excitation function.
Optionally, the training module 503 carries out pondization calculating, specifically includes：
The training module 503 carries out maximizing pondization calculating or average pondization calculates.
Optionally, the training module 503 according to the primary vector, the secondary vector, the third vector and
The loss function specified updates the parameter of the convolutional neural networks, specifically includes：
The training module 503 calculates the first similarity and described of the secondary vector and the primary vector
Second similarity of three vectors and the primary vector；
According to first similarity, second similarity and the loss function specified, the convolutional Neural is updated
The parameter of network.
Optionally, the loss function specifically includes：
Wherein, c represents the primary vector, and w represents the secondary vector, w'mRepresent the corresponding institute of m-th of negative sample word
Third vector is stated, ω represents the weight parameter of convolutional layer, and ζ represents the offset parameter of convolutional layer,Represent the weight ginseng of full articulamentum
Number, τ represent the offset parameter of full articulamentum, and γ represents hyper parameter, and s represents similarity calculation function, and λ represents the number of negative sample word
Amount.
Optionally, term vector of the processing module 504 according to each word and the convolutional neural networks after training,
The training result of the term vector of each word is obtained, is specifically included：
The term vector of each word is inputted the complete of the convolutional neural networks after training by the processing module 504 respectively
Articulamentum is calculated, the vector exported after being calculated, as corresponding term vector training result.
Based on same thinking, this specification embodiment additionally provides a kind of corresponding term vector processing equipment, including：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be performed by least one processor, and described instruction is by described at least one
A processor performs, so that at least one processor can：
Obtain each word segmented to language material；
Establish the term vector of each word；
According to the term vector of the cliction up and down of the term vector of each word and each word in the language material, to volume
Product neural network is trained；
According to the convolutional neural networks after the term vector of each word and training, the term vector of each word is obtained
Training result.
Based on same thinking, this specification embodiment additionally provides a kind of corresponding non-volatile computer storage and is situated between
Matter, is stored with computer executable instructions, and the computer executable instructions are set as：
Obtain each word segmented to language material；
Establish the term vector of each word；
According to the term vector of the cliction up and down of the term vector of each word and each word in the language material, to volume
Product neural network is trained；
According to the convolutional neural networks after the term vector of each word and training, the term vector of each word is obtained
Training result.
It is above-mentioned that this specification specific embodiment is described.Other embodiments are in the scope of the appended claims
It is interior.In some cases, the action recorded in detail in the claims or step can be come according to different from the sequence in embodiment
It performs and still can realize desired result.In addition, the process described in the accompanying drawings not necessarily require show it is specific suitable
Sequence or consecutive order could realize desired result.In some embodiments, multitasking and parallel processing be also can
With or it may be advantageous.
Each embodiment in this specification is described by the way of progressive, identical similar portion between each embodiment
Point just to refer each other, and the highlights of each of the examples are difference from other examples.Especially for device,
For equipment, nonvolatile computer storage media embodiment, since it is substantially similar to embodiment of the method, so the ratio of description
Relatively simple, the relevent part can refer to the partial explaination of embodiments of method.
Device that this specification embodiment provides, equipment, nonvolatile computer storage media with method be it is corresponding, because
This, device, equipment, nonvolatile computer storage media also have the advantageous effects similar with corresponding method, due to upper
Face is described in detail the advantageous effects of method, therefore, which is not described herein again corresponding intrument, equipment, it is non-easily
The advantageous effects of the property lost computer storage media.
In the 1990s, the improvement of a technology can be distinguished clearly be on hardware improvement (for example,
Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).So
And with the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit.
Designer nearly all obtains corresponding hardware circuit by the way that improved method flow is programmed into hardware circuit.Cause
This, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, programmable logic device
(Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable Gate
Array, FPGA)) it is exactly such a integrated circuit, logic function determines device programming by user.By designer
Voluntarily programming a digital display circuit " integrated " on a piece of PLD, designs and make without asking chip maker
Dedicated IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " patrols
Volume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development,
And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language
(Hardware Description Language, HDL), and HDL is also not only a kind of, but there are many kind, such as ABEL
(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description
Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL
(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby
Hardware Description Language) etc., VHDL (Very-High-Speed are most generally used at present
Integrated Circuit Hardware Description Language) and Verilog.Those skilled in the art also should
This understands, it is only necessary to method flow slightly programming in logic and is programmed into integrated circuit with above-mentioned several hardware description languages,
The hardware circuit for realizing the logical method flow can be readily available.
Controller can be implemented in any suitable manner, for example, controller can take such as microprocessor or processing
The computer of computer readable program code (such as software or firmware) that device and storage can be performed by (micro-) processor can
Read medium, logic gate, switch, application-specific integrated circuit (Application Specific Integrated Circuit,
ASIC), the form of programmable logic controller (PLC) and embedded microcontroller, the example of controller include but not limited to following microcontroller
Device：ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, are deposited
Memory controller is also implemented as a part for the control logic of memory.It is also known in the art that in addition to
Pure computer readable program code mode is realized other than controller, can be made completely by the way that method and step is carried out programming in logic
Controller is obtained in the form of logic gate, switch, application-specific integrated circuit, programmable logic controller (PLC) and embedded microcontroller etc. to come in fact
Existing identical function.Therefore this controller is considered a kind of hardware component, and various to being used to implement for including in it
The device of function can also be considered as the structure in hardware component.Or even, the device for being used to implement various functions can be regarded
For either the software module of implementation method can be the structure in hardware component again.
System, device, module or the unit that above-described embodiment illustrates can specifically realize by computer chip or entity,
Or it is realized by having the function of certain product.A kind of typical realization equipment is computer.Specifically, computer for example may be used
Think personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media play
It is any in device, navigation equipment, electronic mail equipment, game console, tablet computer, wearable device or these equipment
The combination of equipment.
For convenience of description, it is divided into various units during description apparatus above with function to describe respectively.Certainly, implementing this
The function of each unit is realized can in the same or multiple software and or hardware during specification.
It should be understood by those skilled in the art that, this specification embodiment can be provided as method, system or computer program
Product.Therefore, this specification embodiment can be used complete hardware embodiment, complete software embodiment or with reference to software and hardware
The form of the embodiment of aspect.Wherein include computer in one or more moreover, this specification embodiment can be used and can be used
It is real in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form for the computer program product applied.
This specification is with reference to the method, equipment (system) and computer program product according to this specification embodiment
Flowchart and/or the block diagram describes.It should be understood that it can be realized by computer program instructions every in flowchart and/or the block diagram
The combination of flow and/or box in one flow and/or box and flowchart and/or the block diagram.These computers can be provided
Program instruction is to the processor of all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices
To generate a machine so that the instruction performed by computer or the processor of other programmable data processing devices generates use
In the dress of function that realization is specified in one flow of flow chart or multiple flows and/or one box of block diagram or multiple boxes
It puts.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that the instruction generation being stored in the computer-readable memory includes referring to
Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or
The function of being specified in multiple boxes.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that counted
Series of operation steps are performed on calculation machine or other programmable devices to generate computer implemented processing, so as in computer or
The instruction offer performed on other programmable devices is used to implement in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include computer-readable medium in volatile memory, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer-readable instruction, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, CD-ROM read-only memory (CD-ROM),
Digital versatile disc (DVD) or other optical storages, magnetic tape cassette, the storage of tape magnetic rigid disk or other magnetic storage apparatus
Or any other non-transmission medium, available for storing the information that can be accessed by a computing device.It defines, calculates according to herein
Machine readable medium does not include temporary computer readable media (transitorymedia), such as data-signal and carrier wave of modulation.
It should also be noted that, term " comprising ", "comprising" or its any other variant are intended to nonexcludability
Comprising so that process, method, commodity or equipment including a series of elements are not only including those elements, but also wrap
Include other elements that are not explicitly listed or further include for this process, method, commodity or equipment it is intrinsic will
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that wanted including described
Also there are other identical elements in the process of element, method, commodity or equipment.
It will be understood by those skilled in the art that this specification embodiment can be provided as method, system or computer program product.
Therefore, the embodiment in terms of complete hardware embodiment, complete software embodiment or combination software and hardware can be used in this specification
Form.Moreover, this specification can be used wherein include the computers of computer usable program code in one or more can
With the computer program product implemented on storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Form.
This specification can be described in the general context of computer executable instructions, such as journey
Sequence module.Usually, program module include routines performing specific tasks or implementing specific abstract data types, program, object,
Component, data structure etc..This specification can also be put into practice in a distributed computing environment, in these distributed computing environment
In, by performing task by communication network and connected remote processing devices.In a distributed computing environment, program module
It can be located in the local and remote computer storage media including storage device.
Each embodiment in this specification is described by the way of progressive, identical similar portion between each embodiment
Point just to refer each other, and the highlights of each of the examples are difference from other examples.Especially for system reality
For applying example, since it is substantially similar to embodiment of the method, so description is fairly simple, related part is referring to embodiment of the method
Part explanation.
The foregoing is merely this specification embodiments, are not limited to the application.For those skilled in the art
For, the application can have various modifications and variations.All any modifications made within spirit herein and principle are equal
Replace, improve etc., it should be included within the scope of claims hereof.
Claims (20)
1. a kind of term vector processing method, including：
Obtain each word segmented to language material；
Establish the term vector of each word；
According to the term vector of the cliction up and down of the term vector of each word and each word in the language material, to convolution god
It is trained through network；
According to the convolutional neural networks after the term vector of each word and training, the training of the term vector of each word is obtained
As a result.
2. the method as described in claim 1, the term vector for establishing each word, specifically include：
Determine the total quantity of each word；
Respectively described each word establishes the term vector that dimension is the total quantity, wherein, the term vector of each word is different,
Element is 1 there are one in the term vector, remaining element is 0.
3. the method as described in claim 1, the term vector and each word according to each word is in the language material
Cliction up and down term vector, convolutional neural networks are trained, are specifically included：
According to the word of cliction up and down in the language material of the term vector of each word and each word and negative sample word to
Amount, is trained convolutional neural networks.
4. method as claimed in claim 3, the term vector and each word according to each word is in the language material
Cliction up and down and negative sample word term vector, convolutional neural networks are trained, are specifically included：
The language material after participle is traversed, the current word traversed is performed：
Determine one or more of the described language material of current word after participle cliction and negative sample word up and down；
The convolutional layer of the term vector input convolutional neural networks of the cliction up and down of current word is subjected to convolutional calculation；
The pond layer that convolutional calculation result is inputted to the convolutional neural networks carries out pondization calculating, obtains primary vector；
The full articulamentum that the term vector of current word is inputted to the convolutional neural networks calculates, obtain secondary vector and
The full articulamentum that the term vector of the negative sample word of current word is inputted to the convolutional neural networks calculates, obtain third to
Amount；
According to the primary vector, the secondary vector, third vector and the loss function specified, the volume is updated
The parameter of product neural network.
5. method as claimed in claim 4, the carry out convolutional calculation, specifically include：
According to equation below, convolutional calculation is carried out：
Wherein, xiRepresent the term vector of about i-th cliction, xi:i+θ-1It represents the term vector of about i-th~i+ θ -1 clictions
Splice obtained vector, yiRepresent i-th of the element of vector obtained by the convolutional calculation, ω represents the weight of convolutional layer
Parameter, ζ represent the offset parameter of convolutional layer, and σ represents excitation function.
6. method as claimed in claim 4, the progress pondization calculates, and specifically includes：
It carries out maximizing pondization calculating or average pondization calculates.
7. method as claimed in claim 4, described vectorial according to the primary vector, the secondary vector, the third, with
And the loss function specified, the parameter of the convolutional neural networks is updated, is specifically included：
Calculate the first similarity of the secondary vector and the primary vector and the third vector and the primary vector
The second similarity；
According to first similarity, second similarity and the loss function specified, the convolutional neural networks are updated
Parameter.
8. method as claimed in claim 4, the loss function specifically includes：
Wherein, c represents the primary vector, and w represents the secondary vector, w'mRepresent m-th of negative sample word corresponding described
Three vectors, ω represent the weight parameter of convolutional layer, and ζ represents the offset parameter of convolutional layer,Represent the weight parameter of full articulamentum, τ
Represent the offset parameter of full articulamentum, γ represents hyper parameter, and s represents similarity calculation function, and λ represents the quantity of negative sample word.
9. the convolutional Neural net after the method as described in claim 1, the term vector according to each word and training
Network obtains the training result of the term vector of each word, specifically includes：
The full articulamentum of the convolutional neural networks term vector of each word inputted respectively after training calculates, and obtains
The vector exported after calculating, as corresponding term vector training result.
10. a kind of term vector processing unit, including：
Acquisition module obtains each word segmented to language material；
Module is established, establishes the term vector of each word；
Training module, according to the term vector of the cliction up and down of the term vector of each word and each word in the language material,
Convolutional neural networks are trained；
Processing module according to the convolutional neural networks after the term vector of each word and training, obtains the word of each word
The training result of vector.
11. device as claimed in claim 10, the term vector established module and establish each word, specifically include：
The total quantity established module and determine each word；
Respectively described each word establishes the term vector that dimension is the total quantity, wherein, the term vector of each word is different,
Element is 1 there are one in the term vector, remaining element is 0.
12. device as claimed in claim 10, the training module exists according to the term vector and each word of each word
The term vector of cliction up and down in the language material, is trained convolutional neural networks, specifically includes：
Up and down cliction and negative sample of the training module according to the term vector and each word of each word in the language material
The term vector of example word, is trained convolutional neural networks.
13. device as claimed in claim 12, the training module exists according to the term vector and each word of each word
The term vector of cliction up and down and negative sample word in the language material, is trained convolutional neural networks, specifically includes：
The training module traverses the language material after participle, and the current word traversed is performed：
Determine one or more of the described language material of current word after participle cliction and negative sample word up and down；
The convolutional layer of the term vector input convolutional neural networks of the cliction up and down of current word is subjected to convolutional calculation；
The pond layer that convolutional calculation result is inputted to the convolutional neural networks carries out pondization calculating, obtains primary vector；
The full articulamentum that the term vector of current word is inputted to the convolutional neural networks calculates, obtain secondary vector and
The full articulamentum that the term vector of the negative sample word of current word is inputted to the convolutional neural networks calculates, obtain third to
Amount；
According to the primary vector, the secondary vector, third vector and the loss function specified, the volume is updated
The parameter of product neural network.
14. device as claimed in claim 13, the training module carries out convolutional calculation, specifically includes：
The training module carries out convolutional calculation according to equation below：
Wherein, xiRepresent the term vector of about i-th cliction, xi:i+θ-1It represents the term vector of about i-th~i+ θ -1 clictions
Splice obtained vector, yiRepresent i-th of the element of vector obtained by the convolutional calculation, ω represents the weight of convolutional layer
Parameter, ζ represent the offset parameter of convolutional layer, and σ represents excitation function.
15. device as claimed in claim 13, the training module carries out pondization and calculates, and specifically includes：
The training module carries out maximizing pondization calculating or average pondization calculates.
16. device as claimed in claim 13, the training module is according to the primary vector, secondary vector, described
Third vector and the loss function specified, update the parameter of the convolutional neural networks, specifically include：
The training module calculate the secondary vector and the primary vector the first similarity and the third vector with
Second similarity of the primary vector；
According to first similarity, second similarity and the loss function specified, the convolutional neural networks are updated
Parameter.
17. device as claimed in claim 13, the loss function specifically includes：
Wherein, c represents the primary vector, and w represents the secondary vector, w'mRepresent m-th of negative sample word corresponding described
Three vectors, ω represent the weight parameter of convolutional layer, and ζ represents the offset parameter of convolutional layer,Represent the weight parameter of full articulamentum, τ
Represent the offset parameter of full articulamentum, γ represents hyper parameter, and s represents similarity calculation function, and λ represents the quantity of negative sample word.
18. device as claimed in claim 10, the processing module is according to after the term vector of each word and training
Convolutional neural networks obtain the training result of the term vector of each word, specifically include：
The term vector of each word is inputted the full articulamentum of the convolutional neural networks after training by the processing module respectively
It is calculated, the vector exported after being calculated, as corresponding term vector training result.
19. a kind of term vector processing method, including：
Step 1, the vocabulary being made up of each word segmented to language material is established, each word is not included in the language material
Occurrence number is less than the word of setting number；Jump procedure 2；
Step 2, the total quantity of each word is determined, identical word is only counted once；Jump procedure 3；
Step 3, the different 1-hot term vector that dimension is the quantity is established respectively for each word；Jump procedure 4；
Step 4, the language material after traversal participle performs step 5 to the current word traversed, step is performed if completion is traversed
6, otherwise continue to traverse；
Step 5, centered on current word, more k words are respectively slid to both sides and establish window, by window in addition to current word
Word as upper and lower cliction, and will it is all up and down clictions term vector input convolutional neural networks convolutional layer carry out convolution meter
It calculates, the pond layer that convolutional calculation result inputs the convolutional neural networks carries out pondization calculating, obtains primary vector；By current word
And the full articulamentum of the term vector input convolutional neural networks of the negative sample word selected in the language material is calculated,
Respectively obtain secondary vector and third vector；According to the primary vector, the secondary vector, third vector and refer to
Fixed loss function updates the parameter of the convolutional neural networks；
The convolutional calculation is carried out according to equation below：
The pondization calculating is carried out according to equation below：
Or
The loss function includes：
Wherein, xiRepresent the term vector of about i-th cliction, xi:i+θ-1It represents the term vector of about i-th~i+ θ -1 clictions
Splice obtained vector, yiRepresent i-th of the element of vector obtained by the convolutional calculation, ω represents the weight of convolutional layer
Parameter, ζ represent the offset parameter of convolutional layer, and σ represents excitation function, and max represents maximizing function, and average expressions ask flat
Mean function, c (j) represent j-th of element of the primary vector that pondization obtains after calculating, and t represents the quantity of cliction up and down,
C represents the primary vector, and w represents the secondary vector, w'mRepresent the corresponding third vector of m-th of negative sample word, ω
Representing the weight parameter of convolutional layer, ζ represents the offset parameter of convolutional layer,Represent the weight parameter of full articulamentum, τ expressions connect entirely
The offset parameter of layer is connect, γ represents hyper parameter, and s represents similarity calculation function, and λ represents the quantity of negative sample word；
Step 6, the full articulamentum of the convolutional neural networks term vector of each word inputted respectively after training is counted
It calculates, obtains corresponding term vector training result.
20. a kind of term vector processing equipment, including：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be performed by least one processor, and described instruction is by least one place
It manages device to perform, so that at least one processor can：
Language material is segmented to obtain each word；
Establish the term vector of each word；
According to the term vector of the cliction up and down of the term vector of each word and each word in the language material, to convolution god
It is trained through network；
According to the convolutional neural networks after the term vector of each word and training, the training of the term vector of each word is obtained
As a result.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711235849.7A CN108170667B (en) | 2017-11-30 | 2017-11-30 | Word vector processing method, device and equipment |
| TW107133778A TWI701588B (en) | 2017-11-30 | 2018-09-26 | Word vector processing method, device and equipment |
| PCT/CN2018/110055 WO2019105134A1 (en) | 2017-11-30 | 2018-10-12 | Word vector processing method, apparatus and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711235849.7A CN108170667B (en) | 2017-11-30 | 2017-11-30 | Word vector processing method, device and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108170667A true CN108170667A (en) | 2018-06-15 |
| CN108170667B CN108170667B (en) | 2020-06-23 |
Family
ID=62524251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711235849.7A Active CN108170667B (en) | 2017-11-30 | 2017-11-30 | Word vector processing method, device and equipment |
Country Status (3)
| Country | Link |
|---|---|
| CN (1) | CN108170667B (en) |
| TW (1) | TWI701588B (en) |
| WO (1) | WO2019105134A1 (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019105134A1 (en) * | 2017-11-30 | 2019-06-06 | 阿里巴巴集团控股有限公司 | Word vector processing method, apparatus and device |
| CN110162770A (en) * | 2018-10-22 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of word extended method, device, equipment and medium |
| CN110502614A (en) * | 2019-08-16 | 2019-11-26 | 阿里巴巴集团控股有限公司 | Text hold-up interception method, device, system and equipment |
| CN112016295A (en) * | 2020-09-04 | 2020-12-01 | 平安科技（深圳）有限公司 | Symptom data processing method and device, computer equipment and storage medium |
| WO2021027218A1 (en) * | 2019-08-12 | 2021-02-18 | 北京国双科技有限公司 | Text classification method and apparatus, and computer readable medium |
| CN114697096A (en) * | 2022-03-23 | 2022-07-01 | 重庆邮电大学 | Intrusion detection method based on space-time characteristics and attention mechanism |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705280A (en) * | 2019-08-23 | 2020-01-17 | 上海市研发公共服务平台管理中心 | Technical contract approval model creation method, device, equipment and storage medium |
| CN110852063B (en) * | 2019-10-30 | 2023-05-05 | 语联网（武汉）信息技术有限公司 | Word vector generation method and device based on bidirectional LSTM neural network |
| CN112988921A (en) * | 2019-12-13 | 2021-06-18 | 北京四维图新科技股份有限公司 | Method and device for identifying map information change |
| CN111241819B (en) * | 2020-01-07 | 2023-03-14 | 北京百度网讯科技有限公司 | Word vector generation method and device and electronic equipment |
| CN111539228B (en) * | 2020-04-29 | 2023-08-08 | 支付宝(杭州)信息技术有限公司 | Vector model training method and device and similarity determining method and device |
| CN111737995B (en) * | 2020-05-29 | 2024-04-05 | 北京百度网讯科技有限公司 | Method, device, equipment and medium for training language model based on multiple word vectors |
| CN111782811A (en) * | 2020-07-03 | 2020-10-16 | 湖南大学 | E-government affair sensitive text detection method based on convolutional neural network and support vector machine |
| CN113961664A (en) * | 2020-07-15 | 2022-01-21 | 上海乐言信息科技有限公司 | Deep learning-based numerical word processing method, system, terminal and medium |
| CN115017915B (en) * | 2022-05-30 | 2023-05-30 | 北京三快在线科技有限公司 | Model training and task execution method and device |
| CN116384515B (en) * | 2023-06-06 | 2023-09-01 | 之江实验室 | Model training method and device, storage medium and electronic equipment |
| CN117522669B (en) * | 2024-01-08 | 2024-03-26 | 之江实验室 | Method, device, medium and equipment for optimizing internal memory of graphic processor |
| CN117573815B (en) * | 2024-01-17 | 2024-04-30 | 之江实验室 | Retrieval enhancement generation method based on vector similarity matching optimization |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105786782A (en) * | 2016-03-25 | 2016-07-20 | 北京搜狗科技发展有限公司 | Word vector training method and device |
| JP2016161968A (en) * | 2015-02-26 | 2016-09-05 | 日本電信電話株式会社 | Word vector learning device, natural language processing device, method, and program |
| CN106295796A (en) * | 2016-07-22 | 2017-01-04 | 浙江大学 | Entity link method based on degree of depth study |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10289957B2 (en) * | 2014-12-30 | 2019-05-14 | Excalibur Ip, Llc | Method and system for entity linking |
| CN108170667B (en) * | 2017-11-30 | 2020-06-23 | 阿里巴巴集团控股有限公司 | Word vector processing method, device and equipment |
-
2017
- 2017-11-30 CN CN201711235849.7A patent/CN108170667B/en active Active
-
2018
- 2018-09-26 TW TW107133778A patent/TWI701588B/en active
- 2018-10-12 WO PCT/CN2018/110055 patent/WO2019105134A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2016161968A (en) * | 2015-02-26 | 2016-09-05 | 日本電信電話株式会社 | Word vector learning device, natural language processing device, method, and program |
| CN105786782A (en) * | 2016-03-25 | 2016-07-20 | 北京搜狗科技发展有限公司 | Word vector training method and device |
| CN106295796A (en) * | 2016-07-22 | 2017-01-04 | 浙江大学 | Entity link method based on degree of depth study |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019105134A1 (en) * | 2017-11-30 | 2019-06-06 | 阿里巴巴集团控股有限公司 | Word vector processing method, apparatus and device |
| CN110162770A (en) * | 2018-10-22 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of word extended method, device, equipment and medium |
| CN110162770B (en) * | 2018-10-22 | 2023-07-21 | 腾讯科技（深圳）有限公司 | Word expansion method, device, equipment and medium |
| WO2021027218A1 (en) * | 2019-08-12 | 2021-02-18 | 北京国双科技有限公司 | Text classification method and apparatus, and computer readable medium |
| CN112395412A (en) * | 2019-08-12 | 2021-02-23 | 北京国双科技有限公司 | Text classification method, device and computer readable medium |
| CN112395412B (en) * | 2019-08-12 | 2024-05-03 | 北京国双科技有限公司 | Text classification method, apparatus and computer readable medium |
| CN110502614A (en) * | 2019-08-16 | 2019-11-26 | 阿里巴巴集团控股有限公司 | Text hold-up interception method, device, system and equipment |
| CN110502614B (en) * | 2019-08-16 | 2023-05-09 | 创新先进技术有限公司 | Text interception method, device, system and equipment |
| CN112016295A (en) * | 2020-09-04 | 2020-12-01 | 平安科技（深圳）有限公司 | Symptom data processing method and device, computer equipment and storage medium |
| CN112016295B (en) * | 2020-09-04 | 2024-02-23 | 平安科技（深圳）有限公司 | Symptom data processing method, symptom data processing device, computer equipment and storage medium |
| CN114697096A (en) * | 2022-03-23 | 2022-07-01 | 重庆邮电大学 | Intrusion detection method based on space-time characteristics and attention mechanism |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108170667B (en) | 2020-06-23 |
| WO2019105134A1 (en) | 2019-06-06 |
| TWI701588B (en) | 2020-08-11 |
| TW201926078A (en) | 2019-07-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108170667A (en) | Term vector processing method, device and equipment | |
| CN110119505A (en) | Term vector generation method, device and equipment | |
| TWI685761B (en) | Word vector processing method and device | |
| CN108734460A (en) | A kind of means of payment recommends method, apparatus and equipment | |
| CN110119507A (en) | Term vector generation method, device and equipment | |
| CN107526667A (en) | A kind of Indexes Abnormality detection method, device and electronic equipment | |
| CN110278175A (en) | Graph structure model training, the recognition methods of rubbish account, device and equipment | |
| CN107368524A (en) | One kind dialogue generation method, device and electronic equipment | |
| CN107506181A (en) | Business processing, data processing method, device and electronic equipment | |
| CN109086961A (en) | A kind of Information Risk monitoring method and device | |
| CN110134668A (en) | Data migration method, device and equipment applied to block chain | |
| CN109034183A (en) | A kind of object detection method, device and equipment | |
| CN110119860A (en) | A kind of rubbish account detection method, device and equipment | |
| CN108021610A (en) | Random walk, random walk method, apparatus and equipment based on distributed system | |
| CN108874765A (en) | Term vector processing method and processing device | |
| CN108346107A (en) | A kind of social content Risk Identification Method, device and equipment | |
| CN107402945A (en) | Word stock generating method and device, short text detection method and device | |
| CN109325508A (en) | The representation of knowledge, machine learning model training, prediction technique, device and electronic equipment | |
| CN110378400A (en) | A kind of model training method and device for image recognition | |
| CN110532295A (en) | A kind of method and device of computer-implemented information processing, information inquiry | |
| CN109948140A (en) | A kind of term vector embedding grammar and device | |
| CN107423269A (en) | Term vector processing method and processing device | |
| CN110119381A (en) | A kind of index updating method, device, equipment and medium | |
| CN108170663A (en) | Term vector processing method, device and equipment based on cluster | |
| CN109492401A (en) | A kind of content vector risk checking method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1255392 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200921 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Patentee after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Patentee before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200921 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Patentee after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Patentee before: Alibaba Group Holding Ltd. |
|
| TR01 | Transfer of patent right |