CN106502983B - The event driven collapse Gibbs sampling method of implicit Di Li Cray model - Google Patents
The event driven collapse Gibbs sampling method of implicit Di Li Cray model Download PDFInfo
- Publication number
- CN106502983B CN106502983B CN201610903756.6A CN201610903756A CN106502983B CN 106502983 B CN106502983 B CN 106502983B CN 201610903756 A CN201610903756 A CN 201610903756A CN 106502983 B CN106502983 B CN 106502983B
- Authority
- CN
- China
- Prior art keywords
- neuron
- layer
- hidden layer
- cynapse
- electric pulse
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
- G06N3/065—Analogue means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Neurology (AREA)
- Feedback Control In General (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The event driven collapse Gibbs sampling method that the present invention provides implicit Di Li Cray model includes: that word each in data set and corresponding document are presented to preset structure neural network one by one, indicates current word and indicates that the visible layer neuron of current document provides electric pulse according to Poisson process with real-time frequency and is transferred to hidden layer along synaptic connections；Each hidden neuron according in the film of change voltage calculate in fact when discharge frequency and with Poisson process provide electric pulse；The feedback periphery inhibition signal that electric pulse mode updates injection hidden neuron is provided according to hidden layer；The relative time difference and the current strength of connection of the cynapse for providing electric pulse according to them from the synaptic connections intensity of a visible layer neuron a to hidden neuron update；Having handled all texts of data set is an iteration, extracts the parameter of implicit Di Li Cray model when the number of iterations reaches preset value from the parameter of neural network.This method space complexity is low, can be used to do low energy consumption calculating on class brain chip in principle.
Description
Technical field
The present invention relates to machine learning and class brain computing technique field more particularly to a kind of things of implicit Di Li Cray model
The collapse Gibbs sampling method of part driving.
Background technique
Implicit topic model can extract wherein hiding semantic structure from complex text set.Use is most wide at present
General implicit topic model is implicit Di Li Cray model (LDA-Latent Dirichlet Allocation).This model
Common training algorithm is collapse gibbs sampler (CGS-Collapsed Gibbs Sampling).Implicit Di Li Cray model
It has been widely used in text analyzing, data visualization, recommender system, information retrieval and network analysis.In recent years, with network
The extensive increase of text data amount has work and shows that extensive implicit Di Li Cray model can promote advertisement and recommendation
The function of system.But currently, extensive implicit Di Li Cray model generally does trained and reasoning, the energy consumption of operation under big cluster
It is very big, so being difficult to be generalized in the rare equipment of the energy, such as mobile phone application.
Class brain calculates the calculation method that low energy consumption is researched and developed by mimic biology neural network, one of important research direction
It is the general low energy consumption processor of manufacture.The prior art provides a TrueNorth that is known as and runs class brain chip,
The energy that consumption is estimated in real-time task is only ten a ten thousandths of conventional central processor.But this type brain chip is not suitable for
The operation of classic computer algorithm is directly run, and is suitble to operation impulsive neural networks dynamics.So current existing classics
Computerized algorithm needs to be realized with impulsive neural networks dynamics again for its principle, could use on class brain chip,
And give full play to the characteristic of chip low energy consumption.
Currently, impulsive neural networks dynamics has been able to achieve the machine learning algorithm of some classics.As the prior art mentions
A kind of neural method of sampling has been supplied, gibbs sampler reasoning can be theoretically done to probability graph.Although by combining collapse Ji cloth
The principle of this sampling, which, which can guarantee, correctly trains implicit Di Li Cray mould with impulsive neural networks dynamics
Type, but the connection number of neuron used in network structure and the total number of word in text have exponent relation, space complexity
It is too high, so it is ineffective to do reasoning to implicit Di Li Cray model with nerve sampling.Complicated network structure in nerve sampling
Derived from its changeless network link parameter.The prior art also with the impulsive neural networks dynamics of adjustable network parameter come
Training mixing multinomial distribution model makes neuron connection can be with self adjustment of environmental change.But implicit Di Li Cray model
It is more complicated than mixing multinomial distribution, it is generalized to this more complicated model and needs to redesign the structure of network.
In consideration of it, how to provide a kind of method of low energy consumption, low spatial complexity, impulsive neural networks can be based on
Dynamics effectively train to implicit Di Li Cray model, and can be used to become the technology for needing to solve at present on class brain chip
Problem.
Summary of the invention
In order to solve the above technical problems, the present invention provides a kind of event driven collapse Ji of implicit Di Li Cray model
The Buss method of sampling can carry out effectively training based on parameter of the impulsive neural networks dynamics to implicit Di Li Cray model,
And it can be used for class brain chip in principle, low energy consumption.
The present invention provides a kind of event driven collapse Gibbs sampling method of implicit Di Li Cray model, comprising:
S1, the neural network for constructing preset structure, the neural network of the preset structure, comprising: two visible layers and one
A hidden layer；Described two visible layers respectively encode word and document according to a hot representation, and the hidden layer is according to one
Hot representation encodes theme, and cynapse is attached to each neuron in hidden layer from each neuron in visible layer；
S2, serially by data set each word and the corresponding document of each word be presented to the nerve net one by one
Network indicates the neuron in the visible layer of current word and indicates current according to the method that visible layer is encoded in step S1
Neuron in the visible layer of document provides electric pulse according to Poisson process with its real-time frequency, and the electric pulse is along synaptic connections
Hidden layer is transferred to from visible layer；
S3, the mode that electric pulse to hidden layer is transmitted according to visible layer in step S2, hidden layer are receiving in electric pulse caudacoria
Voltage is changed, the discharge frequency when neuron in each hidden layer calculates in fact according to voltage in the film after change, and according to this
Real-time discharge frequency provides electric pulse with Poisson process, if expression some tool when some word is presented, in hidden layer
The neuron of body theme provides electric pulse, then it represents that this specific theme has been newly assigned to this word；
S4, the mode that electric pulse is provided according to hidden layer in step S3, update the anti-of the neuron being injected into these hidden layers
It presents periphery and inhibits signal；
S5, the mode that hidden layer in the mode and step S3 of electric pulse provides electric pulse is provided according to visible layer in step S2,
It is sent out from the neuron in a visible layer to the strength of connection of the cynapse of the neuron in a hidden layer according to the two neurons
The strength of connection that the relative time of discharge pulse is poor and the cynapse is current is updated；
S6, S3 are executed parallel to S5, and it is primary for executing above-mentioned steps S2 to S5 and having handled all texts in data set
Iteration, repeat the above steps S2 to S5, until stopping iteration and from the parameter of neural network when the number of iterations reaches preset value
Extract the parameter of implicit Di Li Cray model.
Optionally, from the visible layer that word is encoded be attached to hidden layer weight safeguard three matrixes, respectively from
The visible layer that word is encoded to hidden layer cynapse strength of connection matrix, experience first-moment matrix and experience second moment square
Battle array；
The weight for being attached to hidden layer from the visible layer encoded to document also safeguards such three matrixes, respectively from right
The visible layer that document is encoded to hidden layer cynapse strength of connection matrix, experience first-moment matrix and experience second moment square
Battle array.
Optionally, described one hot representation, comprising:
In the visible layer encoded to word, each neuron indicates a specific word, the nerve of the visible layer
First number is the word number in dictionary；
In the visible layer encoded to document, each neuron indicates a specific document, the nerve of the visible layer
First number is the number of files in collection of document；
In hidden layer, each neuron indicates that a specific theme, the neuron number of the hidden layer are implicit Di Like
The preset number of topics of thunder model.
Optionally, the company of the cynapse of each neuron in neural network from each neuron in visible layer into hidden layer
Knotting strength design, comprising:
Strength of connection from the same visible layer to the cynapse of hidden layer jointly increases an identical parameter preset, described
Parameter preset is normal number.
Optionally, hidden layer is receiving the mode that voltage is changed in electric pulse caudacoria, comprising:
The influence of electric pulse from each cynapse be the auxiliary voltage that one has limit, size constant is superimposed upon it is hidden
In layer and on voltage in the film of the neuron of the synaptic connections, the amplitude of the auxiliary voltage is equal to the strength of connection of the cynapse；
The influence of electric pulse from phase homo-synapse be it is renewable, the influence of the electric pulse from not homo-synapse can mutually add.
Optionally, it is seen that the real-time frequency of each neuron is the reference discharge rate of the neuron multiplied by an overall situation in layer
Trigonometric function tuning signal；
Wherein, it is seen that the reference discharge rate of each neuron is preset constant in excitement in layer, is 0 in tranquillization.
Optionally, the real-time discharge frequency of each neuron is that the reference discharge rate of the neuron is complete multiplied by one in hidden layer
The trigonometric function tuning signal of office；
Wherein, the reference discharge rate of each neuron and voltage in the film of the neuron are exponential relationship in hidden layer.
Optionally, the mode that electric pulse is provided according to hidden layer in step S3, updates the mind being injected into these hidden layers
Feedback periphery through member inhibits signal, comprising:
If there is the neuron in hidden layer provides electric pulse, then signal is inhibited to be increased to preset maximum value, otherwise inhibits letter
Number exponentially decay.
Optionally, described that hidden layer granting electric pulse in the mode and step S3 of electric pulse is provided according to visible layer in step S2
Mode, from the neuron in a visible layer to the strength of connection of the cynapse of the neuron in a hidden layer according to the two mind
The strength of connection that the relative time for providing electric pulse through member is poor and the cynapse is current is updated, comprising:
If the neuron in hidden layer provides electric pulse, the neuron in a visible layer is along prominent between them
Touching is linked with neuron of the voltage influence in this hidden layer, then the strength of connection of this cynapse will increase, otherwise this is prominent
The strength of connection of touching can be reduced；
Wherein, the incrementss of the strength of connection of this cynapse are direction multiplied by size term；Wherein, the direction
The strength of connection current depending on the relative time difference of the two neurons granting electric pulse and the cynapse；The size term by
The experience first moment and experience second moment of the cynapse are estimated that the experience first moment and experience second moment of the cynapse are for estimating
The effective sample volume of corresponding beta distribution, the size term are the inverse for the effective sample volume that estimation obtains；
The reduction amount of the strength of connection of this cynapse is the size term of a unit.
Optionally, the parameter of implicit Di Li Cray model is extracted in the parameter from neural network, comprising:
Theme distribution is extracted into the strength of connection of the cynapse of hidden layer from the visible layer encoded to word, to document
The visible layer encoded extracts theme mixed proportion into the strength of connection of the cynapse of hidden layer；
Estimation is extracted into the experience first order and second order moments of the cynapse of hidden layer from the visible layer encoded to word
Word-theme count matrix, from the visible layer encoded to document into the experience first order and second order moments of the cynapse of hidden layer
Extract document-theme count matrix of estimation.
As shown from the above technical solution, the event driven collapse gibbs sampler of implicit Di Li Cray model of the invention
Method can be to implicit Di Li Cray model by using the thought of impulsive neural networks dynamics and collapse gibbs sampler
Parameter carry out effectively train, guarantee the training effect of implicit Di Li Cray model, and class brain chip can be used in principle,
Low energy consumption；By the present invention in that with can self-control synapse turnover method, reduce the space complexity of network.
Detailed description of the invention
Fig. 1 is that a kind of event driven collapse gibbs for implicit Di Li Cray model that one embodiment of the invention provides is adopted
The flow diagram of quadrat method.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, the technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
It is only a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiment of the present invention, ordinary skill people
Member's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
The event driven collapse gibbs that Fig. 1 shows the implicit Di Li Cray model of one embodiment of the invention offer is adopted
The flow diagram of quadrat method, as shown in Figure 1, the event driven collapse gibbs of the implicit Di Li Cray model of the present embodiment
The method of sampling is as described below.
S1, the neural network for constructing preset structure, the neural network of the preset structure, comprising: two visible layers and one
A hidden layer；Described two visible layers respectively encode word and document according to a hot representation, and the hidden layer is according to one
Hot representation encodes theme, and cynapse is attached to each neuron in hidden layer from each neuron in visible layer.
It should be noted that one scalar of every layer of maintenance of neural network, indicates that this current layer provides the neuron of electric pulse
Subscript, or for sky；This scalar is denoted as in the visible layer encoded to wordIn the visible layer encoded to document
It is denoted asIt is denoted as in the hidden layer of encoding schemesIn addition each visible layer safeguards two scalars, respectively indicating currently has hidden layer
The subscript of the neuron of postsynaptic membrane effect and this postsynaptic membrane effect also want duration, the two scalars are right
The visible layer that word is encoded is denoted as respectivelyWithIt is denoted as respectively in the visible layer encoded to documentWithHidden layer
In addition three vectors are safeguarded, it is electric to the postsynaptic membrane of neuron each in hidden layer to respectively indicate the visible layer encoded to word
Pressure, the visible layer encoded to document are to each nerve in the postsynaptic membrane voltage and hidden layer of neuron each in hidden layer
Voltage in the current film of member.
From the visible layer that word is encoded be attached to hidden layer weight safeguard three matrixes, respectively to word into
The strength of connection matrix of cynapse of the visible layer of capable coding to hidden layerExperience first-moment matrixWith experience second moment
MatrixWherein, l is the row of matrix, and k is matrix column；The power of hidden layer is attached to from the visible layer encoded to document
Weight also safeguards such three matrixes, respectively from the visible layer encoded to document to the strength of connection matrix of the cynapse of hidden layerExperience first-moment matrixWith experience second-order moments matrixWherein, d is the row of matrix, and k is matrix column.
In a particular application, described one hot representation, specifically includes:
In the visible layer encoded to word, each neuron indicates a specific word, the nerve of the visible layer
First number is the word number in dictionary；
In the visible layer encoded to document, each neuron indicates a specific document, the nerve of the visible layer
First number is the number of files in collection of document；
In hidden layer, each neuron indicates that a specific theme, the neuron number of the hidden layer are implicit Di Like
The preset number of topics of thunder model.
In the step S1, each neuron in neural network from each neuron in visible layer into hidden layer
The strength of connection of cynapse designs, and may particularly include:
Strength of connection from the same visible layer to the cynapse of hidden layer jointly increases an identical parameter preset, described
Parameter preset is normal number, can guarantee that all synaptic strengths are positive number in the training process, i.e., to word encoded can
See that layer safeguards a scalar a, the visible layer encoded to document safeguards that scalar a b, a and b are to guarantee all cynapses
The amendment constant introduced for positive number.
S2, serially by data set each word and the corresponding document of each word be presented to the nerve net one by one
Network indicates the neuron in the visible layer of current word and indicates current according to the method that visible layer is encoded in step S1
Neuron in the visible layer of document provides electric pulse according to Poisson process with its real-time frequency, and the electric pulse is along synaptic connections
Hidden layer is transferred to from visible layer.
It is understood that in step s 2, t is presented in each worddispT is shielded after timemaskTime.If when current
Between t belong to the shielding phase, then the neuron in all visible layers does not provide electric pulse；If current time t observes a list
Word document corresponding with the word, the corresponding neuron that this word is encoded and the mind that this document is encoded
Pulse is provided with real-time frequency λ respectively through member, accordinglyOrIt can be updated.
Specifically,OrThe mode that can be updated, may particularly include:
By taking the visible layer encoded to this word as an example, if the layer does not have neuron to provide pulse,IfSoIt remains unchanged；IfSoFor sky；On the contrary, if depositing
Pulse is provided in first of neuron, then Wherein τ is a preset constant.
In a particular application, it is seen that in layer the real-time frequency λ of each neuron be the neuron reference discharge rate multiplied by
One global trigonometric function tuning signal, i.e., if current time t observes word l document d corresponding with the word,
The real-time frequency of the neuron in coding and the visible layer encoded to document d is so carried out to word l are as follows:
λ=λ0Asin(ωt+φ) (1)
Wherein, A, ω, φ are preset constant, it is seen that the reference discharge rate λ of each neuron in layer0It is pre- in excitement
It is 0 in tranquillization if constant.
S3, the mode that electric pulse to hidden layer is transmitted according to visible layer in step S2, hidden layer are receiving in electric pulse caudacoria
Voltage is changed, the discharge frequency when neuron in each hidden layer calculates in fact according to voltage in the film after change, and according to this
Real-time discharge frequency provides electric pulse with Poisson process, if expression some tool when some word is presented, in hidden layer
The neuron of body theme provides electric pulse, then it represents that this specific theme has been newly assigned to this word.
It is understood that in step s3, it can be according to the postsynaptic membrane effect of the visible layer recorded in step S2With
Mode, indicate that each neuron calculates voltage in the film of current time in the hidden layer of themeWith real-time discharge frequencyAnd it updatesIf the neuron for indicating some specific theme in hidden layer provides electricity when some word is presented
Pulse, then it represents that this specific theme has been newly assigned to this word.
In a particular application, in hidden layer each neuron real-time discharge frequency be the neuron reference discharge rate multiplied by
One global trigonometric function tuning signal, i.e., the real-time frequency of the neuron in k-th of hidden layer theme encodedAre as follows:
Wherein, the reference discharge rate of each neuron and voltage in the film of the neuron in hidden layerFor exponential relationship,
A, ω, φ are preset constant.
In the step S3, hidden layer is receiving the mode that voltage is changed in electric pulse caudacoria, may particularly include:
The influence of electric pulse from each cynapse be the auxiliary voltage that one has limit, size constant is superimposed upon it is hidden
In layer and on voltage in the film of the neuron of the synaptic connections, the amplitude of the auxiliary voltage is equal to the strength of connection of the cynapse；
The influence of electric pulse from phase homo-synapse be it is renewable, the influence of the electric pulse from not homo-synapse can mutually add；I.e.
Voltage in the film of k-th of hidden neuronAre as follows:
Wherein, I is the feedback periphery inhibition signal for updating the neuron being injected into these hidden layers.
S4, the mode that electric pulse is provided according to hidden layer in step S3, update the anti-of the neuron being injected into these hidden layers
It presents periphery and inhibits signal I.
In a particular application, the step S4, may particularly include:
If there is the neuron in hidden layer provides electric pulse, then signal is inhibited to be increased to preset maximum value I=Ainh, otherwise
Signal is inhibited exponentially to decay:
I←I+(μihn-I)/τinh (4)
Wherein, AinhIt is constant to inhibit signal preset maximum value；μihnAnd τinhIt is all preset constant.
S5, the mode that hidden layer in the mode and step S3 of electric pulse provides electric pulse is provided according to visible layer in step S2,
It is sent out from the neuron in a visible layer to the strength of connection of the cynapse of the neuron in a hidden layer according to the two neurons
The strength of connection that the relative time of discharge pulse is poor and the cynapse is current is updated.
It is understood that in the step S5, whenWhen for non-empty, the matrix of the strength of connection of all cynapsesWith experience first momentAnd experience second momentIt will be updated.
In a particular application, the step S5, may particularly include:
If the neuron in hidden layer provides electric pulse, the neuron in a visible layer is along prominent between them
Touching is linked with neuron of the voltage influence in this hidden layer, then the strength of connection of this cynapse will increase, otherwise this is prominent
The strength of connection of touching can be reduced；
Wherein, the incrementss of the strength of connection of this cynapse are direction multiplied by size term；Wherein, the direction
The strength of connection current depending on the relative time difference of the two neurons granting electric pulse and the cynapse；The size term by
The experience first moment and experience second moment of the cynapse are estimated that the experience first moment and experience second moment of the cynapse are for estimating
The effective sample volume of corresponding beta distribution, the size term are the inverse for the effective sample volume that estimation obtains；
The reduction amount of the strength of connection of this cynapse is the size term of a unit, the calculating of the size term and above-mentioned increase
Calculating in amount is consistent.
Specifically, the update method of the strength of connection of cynapse is:
When there is V word in dictionary, for l=1 ..., V, ifSo
Otherwise
Wherein:
When a shared K theme, for k=1 ..., K, ifSo
Otherwise
Wherein:
Specifically, the update method of experience first-moment matrix and experience second-order moments matrix is:
When there is V word in dictionary, for l=1 ..., V:
When a shared K theme, for k=1 ..., K:
S6, S3 are executed parallel to S5, and it is primary for executing above-mentioned steps S2 to S5 and having handled all texts in data set
Iteration, repeat the above steps S2 to S5, until stopping iteration and from the parameter of neural network when the number of iterations reaches preset value
Extract the parameter of implicit Di Li Cray model.
In the step S6, the parameter of implicit Di Li Cray model is extracted from the parameter of neural network, can specifically be wrapped
Include the step P1 and P2 for not including in figure:
P1, theme distribution is extracted into the strength of connection of the cynapse of hidden layer from the visible layer encoded to wordFrom
The visible layer encoded to document extracts theme mixed proportion into the strength of connection of the cynapse of hidden layerWherein,It indicates
Specific gravity of first of word in k-th of theme,Indicate specific gravity of k-th of theme in d-th of document；
Wherein:
It is extracted in step P1WithIt should will also extractWithNormalization.
P2, estimation is extracted into the experience first order and second order moments of the cynapse of hidden layer from the visible layer encoded to word
Word-theme count matrix c·,k, from the visible layer encoded to document to the experience first moment and second order of the cynapse of hidden layer
Document-theme count matrix c of estimation is extracted in square·,d。
Wherein, c·,kIndicate the total words for distributing to theme k estimated, c·,dIndicate the word of the document d estimated
Sum.
NIPS number in the machine learning databases using the present embodiment the method analysis University of California at Irvine
When according to collection, in selection parameter tdisp=tmask=τ=10, A=1, ω=100 π, φ=0, a=logV, b=logK, λ0=
1000, Ainh=100, μinh=6, τinhAfter=4, the network puzzlement degree change curve in test data set in the training process
It is close with classical collapse Gibbs sampling method.In the case where taking 200 themes after 400 wheel iteration, our side
The puzzlement degree of method respectively reaches 1969 and 1587.Close to the result 1905 and 1503 of classical way.Our method is average each
Word consumes about 10 electric pulses.
The event driven collapse Gibbs sampling method of the implicit Di Li Cray model of the present embodiment uses pulse nerve
Random chance reasoning is done in random pulses granting and postsynaptic membrane voltage effects in network dynamics；Based on event driven cynapse
Update method records the parameter in random chance reasoning formula；All neurons and cynapse concurrently update the state of itself；Base
The effect of implicit Di Li Cray model is finally trained in the training ideological guarantee of collapse gibbs sampler；The training result energy of network
It is compared with the training result based on classical collapse Gibbs sampling method.It is demonstrated experimentally that this method can reach above
Purpose.
The event driven collapse Gibbs sampling method of the implicit Di Li Cray model of the present embodiment, can pass through processor
It realizes, it, can be to implicit Di Li Cray model by using the thought of impulsive neural networks dynamics and collapse gibbs sampler
Parameter carry out effectively train, guarantee the training effect of implicit Di Li Cray model, and class brain chip can be used for, low energy consumption；
By the present invention in that with can the synapse turnover method of self-control solved previous while not losing reasoning precision substantially
Synaptic connections number exponentially space complexity problem, reduces the space complexity of network in similar network.
It should be understood by those skilled in the art that, embodiments herein can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the application, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including the element.Term " on ", "lower" etc. refer to
The orientation or positional relationship shown is to be based on the orientation or positional relationship shown in the drawings, and is merely for convenience of the description present invention and simplifies
Description, rather than the device or element of indication or suggestion meaning must have a particular orientation, constructed and grasped with specific orientation
Make, therefore is not considered as limiting the invention.Unless otherwise clearly defined and limited, term " installation ", " connected ",
" connection " shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected；It can be
Mechanical connection, is also possible to be electrically connected；It can be directly connected, two can also be can be indirectly connected through an intermediary
Connection inside element.For the ordinary skill in the art, above-mentioned term can be understood at this as the case may be
Concrete meaning in invention.
In specification of the invention, numerous specific details are set forth.Although it is understood that the embodiment of the present invention can
To practice without these specific details.In some instances, well known method, structure and skill is not been shown in detail
Art, so as not to obscure the understanding of this specification.Similarly, it should be understood that disclose in order to simplify the present invention and helps to understand respectively
One or more of a inventive aspect, in the above description of the exemplary embodiment of the present invention, each spy of the invention
Sign is grouped together into a single embodiment, figure, or description thereof sometimes.However, should not be by the method solution of the disclosure
Release is in reflect an intention that i.e. the claimed invention requires more than feature expressly recited in each claim
More features.More precisely, as the following claims reflect, inventive aspect is less than single reality disclosed above
Apply all features of example.Therefore, it then follows thus claims of specific embodiment are expressly incorporated in the specific embodiment,
It is wherein each that the claims themselves are regarded as separate embodiments of the invention.It should be noted that in the absence of conflict, this
The feature in embodiment and embodiment in application can be combined with each other.The invention is not limited to any single aspect,
It is not limited to any single embodiment, is also not limited to any combination and/or displacement of these aspects and/or embodiment.And
And can be used alone each aspect and/or embodiment of the invention or with other one or more aspects and/or its implementation
Example is used in combination.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme should all cover within the scope of the claims and the description of the invention.
Claims (9)
1. a kind of event driven collapse Gibbs sampling method of implicit Di Li Cray model characterized by comprising
S1, construct preset structure neural network, the neural network of the preset structure, comprising: two visible layers and one it is hidden
Layer；Described two visible layers respectively encode word and document according to a hot representation, and the hidden layer is according to a hotlist
Show that mode encodes theme, cynapse is attached to each neuron in hidden layer from each neuron in visible layer；
S2, serially by data set each word and the corresponding document of each word be presented to the neural network one by one,
According to the method that visible layer is encoded in step S1, indicates the neuron in the visible layer of current word and indicate current document
Visible layer in neuron electric pulse is provided according to Poisson process with its real-time frequency, the electric pulse is along synaptic connections from can
See that layer is transferred to hidden layer；
S3, the mode that electric pulse to hidden layer is transmitted according to visible layer in step S2, hidden layer are receiving voltage in electric pulse caudacoria
It is changed, the discharge frequency when neuron in each hidden layer calculates in fact according to voltage in the film after change, and in real time according to this
Discharge frequency provides electric pulse with Poisson process, if some is specific main for expression in hidden layer when some word is presented
The neuron of topic provides electric pulse, then it represents that this specific theme has been newly assigned to this word；
S4, the mode that electric pulse is provided according to hidden layer in step S3 update the feedback week for the neuron being injected into these hidden layers
Side inhibits signal；
S5, the mode that hidden layer in the mode and step S3 of electric pulse provides electric pulse is provided according to visible layer in step S2, from one
The strength of connection of cynapse of neuron in a visible layer to the neuron in a hidden layer provides electricity according to the two neurons
The strength of connection that the relative time of pulse is poor and the cynapse is current is updated；
S6, S3 are executed parallel to S5, and executing above-mentioned steps S2 to S5 and having handled all texts in data set is an iteration,
Repeat the above steps S2 to S5, until stopping iteration when the number of iterations reaches preset value and extracting from the parameter of neural network
The parameter of implicit Di Li Cray model；
Wherein, described one hot representation, comprising:
In the visible layer encoded to word, each neuron indicates a specific word, the neuron number of the visible layer
Mesh is the word number in dictionary；
In the visible layer encoded to document, each neuron indicates a specific document, the neuron number of the visible layer
Mesh is the number of files in collection of document；
In hidden layer, each neuron indicates that a specific theme, the neuron number of the hidden layer are implicit Di Li Cray mould
The preset number of topics of type.
2. the method according to claim 1, wherein being attached to hidden layer from the visible layer encoded to word
Weight safeguards three matrixes, respectively from the visible layer encoded to word to the strength of connection matrix of the cynapse of hidden layer, warp
Test first-moment matrix and experience second-order moments matrix；
The weight for being attached to hidden layer from the visible layer encoded to document also safeguards such three matrixes, respectively to document
The visible layer encoded to hidden layer cynapse strength of connection matrix, experience first-moment matrix and experience second-order moments matrix.
3. the method according to claim 1, wherein from each neuron in visible layer to hidden in neural network
The strength of connection design of the cynapse of each neuron in layer, comprising:
Strength of connection from the same visible layer to the cynapse of hidden layer jointly increases an identical parameter preset, described default
Parameter is normal number.
4. the method according to claim 1, wherein hidden layer is receiving what voltage in electric pulse caudacoria was changed
Mode, comprising:
The influence of electric pulse from each cynapse is that the auxiliary voltage that one has limit, size constant is superimposed upon in hidden layer
With in the film of the neuron of the synaptic connections on voltage, the amplitude of the auxiliary voltage is equal to the strength of connection of the cynapse；It comes from
The influence of the electric pulse of phase homo-synapse be it is renewable, the influence of the electric pulse from not homo-synapse can mutually add.
5. the method according to claim 1, wherein the real-time frequency of each neuron is the nerve in visible layer
The reference discharge rate of member is multiplied by a global trigonometric function tuning signal；
Wherein, it is seen that the reference discharge rate of each neuron is preset constant in excitation time in layer, is 0 in tranquillization.
6. the method according to claim 1, wherein the real-time discharge frequency of each neuron is the mind in hidden layer
Reference discharge rate through member is multiplied by a global trigonometric function tuning signal；
Wherein, the reference discharge rate of each neuron and voltage in the film of the neuron are exponential relationship in hidden layer.
7. the method according to claim 1, wherein the side for providing electric pulse according to hidden layer in step S3
Formula, the feedback periphery for updating the neuron being injected into these hidden layers inhibit signal, comprising:
If there is the neuron in hidden layer provides electric pulse, then signal is inhibited to be increased to preset maximum value, otherwise inhibition signal is in
Exponential damping.
8. the method according to claim 1, wherein the side for providing electric pulse according to visible layer in step S2
Hidden layer provides the mode of electric pulse in formula and step S3, from the neuron in a visible layer to the neuron in a hidden layer
The strength of connection of the cynapse strength of connection current according to the relative time difference and the cynapse that the two neurons provide electric pulse
It is updated, comprising:
If the neuron in hidden layer provides electric pulse, the neuron in a visible layer connects along the cynapse between them
Neuron of the voltage influence in this hidden layer is had, then the strength of connection of this cynapse will increase, otherwise this cynapse
Strength of connection can be reduced；
Wherein, the incrementss of the strength of connection of this cynapse are direction multiplied by size term；Wherein, the direction is depended on
In the current strength of connection of relative time difference and the cynapse that the two neurons provide electric pulse；The size term is dashed forward by this
The experience first moment and experience second moment of touching are estimated that the experience first moment and experience second moment of the cynapse are corresponding for estimating
Beta distribution effective sample volume, the size term be the inverse of effective sample volume for estimating to obtain；
The reduction amount of the strength of connection of this cynapse is the size term of a unit.
9. according to the method described in claim 2, it is characterized in that, extracting implicit Di Like in the parameter from neural network
The parameter of thunder model, comprising:
Theme distribution is extracted into the strength of connection of the cynapse of hidden layer from the visible layer encoded to word, is carried out to document
The visible layer of coding extracts theme mixed proportion into the strength of connection of the cynapse of hidden layer；
The word-of estimation is extracted into the experience first order and second order moments of the cynapse of hidden layer from the visible layer encoded to word
Theme count matrix extracts into the experience first order and second order moments of the cynapse of hidden layer from the visible layer encoded to document and estimates
The document of meter-theme count matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610903756.6A CN106502983B (en) | 2016-10-17 | 2016-10-17 | The event driven collapse Gibbs sampling method of implicit Di Li Cray model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610903756.6A CN106502983B (en) | 2016-10-17 | 2016-10-17 | The event driven collapse Gibbs sampling method of implicit Di Li Cray model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106502983A CN106502983A (en) | 2017-03-15 |
| CN106502983B true CN106502983B (en) | 2019-05-10 |
Family
ID=58295074
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610903756.6A Active CN106502983B (en) | 2016-10-17 | 2016-10-17 | The event driven collapse Gibbs sampling method of implicit Di Li Cray model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106502983B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113449864B (en) * | 2021-07-21 | 2023-08-25 | 北京大学 | Feedback type impulse neural network model training method for image data classification |
| CN114328798B (en) * | 2021-11-09 | 2024-02-23 | 腾讯科技（深圳）有限公司 | Processing method, device, equipment, storage medium and program product for searching text |
| CN114548046B (en) * | 2022-04-25 | 2022-09-27 | 阿里巴巴达摩院(杭州)科技有限公司 | Text processing method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101710333A (en) * | 2009-11-26 | 2010-05-19 | 西北工业大学 | Network text segmenting method based on genetic algorithm |
| CN102591917A (en) * | 2011-12-16 | 2012-07-18 | 华为技术有限公司 | Data processing method and system and related device |
| CN104391902A (en) * | 2014-11-12 | 2015-03-04 | 清华大学 | Maximum entropy topic model-based online document classification method and device |
| CN105701210A (en) * | 2016-01-13 | 2016-06-22 | 福建师范大学 | Microblog theme emotion analysis method based on mixed characteristic calculation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8825648B2 (en) * | 2010-04-15 | 2014-09-02 | Microsoft Corporation | Mining multilingual topics |
| US9767416B2 (en) * | 2015-02-04 | 2017-09-19 | Oracle International Corporation | Sparse and data-parallel inference method and system for the latent Dirichlet allocation model |
-
2016
- 2016-10-17 CN CN201610903756.6A patent/CN106502983B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101710333A (en) * | 2009-11-26 | 2010-05-19 | 西北工业大学 | Network text segmenting method based on genetic algorithm |
| CN102591917A (en) * | 2011-12-16 | 2012-07-18 | 华为技术有限公司 | Data processing method and system and related device |
| CN104391902A (en) * | 2014-11-12 | 2015-03-04 | 清华大学 | Maximum entropy topic model-based online document classification method and device |
| CN105701210A (en) * | 2016-01-13 | 2016-06-22 | 福建师范大学 | Microblog theme emotion analysis method based on mixed characteristic calculation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106502983A (en) | 2017-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jakus et al. | Optimal reconfiguration of distribution networks using hybrid heuristic-genetic algorithm | |
| Nusair et al. | Optimal power flow management system for a power network with stochastic renewable energy resources using golden ratio optimization method | |
| Cai et al. | Bat algorithm with Gaussian walk | |
| Guo et al. | Differential evolution improved with self-adaptive control parameters based on simulated annealing | |
| Liu et al. | Human behavior-based particle swarm optimization | |
| Kang et al. | A hybrid approach for power system security enhancement via optimal installation of flexible AC transmission system (FACTS) devices | |
| CN106502983B (en) | The event driven collapse Gibbs sampling method of implicit Di Li Cray model | |
| CN105184368A (en) | Distributed extreme learning machine optimization integrated framework system and method | |
| Zhang et al. | YinYang bipolar dynamic organizational modeling for equilibrium-based decision analysis: Logical transformation of an indigenous philosophy to a global science | |
| Li et al. | Online static security assessment of power systems based on lasso algorithm | |
| CN106980650A (en) | A kind of emotion enhancing word insertion learning method towards Twitter opinion classifications | |
| Han et al. | Reinforcement-learning-based vibration control for a vehicle semi-active suspension system via the PPO approach | |
| Bi et al. | Bifurcation mechanism of bursting oscillations in parametrically excited dynamical system | |
| Fan et al. | Multi-objective stochastic optimal operation of a grid-connected microgrid considering an energy storage system | |
| Diao et al. | Optimal flood-control operation of cascade reservoirs using an improved particle swarm optimization algorithm | |
| Choi et al. | Optimal charging strategy based on model predictive control in electric vehicle parking lots considering voltage stability | |
| Sulaiman et al. | Artificial Intelligence-Based Secured Power Grid Protocol for Smart City | |
| Abdelsalam et al. | A metaheuristic Harris hawk optimization approach for coordinated control of energy management in distributed generation based Microgrids | |
| Huang et al. | A cuckoo search algorithm with elite opposition-based strategy | |
| Sun | Sports performance prediction based on chaos theory and machine learning | |
| Xie et al. | Transmission Expansion Planning Considering Wind Power and Load Uncertainties | |
| Rosales Muñoz et al. | Application of the Multiverse Optimization Method to Solve the Optimal Power Flow Problem in Alternating Current Networks | |
| Someetheram et al. | Random maximum 2 satisfiability logic in discrete hopfield neural network incorporating improved election algorithm | |
| Fontanari | Social interaction as a heuristic for combinatorial optimization problems | |
| Nguyen et al. | A novel algorithm for optimal operation of hydrothermal power systems under considering the constraints in transmission networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210528 Address after: 100084 a1901, 19th floor, building 8, yard 1, Zhongguancun East Road, Haidian District, Beijing Patentee after: Beijing Ruili Wisdom Technology Co.,Ltd. Address before: 100084 mailbox, 100084-82 Tsinghua Yuan, Beijing, Haidian District, Beijing Patentee before: TSINGHUA University |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20170315 Assignee: Beijing Intellectual Property Management Co.,Ltd. Assignor: Beijing Ruili Wisdom Technology Co.,Ltd. Contract record no.: X2023110000073 Denomination of invention: Event driven Collapsing Gibbs sampling Sampling Method for Implicit Dirichlet Model Granted publication date: 20190510 License type: Common License Record date: 20230531 |