CN101833691A - Realizing method of least square support vector machine serial structure based on EPGA (Filed Programmable Gate Array) - Google Patents

Abstract

The invention discloses a realizing method of a least square support vector machine serial structure based on an EPGA (Filed Programmable Gate Array), comprising the following steps of: constructing a topological structure of a LSSVM (Least Square Support Vector Machine) classified neural network according to number of samples; selecting a radial basis function; discretizing a dynamic equation, and setting the value of step length deltaT; selecting the number of bits encoded by binary and having complementary codes, wherein the number of bits consists of number of integer bits and number of decimal bits; constructing a basic database which comprises an arithmetic element unit, a storage unit and a control unit; constructing a neuro neuron unit of the LSSVM classified neural network, constructing a Bias_b neuron unit of the LSSVM classified neural network, and calculating the value of b after solving the value of the last a; constructing a control module for the whole network, then making the network perform recursive operation, and solving a convergence parameter (a and b) when the network is stable. The invention not only reduces the consumption of FPGA resource, but also overcomes the shortcoming of poor flexibility of an analogue circuit, thereby preferably accommodating varying of application environment.

Description

A kind of least square method supporting vector machine serial structure implementation method based on FPGA

Technical field

The invention belongs to mode identification technology, relate to a kind of least square method supporting vector machine serial structure implementation method based on FPGA.

Background technology

Support vector machine (Support Vector Machines, SVM) be a kind of machine Learning Theory based on structural risk minimization, it combines the theory of aspects such as statistical learning, machine learning and neural network, and be proved to be when minimizing structure risk, can improve the popularization ability of algorithm effectively.Research to support vector machine at present mainly concentrates on theoretical research and optimization Algorithm aspect.By comparison, its applied research is then relative less with the research that algorithm is realized, has only comparatively limited experimental study report at present.Most general-purpose computers software of these algorithms is realized simultaneously, and is not suitable for the realization of analog hardware, and this has obviously limited SVM application in practice greatly.

The method that at present existing neural network combines with support vector machine, and realized classified calculating to a small amount of training sample by building mimic channel, but the mimic channel of realizing the support vector machine learning neural network is very big to the dependence of resistance value, and different classified instance can cause different neural network structures and simulation circuit structure.The development of FPGA technology is very fast in recent years, FPGA itself has restructural, it is low to consume, adaptability strong and advantage such as simplicity of design, have the support vector machine neural network implementation method of the parallel organization under the FPGA environment at present, the method has overcome the dependence to hardware under the mimic channel environment, improved the practicality of SVM, but owing to adopted parallel organization, cause the waste of FPGA resource, be difficult to realize for the training of extensive sample.(English spelling is Least Square Support Vector Machines to least square method supporting vector machine, hereinafter to be referred as being abbreviated as LSSVM), therefore proposing a kind of method based on least square method supporting vector machine, both can reach requirement and reduce the utilization of resources, is very necessary.

Summary of the invention

The objective of the invention is, a kind of least square method supporting vector machine serial structure implementation method based on FPGA is provided,, improve hard-wired dirigibility so that reduce consumption to the FPGA resource.

The technical solution used in the present invention is, a kind of least square method supporting vector machine serial structure implementation method based on FPGA, and this method is implemented according to the following steps:

Step 1, according to the topological structure of sample size structure LSSVM Classification Neural:

Topological structure according to given sample size structure LSSVM classification learning recurrent neural network is: will

1 ,-γ ^-1α _jAnd-by _iInsert in the ∑ and sue for peace, the output terminal of ∑ inserts integrator ∫ 1, and the output of integrator ∫ 1 is α _j, simultaneously will

Insert integrator ∫ 2, output b value, α _jAgain through weights-q _IjFeed back in the corresponding ∑, form a recurrent neural network;

Given classification based training collection (z _i, y _i) be one group of sample to be classified, i=1,2 ... N, for all z _i∈ R ^NY is all arranged _i∈ (+1 ,-1) is the corresponding classification of sample, and its categorised decision face is Wherein W is a weight vector, and b is a threshold value,

The Nonlinear Mapping of expression training sample from the input space to the feature space, the classification learning of LSSVM promptly solves following affined optimization problem:

$\underset{W,e}{\min }J(W,e)=\frac{1}{2}{W}^{T}W+\mathrm{\γ}\underset{i=1}{\overset{l}{\mathrm{\Σ}}}{e}_i^2---\left(1\right)$

Constraint condition:

i＝1，...l (2)

Find the solution this problem and introduce the Lagrange function:

α wherein _iBe the Lagrange multiplier, utilize the KKT condition respectively each variable of Lagrange function to be asked local derviation, obtain the optimal conditions of this problem:

$1-b{y}_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j{q}_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i=0---\left(4\right)$

$\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_i{y}_i=0---\left(5\right)$

Q wherein _Ij=y _iy _jK _Ij, and

Be defined as kernel function,

Obtain the dynamic equation of LSSVM classification learning neural network model:

$b=\frac{\∂J}{\∂b}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_i{y}_i---\left(6\right)$

${\mathrm{\α}}_i=-\frac{\∂J}{\∂{\mathrm{\α}}_i}=1-b{y}_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j{q}_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i---\left(7\right)$

α wherein _iBe the Lagrange multiplier, b is a threshold value, (z _i, y _i) be one group of sample to be classified, i=1,2 ... N;

Step 2, select gaussian kernel function for use, select parameter γ ^-1=1, σ=1, and calculate:

The result of step 3, integrating step 2 carries out discretize with dynamic equation (6), (7) and handles, and sets step delta T value;

Equation (6), (7) are carried out obtaining Discrete Dynamic equation (8), (9) after discretize is handled

$b(t+\mathrm{\ΔT})=\mathrm{\ΔT}\underset{i=1}{\overset{T}{\mathrm{\Σ}}}{\mathrm{\α}}_i\left(t\right)y_i\left(t\right)+b\left(t\right)---\left(8\right)$

${\mathrm{\α}}_i(t+\mathrm{\ΔT})=\mathrm{\ΔT}(1-b\left(t\right)y_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j\left(t\right)q_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i\left(t\right))+{\mathrm{\α}}_i\left(t\right);---\left(19\right)$

The digits of binary coding of step 4, setting band complement code is comprising the figure place of integer-bit and the figure place of decimal place;

Step 5, the basic component library of digits of binary coding structure that sets according to step 4 comprise arithmetic element, storage unit and control module;

Arithmetic element comprises multiply accumulating unit MAC, multiplier unit Mul, subtractor unit Sub, accumulator element ADD;

Storage unit comprises ROM and RAM;

Control module calls IP kernel among the ISE9.1, realizes MAC, Mul, Sub, ADD and ROM;

Step 6, the component library that utilizes step 5 to obtain make up the neuro neuron elements;

Step 7: the component library that utilizes step 5 to obtain makes up the Bisa_b neuron elements;

Step 8: the structure of finishing the serial computing functional module:

Make each neuron be controlled by a clock control signal, after calculating finishes in each neuron one-period, produce an effective control signal, when all neuronic control signals that receive when the ANN (Artificial Neural Network) Control unit are effective, producing a total effective control signal makes whole neurons enter the computing of following one-period, when network carries out recursive operation when stablizing, promptly try to achieve the convergence parameter alpha, b.

The inventive method has realized the function of neural network least square method supporting vector machine with the form of serial structure, compare with the parallel organization of prior art, not only reduced consumption, also improved the dirigibility of circuit, made the application of LSSVM more extensive the FPGA resource.

Description of drawings

Fig. 1 is existing LSSVM classification learning neural network topology structure figure;

Fig. 2 is a kind of serial neural network example structure figure that realizes the least square method supporting vector machine function in the inventive method;

Fig. 3 is the neural network LSSVM serial algorithm process flow diagram in the inventive method;

Fig. 4 is the binary coding synoptic diagram of existing 16 band complement code;

Fig. 5 is the flow graph of coded signal between arithmetic element in the inventive method;

Fig. 6 is a processing procedure synoptic diagram in the existing coded data training;

Fig. 7 is the neural network LSSVM realization flow figure in the inventive method;

Fig. 8 is the neuro neuron FPGA implementation structure figure of the neural network LSSVM in the inventive method;

Fig. 9 is the bias_b neuron FPGA implementation structure figure of the neural network LSSVM in the inventive method;

The value of each parameter that the FPGA implementation method of Figure 10 existing neural network LSSVM parallel computation structure for present embodiment adopts obtains.

Curve among Figure 11 is by the convergence parameter alpha among Figure 10 the classification lineoid that b tries to achieve, the wherein positive class sample of " o " representative, the negative class sample of " * " representative.

The value of each parameter that Figure 12 obtains for the FPGA implementation method of the neural network LSSVM serial computing of the embodiment of the invention.

Curve among Figure 13 is by the convergence parameter alpha among Figure 12 the classification lineoid that b tries to achieve, the wherein positive class sample of " o " representative; The negative class sample of " * " representative.

Embodiment

The present invention is described in detail below in conjunction with the drawings and specific embodiments.

The present invention is based on the least square method supporting vector machine serial structure implementation method of FPGA, specifically implements according to following steps:

Step 1, according to the topological structure of sample size structure LSSVM Classification Neural:

Given classification based training collection (z _i, y _i) be one group of sample to be classified, i=1,2 ... N, for all z _i∈ R ^NY is all arranged _i∈ (+1 ,-1), its categorised decision face can be described as

Wherein W is a weight vector, and b is a threshold value,

Be used for representing the Nonlinear Mapping of training sample from the input space to the feature space.The classification learning of LSSVM promptly solves following affined optimization problem:

Constraint condition is:

I=1 ... l (2)

Find the solution this problem and can introduce the Lagrange function:

α wherein _iBe the Lagrange multiplier, utilize the KKT condition to ask local derviation can obtain the optimal conditions of this problem to each variable of Lagrange function respectively:

$1-b{y}_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j{q}_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i=0---\left(4\right)$

$\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_i{y}_i=0---\left(5\right)$

Q wherein _Ij=y _iy _jK _Ij, and

Be defined as kernel function, if kernel function satisfies the Merce condition, and symmetrical matrix Q _c=[q _Ij] be positive definite, then this problem is an optimized protruding problem, promptly it has only an overall situation to separate.

Neural network model is described by following dynamic equation:

$b=\frac{\∂J}{\∂b}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_i{y}_i---\left(6\right)$

${\mathrm{\α}}_i=-\frac{\∂J}{\∂{\mathrm{\α}}_i}=1-b{y}_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j{q}_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i---\left(7\right)$

Fig. 1 is the topology diagram of existing LSSVM classification learning neural network ,-α ₁q _I1-α _Nq _IN, 1 ,-γ ^-1α _iAnd-by _iInsert in the ∑ and sue for peace, the output terminal of ∑ inserts integrator ∫, and the output of integrator ∫ is α _i, and α _iAgain through weights-q _IjFeed back in each corresponding ∑, form a recurrent neural network.

The topology diagram of Fig. 1, its design philosophy are parallel computation m α, calculate the b value according to one group of α value of real-time change simultaneously, and the result who does has like this caused shared resource in the core of its calculating and sample number m that direct relation is arranged.When sample number changes, must increase the realization module of hardware simultaneously, though on speed, obtained very big advantage like this, because the complexity of the variation of resource and structure can make it that significant limitation is arranged when using.Therefore, when sample number increases, the LSSVM that this structure realized will consume the resource of FPGA in a large number, and the complexity of structure is increased.Suppose that the sample number that needs is m, it is k% that each sample is carried out the FPGA resource that training need takies, m sample will take the resource of m * k% in the FPGA altogether so, be easy to cause the utilization of resources to exceed the problem of real resource total amount like this, so also will limit its application in embedded system.

Fig. 2 is the employed structural drawing of the inventive method, and the topological structure of constructing LSSVM classification learning recurrent neural network according to given sample size is: will

1 ,-γ ^-1α _jAnd-by _iInsert in the ∑ and sue for peace, the output terminal of ∑ inserts integrator ∫ 1, and the output of integrator ∫ 1 is α _j, simultaneously will

Insert integrator ∫ 2, output b value, α _jAgain through weights-q _IjFeed back in the corresponding ∑, form a recurrent neural network.

Fig. 2 optimizes the structure of Fig. 1, though be to calculate in the mode of serial, but do not change the parallel organization of neural network LSSVM, so just avoided the direct relation of side circuit and sample number, can finish the uncertain training work of actual sample number under the situation that does not change side circuit, its core algorithm as shown in Figure 3.Calculate the b value by all α values again after each as seen from Figure 3 first cycle calculations n α value, so just avoid setting up a plurality of module training α values and caused the wasting of resources, and each α value of calculating is not subjected to the influence of other α values of changing at any time yet, because the α value of this calculating is relevant with n α value of last computation of Period.Certainly, set the number of training of a maximum,,, recycle same resource, can make that hardware area obtains reducing by the computing method of serial as long as number of training is no more than the scope of this setting according to the resource of FPGA.

Step 2, the existing method of basis are selected suitable kernel function and kernel function parameter for use, and are calculated:

Can select gaussian kernel function for use, select parameter γ ^-1=1, σ=1;

Step 3, discretize carried out in dynamic equation (6), (7) handle, and set step delta T value:

Because FPGA itself can not directly realize continuous dynamic equation, so Discrete Dynamic equation (8), (9) are carried out obtaining after discretize is handled in dynamic equation (6), (7):

$b(t+\mathrm{\ΔT})=\mathrm{\ΔT}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_i\left(t\right)y_i\left(t\right)+b\left(t\right)---\left(8\right)$

${\mathrm{\α}}_i(t+\mathrm{\ΔT})=\mathrm{\ΔT}(1-b\left(t\right)y_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j\left(t\right)q_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i\left(t\right))+{\mathrm{\α}}_i\left(t\right)---\left(9\right)$

Δ T is the time interval of sampling in the dynamic equation (8), (9), and how the back will pair realize that with FPGA dynamic equation (8), (9) are elaborated in step 6, step 7.

The digits of binary coding of step 4, select tape complement code, comprising the figure place of integer-bit and the figure place of decimal place:

Element in the component library that the inventive method is created, input is all adopted the binary coding of being with complement code with output signal.The computing of corresponding primary element is fixed-point arithmetic, and the position of radix point is self-determining by the deviser in fixed-point arithmetic, the position that radix point is deposited in the unit of not storing separately in the hardware of reality is realized.For the deviser, it is a virtual existence, only needs the deviser to remember its position, is not influence in the calculating process of reality.The code length scope of signal can by deviser's experience or software emulation obtains.

Another important technological problems in the inventive method is that intermediate data is handled, because the network that is proposed all has recursive nature, for this class network, import given back training process and be oneself a continuous iterative process, be difficult to carry out human intervention in this process.Comprised arithmetic elements such as multiplier and multiply accumulating device for the computing of neuron in one-period, this class arithmetic element can cause expression way inconsistent of output signal and input signal.Figure 4 shows that the binary coded number of one 16 band complement code, S conventional letter position wherein, F is an integer-bit, I is a decimal place.With reference to Fig. 5, describe with the example that is combined as of a multiplier and a subtracter, the input signal a of multiplier (Mul) and b are that length is the binary coding of 16 band complement code, then its output signal C is that length is the binary coding of 32 band complement code.And will be as an input signal of subtracter (Sub) for the output signal C of multiplier, but the input signal code length that subtracter (Sub) is set is the binary coding of 16 bit strip complement codes, so will the binary coding C of 32 band complement codes be converted to the binary coding C of 16 band complement code through a data processing procedure before this; With reference to Fig. 6,32 coding C are done following processing, give up wherein part integer-bit and part decimal place, if the residue integral part can be represented all integer-bit, casting out integer-bit part does not so influence the result; For the fractions omitted part, if it is abundant that the signal decimal place obtains, give up fraction part shown in Figure 4 so, influence to the result is very little, can ignore, therefore, when through after part shown in Figure 6 casts out processing, just can obtain binary coding with the band complement code of 16 of Fig. 4 same form.

Step 5, the basic component library of structure comprise arithmetic element, storage unit and control module:

The programmability of FPGA itself has strengthened the dirigibility of circuit design, so the main means in the inventive method are VHDL programming languages that FPGA adopts usually, it is how to be used in the LSSVM classification learning neural network that Fig. 7 further specifies VHDL language.Two of the bottom primary element storehouses that the square frame representative makes up among Fig. 7: mainly comprise multiply accumulating unit (Mac, Bias), multiplier unit (Mul), subtractor unit (Sub), accumulator element (Add) and storage unit ROM and RAM.On the realization means of primary element, some adopts the VHDL language programming to realize, also has some then to adopt third-party IP kernel to realize.IP kernel involved in the present invention is soft IP kernel, does not contain concrete physical message.The square frame of the second layer is represented neuron, and wherein neuro and bias_b represent the neuron that calculates α and b respectively, and they all are to call in the bottom element in the primary element storehouse and realize by writing the vhd file.Uppermost square frame is represented whole LSSVM neural network, and it also is to realize by writing the neuron file that the vhd file calls the second layer.

The neuro neuron elements of step 6, structure LSSVM Classification Neural:

Fig. 8 is the neuronic FPGA implementation structure of the neuro among Fig. 4 figure, and this module is carried out the function of calculating α, also is the core calculations part of whole LSSVM framework, and it has finished the calculating of dynamic equation (9).Comprising multiply accumulating unit (MAC), multiplier unit (Mul), subtractor unit (Sub), accumulator element (ADD) and storage unit ROM, RAM and RAM1.The q that deposits among the ROM _IjBe the weights of network, Mul1, Mul2 and Mul3 are the mapping elements to the element Mul in the primary element storehouse, are used for calculating b (t) y in the dynamic equation 9 respectively _i, γ ^-1α _i(t),

Sub1, Sub2 and Sub3 are the mapping elements to the element Sub in the primary element storehouse, are used for calculating 1-b (t) y respectively _i,

The y of the input port y input of Mul1 _iBe the sign of classification, the γ of the input port b input of Mul2 is a penalty factor.What deposit among RAM and the RAM1 is one group of α value, and the b value of the input port b of Mul1 input is the threshold value of table network, but all α values and b value people are for being set at arbitrary value initially the time, and it is 0 that the present invention is provided with initial value.Know that by step 4 the shift unit is to be used for data limit in the scope of appointment.Be different from common neural network, for neural network LSSVM with recursive nature, the weights q of network _IjBe by the decision of the value of training sample, and the weights of network not real-time update.For a specific problem, the value of training sample is fixed, i.e. the network weight q of neural network LSSVM _IjIn the process of training, do not change.So before network training, calculated weights q according to training sample _Ij, leave in the ROM cell.

After the FPGA implementation structure was set up, network entered training process.Because present networks has recurrence, so running describes to the neuron in the minor cycle at this.Big training sample cycle of definition earlier, a little training sample cycle was for calculating one of them α in order to have calculated the one-period of all α values _iIn the cycle of value, when a last large period finishes, produce one group of training parameter α ₁(t), α ₂(t) ... α _i(t), and this group parameter value is deposited in the RAM1 unit, this is to consider the saving problem of resource and the simplification of structure, and the pipelining of using, and has also guaranteed the concurrency of network simultaneously.When subsequent calculations need be used these parameter values, from RAM1, read again.As Fig. 8, can be divided into following six stages to the FPGA implementation procedure of whole neural network LSSVM alpha value calculated:

1), finish when data storage, this moment, reading of data simultaneously from ROM and RAM1 was input in the MAC unit, wherein a triggering along in α _j(t) and q _IjBe corresponding mutually.Meanwhile, calculate b (t) y by Mul1 and Mul2 respectively _i, γ ^-1α _i(t), so, when two or several module parallel computation, just can save the training time of network;

2), wait for that the Mac unitary operation finishes, with the result

And b (t) y _iBe input to the map unit Sub1 of Sub, finish 1-b (t) y _iCalculating;

3), wait for that Mac unit and Sub1 unitary operation finish, and incite somebody to action

And 1-b (t) y _iSend into Sub2, finish

Calculating;

4), the result of Sub2 unit and the result of Mul2 unit are sent into Sub3, finish

Calculating;

5), again the result of Sub3 and the AT that configures are sent into Mul3, calculate Δ T with

Product.

6), finish the ADD accumulating operation because dynamic equation (7) is carried out after discretize handles, last integral unit has become the unit that adds up, so ADD is used for α that this is drawn _i ^N+1The α that a value and a last large period obtain _i ⁿValue adds up (n is the large period number of times here), and the value that will upgrade deposits in the ram cell.Because calculating and comparison procedure need be used the α that a large period calculates _i ⁿSo, utilize RAM and RAM1 to preserve one group of α of this calculating ^N+1One group of α that a value and a last large period calculate ⁿValue.For convenience of comparing and adding up, according to calculating each α _iThe minor cycle of value, will go up the α of a large period temporarily _iValue is kept among the A_in.Because what adopt is fixed-point arithmetic, so the shift unit is used for data limit in the scope of appointment, and in order not influence the precision of data, the scope of restriction is selected (15,15) for use.Calculating n+1 step α _iAfter the value, calculate n+1 step α again _I+1,, promptly finished the serial computing mode up to having calculated k α value (k is a number of training).

The Bias_b neuron elements of step 7, structure LSSVM Classification Neural:

Fig. 9 is the neuronic FPGA implementation structure of the bias_b among Fig. 4 figure.

This module is carried out the function of calculating b, finishes the calculating of dynamic equation (8).Comprising multiply accumulating unit (Bias), multiplier unit (Mul), accumulator element (ADD) and storage unit RAM1.Wherein the Bias unit is used for calculating

The Mul unit calculates

The ADD unit b value that is used for adding up.Storage unit RAM1 wherein calls is storage unit RAM1 among Fig. 8.Know that by step 4 the shift unit is to be used for data limit in the scope of appointment.

At first read one group of α value and put into a port of Bias module in order from RAM1, the b port of Bias module is imported the other y value of a category in order simultaneously, and finishes

Calculating.Afterwards the result of calculation of Bias module is sent to a port of Mul unit, the Δ T value that the b port of Mul unit is just imported for beginning is finished in the Mul unit

Calculating.Result of calculation with the Mul unit is kept in the b unit temporarily then.What the b1 unit was preserved is the b value of a last large period, if calculate for the first time, then initial b value can be made as zero.At last two b values of b unit and b1 unit are sent into a port and the b port of ADD unit respectively, finished accumulation calculating, and the b value that will add up at last puts into the b1 unit, wait for the b value accumulation calculating of next large period.

Because the calculating of b value and all α of a last large period _iBe worth relevant, and just one of them α that in each minor cycle, is calculated _iValue, i.e. α _iTherefore value is placed on the α of last minor cycle along with the minor cycle upgrades at any time for the calculating unification of b value _iValue is calculated when finishing, and the renewal of this moment will be at one group of new α _iThe new b value that obtains under the value situation, and this b value will be saved in last minor cycle of next large period always, calculate up to next new b value.Because last link of calculating to the b value need add up with the b value of a last large period, therefore after each renewal b value, it is stored in the b1 module of Fig. 9 temporarily.

Step 8, after having made up above two neuron elements, make network carry out recursive operation, when network stabilization, try to achieve the convergence parameter alpha, b.

When using VHDL language to realize, neuron neuro and bias_b are used as a solid element respectively.Make up neuro.vhd and bias.vhd file, call element in the component library by the component statement, do like this when needs carry out the part change to network, only need the elementary cell in change or the replacement component library can constitute new network, have better expansibility.

Just finished the structure of serial computing functional module by the flow process of Fig. 8 and Fig. 9.In order well to finish the computing function of network, make each neuron be controlled by a clock control signal in the present invention, after calculating finishes in each neuron one-period, the capital produces an effective control signal, when all neuronic control signals that receive when the ANN (Artificial Neural Network) Control unit are effective, will produce a total effective control signal and make whole neurons enter the computing of following one-period.The same component statement that uses calls the neuron module, and the method is convenient to increase and decrease the number of training sample in the neural network so that the variation of network size.

Therefore, the FPGA implementation method of the neural network least square support vector classifier based on serial computing of the present invention, implement according to the following steps:

Step 1: according to the topological structure of sample size structure LSSVM Classification Neural, as shown in Figure 2;

Step 2: select gaussian kernel function for use, select parameter γ ^-1=1, σ=1, and calculate:

Step 3: discretize is carried out in dynamic equation (6), (7) handle, and set step delta T value;

Step 4: the digits of binary coding of select tape complement code, comprising the figure place of integer-bit and the figure place of decimal place, as select 1 bit sign position, 14 integer-bit and 15 decimal places;

Step 5: construct basic component library, comprise arithmetic element, storage unit and control module;

Step 6:, make up the neuro neuron elements of LSSVM Classification Neural according to Fig. 8 flow process;

Step 7: according to Fig. 9 flow process, make up the Bias_b neuron elements of LSSVM Classification Neural, after step 6 is finished last α value calculating, calculate the b value;

Step 8: regard step 6 and the neuron that step 7 makes up as primary element, make up the control module of whole network, make network carry out recursive operation then, when network stabilization, try to achieve the convergence parameter alpha, b.

Embodiment

" R ²The checking example of 7 sample point linear separabilities ", z is set ₁=(3,1), z ₂=(3,2), z ₃=(3,3), z ₄=(5,1), z ₅=(5,2), z ₆=(5,3), z ₇=(5,1.5), corresponding classification are (+1 ,+1 ,+1 ,-1 ,-1 ,-1 ,-1).

Step 1: according to the topological structure of 7 sample architecture LSSVM classification learning recurrent neural networks;

Step 2: adopt gaussian kernel function, select γ ^-1=1, σ=1,

And calculate

Step 3: dynamic equation (6) (7) is carried out discretize handle, get step delta T=2 ^-3S;

Step 4: the digits of binary coding of selecting employed band complement code is 30, comprising 1 bit sign position, 14 integer-bit and 15 decimal places;

Step 5: structure multiply accumulating unit (MAC, Bias), multiplier unit (Mul), subtractor unit (Sub), accumulator element (ADD) and storage unit ROM, RAM and RAM1, wherein call IP kernel realization MAC, Mul, Sub, ADD and ROM among the ISE9.1, write the vhd file and realize;

Step 6: according to Fig. 8 flow process, the q that will obtain according to sample calculation earlier _IjBe stored among the ROM, design connection and trigger sequence between each arithmetic element according to computation sequence then;

Step 7:, after step 6 is finished last α value calculating, calculate the b value according to Fig. 9 flow process;

Step 8: regard step 6 and the neuron that step 7 makes up as primary element, make up the control module of whole network.Make network carry out recursive operation, when network stabilization, try to achieve the convergence parameter alpha, b.

The value of each parameter that the FPGA implementation method of Figure 10 existing neural network LSSVM parallel computation structure for present embodiment adopts obtains.

Curve among Figure 11 is by the convergence parameter alpha among Figure 10 the classification lineoid that b tries to achieve.

The value of each parameter that Figure 12 obtains for the FPGA implementation method of the neural network LSSVM serial computing of the embodiment of the invention.

Curve among Figure 13 is by the convergence parameter alpha among Figure 12 the classification lineoid that b tries to achieve.

This embodiment based on these two kinds of implementation method required times of LSSVM classification learning neural network and occupation condition is: it is 2.346 * 10 that parallel organization is tested the used time ^-4S, resource occupation are 103%; It is 3.108 * 10 that serial structure is tested the used time ^-4S, resource occupation are 25%.From the classifying quality of last classification lineoid, two kinds of implementation methods have all been finished classification purpose of the present invention preferably, but use FPGA resource that method of the present invention consumes than existing method still less.

In sum, the FPGA implementation method of neural network LSSVM serial computing of the present invention, dynamic equation after mainly having adopted structurized VHDL language to the network discretize is described, according to from up to down making up of primary element, neuron, neural network, not only reduced the consumption of FPGA resource, and overcome the shortcoming that mimic channel is realized the dirigibility deficiency, and can adapt to the change of applied environment preferably, make the range of application of support vector machine more extensive.

Claims (4)

1. least square method supporting vector machine serial structure implementation method based on FPGA is characterized in that this method is implemented according to the following steps:

Step 1, according to the topological structure of sample size structure LSSVM Classification Neural:

Topological structure according to given sample size structure LSSVM classification learning recurrent neural network is: will 1 ,-γ ^-1α _jAnd-by _iInsert in the ∑ and sue for peace, the output terminal of ∑ inserts integrator ∫ 1, and the output of integrator ∫ 1 is α _j, simultaneously will

Insert integrator ∫ 2, output b value, α _jAgain through weights-q _IjFeed back in the corresponding ∑, form a recurrent neural network;

Given classification based training collection (z _i, y _i) be one group of sample to be classified, i=1,2 ... N, for all z _i∈ R ^NY is all arranged _i∈ (+1 ,-1) is the corresponding classification of sample, and its categorised decision face is

Wherein W is a weight vector, and b is a threshold value,

The Nonlinear Mapping of expression training sample from the input space to the feature space, the classification learning of LSSVM promptly solves following affined optimization problem:

$\underset{W,e}{\min }J(W,e)=\frac{1}{2}{W}^{T}W+\mathrm{\γ}\underset{i=1}{\overset{l}{\mathrm{\Σ}}}{e}_i^2---\left(1\right)$

Constraint condition:

Find the solution this problem and introduce the Lagrange function:

α wherein _iBe the Lagrange multiplier, utilize the KKT condition respectively each variable of Lagrange function to be asked local derviation, obtain the optimal conditions of this problem:

$1-{\mathrm{by}}_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j{q}_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i=0---\left(4\right)$

$\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_i{y}_i=0---\left(5\right)$

Q wherein _Ij=y _iy _jK _Ij, and Be defined as kernel function,

Obtain the dynamic equation of LSSVM classification learning neural network model:

$\stackrel{.}{b}=\frac{\∂J}{\∂b}=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_i{y}_i---\left(6\right)$

${\stackrel{.}{\mathrm{\α}}}_i=-\frac{\∂J}{\∂{\mathrm{\α}}_i}=1-{\mathrm{by}}_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j{q}_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i---\left(7\right)$

α wherein _iBe the Lagrange multiplier, b is a threshold value, (z _i, y _i) be one group of sample to be classified, i=1,2 ... N;

Step 2, select gaussian kernel function for use, select parameter γ ^-1=1, σ=1, and calculate:

The result of step 3, integrating step 2 carries out discretize with dynamic equation (6), (7) and handles, and sets step delta T value;

Equation (6), (7) are carried out obtaining Discrete Dynamic equation (8), (9) after discretize is handled

$b(t+\mathrm{\ΔT})=\mathrm{\ΔT}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_i\left(t\right)y_i\left(t\right)+b\left(t\right)---\left(8\right)$

${\mathrm{\α}}_i(t+\mathrm{\ΔT})=\mathrm{\ΔT}(1-b\left(t\right)y_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j\left(t\right)q_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i\left(t\right))+{\mathrm{\α}}_i\left(t\right);---\left(9\right)$

The digits of binary coding of step 4, setting band complement code is comprising the figure place of integer-bit and the figure place of decimal place;

Step 5, the basic component library of digits of binary coding structure that sets according to step 4 comprise arithmetic element, storage unit and control module;

Arithmetic element comprises multiply accumulating unit MAC, multiplier unit Mul, subtractor unit Sub, accumulator element ADD;

Storage unit comprises ROM and RAM;

Control module calls IP kernel among the ISE9.1, realizes MAC, Mul, Sub, ADD and ROM;

Step 6, the component library that utilizes step 5 to obtain make up the neuro neuron elements;

Step 7: the component library that utilizes step 5 to obtain makes up the Bisa_b neuron elements;

Step 8: the structure of finishing the serial computing functional module

Make each neuron be controlled by a clock control signal, after calculating finishes in each neuron one-period, produce an effective control signal, when all neuronic control signals that receive when the ANN (Artificial Neural Network) Control unit are effective, producing a total effective control signal makes whole neurons enter the computing of following one-period, when network carries out recursive operation when stablizing, promptly try to achieve the convergence parameter alpha, b.

2. method according to claim 1 is characterized in that, in the described step 4, concrete steps are:

The input signal a and the b that set multiplier Mul are that length is the binary coding of 16 band complement code, then output signal C is that length is the binary coding of 32 band complement code, the binary coding C of 32 band complement codes is converted to the binary coding C of 16 band complement code, give up wherein part integer-bit and part decimal place, obtain the binary coding of 16 band complement code.

3. method according to claim 1 is characterized in that, described step 6 makes up the neuro neuron elements, and concrete steps are:

Make up LSSVM classification learning neuron module, this module is carried out the function of calculating α, finish the calculating of dynamic equation (9), comprising multiply accumulating unit (MAC), multiplier unit (Mul), subtractor unit (Sub), accumulator element (ADD) and storage unit ROM, RAM and RAM1; The q that deposits among the ROM _IjBe the weights of network, Mul1, Mul2 and Mul3 are the mapping elements to the element Mul in the primary element storehouse, are used for calculating b (t) y in the dynamic equation (9) respectively _i, γ ^-1α _i(t),

$\mathrm{\ΔT}(1-b\left(t\right)y_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j\left(t\right)q_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i\left(t\right));$

Sub1 wherein, Sub2 and Sub3 are the mapping elements to the element Sub in the primary element storehouse, are used for calculating 1-b (t) y respectively _i,

And

$1-b\left(t\right)y_i-\underset{j=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{\α}}_j\left(t\right)q_{\mathrm{ij}}-{\mathrm{\γ}}^{-1}{\mathrm{\α}}_i\left(t\right),$

The y of the input port y input of Mul1 _iIt is the sign of classification, the γ of the input port b input of Mul2 is a penalty factor, what deposit among RAM and the RAM1 is one group of α value, the b value of the input port b input of Mul1 is the threshold value of table network, know that by step 4 the shift unit is to be used for data limit in the scope of appointment, for neural network LSSVM with recursive nature, the weights q of network _IjBy the decision of the value of training sample, and the weights of network are not real-time update, and for a specific problem, the value of training sample is fixed, i.e. the network weight q of neural network LSSVM _IjIn the process of training, do not change, so before network training, calculated weights q according to training sample _Ij, leave in the ROM cell, specifically may further comprise the steps:

A1, finish when data storage, this moment, reading of data simultaneously from ROM and RAM1 was input in the MAC unit, wherein a triggering along in α _j(t) and q _IjBe corresponding mutually, meanwhile, calculate b (t) y by Mul1 and Mul2 respectively _i, γ ^-1α _i(t);

A2, wait MAC unitary operation finish, with the result

And b (t) y _iBe input to the map unit Sub1 of Sub, finish 1-b (t) y _iCalculating;

A3, wait Mac unit and Sub1 unitary operation finish, will

And 1-b (t) y _iSend into SUB2, finish Calculating;

A4, the result of Sub2 unit and the result of Mul2 unit are sent into Sub3, finish

Calculating;

A5, again the result of Sub3 and the Δ T that configures are sent into Mul3, calculate Δ T with

Product;

A6, finish the ADD accumulating operation, because dynamic equation (7) is carried out after discretize handles, last integral unit has become the unit that adds up, so ADD is used for α that this is drawn _i ^N+1The α that a value and a last large period obtain _i ⁿValue adds up, and n is the large period number of times, and the value that will upgrade deposits in the ram cell, owing to calculate and comparison procedure need be used the α that a large period calculates _i ⁿSo, utilize RAM and RAM1 to preserve one group of α of this calculating ^N+1One group of α that a value and a last large period calculate ⁿValue is according to calculating each α _iThe minor cycle of value, will go up the α of a large period temporarily _iValue is kept among the A_in, and the shift unit is used for data limit in the scope of appointment, and the scope that is limited is (15,15), is calculating n+1 step α _iAfter the value, calculate n+1 step α again _I+1, up to having calculated k α value, k is a number of training, promptly finishes the serial computing mode.

4. method according to claim 1 is characterized in that, described step 7 makes up the Bisa_b neuron elements, and concrete steps are:

This module is carried out the function of calculating b, finishes the calculating of dynamic equation (8), and comprising multiply accumulating unit (Bias), multiplier unit (Mul), accumulator element (ADD) and storage unit RAM1, wherein the Bias unit is used for calculating The Mul unit is used for calculating

The ADD unit b value that is used for adding up,

At first read one group of α value and put into a port of Bias module in order from RAM1, the b port of Bias module is imported the other y value of a category in order simultaneously, and finishes Calculating; Afterwards the result of calculation of Bias module is sent to a port of Mul unit, the Δ T value that the b port of Mul unit is just imported for beginning is finished in the Mul unit

Calculating; Result of calculation with the Mul unit is kept in the b unit temporarily then, and what the b1 unit was preserved is the b value of a last large period, if calculate for the first time, then initial b value is made as zero; At last two b values of b unit and b1 unit are sent into a port and the b port of ADD unit respectively, finished accumulation calculating, and the b value that will add up at last puts into the b1 unit, wait for the b value accumulation calculating of next large period;

Because the calculating of b value and all α of a last large period _iBe worth relevant, and just one of them α that in each minor cycle, is calculated _iValue, i.e. α _iTherefore value is placed on the α of last minor cycle along with the minor cycle upgrades at any time for the calculating unification of b value _iValue is calculated when finishing, and the renewal of this moment will be at one group of new α _iThe new b value that obtains under the value situation, and this b value will be saved in last minor cycle of next large period always, calculate up to next new b value, because last link of calculating to the b value need add up with the b value of a last large period, therefore after each renewal b value, it is stored in the b1 module temporarily.

Images