CN110427960B - Reconfigurable multi-classification support vector machine system - Google Patents

Reconfigurable multi-classification support vector machine system Download PDF

Info

Publication number
CN110427960B
CN110427960B CN201910528797.5A CN201910528797A CN110427960B CN 110427960 B CN110427960 B CN 110427960B CN 201910528797 A CN201910528797 A CN 201910528797A CN 110427960 B CN110427960 B CN 110427960B
Authority
CN
China
Prior art keywords
support vector
vector machine
class
module
kernel function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910528797.5A
Other languages
Chinese (zh)
Other versions
CN110427960A (en
Inventor
李丽
孙瑞
傅玉祥
陈辉
高珺
何书专
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University
Original Assignee
Nanjing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University filed Critical Nanjing University
Priority to CN201910528797.5A priority Critical patent/CN110427960B/en
Publication of CN110427960A publication Critical patent/CN110427960A/en
Application granted granted Critical
Publication of CN110427960B publication Critical patent/CN110427960B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention relates to a reconfigurable multi-class support vector machine system, comprising: the device comprises a main control module, a storage control module, a kernel function operation module, a category operation module and a result comparison module. The main control module provides control information and reconstruction information for the whole decision process; the storage control module controls the storage of data; the kernel function operation module calculates a kernel function between the test data and the support vector; the class operation module calculates decision values and classification classes; and the result comparison module compares decision values calculated by different models to obtain a final classification result of the test data. Compared with the traditional method, the method fully utilizes the parallelism of hardware, accelerates the operation speed of supporting the classification of the vector machine, shares the calculation resources with the kernel function operation module and the class operation module, supports the reconstruction of the hardware, and has good flexibility for samples with different feature numbers.

Description

Reconfigurable multi-classification support vector machine system
Technical Field
The invention belongs to the field of hardware implementation of machine learning algorithms, and particularly relates to a reconfigurable multi-classification support vector machine system.
Background
The support vector machine (support vector machine, SVM) is a supervised machine learning algorithm, can be used for data analysis, recognition mode, data classification and regression analysis, and has wide application. The support vector machine algorithm can be divided into two parts of training and decision making, wherein the training part is generally carried out on a high-performance server due to complex calculation mode and large calculation amount; the decision-making part classifies or carries out regression analysis on the sample data by using a trained model, and the calculation process comprises a large number of multiply-add operations. The input sample is recorded as x, and the ith support vector is recorded as x i Corresponding coefficient is alpha i The corresponding label is y i Offset b, the kernel function between the two vectors is k (x i ,x j ) The decision function is:
Figure BDA0002099056460000011
the above procedure addresses the two classification problem. When the support vector machine is used for data classification in a real task, the problem of multiple classification can be encountered. At present, two main methods for constructing SVM multi-classifier are: one is a direct method, which directly constructs a model to perform multi-classification, and the method is seemingly simple, but has high calculation complexity and difficult realization. Therefore, the indirect method is generally adopted: the method is to train a plurality of two-classification SVMs and then to make multi-classification decisions according to a certain mechanism. The usual methods are: one-to-many (OVR), one-to-one (one-versus-one, OVO), binary Tree (BT), directed acyclic graph (Directed Acyclic Graph, DAG).
With the rapid development of the integrated circuit industry, high performance and real-time performance are constantly pursued in the embedded field. Currently, most of hardware implementation of support vector machines is designed based on CPU and GPU platforms. The CPU has a large number of caches and control units, the number of ALUs used for calculation is small, and the GPU has a large number of calculation resources and storage resources, so that the control is simple, and for a computationally intensive algorithm, the GPU has a faster operation speed than the CPU, but the GPU has a larger area and high energy consumption, and is not suitable for embedded real-time operation.
Disclosure of Invention
The invention aims to overcome the defects of the implementation mode of the support vector machine in the background technology, aims at a decision part, adopts a one-to-one method to carry out multi-classification, and provides a reconfigurable hardware implementation of the multi-classification support vector machine, which can support the reconfiguration of computing resources, the cutting of hardware and the parallel and pipeline operation, thereby improving the computing speed. The method is realized by the following technical scheme:
the reconfigurable multi-class support vector machine system comprises:
the main control module is used for controlling the whole calculation flow and providing reconstruction information of calculation resources;
the storage control module is used for controlling the data storage of each operation module;
the kernel function operation module receives the test data and the support vector, calculates a kernel function between the test data and the support vector, and obtains an operation result;
the class operation module calculates a multiplication and accumulation result of the kernel function and the corresponding Lagrange multiplier coefficient according to the operation result, adds bias to the multiplication and accumulation result to obtain a decision value and class, and adds 1 to the corresponding class ticket number;
and the result comparison module is used for comparing the ticket numbers of all the categories to obtain the final classification result of the test data.
The reconfigurable multi-classification support vector machine system is further designed in that a main control module receives a system starting signal to finish ticket taking classification operation of a single vector machine, and the operation specifically comprises the following steps: starting a kernel function operation module, starting a category operation module after kernel function operation is finished, calculating to obtain categories, adding 1 to corresponding category ticket numbers, storing by a storage control module, judging whether the current support vector machine is the last support vector machine by a main control module, and starting a result comparison module if the current support vector machine is the last support vector machine to obtain a final classification result; otherwise, repeating the ticket taking and classifying operation of the single vector machine.
The reconfigurable multi-class support vector machine system is further designed in that the kernel function operation module and the class operation module share computing resources, and the kernel function operation module and the class operation module are respectively formed by changing the topological structure and the interconnection relation of basic operation units of the computing resources.
The reconfigurable multi-class support vector machine system is further designed in that the basic operation unit comprises an adder, a multiplier, an accumulator and an exponential operator.
The reconfigurable multi-class support vector machine system is further designed in that in the kernel function operation module, the computing resource is reconfigured into a plurality of parallel computing units, each parallel computing unit consists of an adder, a multiplier, an accumulator and an exponential operator, and each parallel computing unit calculates a kernel function.
The reconfigurable multi-classification support vector machine system is further designed in that the class operation module reconstructs computing resources into a multi-input multiply-add tree structure and carries out parallel acceleration on the computation of decision values.
The invention has the beneficial effects that:
the reconfigurable multi-classification support vector machine system fully utilizes the parallelism of hardware, supports the reconfiguration of computing resources, enables the hardware to be cut, supports parallel and pipeline operation, improves the operation speed of the classification of the support vector machine, and has good flexibility for samples with different feature numbers.
Drawings
Fig. 1 is a diagram of a support vector machine hardware implementation architecture.
Fig. 2 is a block diagram of a kernel operation module.
FIG. 3 is a schematic diagram of a path computation unit of the kernel function operation module.
Fig. 4 is a schematic diagram of support vector data arrangement.
Fig. 5 is a diagram of a kernel data arrangement.
Fig. 6 is a schematic diagram of the arrangement of coefficient data.
Fig. 7 is a block diagram of a class operation module multiply-add tree.
Detailed Description
The present invention will be described in detail with reference to the accompanying drawings.
In the multi-classification support vector machine system provided in this embodiment, a sample to be classified and a support vector of a support vector machine model are input, a kernel function between a Test Vector (TV) and all Support Vectors (SVs) is calculated, decision is made after the kernel function calculation is completed, multiply-accumulate calculation is performed on the kernel function value and a corresponding coefficient, and bias is added after the completion to obtain a decision value. For the multi-classification support vector machine, a plurality of models are trained, after the calculation of the decision value is completed, whether the model is the last support vector machine is judged, if yes, a result comparison module is entered, the decision value of each model is compared, and the category with the largest decision value is used as the category of the final test vector. Otherwise, the next support vector machine model is calculated, and the calculation process and the previous process are performed, wherein different models have different support vectors and corresponding coefficients.
The reconfigurable multi-class support vector machine system of the embodiment mainly comprises a main control module, a storage control module, a kernel function operation module, a class operation module and a result comparison module.
The main control module receives a system starting signal, starts the kernel function operation module, starts the category operation module based on data delay, obtains categories, and adds 1 to the corresponding category ticket number to be stored in the storage unit. And after the calculation is finished, the control module judges whether the current support vector machine is the last support vector machine, if so, the result comparison module is started, and if not, the next kernel function operation module is started. When all support vector machines calculate the category, the final classification result is obtained by the result comparison module after the category ticket number is obtained, the main control module simultaneously provides reconstruction information, and the calculation resources are dynamically reconstructed into calculation structures required by different calculation modules by changing the topological structure and the interconnection relation of the basic calculation units.
The storage control module controls the storage of data and comprises a support vector storage module, a kernel function storage module, a coefficient storage module, a test vector storage unit, an offset storage unit and a category ticket number storage unit.
The kernel function operation module is responsible for receiving the starting signal of the main control module and performing kernel function calculation, and the whole structure is formed by multiple paths of operation units in parallel. When the module is operated, the computing resources are reconstructed into operation units, and each operation unit sequentially performs subtraction, squaring, accumulation, offset addition and exponential operation, and a kernel function is obtained through calculation. The operation unit has 16 paths in total and calculates the kernel functions of the test vector and the 16 support vectors in parallel. The test vector is stored in one storage unit, the support vector data is stored in 16 storage units, all eigenvalues of the same support vector are stored in one storage unit, each address stores two eigenvalues, the first support vector is stored in storage unit 1, the second support vector is stored in storage unit 2, after 16 support vectors are stored, the 17 th support vector is continuously stored in storage unit 1, and so on. After the calculation is completed, the kernel function results are stored in 16 storage units, two kernel functions are stored in each address, the kernel functions of the test vector and the first and second support vectors are stored in the storage unit 1, the kernel functions of the test vector and the third and fourth support vectors are stored in the storage unit 2, after 32 kernel functions are stored, the kernel functions of the test vector and the 33 th and 34 th support vectors are stored in the storage unit 1 again, and so on.
The class operation module starts to start after receiving the end signal of the kernel function operation module. When the module is operated, the computing resource is reconstructed into a multiplication and addition tree structure, 32 kernel function values in 16 kernel function storage units and 32 coefficient values in 16 coefficient storage units are parallelly taken out during computation, the coefficient 1 and the coefficient 2 are stored in the storage unit 1, the coefficient 3 and the coefficient 4 are stored in the storage unit 2, and after 16 storage units are fully stored, the coefficient 33 and the coefficient 34 are continuously stored in the storage unit 1, and the like. The kernel function and the coefficient are respectively used as source data 1 and source data 2 of the multiplier, the result of the multiplier is accumulated through an adder tree, and the specific multiplier-adder tree structure is of 7 stages in total and comprises 32 multipliers, 16 adders, 8 adders, 4 adders, 2 adders, 1 adder and 1 accumulator. And adding bias after all the kernel functions and the coefficient multiplication accumulation calculation are finished to obtain decision values and categories, adding 1 to the ticket number of the corresponding category, and storing the ticket number in a category ticket number storage unit.
The result comparison unit is started after receiving the starting signal given by the control module, compares all the decision values stored in the decision value storage unit, and takes the category to which the maximum decision value belongs as the final category.
A specific example is given below, in which the kernel function operation module is composed of 16-way calculation units. The class operation module consists of a 32-input multiply-add tree. The feature numbers of the test vector and the support vector are 8, and 1,2 and 3 categories are all provided, wherein the 1 category and the 2 category train one support vector machine, which is called a support vector machine 1, and the number of the support vectors is 66; class 2 and class 3 train a support vector machine, called support vector machine 2, the number of support vectors is 66; class 1 and class 3 train a support vector machine, called support vector machine 3, with 66 support vectors. The kernel function uses a gaussian kernel function
Step 1) inputting a sample to be tested and support vector data of a support vector machine 1, wherein the support vector is stored by using 16 storage units, and the test sample is stored by using 1 storage unit, and the storage mode is consistent with that of the support vector storage unit.
And 2) subtracting each component of the sample and the support vector, accumulating after squaring the subtracted result, multiplying the accumulated result by a kernel parameter, and obtaining an index to obtain a kernel function.
And 3) after the kernel function calculation is completed, carrying out a category operation module. The inputs of the multiply-add tree in the module are kernel functions and coefficients respectively, and for the specific example discussed above, bias is added after multiply-add calculation is completed to obtain a class, which is a class 1 and class 2 support vector machine at this time, and the class is determined to be class 1, and the number of votes in class 1 is added by 1.
And 4) judging that the support vector machine is not the last support vector machine currently, repeating the step 1), the step 2) and the step 3) until the classification of all the support vector machines is completed, and paying attention to the support vector machines and the coefficients respectively corresponding to the support vector machines when each repetition is carried out.
And 5) comparing the ticket values of all the categories of the current category ticket number storage unit, wherein the category 1 ticket, the category 2 ticket, the category 3 ticket and the category 0 ticket are the category 1.
The reconfigurable multi-classification support vector machine system of the embodiment fully utilizes the parallelism of hardware, supports the reconfiguration of computing resources, enables the hardware to be cut, supports parallel and pipeline operation, improves the operation speed of support vector machine classification, and has good flexibility for samples with different feature numbers.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (5)

1. A reconfigurable multi-class support vector machine system, comprising:
the main control module is used for controlling the whole calculation flow and providing reconstruction information of calculation resources;
the storage control module is used for controlling the data storage of each operation module;
the kernel function operation module receives the test data and the support vector, calculates a kernel function between the test data and the support vector, and obtains an operation result;
the class operation module calculates a multiplication and accumulation result of the kernel function and the corresponding Lagrange multiplier coefficient according to the operation result, adds bias to the multiplication and accumulation result to obtain a decision value and class, and adds 1 to the corresponding class ticket number;
the result comparison module is used for comparing the ticket numbers of all the categories to obtain the final classification result of the test data;
the main control module receives a system starting signal and completes ticket taking classification operation of a single vector machine, and the operation is specifically as follows: starting a kernel function operation module, starting a category operation module after kernel function operation is finished, calculating to obtain categories, adding 1 to corresponding category ticket numbers, storing by a storage control module, judging whether the current support vector machine is the last support vector machine by a main control module, and starting a result comparison module if the current support vector machine is the last support vector machine to obtain a final classification result; otherwise, repeating the ticket taking and classifying operation of the single vector machine.
2. The reconfigurable multi-class support vector machine system of claim 1, wherein: the kernel function operation module and the class operation module share computing resources, and the kernel function operation module and the class operation module are respectively formed by changing the topological structure and the interconnection relation of basic operation units of the computing resources.
3. The reconfigurable multi-class support vector machine system of claim 2, wherein: the basic operation unit comprises an adder, a multiplier, an accumulator and an exponent operator.
4. The reconfigurable multi-class support vector machine system of claim 2, wherein: in the kernel function operation module, the computing resource is reconstructed into a plurality of parallel computing units, each parallel computing unit consists of an adder, a multiplier, an accumulator and an exponential arithmetic unit, and each parallel computing unit calculates a kernel function.
5. The reconfigurable multi-class support vector machine system of claim 2, wherein: and the class operation module reconstructs the computing resources into a multi-input multiply-add tree structure and carries out parallel acceleration on the computation of the decision value.
CN201910528797.5A 2019-06-18 2019-06-18 Reconfigurable multi-classification support vector machine system Active CN110427960B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910528797.5A CN110427960B (en) 2019-06-18 2019-06-18 Reconfigurable multi-classification support vector machine system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910528797.5A CN110427960B (en) 2019-06-18 2019-06-18 Reconfigurable multi-classification support vector machine system

Publications (2)

Publication Number Publication Date
CN110427960A CN110427960A (en) 2019-11-08
CN110427960B true CN110427960B (en) 2023-04-25

Family

ID=68408684

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910528797.5A Active CN110427960B (en) 2019-06-18 2019-06-18 Reconfigurable multi-classification support vector machine system

Country Status (1)

Country Link
CN (1) CN110427960B (en)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016108962A1 (en) * 2014-12-30 2016-07-07 Battelle Memorial Institute Fast efficient evaluation of messages on automotive networks using look-up tables
CN106845401B (en) * 2017-01-20 2020-11-03 中国科学院合肥物质科学研究院 Pest image identification method based on multi-space convolution neural network
CN106874958B (en) * 2017-02-28 2019-09-10 中南大学 A kind of data processing method of the system based on economical CPU platform

Also Published As

Publication number Publication date
CN110427960A (en) 2019-11-08

Similar Documents

Publication Publication Date Title
Zhang et al. BoostGCN: A framework for optimizing GCN inference on FPGA
WO2022068623A1 (en) Model training method and related device
US20230297846A1 (en) Neural network compression method, apparatus and device, and storage medium
CN111967468A (en) FPGA-based lightweight target detection neural network implementation method
EP4080416A1 (en) Adaptive search method and apparatus for neural network
CN109543662A (en) Object detection method, system, device and the storage medium proposed based on region
CN110991658A (en) Model training method and device, electronic equipment and computer readable storage medium
CN111709295A (en) SSD-MobileNet-based real-time gesture detection and recognition method and system
CN112116001A (en) Image recognition method, image recognition device and computer-readable storage medium
EP3649582A1 (en) System and method for automatic building of learning machines using learning machines
RU2013134325A (en) DEVICE AND METHOD FOR RECOGNITION OF GESTURES ON THE BASIS OF ANALYSIS OF MANY POSSIBLE SECTION BORDERS
Nazemi et al. Nullanet: Training deep neural networks for reduced-memory-access inference
Dong et al. Heatvit: Hardware-efficient adaptive token pruning for vision transformers
Wang et al. Briefly Analysis about CNN Accelerator based on FPGA
Dias et al. Deep learning in reconfigurable hardware: A survey
CN114462591A (en) Inference method for dynamic quantitative neural network
CN110427960B (en) Reconfigurable multi-classification support vector machine system
CN111815209A (en) Data dimension reduction method and device applied to wind control model
Zhuang et al. Vlsi architecture design for adder convolution neural network accelerator
WO2022188711A1 (en) Svm model training method and apparatus, device, and computer-readable storage medium
CN115392441A (en) Method, apparatus, device and medium for on-chip adaptation of quantized neural network model
Yi A Hardware Implementation of SOM Neural Network Algorithm
CN110415006B (en) Advertisement click rate estimation method and device
Tajiri et al. FPGA acceleration of a composite kernel SVM for hyperspectral image classification
US20240054369A1 (en) Ai-based selection using cascaded model explanations

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant