CN110909793B - Decision forest system and decision forest reasoning method based on FPGA - Google Patents

Abstract

The invention discloses a decision forest system and a decision forest reasoning method based on FPGA (field programmable gate array), belonging to the field of machine learning. The invention realizes the decision tree on the FPGA based on the architecture of the path backtracking module and the classification module, is beneficial to automatically converting the MATLAB code of the trained decision tree into the Verilog code, and does not need to manually realize a large number of decision trees in an EDA tool manually. And a top-down design structure is adopted, each decision tree is used as a submodule to run in parallel, the precision is ensured while the decision trees are deployed rapidly, and less hardware resources are consumed. Based on a pipeline technology, a majority voting method is realized on the FPGA, a decision tree submodule is called in a top-level module, and the results of the submodule are processed in a unified mode, so that the reasoning speed is improved. The asynchronous FIFO module is used for transmitting data across clock domains, so that the application of the asynchronous FIFO module is wider. The inference implementation method provided by the invention is suitable for the decision tree generated by any training algorithm in principle.

Description

Decision forest system and decision forest reasoning method based on FPGA

Technical Field

The invention belongs to the technical field of machine learning, and particularly relates to a decision forest system and a decision forest reasoning method based on FPGA.

Background

Ensemble learning is an important method in machine learning. Ensemble learning often achieves significantly superior generalization performance over a single learner by combining multiple learners. The entire ensemble learning does not rely on any single model, but rather makes predictions collectively. Decision forests are one of the best-known representatives of the parallel ensemble learning method, and their diversity is obtained by using different subsets of training data. The decision forest is an integrated learning method which is formed by taking a decision tree as a base learning device and taking a majority voting method as an integrated mode, and random attribute selection can be further introduced in the training process of the decision tree to enhance generalization performance.

Although decision forests exhibit great performance in many real-world tasks, its shortcomings have hampered the widespread use of decision forests, particularly in hardware-based scenarios. To generate a classification of unlabeled data instances, the instances need to be classified per model before a final classification is obtained. Thus, for some applications where reasoning speed is a major issue (e.g., mining data streams), classification integration may not be timely generated.

The decision tree is used as a base learner in ensemble learning, and the decision is judged and decided by utilizing a tree structure, wherein the decision tree can be a classification tree or a regression tree. In the training process of the decision tree, the nodes need to be subjected to optimal attribute division, and there are many common optimal attribute division strategies, such as based on information gain, a kini coefficient, and the like. In the prior art, algorithms used in various decision forest training processes are disclosed, but the implementation method of reasoning in the FPGA is not mature yet and needs to be further improved.

Disclosure of Invention

Aiming at the defects and improvement requirements of the prior art, the invention provides a decision forest system and a decision forest reasoning method realized based on an FPGA (field programmable gate array), aiming at greatly improving the reasoning speed of a decision forest in hardware deployment on the premise of ensuring precision, solving the performance requirement of the decision forest and simultaneously enabling the decision forest and similar technologies to be widely and practically applied through the design of a clock domain crossing.

To achieve the above object, according to a first aspect of the present invention, there is provided a decision forest system implemented based on an FPGA, the decision forest system comprising:

the system comprises a plurality of decision tree subsystems based on the FPGA, an asynchronous FIFO module and a majority voting method subsystem based on the FPGA;

each decision tree subsystem is independent and runs in parallel, and each decision tree subsystem comprises: the system comprises a first clock controller, a path backtracking module and a classification module; wherein the content of the first and second substances,

the first clock controller is a synchronous clock and is used for controlling the path backtracking module and the classification module;

the path backtracking module is used for acquiring all backtracking paths from leaf nodes to root nodes in a trained decision tree and obtaining a classification path of data to be classified in the decision tree;

the classification module is used for obtaining a classification result of the data to be classified in the decision tree according to the classification path output by the path backtracking module and transmitting the classification result to a majority voting method subsystem based on the FPGA;

the asynchronous FIFO module is used for transmitting data across clock domains between the decision tree subsystems and the majority voting method subsystems;

the majority voting subsystem comprises: the second clock controller, count the module, comparator and output module of voting result of the classification quantity; wherein the content of the first and second substances,

the second clock controller is a synchronous clock and is used for controlling the counting module, the category number comparator and the voting result output module;

the counting module is used for counting the classification results obtained by each classification module according to the categories;

the category number comparator is used for finding out the maximum number value from the counting result obtained by the counting module;

and the voting result output module is used for outputting the classification category corresponding to the maximum quantity.

Specifically, for the "and" of the node phase in a single path, and for the "or" of the different path phases of the same classification category, the classification path of the classification category is obtained.

Specifically, the variable corresponding to which classification path is "true", that is, the classification result is the category corresponding to the classification path.

Specifically, the clock frequency of the second clock controller is ≧ the clock frequency of the first clock controller.

Specifically, the entire majority voting subsystem is implemented using pipelining, which requires N stages if the module needs to consume N clock cycles.

To achieve the above object, according to a second aspect of the present invention, a decision forest reasoning method for a decision forest system according to the first aspect comprises the steps of:

s1, performing parallel classification prediction on data to be classified through a plurality of decision trees constructed by a decision tree subsystem to obtain a plurality of classification results;

s2, obtaining the maximum numerical value in the classification results of the decision trees through the pipeline operation in the majority voting subsystem;

and S3, taking the classification category corresponding to the maximum quantity value as an inference result of the whole decision forest system.

Specifically, step S2 includes the steps of:

s21, obtaining data streams from a decision tree subsystem through asynchronous FIFO module cache to realize clock domain crossing transmission of data;

s22, counting the classification results obtained by the decision trees in different classes by a counting module, and using multi-stage flowing water;

and S23, finding out the maximum value of the number from the counting result obtained by the counting module through the category number comparator, and using multi-stage running water.

Generally, by the above technical solution conceived by the present invention, the following beneficial effects can be obtained:

(1) the invention realizes the decision tree on the FPGA based on the architecture of the path backtracking module and the classification module, and the architecture is favorable for automatically converting the MATLAB code of the trained decision tree into the Verilog code without manually realizing a large number of decision trees in an EDA tool.

(2) The invention adopts a top-down design structure, and each decision tree is used as a submodule to run in parallel, thereby realizing large-scale rapid deployment in the FPGA and simultaneously ensuring the precision, and less hardware resources are consumed.

(3) The invention realizes a majority voting method on the FPGA based on a pipeline technology, calls a decision tree submodule in a top-level module of the majority voting method and performs unified processing on the result of the submodule, thereby improving the reasoning speed.

(4) The invention meets the requirements of different clock domains of a decision tree subsystem and a majority voting method subsystem which are classified in parallel through an asynchronous FIFO module, achieves the purpose of cross-clock domain data transmission, and has wider application.

(5) The inference implementation method provided by the invention is suitable for the decision tree generated by any training algorithm in principle.

Drawings

Fig. 1 is a schematic structural diagram of a decision forest system implemented based on an FPGA according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a decision tree subsystem according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a majority voting subsystem according to an embodiment of the present invention;

FIG. 4 is a Verilog pseudo-code description diagram of an embodiment of a decision tree provided by an embodiment of the present invention;

fig. 5 is a diagram of a classification simulation result of the decision forest embodiment provided in the embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

The overall concept of the invention is as follows: the decision forest reasoning is carried out in the FPGA by constructing two important components of a decision forest system, namely a decision tree subsystem and a majority voting method subsystem. The decision tree is realized based on the architecture of the path backtracking module and the classification module, and the majority voting method is realized based on the pipeline technology.

As shown in fig. 1, the present invention provides a decision forest system implemented based on an FPGA, which includes:

a plurality of decision tree subsystems based on FPGA, each decision tree subsystem being independent and operating in parallel, as shown in fig. 2, each decision tree subsystem comprising: the system comprises a first clock controller, a path backtracking module and a classification module;

the first clock controller is a synchronous clock and is used for controlling a path backtracking module and a classification module, the path backtracking module is used for acquiring all backtracking paths from leaf nodes to root nodes in a trained decision tree and acquiring classification paths of data to be classified in the decision tree, and the classification module is used for acquiring classification results of the data to be classified in the decision tree according to the classification paths output by the path backtracking module and transmitting the classification results to a majority voting method subsystem based on an FPGA;

an FPGA-based majority voting subsystem, as shown in fig. 3, comprising: the second clock controller, count the module, comparator and output module of voting result of the classification quantity;

the second clock controller is a synchronous clock and is used for controlling a counting module, a category number comparator and a voting result output module, the counting module is used for counting the classification results obtained by the classification modules according to categories, the category number comparator is used for finding out the maximum number value from the counting results obtained by the counting module, and the voting result output module is used for outputting the classification category corresponding to the maximum number value.

An asynchronous FIFO module for cross-clock-domain data transfer between the plurality of decision tree subsystems and the majority voting subsystem.

In the invention, a decision forest system adopts a top-down design structure, each decision tree is used as a submodule to run in parallel, and then the submodules are called in a top-level module of a majority voting method and the results of the submodules are processed in a unified way. Specifically, in the implementation of the FPGA, in order to meet the requirements of different clock domains where a decision tree subsystem and a majority voting subsystem which are classified in parallel are located, an asynchronous FIFO is deployed to achieve the purpose of cross-clock domain data transmission.

A decision tree is a common machine learning method, which can be used for regression or classification, and is usually generated by steps of feature selection, decision tree generation, decision tree pruning and the like, and a plurality of decision trees can be generated by pre-training using an MATLAB tool, wherein each trained decision tree corresponds to one decision tree subsystem.

Specifically, the selection of the FPGA needs to meet the resource consumption requirements of all implemented decision tree logic occupation and IO port occupation, and the resources of the FPGA include an LUT, a Register, a BRAM, an IOB, and the like.

The decision tree subsystem includes: the system comprises a first clock controller, a path backtracking module and a classification module.

Specifically, the clock generated by the first clock controller can meet the combinational logic delay requirement of the path backtracking module, and furthermore, the larger combinational logic can be split by utilizing the hierarchical pipeline to ensure high-speed performance.

Specifically, the path backtracking module finds a path from the root node to the leaf node by backtracking each leaf node in the decision tree, and performs an or operation on the paths of the same classification category in the leaf nodes (actually, the whole is boolean logic, and a logic synthesizer in the EDA tool automatically optimizes the boolean logic) to obtain a plurality of classification paths. The paths from the root node to the leaf nodes are all AND operations after comparison between the attribute variables and the separation values, and the basis for the separation of the nodes in the tree is generally a Keyny coefficient.

Specifically, the classification module performs time sequence determination on the logic obtained by the path backtracking module, and at a certain time, a variable corresponding to which classification path is "true", that is, the classification result is a category corresponding to the classification path.

A majority voting method, wherein if the statistical number (namely the number of votes) of a certain classification category is the maximum, the classification category is used as the output of a decision forest; if there are multiple classification categories for which the highest ticket is obtained at the same time, then the classification category for which the earliest statement is output in the order of the statements, which may be described as

x represents data to be classified, h represents decision tree, T represents number of decision trees in decision forest, c_jRepresenting the jth category and H the decision forest.

The majority voting system comprises: a second clock controller, a counting module, and a class number comparator.

In particular, the clock frequency of the second clock controller is not necessarily exactly the same as in a decision tree system. In general, pipeline-based majority voting systems can achieve higher clock frequencies than decision tree systems because the combinatorial logic delay of each block in the majority voting system is relatively low.

Specifically, the counting module is configured to count classification results obtained from the plurality of decision trees in different categories. When the number of decision trees is larger, the number of clock cycles required by the counting module is correspondingly increased. For example, when there are 10 decision trees, the counting module takes 2 clock cycles; if there are 200 decision trees, the counting module takes 4 clock cycles, i.e. four levels are needed for the flow.

Specifically, the category number comparator is used for finding out the maximum value of the number from the counting result obtained by the counting module. The number of clock cycles required by the counting module varies according to the category, which is a flexible design. For example, the class is only 2, the counting module takes 1 clock cycle; if the class has 4 classes, the counting module takes 2 clock cycles, i.e. two stages are required for the flow.

The whole majority voting subsystem is realized by a pipeline technology, and if a module needs to consume N clock cycles, the pipeline needs N stages.

The asynchronous FIFO module comprises a dual-port RAM, is used for data recording and transmission between clock domains, ensures correct transmission between data, and can directly call an IP core in an EDA tool to realize.

The invention also provides a decision forest reasoning method based on the system, which comprises the following steps:

s1, carrying out parallel classification prediction on data to be classified through a plurality of decision trees constructed by a decision tree subsystem to obtain a plurality of classification results.

This is further illustrated by the following examples: assume that the decision tree input attribute is expressed as { x }₁,x₂,x₃,x₄And the classification category is {0,1,2}, and three variables representing the classification category are represented as { class0, class1, class2 }. If the leaf node with classification category 0 has N₀Then there is N₀The associated paths are respectively expressed as {1 }₀,2₀,...,N₀}; similarly, a leaf node with a classification category of 1 has N₁The leaf node with classification category 2 has N₂And (4) respectively. The Verilog pseudo code description of the embodiment of the decision tree is shown in fig. 4, wherein the path backtracking module mainly comprises combinational logic, and the classification module mainly comprises sequential logic.

Further, lines 1-3 of FIG. 4 show the traversal of the path for each classification category, and the OR operation of all paths for the same classification. Unless otherwise specified, only one of class0, class1, class2 is "true" and the remaining two are "false" after the test data is entered, i.e., the classification result must be unique. Lines 4-9 represent the class category that is found to be "true" out of three of class0, class1, class2 as output, at the rising edge of each clock.

And S2, obtaining the maximum numerical value in the classification results of the decision trees through the pipeline operation in the majority voting method subsystem.

Step S2 includes the following steps:

and S21, obtaining data stream from the decision tree subsystem through asynchronous FIFO module cache, and realizing clock domain crossing transmission of data.

S22, counting the classification results obtained by the decision trees in different classes through a counting module, and using multi-stage flowing water.

And S23, finding out the maximum value of the number from the counting result obtained by the counting module through the category number comparator, and using multi-stage running water.

And S3, taking the classification category corresponding to the maximum quantity value as an inference result of the whole decision forest system.

The results are further illustrated by the following examples.

The conditions of this example are as follows: the Fisher's Iris dataset has 4 input attributes and 3 classification categories, denoted as {0,1,2}, and has 150 groups of data. And randomly extracting 135 groups from the decision tree, and pre-training the groups according to a decision forest algorithm to obtain 10 decision trees. Based on the 10 decision trees, the decision forest reasoning process is realized on an XC7A35TFTG256-1 development board in a XILINX Artix-7 series, and a system realized by the FPGA is tested by using the remaining 15 groups of data.

Further, there are 10 decision trees in this embodiment, 3 classification categories. The counting module of the majority voting system in this embodiment uses two-stage pipelining, and the category number comparator also uses two-stage pipelining.

Further, the embodiment realizes a high-speed system with a clock frequency of 100MHz, and the test classification simulation result of the whole decision forest embodiment is shown in fig. 5. Four black bold lines are identified in the figure, the first line from left to right representing the classification of the first set of test data by the decision tree from this moment; then, the time when the second line is reached after 6 clock cycles (6 pipelines) is passed, which represents that the result of the first voting is started to be output; a clock cycle is formed between the second line and the third line, the voting result of the first group of data lasts for one clock cycle, and then the second voting result is output; the second voting result is output until the fourth line, and the voting results of the 15 groups of test data are output one by one in sequence. From the waveform diagram, the results of 15 votes are 021111220000110 in turn, and the results of the majority voting method can be found to be expected by comparing the classification category results of the next 10 decision trees.

It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (6)

1. A decision forest system realized based on FPGA is characterized by comprising:

the system comprises a plurality of decision tree subsystems based on the FPGA, an asynchronous FIFO module and a majority voting method subsystem based on the FPGA;

each decision tree subsystem is independent and runs in parallel, and each decision tree subsystem comprises: the system comprises a first clock controller, a path backtracking module and a classification module; wherein the content of the first and second substances,

the first clock controller is a synchronous clock and is used for controlling the path backtracking module and the classification module;

the path backtracking module is used for acquiring all backtracking paths from leaf nodes to root nodes in a trained decision tree and obtaining classification paths of data to be classified in the decision tree, and the specific steps are as follows:

for the 'AND' of the node phase in the single path, for the 'OR' of the different path phases of the same classification category, obtaining the classification path of the classification category;

the classification module is used for obtaining a classification result of the data to be classified in the decision tree according to the classification path output by the path backtracking module and transmitting the classification result to a majority voting method subsystem based on the FPGA;

the asynchronous FIFO module is used for transmitting the cross-clock-domain data between the decision tree subsystems and the majority voting method subsystems;

the majority voting subsystem comprises: the second clock controller, count the module, comparator and output module of voting result of the classification quantity; wherein, the first and the second end of the pipe are connected with each other,

the second clock controller is a synchronous clock and is used for controlling the counting module, the category number comparator and the voting result output module;

the counting module is used for counting the classification results obtained by the classification modules according to the categories;

the category number comparator is used for finding out the maximum number value from the counting result obtained by the counting module;

and the voting result output module is used for outputting the classification category corresponding to the maximum quantity.

2. The system of claim 1, wherein the variable corresponding to which classification path is true indicates that the classification result is the class corresponding to the classification path.

3. The system of claim 1, wherein the clock frequency of the second clock controller is greater than or equal to the clock frequency of the first clock controller.

4. The system of claim 1, wherein the entire majority voting subsystem is implemented in a pipelined technique, wherein if a module takes N clock cycles, then N stages are required for pipelining.

5. A decision forest reasoning method based on a decision forest system as claimed in any one of claims 1 to 4, characterised in that the method comprises the steps of:

s1, performing parallel classification prediction on data to be classified through a plurality of decision trees constructed by a decision tree subsystem to obtain a plurality of classification results;

s2, obtaining the maximum numerical value in the classification results of the decision trees through the pipeline operation in the majority voting subsystem;

and S3, taking the classification category corresponding to the maximum quantity value as an inference result of the whole decision forest system.

6. The method of claim 5, wherein the step S2 includes the steps of:

s21, obtaining data streams from a decision tree subsystem through asynchronous FIFO module cache to realize clock domain crossing transmission of data;

s22, counting the classification results obtained by the decision trees in different classes by a counting module, and adopting multi-stage flow;

and S23, finding out the maximum value of the number from the counting result obtained by the counting module through the category number comparator, and adopting multi-stage running water.

Images