CN107862381A - A kind of FIR filter suitable for a variety of convolution patterns is realized - Google Patents

Abstract

The invention discloses a FIR filter applicable to multiple convolution modes and its hardware implementation. The structure can support the mainstream convolution operations in the current convolutional neural network, such as 3*3 and 5 with a step size of 1. *5 convolution calculation and 3*3 convolution operation with a step size of 2, etc., and use 6 parallel fast FIR algorithms to reduce hardware consumption, reduce convolution calculation complexity, and improve data throughput. The present invention has completed the derivation of the hardware structure of three parallel convolution operations with steps of 2, and combines it with the hardware structure of 6 parallel fast FIR filters on the basis of not increasing the adder and multiplier, so that the structure can be adapted to In each mode, hardware resources are greatly utilized. The present invention can realize most of the current mainstream convolutional neural network calculations through different configurations of the single hardware structure, improves hardware utilization, has high versatility, and simplifies the hardware implementation of the convolutional neural network.

Description

A FIR Filter Implementation Suitable for Various Convolution Modes

technical field

The present invention relates to the field of computer and electronic science, in particular to the hardware implementation of convolutional neural network in the field of deep learning, a general architecture and hardware implementation compatible with step 1 convolution calculation and step 2 convolution operation.

Background technique

Convolutional Neural Network (CNN) has become one of the most popular and widely used deep learning algorithms due to its excellent performance in image, audio and other fields. With the rapid development of convolutional neural networks in recent years, the application of large convolution kernels in models has become less and less. At present, the most widely used convolution operations in various models are 3*3 and 5*5 convolution operations, and the pace The convolution operation of 2 is also used by more and more models. However, for the convolution operation with a step of 2, there has not been a good hardware implementation optimization solution. The traditional convolution operation with a stride of 1 can use the fast FIR algorithm to improve parallelism and reduce multiplier resources.

The polynomial representation of an N-tap FIR filter in the time domain is:

Or in the z domain it can be expressed as

If the FIR filter coefficient sequence {h(n)} with a length of N is used as the coefficient of N-dimensional discrete convolution, the FIR filter can realize an N-dimensional convolution operation. Through the combination of N filters, the convolution operation of N*N in the convolutional neural network can be realized. The fast FIR algorithm can achieve high parallelism, and achieve low complexity by adding adders and reducing multipliers. However, this method is not suitable for the convolution operation with a step size of 2. Using this method to calculate and selectively output the convolution with a step size of 2 will cause a serious waste of hardware resources. There are About 50% of the hardware resources have no effect on the calculation results. Therefore, a general-purpose hardware architecture that can realize both the traditional convolution operation with a step size of 1 and the convolution operation with a step size of 2, and has low complexity, high parallelism, and high hardware resource utilization will become a need.

Contents of the invention

Aiming at the above problems, the present invention proposes a convolution calculation framework compatible with a step size of 1 and a step size of 2 on the fast FIR algorithm framework and its hardware implementation. The present invention implements three calculation modes on a hardware architecture, which are 6-tap 6 parallel convolution calculation, three independent 3-tap 3 parallel convolution operations, and 2 independent 3-tap 3-tap with a step size of 2. Parallel convolution computation. The present invention has high versatility, and can realize most of the current mainstream convolution operations through different configurations of the hardware architecture. Concrete invention content is as follows:

A FIR filter applicable to multiple convolution modes, its hardware architecture includes:

1) The data input selection unit, for different convolution modes, reselects and arranges the input data and inputs it to the corresponding convolution calculation module.

2) The convolution calculation unit, the basic unit is 3 parallel 3-tap fast FIR filters, and a data selector is inserted to control the data flow for different convolution operations.

3) The post-convolution calculation unit processes and calculates the output of the convolution calculation unit so as to realize the cascading of multiple independent constituent units in the convolution calculation unit to form a multi-parallel multi-tap fast FIR filter.

4) The data output selection unit, for different convolution modes, selects the corresponding calculation results as the module output.

The second calculation mode of the present invention is three independent 3-tap 3-parallel fast FIR algorithm hardware structures, wherein the 3-tap 3-parallel fast FIR hardware structure is the most basic component module, and each output Y and input can be obtained by formula derivation The relationship between X:

Y ₀ ＝H ₀ X ₀ -z ^-3 H ₂ X ₂ +z ^-3 [(H ₁ +H ₂ )(X ₁ +X ₂ )-H ₁ X ₁ ]

Y ₁ ＝[(H ₀ +H ₁ )(X ₀ +X ₁ )-H ₁ X ₁ ]-[H ₀ X ₀ -z ^-3 H ₂ X ₂ ]

Y ₂ ＝[H ₀ +H ₁ +H ₂ )(X ₀ +X ₁ +X ₂ )]-[(H ₀ +H ₁ )(X ₀ +X ₁ )-H ₁ X ₁ ]-[(H ₁ +H ₂ )(X ₁ +X ₂ )-H ₁ X ₁ ]

For a 3-tap 3 parallel convolution operation with a step size of 2, the relationship between each output Y and input X can be deduced as:

Y ₀ ＝H ₀ X ₀ +z ^-6 (H ₂ X ₄ +H ₁ X ₅ )

Y ₁ ＝H ₂ X ₀ +H ₁ X ₁ +H ₀ X ₂

Y ₂ =H ₂ X ₂ +H ₁ X ₃ +H ₀ X ₄

Through the multiplexing of multipliers and adders, and adding a multiplexer, the convolution operation with a step size of 2 and the fast FIR algorithm are combined to achieve high hardware resource utilization, high parallelism, and high versatility. The revolutionary hardware architecture of convolution operation.

Description of drawings

Figure 1 is a hardware structure diagram of 3 parallel convolution calculations with a step size of 2.

Fig. 2 is a 3-tap 3-parallel fast FIR filter calculation hardware structure diagram.

Fig. 3 is an overall structure diagram of the present invention.

FIG. 4 is a hardware structure diagram of an FIR filter applicable to multiple convolution modes according to the present invention.

Detailed ways

The specific implementation of the present invention will be described further below in conjunction with the accompanying drawings. The convolution operation with a step size of 2 and the convolution operation with a step size of 1 are very different in implementation, because each input data and The last data interval is 2, which makes half of the calculation results useless data due to the discontinuity of the data when the fast FIR algorithm is used for parallel calculations. If the fast FIR algorithm is not used, the relationship between input and output:

Y ₀ ＝H ₀ X ₀ +z ^-6 (H ₂ X ₄ +H ₁ X ₅ )

Y ₁ ＝H ₂ X ₀ +H ₁ X ₁ +H ₀ X ₂

Y ₂ =H ₂ X ₂ +H ₁ X ₃ +H ₀ X ₄

It can be obtained that if the convolution operation with a step size of 2 is made into 3 parallels, its hardware structure is shown in Figure 1.

The hardware structure uses 9 multipliers and 6 adders in total, and every 6 data inputs are calculated to obtain 3 outputs. Compared with the fast FIR algorithm, it achieves high utilization of hardware resources under the same parallel conditions.

A hardware structure of a fast FIR filter is shown in Figure 2. The hardware structure reduces the complexity of hardware implementation and achieves high parallelism by adding adders and reducing multipliers. For the traditional convolution operation with a step size of 1, this hardware structure can realize efficient operation.

The present invention realizes the efficient support for the convolution operation with a step of 1 and a step of 2 by combining the above two hardware structures, and has low complexity, high hardware utilization rate and strong versatility under the premise of high parallelism Features. Fig. 3 is the hardware architecture of the present invention, as shown in the figure, the architecture includes four modules:

1) The data input selection unit, for different convolution modes, reselects and arranges the input data and inputs it to the corresponding convolution calculation module.

2) The convolution calculation unit, the basic unit is 3 parallel 3-tap fast FIR filters, and a data selector is inserted to control the data flow for different convolution operations.

3) The post-convolution calculation unit processes and calculates the output of the convolution calculation unit so as to realize the cascading of multiple independent constituent units in the convolution calculation unit to form a multi-parallel multi-tap fast FIR filter.

4) The data output selection unit, for different convolution modes, selects the corresponding calculation results as the module output.

The specific hardware circuit structure is shown in Figure 4. In the present invention, by multiplexing multipliers and adders and adding multiplexers, the fusion of three parallel convolution computing hardware structures with a step size of 2 and three parallel fast FIR filters is realized without adding a multiplication Compatible with two convolution modes with a stride of 2 and a stride of 1 are realized under the condition of an adder and an adder. Three 3-parallel fast FIR filters are cascaded to realize a 6-parallel fast FIR filter. The present invention has three working modes to choose from. The first one is three independent 3-parallel 3-tap fast FIR filters. By setting the tap coefficient H, a 3*3 convolution operation can be realized. The second is two independent 3-tap convolution operations with a stride of 2. The third is a single 6-parallel fast FIR filter. A 5*5 convolution operation can be realized by setting the last tap coefficient of the third mode to 0. Just input data X, tap coefficient H, and working mode M into the hardware circuit, the present invention will automatically allocate X and H according to the input mode, input them to each sub-module for calculation, and obtain the result The data matching the pattern is selected by the multiplexer as the result output.

Of course, the present invention is not only applicable to the calculation of the convolutional neural network, but also has many applicable scenarios in the fields of digital image processing, digital signal processing, wireless communication, etc., and the multiplier and adder are multiplexed and inserted into the multiplexer The improvement of some other filters can realize the matching of more convolution calculation modes. For those skilled in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications should also be regarded as the protection scope of the present invention. All components that are not specified in this embodiment can be realized by existing technologies.

Claims (1)

1. A kind of FIR filter applicable to multiple convolution modes, its hardware architecture includes: 1) The data input selection unit, for different convolution modes, reselects and arranges the input data and inputs it to the corresponding convolution calculation module. 2) The convolution calculation unit, the basic unit is 3 parallel 3-tap fast FIR filters, and a data selector is inserted to control the data flow for different convolution operations. 3) The post-convolution calculation unit processes and calculates the output of the convolution calculation unit so as to realize the cascading of multiple independent constituent units in the convolution calculation unit to form a multi-parallel multi-tap fast FIR filter. 4) The data output selection unit, for different convolution modes, selects the corresponding calculation results as the module output.