CN113541647A

CN113541647A - Filter and design method thereof

Info

Publication number: CN113541647A
Application number: CN202110745358.7A
Authority: CN
Inventors: 谷涛; 黄源浩; 肖振中
Original assignee: Orbbec Inc
Current assignee: Orbbec Inc
Priority date: 2021-07-01
Filing date: 2021-07-01
Publication date: 2021-10-22
Also published as: WO2023273095A1

Abstract

The application is applicable to the technical field of digital signal processing, and particularly relates to a filter and a design method thereof, wherein the method comprises the following steps: acquiring a first filter, and carrying out zero calculation on the first filter; and grouping the zero points of the first filter according to a preset cascade filter architecture, and acquiring a second filter according to a grouping result. The embodiment of the application can reduce the occupied chip area and reduce the power consumption.

Description

Filter and design method thereof

Technical Field

The present application relates to the field of digital signal processing technologies, and in particular, to a filter and a design method thereof.

Background

In image processing, it is often necessary to perform filtering processing on an image. The filtering process is usually implemented by a two-dimensional (2D) image filter such as gaussian filtering or mean filtering in an Application Specific Integrated Circuit (ASIC) chip of an image processing system. When processing images of different resolutions, filters with different convolution kernel sizes (sizes) are often required, for example, 5 × 5 filters for video image adapter (VGA) images, 3 × 3 filters for quarter-size VGA (QVGA) images, and 9 × 9 filters for high resolution (HD) images.

In the ASIC chip design, in order to satisfy the difference of image resolutions at the same time, the first scheme often needs to implement filters with various convolution kernel sizes such as 3 × 3, 5 × 5, 7 × 7 and 9 × 9 at the same time; the second scheme, for example, designs a 9 × 9filter, and implements the filtering function of a 3 × 3, 5 × 5, or 7 × 7filter by configuring the filter coefficients.

In the first scheme, several filters of convolution kernel size are implemented at the same time, but the more kinds of convolution kernels are designed, the larger chip area is consumed. In the second scheme, although the consumed area is small, the power consumption in the actual filtering process is high, and especially when a large convolution kernel filter is used to perform only a small convolution kernel filter function, the power consumption is higher, for example, when a 9 × 9filter is used to perform a 3 × 3filter function, coefficients with (9 × 9-3 × 3)/(9 × 9) ═ 88.9% in the filter weights are all zero, and the utilization rate of a Multiply Accumulate (MAC) unit is only 11.1%, which is inefficient, resulting in increased power consumption.

Disclosure of Invention

In view of this, embodiments of the present application provide a filter and a design method thereof, which can reduce occupied chip area and reduce power consumption.

In a first aspect, an embodiment of the present application provides a filter design method, including:

acquiring a first filter, and carrying out zero calculation on the first filter;

and grouping the zero points of the first filter according to a preset cascade filter architecture, and acquiring a second filter according to a grouping result.

In this embodiment, according to the first filter (or called original filter), the second filter (or called target filter) having the filtering effect equivalent to that of the original filter is obtained in the preset cascaded filter architecture, so that the occupied chip area is reduced, and the power consumption is reduced.

As an implementation manner of the first aspect, the performing zero point calculation on the first filter includes:

if the first filter is a row-column separable filter, performing row-column separation on the first filter to obtain a row coefficient vector and a column coefficient vector;

and respectively constructing a first sub-filter and a second sub-filter by using the column coefficient vector and the row coefficient vector, and performing zero point calculation.

As an implementation manner of the first aspect, if the first filter is a row-column separable filter, the grouping zeros of the first filter according to a preset cascade filter architecture, and acquiring the second filter according to a grouping result includes:

grouping zeros of the first sub-filter and the second sub-filter according to a preset cascade filter architecture to construct a third sub-filter and a fourth sub-filter;

recombining said third sub-filter and said fourth sub-filter to obtain a second filter bank.

As an implementation manner of the first aspect, the constructing a first sub-filter and a second sub-filter by using the column coefficient vector and the row coefficient vector, and performing zero point calculation includes:

respectively constructing a first sub-filter and a second sub-filter by utilizing the column coefficient vector and the row coefficient vector;

and respectively carrying out zero point calculation on the first sub-filter and the second sub-filter.

As an implementation manner of the first aspect, the performing zero point calculation on the first filter further includes:

and if the first filter is a filter with inseparable ranks, carrying out zero point calculation on the first filter.

As an implementation manner of the first aspect, the preset cascaded filter architecture includes three filtering units and a selecting unit;

the three filtering units comprise a first 5 multiplied by 5 filtering unit, a first 3 multiplied by 3 filtering unit and a second 3 multiplied by 3 filtering unit; the selection unit comprises a first multi-path selection module and a second multi-path selection module;

the first multi-path selection module comprises a first path of signal and a second path of signal, and the second multi-path selection module comprises a third path of signal, a fourth path of signal and a fifth path of signal;

the input end of the first 5 × 5 filtering unit is connected to the first path of signals, the output end of the first 5 × 5 filtering unit is connected to the input end of the first 3 × 3 filtering unit and the fifth path of signals, the output end of the first 3 × 3 filtering unit is connected to the second path of signals and the fourth path of signals, the input end of the second 3 × 3 filtering unit is connected to the first multi-path selection module, and the output end of the second 3 × 3 filtering unit is connected to the third path of signals.

As an implementation manner of the first aspect, the preset cascaded filter architecture includes four filtering units and a selecting unit;

the four filtering units comprise a third 3 × 3 filtering unit, a fourth 3 × 3 filtering unit, a fifth 3 × 3 filtering unit and a sixth 3 × 3 filtering unit; the selection unit comprises a third multi-path selection module;

the third multi-path selection module comprises a sixth path of signal, a seventh path of signal, an eighth path of signal and a ninth path of signal;

the output end of the third 3 × 3 filtering unit is respectively connected with the input end of the fourth 3 × 3 filtering unit and the ninth signal; the output end of the fourth 3 × 3 filtering unit is respectively connected with the input end of the fifth 3 × 3 filtering unit and the eighth path of signal; the output end of the fifth 3 × 3 filtering unit is respectively connected with the input end of the sixth 3 × 3 filtering unit and the seventh path of signal; and the output end of the sixth 3 × 3 filtering unit is connected with the sixth path of signals.

In a second aspect, an embodiment of the present application provides a filter, including a zero point calculation module, an obtaining module, and a cascade filtering module, where:

the zero point calculation module is used for acquiring a first filter and performing zero point calculation on the first filter;

the obtaining module is configured to group the zero points of the first filter according to the cascade filtering module, and obtain a polynomial coefficient matrix corresponding to the second filter after the cascade connection according to a grouping result.

The cascade filtering module comprises at least three filtering units and a selection unit, wherein the filtering units are cascaded through the selection unit according to the corresponding polynomial coefficient matrix to obtain the second filter, so that the filtering effect of the second filter is equal to that of the first filter.

In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the filter design method according to the first aspect or any implementation manner of the first aspect when executing the computer program.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the filter design method according to the first aspect or any implementation manner of the first aspect.

In a fifth aspect, an embodiment of the present application provides a computer program product, which, when run on an electronic device, causes the electronic device to execute the filter design method according to the first aspect or any implementation manner of the first aspect.

It is understood that the beneficial effects of the second aspect to the fifth aspect can be seen from the description of the first aspect, and are not described herein again.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic implementation flow diagram of a filter design method according to an embodiment of the present application;

fig. 2 is a schematic flowchart illustrating an implementation of step S110 in a filter design method according to an embodiment of the present application;

fig. 3 is a schematic flowchart illustrating a specific implementation of step S113 in a filter design method according to an embodiment of the present application;

fig. 4 is a schematic flowchart illustrating an implementation of step S120 in a filter design method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a cascade of filters provided by an embodiment of the present application;

fig. 6 is a schematic structural diagram of a default cascaded filter architecture according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of another default cascaded filter architecture according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a filter according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a zero point calculation module in a filter according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a zero point calculation module in another filter according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

The term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.

Further, in the description of the present application, "a plurality" means two or more. The terms "first," "second," "third," and "fourth," etc. are used merely to distinguish one description from another, and are not to be construed as indicating or implying relative importance.

It should be noted that the drawings provided in the embodiments of the present application are only schematic representations for explaining the basic idea of the present application, so that the drawings only show the components related to the present application and are not drawn according to the number, shape and size of the components in actual implementation, the type, number and ratio of the components in actual implementation can be changed freely, and the layout of the components may be more complicated.

In conventional filter design, two schemes are usually adopted. In the first scheme, assuming that a filter is designed according to a convolution kernel of 9 × 9 at maximum, one pixel is output for filtering processes of different convolution kernels, the Mac number of times required by the filter is 9 × 9-81, and the area scale coefficient is 9 × 9-81. In the second scheme, if a plurality of filters are designed according to different convolution kernels, and if a plurality of filters, 3 × 3, 5 × 5, 7 × 7 and 9 × 9, are designed, the number of times that the filters need MACs is different when different filtering is performed, and the area ratio coefficient is 3 × 3+5 × 5+7 × 7+9 × 9 ═ 164, which greatly increases the occupied area of the chip. In fact, in the chip, the area proportion coefficient corresponds to the size of the area, and the smaller the area proportion coefficient, the better the area proportion coefficient; the smaller the MAC times corresponding to the calculation power consumption of the chip, the better.

Based on the above problems, the present application provides a filter design method. Fig. 1 is a schematic implementation flow diagram of a filter design method according to an embodiment of the present application. The filter design method in this embodiment may be performed by an electronic device. Electronic devices include, but are not limited to, computers, tablets, servers, cell phones, cameras, wearable devices, or the like. The server includes but is not limited to a stand-alone server or a cloud server, etc. The filter design method in this embodiment is suitable for the case where the data needs to be filtered. As shown in fig. 1, the filter design method may include steps S110 to S120.

And S110, acquiring a first filter, and performing zero calculation on the first filter.

In this embodiment, the first filter is an original filter, and is usually a filter to be used. And acquiring a first filter, and carrying out zero point calculation on the first filter.

As a non-limiting example, assume that the transfer function of the first filter is:

the first filter zero is decomposed into a cascaded form of a plurality of low order filters by computation and combination thereof. In one embodiment, it is preferred that:

the mathematical representation is as follows:

further, the zero point of the first filter can be obtained from the cascade format.

And S120, grouping the zero points of the first filter according to a preset cascade filter architecture, and acquiring a second filter according to a grouping result.

In this embodiment, the electronic device is preset with a cascaded filter architecture, and the preset cascaded filter architecture may be stored in advance on a processor of the electronic device, and is represented in the form of an integrated circuit, which is called when used. The preset cascade filter architecture can be set by default of the system or can be designed by user self-definition, and the preset cascade filter architecture is not limited by the application. The preset cascaded filter architecture comprises a plurality of filtering units and a selection unit, and the convolution kernels of the filtering units are generally smaller than those of the original filter. The multiple filtering units may be different convolution kernel sizes or the same convolution kernel size. The selection unit is used for cascading the filtering units.

As a non-limiting example, in some embodiments, the preset cascaded filter architecture includes three filtering units, a first 5 × 5 filtering unit, a first 3 × 3 filtering unit, and a second 3 × 3 filtering unit, and a selection unit. In some other embodiments, the predetermined cascaded filtering architecture includes four 3 × 3 filtering units and a selection unit, where the four filtering units are a third 3 × 3 filtering unit, a fourth 3 × 3 filtering unit, a fifth 3 × 3 filtering unit, and a sixth 3 × 3 filtering unit, respectively.

In step S120, the zeros of the first filter may be grouped according to the architecture of the preset cascade filter, a polynomial coefficient matrix matched with the architecture of the preset cascade filter is obtained, and the second filter is constructed by using the polynomial coefficient matrix. It should be noted that the architecture of the second filter is the architecture of the predetermined cascade filter.

In some embodiments, zero grouping is performed according to filter types included in a preset cascade filter architecture, according to a cascade structure obtained after the zero grouping, a polynomial coefficient matrix is formed by obtaining coefficients corresponding to (or matched with) the cascade structure in the preset cascade filter architecture, and a second filter is constructed by using the polynomial coefficient matrix.

As a non-limiting example, assuming that the preset cascaded filter architecture includes three types of filters, a first 5 × 5filter unit, a first 3 × 3filter unit, and a second 3 × 3filter unit, zero-point grouping is performed according to the three types of filters.

In this embodiment, according to the size of the convolution kernel and the coefficient matrix of the original filter, the second filter (or called target filter) with the filtering effect equal to that of the original filter is obtained in the preset cascade filter architecture, so that the occupied chip area is reduced, and the power consumption is reduced.

On the basis of the embodiment shown in fig. 1, in some other embodiments, specifically, as shown in fig. 2, the step S110 may include steps S111 to S113.

S111, acquiring the first filter, and judging whether the first filter is separable in row and column. If yes, i.e. the first filter is a row-column separable filter, continue to execute step S112 and the following steps; if not, that is, the first filter is a filter whose rows and columns are not separable, then zero point calculation is performed on the first filter, and step S120 is continuously performed.

And S112, separating the rows and the columns of the separable first filter to obtain a column coefficient vector and a row coefficient vector.

In some embodiments, the convolution kernel of the separable first filter is assumed to be a two-dimensional Gaussian convolution kernel, and more particularly, the two-dimensional Gaussian convolution kernel function is assumed to be

f[i，j]Characterizing an image, and performing two-dimensional Gaussian filtering on the image, namely:

wherein the content of the first and second substances,

m, n represents the order of the filter. Neglecting the influence of the constant K, the above equation represents a two-dimensional Gaussian for an imageFiltering, i.e. first performing a one-dimensional Gaussian filter on the columns, g_c[j]For the column coefficient vector, the rows are again one-dimensional Gaussian filtered, g_r[i]Is a row coefficient vector, and the variance of the gaussian kernel of each row and column is the same as the variance of the two-dimensional gaussian kernel. It should be noted that, in some other embodiments, the rows of the image may be subjected to one-dimensional gaussian filtering first, and then the columns of the image may be subjected to one-dimensional gaussian filtering. The dimension of the coefficient matrix of the first filter can be set to (2M-1) × (2M-1), and the dimension of the coefficient matrix can also be other expressions, which is not limited herein.

It should be understood that in some other embodiments, the row-column separable two-dimensional filter may be a gaussian filter, and may also be other row-column separable filters such as a mean filter, and the filter is not particularly limited in this application.

And S113, respectively constructing a first sub-filter and a second sub-filter by using the column coefficient vector and the row coefficient vector, and performing zero point calculation.

In some embodiments, more specifically, as shown in fig. 3, step S113 includes steps S1131 to S1132.

S1131, a first sub-filter and a second sub-filter are respectively constructed by using the column coefficient vector and the row coefficient vector.

In some embodiments, the first sub-filter and the second sub-filter may be respectively constructed by using a column coefficient vector and a row coefficient vector obtained by performing row-column decomposition on the first filter based on step S112. More specifically, the first filter is a two-dimensional filter with dimensions of (2M-1) × (2M-1), and the first sub-filter has dimensions of 1 × (2M-1), and its coefficient matrix is a column coefficient vector; the dimension of the second sub-filter is (2M-1) multiplied by 1, and the coefficient matrix is a row coefficient vector. Further, the cascading of the first sub-filter and the second sub-filter may be equivalent to the first filter, that is, the filtering results of the cascaded first sub-filter and the cascaded second sub-filter are equivalent to the filtering results of the first filter.

It should be noted that, in some embodiments, the coefficient matrix of the first sub-filter may be a column coefficient vector, and the coefficient matrix of the second sub-filter is a row coefficient vector; in other embodiments, the coefficient matrix of the first sub-filter may be a row coefficient vector, and the coefficient matrix of the second sub-filter may be a column coefficient vector, which is not limited herein.

S1132, respectively performing zero point calculation on the first sub-filter and the second sub-filter.

In some embodiments, based on the row coefficient vector of the first sub-filter and the column coefficient vector of the second sub-filter obtained in step S112, the transfer function G of the first sub-filter_r(Z) and transfer function G of the second sub-filter_c(Z) is respectively:

assuming that the first sub-filter and the second sub-filter are both gaussian filters, and the gaussian filters are Finite Impulse Response (FIR) filters, both the first sub-filter and the second sub-filter can be decomposed into a cascade form of a plurality of low-order filters by calculation and combination of zeros. In one embodiment, it is preferred that:

more specifically, assuming that the first sub-filter and the second sub-filter are both high-order filters, they can be decomposed into a cascade architecture form of a plurality of second-order filters, and the second-order cascade, i.e. the above transfer function is decomposed into a form of second-order polynomial multiplication, which can be expressed as:

wherein each of the second order polynomials controls a pair of zeros. It should be noted that the transfer function can also be decomposed into a plurality of first-order polynomials and a cascade form of second-order polynomials, and is not limited herein.

The first sub-filter and the second sub-filter can be decomposed according to a preset cascade filter framework, and the zero points of the first sub-filter and the second sub-filter are calculated according to the decomposed cascade structure (i.e. polynomial).

Based on the embodiment shown in fig. 3, in some embodiments, if the first filter is a filter capable of column separation, as shown in fig. 4, the step S120 more specifically includes steps S121 to S122.

S121: and grouping the zero points of the first sub-filter and the second sub-filter according to a preset cascade filter architecture to construct a third sub-filter and a fourth sub-filter.

In some embodiments, if the dimensions of the first sub-filter and the second sub-filter are 1 × (2M-1) and (2M-1) × 1, the dimensions of the third sub-filter and the fourth sub-filter can be converted to M × M by calculating the zeros and grouping the zeros to reconstruct the first sub-filter and the second sub-filter, respectively, to obtain the third sub-filter and the fourth sub-filter; further, the third sub-filter and the fourth sub-filter can be recycled to perform zero point calculation and grouped to further decompose into two cascaded [ (M +1)/2] × [ (M +1)/2] filters, respectively, without limitation. As an example, a cascaded structure of filters may be as shown in fig. 5.

More specifically, if the zero point is found by the same polynomial as the first-order sub-filter, the zero point obtained by the plurality of polynomials is the first-order sub-filter. The first sub-filter and the second sub-filter can be decomposed in a form of a plurality of polynomial cascades, namely, the first sub-filter is divided into a plurality of polynomial cascades, and a zero point obtained by the same polynomial is a first-stage sub-filter, so that a multi-stage third sub-filter can be obtained; the second sub-filter is the same as the first sub-filter, and a plurality of stages of fourth sub-filters can also be obtained, which is not described herein again.

Reconstructing the first sub-filter and the second sub-filter based on the zero in the polynomial calculated by the cascade architecture decomposed by the first sub-filter and the second sub-filter according to the preset cascade filter architecture in the step S1132, and according to the position of the zero in the decomposed cascade architecture, and obtaining a third sub-filter and a fourth sub-filter; if the preset filter architecture is preferably a cascade combination of one 5 × 5filter and two 3 × 3 filters or a cascade combination of four 3 × 3 filters, the corresponding third sub-filter and fourth sub-filter are also decomposed into a cascade combination of one 5 × 5filter unit and two 3 × 3filter units or a cascade combination of four 3 × 3filter units, which is not limited herein.

And S122, recombining the third sub-filter and the fourth sub-filter to obtain a second filter bank.

In some embodiments, based on the third sub-filter and the fourth sub-filter obtained in step S121, the first sub-filter is formed by a row coefficient vector and the second sub-filter is formed by a column coefficient vector, the reconstructed first sub-filter and second sub-filter, that is, the third sub-filter and fourth sub-filter are combined according to a preset cascade filter architecture, that is, the row coefficient vector and the column coefficient vector are multiplied to obtain a plurality of two-dimensional filters, and the plurality of two-dimensional filters are combined to obtain a second filter bank, where the filtering effect of the second filter bank is equivalent to that of the first filter.

Based on the filter cascade decomposition method, a two-dimensional (2M-1) x (2M-1) filter with separable rows and columns can be converted into two cascaded M x M sub-filters by the method, and the filtering result is equivalent to the two-dimensional filter; furthermore, the M × M filter can be further converted into two cascaded [ (M +1)/2] × [ (M +1)/2] filters, so as to reduce the occupied area and the power consumption.

In this embodiment, according to the size of the convolution kernel and the coefficient matrix of the original filter, the filtering units meeting the requirement of the original filter are selected from the preset cascade filter architecture to be combined, so as to obtain the target filter with the filtering effect equal to that of the original filter, thereby reducing the occupied chip area and reducing the power consumption.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

The following describes advantageous effects produced by the filter design method provided in the embodiments of the present application through specific application scenarios. It should be understood that the specific application scenario is not to be construed as a limitation of the present application.

In some embodiments, the preset cascaded filter architecture comprises three filtering units (or filters) and a selection unit. The three filtering units are a first 5 × 5 filtering unit, a first 3 × 3 filtering unit and a second 3 × 3 filtering unit, respectively. The selection unit comprises a first multi-path selection module and a second multi-path selection module, wherein the first multi-path selection module comprises a first path of signal and a second path of signal, and the second multi-path selection module comprises a third path of signal, a fourth path of signal and a fifth path of signal.

As shown in fig. 6, the three filtering units are a first 5 × 5 filtering unit, a first 3 × 3 filtering unit a, and a second 3 × 3 filtering unit B, respectively. The first multiplexing module includes a first input select pin Mux Sel0, and the first input select pin Mux Sel0 includes a first signal 0 and a second signal 1. The second multiplexing module includes a second input select pin Mux Sel1, and the second input select pin Mux Sel1 includes a third signal 0, a fourth signal 1, and a fifth signal 2.

Specifically, the input end of the first 5 × 5 filtering unit is connected to the first signal 0 in the first multi-path selecting module Mux Sel0, and the output end of the first 5 × 5 filtering unit is respectively connected to the input end of the first 3 × 3 filtering unit a and the fifth signal 2 in the second multi-path selecting module Mux Sel 1; the output end of the first 3 × 3 filtering unit a is connected to the second signal 1 in the first multi-path selecting module Mux Sel0 and the fourth signal in the second multi-path selecting module Mux Sel 1; the input end of the second 3 × 3 filtering unit B is connected to the first multi-path selecting module Mux Sel0, and the output end of the second 3 × 3 filtering unit B is connected to the third signal 0 of the second multi-path selecting module Mux Sel 1.

Based on the filter design method provided in the foregoing embodiment, in a preset cascaded filter architecture, a second filter or a second filter group is determined, where the second filter or the second filter group is a filter or a filter combination equivalent to the original filter (or referred to as the first filter). Through the selection unit, an appropriate filtering unit, i.e. a second filter or a second filter bank, within a preset cascaded filter architecture can be cascaded to implement a filter equivalent to the original filter.

As an example, continuing with fig. 6. If the convolution kernel of the original filter is 9 × 9, the 9 × 9filter can be decomposed into a combination of 5 × 5 and two 3 × 3 filters based on the filter design method provided in the above embodiment. The second signal 1 of the first multi-path selection module Mux Sel0 and the third signal 0 of the second multi-path selection module Mux Sel1 select one 5 × 5 and two 3 × 3 filtering units a and B to be combined, and a filtering unit combination equivalent to the 9 × 9 original filter is obtained. If the convolution kernel of the original filter is 7 × 7, the 7 × 7filter can be decomposed into a combination of 5 × 5 and 3 × 3 filters based on the filter design method provided in the above embodiment. The second signal 1 of the first multi-path selection module Mux Sel0 and the fourth signal 1 of the second multi-path selection module Mux Sel1 select a 5 × 5 and a 3 × 3filter unit a to be combined, and a filter unit combination equivalent to the original filter of 7 × 7 is obtained.

It should be noted that, in the example shown in fig. 6, the original filter is a 9 × 9filter, and if the filter is designed by using a conventional method according to a filter convolution kernel of 9 × 9, the original filter outputs one pixel, the required MAC number of times is 81, and the area scaling coefficient is 81. By using the filter design method provided in an embodiment of the present application, the original filter is decomposed to obtain a filter combination equivalent to the original filter, and the MAC number of times required for outputting one pixel by the filter combination is 5 × 5+3 × 3+3 × 3 — 43, that is, the area scaling coefficient is 43. Therefore, by adopting the filter design method provided by the embodiment of the application, the chip area is reduced, and the power consumption is reduced.

As shown in the following table one, in the case that the preset cascaded Filter architecture shown in fig. 6 includes 1 5 × 5 and 2 3 × 3Filter units (filters) a and B, the Filter design method provided in an embodiment of the present application is adopted to respectively implement result comparison of 3 × 3 filters, 5 × 5 filters, 7 × 7 filters, and 9 × 9 filters in the preset cascaded Filter architecture. Wherein ON represents a Filter requiring cascade connection, and OFF represents a Filter not requiring cascade connection. "-" indicates a mux block unselect signal.

Watch 1

As can be seen from table one, after the filtering scheme of the embodiment of the present application is adopted, the number of MAC times required for calculating one output pixel of the 9 × 9filter is 5 × 5+3 × 3+3 × 3, which is 43.

For the calculation of the 7 × 7filter output one pixel, the MAC number of times required is 5 × 5+3 × 3 — 34.

For the calculation of the 5 × 5filter output one pixel, the MAC number of times required is 25 × 5.

For the calculation of the 3 × 3filter output one pixel, the MAC number of times required is 3 × 3 — 9.

The predetermined cascaded filter architecture shown in fig. 6 has an area scaling factor of 5 × 5+3 × 3+3 × 3 — 43.

In other embodiments, as shown in fig. 7, the preset cascaded filter architecture includes four filtering units and a selecting unit. The four filtering units are four 3 × 3 filtering units, namely a third 3 × 3 filtering unit a, a fourth 3 × 3 filtering unit B, a fifth 3 × 3 filtering unit C and a sixth 3 × 3 filtering unit D. The selection unit comprises a third multiplexing module comprising a third input select pin Mux Sel. The third multi-path selection module comprises a sixth path of signal 0, a seventh path of signal 1, an eighth path of signal 2 and a ninth path of signal 3. Preferably, the output end of the third 3 × 3 filtering unit a is respectively connected to the input end of the fourth 3 × 3 filtering unit B and the ninth signal 3 of the third multi-path selecting module Mux Sel, and a 3 × 3 filtering combination can be output through the ninth signal 3; the output end of the fourth 3 × 3 filtering unit B is respectively connected to the input end of the fifth 3 × 3 filtering unit C and the eighth signal 2 of the third multi-path selecting module Mux Sel, and a 3 × 3+3 × 3 filtering combination can be output through the eighth signal 2, that is, 2 cascaded 3 × 3filter combinations, and the filtering effect of the filtering combination is equivalent to a 5 × 5 filter; the output end of the fifth 3 × 3 filtering unit C is connected to the input end of the sixth 3 × 3 filtering unit D and the seventh signal 1 of the third multi-path selecting module Mux Sel, and 3 × 3 × 3 filtering combinations, that is, 3 cascaded 3 × 3filter combinations can be output through the seventh signal 1, and the filtering effect is equivalent to that of a 7 × 7 filter; the output end of the sixth 3 × 3 filtering unit D is connected to the sixth signal 0 of the third multi-path selecting module Mux Sel, and 4 × 3 × 3 filtering combinations, that is, 4 cascaded 3 × 3filter combinations, can be output through the sixth signal 0, and the filtering effect thereof is equivalent to a 9 × 9 filter.

As shown in the following table two, in the case that the preset cascaded Filter architecture shown in fig. 7, that is, the preset cascaded Filter architecture includes 43 × 3Filter units (filters) A, B, C and D, by using the Filter design method provided in an embodiment of the present application, the results of 3 × 3 filters, 5 × 5 filters, 7 × 7 filters, and 9 × 9 filters are respectively compared in the preset cascaded Filter architecture. Wherein ON represents a Filter requiring cascade connection, and OFF represents a Filter not requiring cascade connection.

Watch two

As can be seen from table two, after the filtering scheme of the embodiment of the present application is adopted, for the calculation of outputting one pixel by the 9 × 9filter, the MAC number required is 4 × 3 × 3 — 36.

For the calculation of one output pixel of the 7 × 7filter, the required MAC number is 3 × 3 × 3 — 27.

For the calculation of the 5 × 5filter output one pixel, the MAC number of times required is 18 × 3 × 3.

The predetermined cascaded filter architecture shown in fig. 7 has an area scaling factor of 4 × 3 × 3 — 36.

It should be understood that the predetermined cascaded filter architecture may also take other forms, and basically can satisfy all kinds of filter requirements, and is not limited herein.

With the first approach in conventional filter design, assume a situation where the filter is designed with a convolution kernel of 9 × 9 at maximum. For the calculation of the 9 × 9filter output one pixel, the MAC number of times required is 81 × 9. For the calculation of the 7 × 7filter output one pixel, the MAC number of times required is 81 × 9. For the calculation of the 5 × 5filter output one pixel, the MAC number of times required is 81 × 9. For the calculation of the 3 × 3filter output one pixel, the MAC number of times required is 81 × 9. The area scaling factor of this architecture is 81 × 9.

And 3 × 3, 5 × 5, 7 × 7 and 9 × 9 filters are designed respectively using the second scheme in the conventional filter design. For the calculation of the 9 × 9filter output one pixel, the MAC number of times required is 81 × 9. For the calculation of the 7 × 7filter output one pixel, the MAC number of times required is 49 — 7 × 7. For the calculation of the 5 × 5filter output one pixel, the MAC number of times required is 25 × 5. For the calculation of the 3 × 3filter output one pixel, the MAC number of times required is 3 × 3 — 9. The area scaling factor of this architecture is 3 × 3+5 × 5+7 × 7+9 × 9 ═ 164.

The following table three shows a comparison table of MAC times and area proportionality coefficients required for respectively implementing 3 × 3Filter, 5 × 5Filter, 7 × 7Filter and 9 × 9Filter under four different implementation architectures.

Watch III

It can be seen that, compared with the conventional design method, the embodiment of the present application implements a unified solution for implementing multiple Kernel sizes of a rank-separable 2D filter, including a filter coefficient conversion algorithm and a chip hardware implementation architecture, and when the flexibility of the Kernel sizes of the filter is satisfied, it is ensured that the area is small and the power consumption is low, and the effect is equal to separately designing multiple filters of different Kernel sizes, and for any rank-separable 2D filter of M × M Size, as long as the rank-separable 2D filter is within the range supported by the design specification and the architecture, the implementation can be implemented by using the cascade combination of multiple filters, and at the same time, the area implemented by the chip is small and the power consumption is low.

An embodiment of the present application further provides a filter design apparatus. The filter design apparatus is not described in detail in the above embodiments of the method.

Referring to fig. 8, fig. 8 is a schematic block diagram of a filter according to an embodiment of the present application. The filter includes: a zero point calculation module 71, an acquisition module 72 and a cascade filtering module 73.

The zero point calculation module 71 is configured to acquire a first filter, and perform zero point calculation on the first filter.

And an obtaining module 72, configured to group the zeros of the first filter according to the cascade filtering module 73, and obtain a polynomial coefficient matrix corresponding to the cascaded second filter according to a grouping result.

The cascade filtering module 73 includes at least three filtering units and a selecting unit, and the filtering units are cascaded through the selecting unit according to the corresponding polynomial coefficient matrix to obtain a second filter, so that the filtering effect of the second filter is equal to that of the first filter.

In some embodiments, as shown in fig. 9, the zero point calculation module 71 includes a separation submodule 711 and a first calculation submodule 712.

The separation submodule 711 is configured to, if the first filter is a row-column separable filter, perform row-column separation on the first filter to obtain a row coefficient vector and a column coefficient vector;

and a first calculating sub-module 712, configured to respectively construct a first sub-filter and a second sub-filter by using the column coefficient vector and the row coefficient vector, and perform zero point calculation.

In some embodiments, if the first filter is a row-column separable filter, the obtaining module 72 is specifically configured to:

and recombining the third sub-filter and the fourth sub-filter to obtain a polynomial coefficient matrix corresponding to the cascaded second filter.

In some embodiments, the first calculating submodule 712 is specifically configured to:

In some embodiments, as shown in fig. 10, the zero point calculation module 71 further includes a second calculation submodule 713 on the basis of fig. 9.

The second calculation submodule 713 is configured to perform zero calculation on the first filter if the first filter is a row-column inseparable filter.

In some embodiments, the cascaded filtering module 73 includes three filtering units and a selecting unit;

In some embodiments, the cascaded filtering module 73 includes four filtering units and a selecting unit;

An embodiment of the present application further provides an electronic device, as shown in fig. 11, the electronic device may include one or more processors 90 (only one is shown in fig. 11), a memory 91, and a computer program 92 stored in the memory 91 and operable on the one or more processors 90, for example, a program for performing filtering processing. The steps in an embodiment of a filter design method may be implemented by one or more processors 90 executing a computer program 92. Alternatively, the functions of the modules/units in the filter design apparatus embodiments may be implemented by one or more processors 90 executing computer programs 92, which are not limited herein.

Those skilled in the art will appreciate that fig. 11 is merely an example of an electronic device and is not intended to limit the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or different components, e.g., the electronic device may also include input-output devices, network access devices, buses, etc.

In one embodiment, the Processor 90 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

In one embodiment, the storage 91 may be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The memory 91 may also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Memory Card (SMC), a Secure Digital (SD) card, a flash memory card (flash card), and the like provided on the electronic device. Further, the memory 91 may also include both an internal storage unit of the electronic device and an external storage device. The memory 91 is used for storing computer programs and other programs and data required by the electronic device. The memory 91 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules, so as to perform all or part of the functions described above. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program can implement the steps in the filter design method embodiment.

An embodiment of the present application provides a computer program product, which when run on an electronic device, enables the electronic device to implement the steps in the filter design method embodiment.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/electronic device and method may be implemented in other ways. For example, the above-described apparatus/electronic device embodiments are merely illustrative, and for example, a module or a unit may be divided into only one logic function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above method embodiments may be implemented by a computer program, which may be stored in a computer readable storage medium and executed by a processor, so as to implement the steps of the above method embodiments. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, in accordance with legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunications signals.

The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims

1. A method of filter design, comprising:

2. The filter design method of claim 1 wherein said performing a zero computation on said first filter comprises:

3. The method of claim 2, wherein if the first filter is a row column separable filter, the grouping zeros of the first filter according to a predetermined cascaded filter architecture, and obtaining a second filter according to the grouping result comprises:

4. The filter design method of claim 1 or 2, wherein the constructing a first sub-filter and a second sub-filter using the column coefficient vector and the row coefficient vector, respectively, and performing zero point calculation comprises:

5. The filter design method of claim 2 wherein said performing a zero computation on said first filter further comprises:

6. A method as claimed in any one of claims 1 to 3, wherein the predetermined cascaded filter architecture comprises three filter units and a selection unit;

7. A method as claimed in any one of claims 1 to 3, wherein the predetermined cascaded filter architecture comprises four filter units and a selection unit;

8. A filter is characterized by comprising a zero point calculation module, an acquisition module and a cascade filtering module, wherein:

9. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the filter design method according to any one of claims 1 to 7 when executing the computer program.

10. A computer storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the filter design method of any one of claims 1 to 7.