EP3841495A1

EP3841495A1 - Convolution-based processing

Info

Publication number: EP3841495A1
Application number: EP18762784.9A
Authority: EP
Inventors: Rolf Koehler; Pia PETRIZIO; Matthias NAHR; Jan Buchholz
Original assignee: Robert Bosch GmbH; Renesas Electronics Corp
Current assignee: Robert Bosch GmbH; Renesas Electronics Corp
Priority date: 2018-08-20
Filing date: 2018-08-20
Publication date: 2021-06-30
Also published as: WO2020038551A1

Abstract

Apparatus is disclosed comprising a hardware processing module and an external memory controller coupled to the hardware processing module. The hardware processing module comprises an input memory controller configured to receive data from the memory controller, a set of convolution units each convolution unit capable of performing a convolution on a respective portion of the data using a filter having size u x v, wherein u and v are each a positive integer equal to or greater than one, and an output memory controller configured to output data derived from the convolution units. The data is divided or divisible into k channels, where k is an integer greater than or equal to two. The apparatus is configured to provide interleaved lines of the data, sequentially interleaved from the k channels, to the input memory controller.

Description

Convolution-based processing

Field of the Invention

The present invention relates to convolution-based processing particularly, but not exclusively, for convolution neural networks.

Background

The core building blocks of convolutional neural networks (CNNs) are convolutional layers which comprise filters. The dimensions (or“size”) of the filter are usually odd numbers and there is trend of using small filters, such as 3 x 3 or 1 x 1.

Specially-designed hardware can be used to execute an already-trained neural network. The hardware, however, is normally designed to operate with a filter having particular dimensions, and so this can place limits on the CNN architecture which can be executed efficiently by the hardware. For example, if the hardware is intended to use filters of a particular size but it is desirable to use smaller filters, then there may be no

computational benefit (e.g. increased bandwidth) to using smaller filters.

Summary

According to a first aspect of the present invention there is provided apparatus comprising a hardware processing module (such as a hardware convolution neural network module) and an external memory controller (such as a direct memory access module) coupled to the hardware processing module. The hardware processing module comprises an input memory controller configured to receive data from the memory controller, a set of convolution units, each convolution unit capable of performing a convolution on a respective portion of the data using a filter having size u x v, wherein u and v are each a positive integer equal to or greater than one, preferably odd and preferably equal to or greater than three (for example, u = v = 5), and an output memory controller configured to output data derived from the convolution units. The data is divided or divisible into k channels, where k is an integer greater than or equal to two, for example, 2, 3 or 4. The apparatus is configured to provide interleaved lines of the data, sequentially interleaved from the k channels, to the input memory controller. The input memory controller is configured to provide successive windows of data to a respective convolution unit, the window taken across a set of k lines, in line- wise increments of k lines. The convolution unit is arranged to convolve a series of windows forming a tile with a filter having a size k x Y, where Y is equal to or less than u.

Thus, smaller convolutional filters can be executed more efficiently.

The value of u may be equal to the value of v. The value of u may be 5. The apparatus may further comprise scratchpad memory, and the external memory controller may be interposed between the external memory controller and input memory controller. The scratchpad memory may take the form of static random-access memory (SRAM). The apparatus may further comprise memory, and the external memory controller may be coupled to the memory. The memory may take the form of dynamic random-access memory (DRAM), such as double data rate synchronous random-access memory (DDR SDRAM). The external memory controller may be configured to receive data divided or divisible into k channels and to interleave the data from the k channels to provide the

interleaved lines of the data. The output memory controller may be configured to receive output data and to interleave the output data to provide the interleaved lines of the data.

The apparatus may be further configured to write out the interleaved lines with stride k. The value of k may be equal to or greater than two. The value of k may be two, three or four. The value of k may be equal to or greater than five.

The hardware processing module may be a hardware convolutional neural network module.

According to a second aspect of the present invention there is provided an integrated circuit comprising the apparatus of the first aspect of the present invention.

The integrated circuit may be a system-on-chip (SoC) or microcontroller. The integrated circuit may further comprise a CPU subsystem in communication with the convolutional neural network module. The integrated circuit may further comprise memory.

According to a third aspect of the present invention there is provided a head unit comprising the integrated circuit of the second aspect of the present invention.

The head unit may be operable to receive images from a digital camera and to pass the images to the hardware circuit. According to a fourth aspect of the present invention there is provided a system comprising the head unit of the third aspect of the present invention and a digital camera configured to transmit images to the head unit for processing by the hardware circuit. The system may further comprise a communications bus for transmitting images from the digital camera to the head unit. The system may be a driver or operator assistance system.

According to a fifth aspect of the present invention there is provided vehicle or plant comprising the driver or operator assistance system.

According to a sixth aspect of the present invention there is provided motor vehicle comprising the driver assistance system. The motor vehicle may be a motorcycle, an automobile (sometimes referred to as a“car”), a minibus, a bus, a truck or lorry. The motor vehicle may be powered by an internal combustion engine and/or one or more electric motors.

According to a seventh aspect of the present invention there is provided a method of convolution-based processing. The method comprises receiving data, divided or divisible into k channels, where k is an integer equal to or greater than one, using a set of convolution units, each convolution unit capable of performing a convolution on a respective portion of the data using a filter having size u x v, wherein u and v are each a positive integer equal to or greater than one, providing interleaved lines of the data, sequentially interleaved from the k channels, providing successive windows of interleaved lines of data a respective convolution unit, the window taken across a set of k lines, in increments of k lines, convolving a series of windows forming a tile with a filter having a size k x Y, where Y is equal to or less than v to provide convolution data, and outputting data in dependence upon convolutions performed by the convolution units.

The value of u may be equal to the value of u. The value of u may be 5.

The method is preferably a hardware-implemented method. Brief Description of the Drawings

Certain embodiments of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which:

Figure 1 is a schematic block diagram of a hardware-based convolutional neural network (CNN) module intended to perform 5 x 5 convolutions;

Figure 2 shows a 5 x 5 filter typically used in the CNN module shown in Figure 1;

Figure 3 shows a 1 x 1 filter usable in the CNN module shown in Figure 1;

Figure 4 illustrates typical handling of an activation map whereby each channel is supplied to a respective input channel;

Figure 5a shows a 1 x 3 filter usable in the CNN module shown in Figure 1;

Figure 5b shows a 5 x 3 filter usable in the CNN module shown in Figure 1;

Figure 6 illustrates re-shaping an activation map whereby lines from three channels are interleaved and supplied to one input channel;

Figure 7 illustrates re-organising of memory and loading of data into convolution unit; Figure 8 illustrates 1 x 3 convolution and readout, followed by 3-line pixel shift;

Figure 9 illustrates striding and interleaving of input data; and

Figure 10 illustrates re-shaping an activation map whereby lines from three channels are written out with stride and interleaved;

Figure 11 is a schematic block diagram of system memoiy, a direct memory access module, scratchpad memory and a CNN input a direct memory access module;

Figure 12 is a schematic block diagram of a CNN output a direct memory access module and system memory;

Figure 13 is a schematic block diagram of the hardware-based CNN module shown in Figure 1 performing 1 x 3 convolutions;

Figure 14 illustrates re-ordering of data by a direct memory access module;

Figure 15 illustrates re-ordering of data by a CNN output direct memory access module; Figure 16 illustrates implementing a first linear CNN using a CNN module;

Figure 17 illustrates implementing a second linear CNN using a CNN module;

Figure 18 illustrates implementing a branch CNN using a CNN module; and

Figure 19 illustrates a motor vehicle which comprises a system which includes a hardware CNN module. Detailed Description of Certain Embodiments

CNN module l

Referring to Figure l, a dedicated hardware convolutional neural network (CNN) module l (which may be referred to as a“CNN IP”,“CNN core” or“CNN accelerator”) is shown which is designed to handle 5 x 5 convolutions.

The CNN module 1 is coupled to a memory system 2 which includes system memory 3 typically in the form of dynamic RAM (herein referred to as“DDR memory” or simply “DDR”), a direct memory access (DMA) module 4 (herein referred to as“IMP-DMA”) and relatively small, but fast (compared to system memory 3) scratchpad memory 5 (which is herein referred to as“cache memory”,‘TMPC memory” or simply“IMPC”). The system memory 3 may take the form of external off-chip memory (i.e., not on the same chip as the CNN modulei) or on-chip, peripheral memory. The scratchpad memory 5 may take the form of SRAM, e.g. 2 MB of SRAM.

The memory system 2 feeds data arranged in k input/feature map channels to the CNN module 1 for processing. There may be two or more input/feature map channels (i.e., k ³ 2), for example, there may be 2, 3 or 4 input/feature map channels (i.e., k = 2, 3 or 4). In this example, k = 4. The input/feature map channel is referred to as“an input channel” when arrange to provide input data to the first CNN layer and is referred to as “a feature map channel” when used to supply data to the second and each subsequent CNN layer. For brevity, however, an input/feature map channel may be hereinafter referred to as“an input channel”. The CNN module 1 includes an input DMA and input register module 8 (herein simply referred to as“CNN input DMA module” or“input DMA module”) which feeds data via a set of line memories 9 (or“buffers”) to corresponding multiplexers 10 which in turn feeds the data to a set of convolution units 11 (or“processing nodes” or simply“nodes”) which are fed with convolution weights 12 from weights register 13. The convolution weights 12 may be transferred to the weights register 13 from non-volatile memory 27, for example in the form of flash memory, outside the module 1.

The convolution units 11 are designed to perform u x v convolutions, in this case u = v = 5, i.e., 5 x 5 convolutions. However, as will be explained in more detail hereinafter, by re-organising the input data and loading suitably re-ordered input data into the convolution units 11, the convolution units 11 can carry out, for example, 1 x 3 convolutions, which can be computationally more efficient.

The outputs of the convolution units 11 are fed into a set of m adder trees 14, wherein bitwise bias and carry bit shifts are carried out using bias 15 and carry data 16 from respective registers 17, 18. The output from each adder tree 14 is fed into a respective saturation bit-shifter module 19. The output of each bit-shifter module 19 is fed into respective rectified linear units 20 (herein also referred to as“Relu”) which perform an activation function.

The outputs of the rectified linear units 19 are fed into a ranking module 21 which can perform labelling. The outputs of the ranking module 20 are fed into a down-sampling module 22 (herein also referred to as an“X/Y pool”) and the output of the down- sampling module 21 is fed into an output register and output DMA module 23.

The output register and output DMA module 23 provides output data which is arranged in m output channels 25 to the memory system 2 and which may be fed back as input date to the CNN module 1, to another, different CNN module 1, and/or be stored in system memory 4. There may be one or more output channels 25 (z^'.e., m > 1), preferably 4, 5, 6, 7 or 8 channels (z^'.e., m =4, 5, 6, 7 or 8). In this example, m = 8.

There may be, however, more than eight output channels.

Operation

A CNN performs feature extraction using a series of computational layers which include one or more convolutional layers, each convolutional layer employing a set of filters (or “kernels”) to extract features and produce corresponding activation map (or“feature map”), and one or more pooling/sub-sampling layers for down-sampling images or feature maps. The output of one computational layer may be used as the input activation map for a subsequent computational layer. As will be explained in more detail hereinafter, the output of the CNN module 1 may be fed back into the CNN module 1 or into another CNN module 1 as an input and, thus, execute a multiple-level CNN.

Each convolutional layer receives an input in the form of an activation map of size h x w x ch, where h and w are the number of height values (e.g., number of pixels) and the number of width values ( e.g ., number of pixels) respectively, and ch is the number of input/feature map channels (z^'.e., ch = k). The convolutional layer can process ri_m input/feature map channels in one run, where ri_m is a fixed, positive, non-zero integer number. In each input/feature map channel, the activation map is convolved with sets of weights, for example, previously-determined weights. Pixel (or other form of data) may have a size of 2 bytes. A smaller pixel (or other data unit) of 1 byte (8 bits), half a byte (4 bits) or smaller pixels having few bits may be used.

The CNN module 1 may not be able process a whole h x w x i activation map (z^'.e., an activation map for one input/feature map channel) in one go. Instead, however, it can process a tile that can fit into the line memory Im having a size lm_max_w and lm_mox_h.

Standard approach to convolution

Referring to Figures 2 and 3, normally if a CNN module is intended to process a certain size of a filter, for example a 5 x 5 filter 12_A, then it is also possible to process a smaller filter, such as a 1 x 1 filter 12B, by filing unused elements (or“entries”) of the 5 x 5 with zeros. The execution times using the two filters 12A, 12B are about the same. However, when a 1 x 1 filter 12B is employed most hardware resources are left unused. Thus, there may be little or no benefit to using a smaller filter in this way using hardware which is able to process a larger filter, although less storage for the weights can be used if the weights are stored outside the CNN module, for example, in Flash memory.

Referring also to Figure 4, in a single pass through the CNN module 1, only in_ch hardware input channels from an activation map 32 can be processed. For four hardware input channels (z^'.e., in_ch = 4), single-channel activation maps 32₀, 32₁, 32₂, 32₃ are arranged into groups of four and each single-channel activation map 32₀, 32₁, 32₂, 32₃ is passed to a respective input channel. Increasing convolution bandwidth

A 1 x 1 convolution can be done more efficiently by performing three 1 x 1 convolutions in one go, thereby increasing the bandwidth by a factor of three.

Referring to Figure 5a, this approach involves using a modified filter 12c and reshaping the input activation map in memory. In particular, the modified filter 12c filter, which includes five rows and five columns comprising twenty-five entries and includes seventh, twelfth and seventeenth entries, herein labelled w₀ __{cho? Wo chi Wo-ch2} respectively, lying in the second, third and fourth rows in the third column whose values can be set accordingly, i.e., may take non-zero, real value. The other entries in the filter are set to zero.

Referring also to Figures 6, 7, 8 or 9, instead of processing only one channel, three channels are processed as three 1 x 1 convolutions, each channel having a respective weight and results are accumulated in the same way as for typical 3D-convolution. In other words, activation maps 32₀, 32₁, 32₂ for three channels are grouped together and reordered to form a re-ordered activation map 32_R for an input channel. The height of the re-shaped activation map 32_R is three times bigger {i.e., 3 h) than a normal single- channel activation map, while its width remains the same. The reshaped activation map 32_R consists of interleaved lines 33* , , 33ch , , 33LI,₂, 33L_0>2, 33LI,₂, 33A₂,₂ from the three channels. Thus, the reshaped activation map 32_R starts with the first line 33*_0,1 from the first channel ch o and then is followed by the first line 33*_1,1 from the second channel ch 1, the first line 33*₂,_I from the third channel ch 2, the second line 33*_{0, 2} from the first channel ch o, the second line 33*_{1, 2} from the second channel ch 1, the second line 33*_2,2 from the third channel ch 2, the third line (not shown) from the first channel ch o and so on.

If there are four hardware input channels {i.e., in_ch = 4), then this would result in 12 channels of activation map being processed in the same pass by the CNN module 1. The grouping factor (in this case, three) depends on the predefined filter size and the maximum number of lines in line memory. For example, if the filter size is five and the maximum number of lines in the line memory is seven, then a grouping factor of three {i.e., 1 + (7 - 5) = 3) results in the maximum increase in bandwidth without causing stalling.

Referring to Figure 5b, it is possible to speed up not only 1 x 1 convolutions, but also 1 x Y convolutions, where Y is 1, 2, 3, 4, or 5. For example, as shown in Figure 5b, a 1 x 5 filter 12_D may be used. As shown in Figure 8, after performing a convolution, the window 36 moves down (z^'.e., in the y-direction) by 3 lines (as opposed to along the lines, z^'.e., in the x-direction) and the next convolution is performed and so on. As explained earlier, the process can be used to increase bandwidth by a fact of three. For example, using the standard approach to convolution employing a 5 x 5 filter or other similar filter, such as 3 x 3 or 1 x 1 filter, and assuming a unit of input data is 2 bytes, 8 bytes of data can be processed per clock cycle, z^'.e., 2 bytes x 4 input channels. By comparison, using the same input data, but the approach described herein using a 3 x i filter, then the bandwidth can be increased to 24 bytes per clock cycle.

Increasing convolution bandwidth further

Referring also Figures 9 and 10, if the CNN module 1 (Figure 1) has the ability to write out each line with a stride, then more 1 x 1 filters can be processed in one step. For example, nine 1 x 1 filter operations can be done in one step, after which pixels are written out and re-ordered, as hereinbefore described. Reordering can be done in the preceding convolutional layer, in particular, in the output DMA module 23. For hardware designed to handle 5 x 5 convolutions, up to twenty-five 1 x 1 filter operations can be done in one step since.

Memory re-ordering

As explained earlier, to enable a modified filter to be used in convolution, the input data is re-ordered by interleaving lines of data from different channels into one input channel.

Referring to Figures 11 and 12, re-ordering can be carried out by a re-ordering function 41 (or“re-ordering block”) in the DMA module 4 (z^'.e., IMP DMA 4). The DMA module 4 may also perform a striding function 42. The input DMA 8 includes a pixel skipping function 43 for incrementing which data is read out in steps of three lines in the y- direction. However, the output DMA module 23 may perform re-ordering 44 and/or striding 45 on output data.

Referring to Figures 13 and 14, the DMA module 4 can re-organise input data in, for example, 12 channels into four input channels. Referring to Figures 13 and 15, the CNN module 1 may process input data in standard way, but then re-organise the output data, ready for the next layer.

Examples of re-ordering data will now be described.

Referring to Figures 9, 11, 13, 14 and 16, non-interleaved data 6 (e.g. image data) can be received and re-ordered by DMA module 4 into interleaved data 6’. The interleaved data 6’ is then supplied to the CNN module 1 which performs a convolution using a first modified filter 12C, 12D, such as a 1 x 3 filter. The output is still interleaved and the interleaved data 6’ can be fed back into the CNN module 1 or another CNN module 1, without further re-ordering by the DMA module 4, and the CNN module 1 performs a convolution with a second, different modified kernel 12C, 12D.

Referring to Figures 12, 13, 15 and 17, non-interleaved data 6 (e.g. image data) can be received by DMA module 4 and supplied to the CNN module 1, without reordering, which performs a convolution using a kernel 12A, 12B, e.g. a s x s or i x i filters. The output DMA module 23 re-orders the output data. The interleaved data 6’ can be fed into the CNN module 1 or another CNN module 1, without further reordering by the DMA module 4, and the CNN module 1 performs a convolution with a second, different modified kernel 12C, 12D.

Interleaving and de-interleaving, and convolutions using different forms of filters, together branches, can be used to implement a branched CNN employing efficient filters can be used.

Referring to Figures 11, 12, 13, 15 and 17, first and second sets of non-interleaved data 6 can be re-ordered by DMA module 4 into first and second sets of interleaved data 6’. The CNN module 1 can process the first set of interleaved data 6 first to produce interleaved output data and then process the interleaved output data and the second set of interleaved input data.

ADAS system 101

Referring to Figure 19, a motor vehicle 101 is shown. The motor vehicle 101 includes an advanced driver assistance system (ADAS) 102 or autonomous car system (not shown) which performs, for example, semantic segmentation to classify objects (such as the road, cars, pedestrians, trees etc.) in images 103 captured by an on-board digital camera 104.

The system 102 includes a head unit 105 connected to an in-vehicle communications bus 106. The system 102 may include a display 107 connected to the head unit 105.

The display 107 can take the form of a flat-panel display located, for example, in a centre console (not shown) or dashboard (not shown).

The digital camera 104 is capable of capturing images in a sequence of image frames 103 (or“pictures”) for example, at a rate of 60 fps. The image frame 103 is transmitted in one or more data frames 109 via the in-vehicle communications bus 106 to the head unit 105.

The head unit 105 includes a processor-based system 110 which may take the form of a system-on-a-chip (SoC), microcontroller or other similar microprocessor-based system, and system memory 3. The SoC 110 includes a central processing unit (CPU) sub- system 112 which includes at least one processing core (not shown), scratchpad memory 5 and the CNN module 1 in the form of a hardware-implemented peripheral module. The SoC 110 includes other peripheral modules, such as other

communications network controllers (for other different types of communications network), timers etc. The head unit 105 also includes flash memory 27 which can be used to store weights 12 (Figure 1) for convolutions. The processor-based system 110 may house the flash memory 27. Uses

An advanced driver assistance system (ADAS) 102 or autonomous car system (not shown) can perform, for example, semantic segmentation to classify objects (such as the road, cars, pedestrians, trees etc.) in images 103 captured by an on-board digital camera 104 and, based on an output of the CNN module 1, generate control signals for automatically controlling the motor vehicle 101 and/or providing data to the driver of the motor vehicle 101 for assisting the driver.

For example, the camera 104 may capture images which are classified using the CNN module 101, which, using the approach herein before described, can be performed more efficiently and, thus, for example, more quickly and/ or process more images.

Classification data can be output by the CNN module 1 to the processor-based system no which can generate control signals which can be provided to motors (not shown) in dependence upon the classification data. This can be used to help steer the motor vehicle 101 and so help to avoid a collision. Similarly, for example, the camera 104 may capture images which are classified using the CNN module 101, which, using the approach herein before described, can be performed more efficiently and, thus, for example, more quickly and/or process more images. Classification data can be output by the CNN module 1 to the processor-based system 110 which can generate control signals which can be provided to motors (not shown) and actuators (no shown) in dependence upon the classification data. This can be used to cause the motor vehicle 101 to break and/or steer.

The CNN module 1 can be used in an industrial system (not shown) which employs cameras (not shown) to monitor a process and, for example, to provide data for identifying an object or part of an object (such as a product being manufactured or processed). The CNN module 1 can be used, for example, correct positioning of the object, completion of a process or performing quality control. If the CNN module 1 has, by classifying or performing some other form of image recognition, an expected or learnt condition, then it may trigger the processor-based system 110 to perform a predetermined step, such as release the object for a subsequent processing step.

Modifications

It will be appreciated that various modifications may be made to the embodiments hereinbefore described. Such modifications may involve equivalent and other features which are already known in the design, manufacture and use of convolution-based processors, such as neural networks, and component parts thereof and which may be used instead of or in addition to features already described herein. Features of one embodiment may be replaced or supplemented by features of another embodiment. The hardware processing module may take the form of a recurrent neural network (RNN) module or autoencoder module.

The approach herein described may be used in, for example, speech processing (such as automatic translation or automatic subtitling), image processing (such as automated driving, image enhancement, object detection, or surveillance). Although claims have been formulated in this application to particular combinations of features, it should be understood that the scope of the disclosure of the present invention also includes any novel features or any novel combination of features disclosed herein either explicitly or implicitly or any generalization thereof, whether or not it relates to the same invention as presently claimed in any claim and whether or not it mitigates any or all of the same technical problems as does the present invention. The applicants hereby give notice that new claims may be formulated to such features and/or combinations of such features during the prosecution of the present application or of any further application derived therefrom.

Claims

1. Apparatus comprising:

a hardware processing module (1); and

an external memory controller (4) coupled to the hardware processing module; wherein the hardware processing module comprises:

an input memory controller (8) configured to receive data from the memory controller;

a set of convolution units (11), each convolution unit capable of performing a convolution on a respective portion of the data using a filter having size u x v, wherein u and v are each a positive integer equal to or greater than one; and

an output memory controller (23) configured to output data in dependence upon convolutions performed by the convolution units;

wherein the data is divided or divisible into k channels, where k is an integer equal to or greater than one;

wherein the apparatus is configured to provide interleaved lines (33) of the data, sequentially interleaved from the k channels, to the input memory controller;

wherein the input memory controller is configured to provide successive windows (36) of data to a respective convolution unit, the window (36) is taken across a set of k lines (33), in increments of k lines; and

wherein the convolution unit is arranged to convolve a series of windows forming a tile (37) with a filter having a size k x Y, where Yis equal to or less than u.

2. The apparatus of claim 1, wherein u = v.

3. The apparatus of claim 1 or 2, wherein u = 5.

4. The apparatus of any one of claims 1 to 3, further comprising:

scratchpad memory (5) wherein the external memory controller (4) is interposed between the external memory controller (4) and input memory controller (8).

5. The apparatus of any one of claims 1 to 4, further comprising:

memory (3), wherein the external memory controller (4) is coupled to the memory.

6. The apparatus of any one of claims l to 5, wherein the external memory controller (4) is configured to receive data divided or divisible into k channels and to interleave the data from the k channels to provide the interleaved lines of the data.

7. The apparatus of any one of claims 1 to 5, wherein the output memory controller

(23) is configured to receive output data and to interleave the output data to provide the interleaved lines of the data.

8. The apparatus of any one of claims 1 to 7, wherein the apparatus is further configured to write out the interleaved lines with stride k.

9. The apparatus of any one of claims 1 to 7, wherein k = 2, 3 or 4.

10. The apparatus of any one of claims 1 to 9, wherein the hardware processing module is a hardware convolutional neural network module.

11. An integrated circuit comprising the apparatus of one of claims 1 to 10.

12. The integrated circuit of claim 11, which is a system-on-chip (SoC) or

microcontroller which further comprises a CPU subsystem in communication with the convolutional neural network module.

13. The integrated circuit of claim 12, further comprising memory (3). 14. A head unit (105) comprising the integrated circuit of any one of claims 11, 12 or

IS

IS. The head unit of claim 14 operable to receive images from a digital camera (104) and to pass the images to the hardware circuit (1).

16. A system comprising:

the head unit (105) of claim 14 or 15; and

a digital camera (104) configured to transmit images to the head unit (105) for processing by the hardware circuit.

17. The system of claim 16, further comprising: a communications bus (106) for transmitting images from the digital camera to the head unit.

18. The system of claim 17, which is a driver or operator assistance system.

19. A vehicle or plant comprising the system of claim 18.

20. A motor vehicle comprising the system of claim 18.

21. A method comprising:

receiving data, divided or divisible into k channels, where k is an integer equal to or greater than one;

using a set of convolution units (11), each convolution unit capable of performing a convolution on a respective portion of the data using a filter having size u x v, wherein u and v are each a positive integer equal to or greater than one;

providing interleaved lines (33) of the data, sequentially interleaved from the k channels;

providing successive windows (36) of interleaved lines (33) of data a respective convolution unit, the window (36) taken across a set of k lines (33), in increments of k lines

convolving a series of windows forming a tile (37) with a filter having a size k x Y, where Y is equal to or less than v to provide convolution data; and

outputting data in dependence upon convolutions performed by the convolution units.

22. The method of claim 21, wherein u = v.

23. The method of claim 21 or 22, wherein u = 5.

24. The method of any one of claims 21 to 23, which is hardware-implemented.