KR20200089302A - Systems and methods for converting matrix inputs to vectorized inputs for matrix processors - Google Patents

Abstract

A system and method for accelerating the convolution of images and similar arithmetic operations by using hardware-specific circuitry that allows multiple operations to be performed in parallel over a large data set is provided. In various embodiments, arithmetic operations are further enhanced by reusing data and eliminating redundant steps of storing and fetching intermediate results from registers and memory when performing arithmetic operations.

Description

Systems and methods for converting matrix inputs to vectorized inputs for matrix processors

This application is Peter Joseph Bannan, William A. McGee, Emil Thalps, listed as the inventor, under the name "SYSTEMS AND METHODS FOR CONVERTING A MATRIX INPUT TO A VECTORIZED INPUT FOR A MATRIX PROCESSOR" Claims priority to co-owned U.S. Patent Application No. 15/839,234 (Docket No. 20150-2166) filed on December 12, 2017. Each of the aforementioned patent documents is incorporated herein by reference in its entirety.

The present disclosure relates to transforming an MxN data matrix into a vectorized input for a high-performance matrix processor, and more specifically, two-dimensional and three-dimensional matrices with large complex mathematical operations. It relates to a method and system for mapping an MxN matrix to a one-dimensional vector that is aligned with respect to a matrix processor so that it can be performed in a.

Those skilled in the art will be aware of the ever-increasing demands of speed and performance for common processors and systems used to implement time-sensitive complex mathematical operations. Because these typical systems are used to process large amounts of data and perform complex mathematical operations, computational resources and computational speed are limited to the ability of conventional general hardware designs to perform these calculations. For example, general purpose computing devices and processors that perform matrix operations may not be able to perform these operations in a timely manner under certain circumstances. Many conventional multipliers that perform digital signal processing operations rely on a series of software and hardware matrix manipulation steps and can be a bottleneck in time sensitive systems. Often these steps involve the arithmetic functions of the processor that produce the intermediate result, with the added step of storing and fetching the intermediate results from the various locations that complete the operation. You pay for the resulting computational time. In one example, the input matrix often needs to be converted to a different format or architecture to be input into a processing element. This conversion process can cause significant delay in the system because the inputs need to be formatted into a particular structure that is subsequently processed by a computational processor or other arithmetic logic.

1 shows an example of a conventional matrix multiplication system. System 100 is a scalar machine that includes a computing unit 102, a register 104, a cache 106, and a memory 108. In operation, computation unit 102 uses register 104 and cache 106 to process data stored in memory 108. The processed data are, for example, image data and weights used in a convolution operation to process an image. Typically, the calculation unit 102 can perform a matrix multiplication on the input matrix to obtain the resulting matrix, for example by converting the multiplication to addition and outputting the result to some internal register, such as a CPU or GPU. It is the same microprocessor.

For example, dot products representing the output pixels of an image are typically created by dot-multiplying individual matrix elements from two matrices to obtain partial results, which results in the final dot product ( final dot product). Multiplication of individual matrix elements (e.g., scalar multiplication) is typically by breaking up a dot multiplication into a series of individual sub-operations, It is performed on individual data elements. Consequently, partial products must be stored and fetched from one or more of registers 104, cache 106, and memory 108 to complete a single arithmetic operation.

Applications that are computationally burdensome, such as convolution, often require software functionality that is used to transform the convolution operation into an alternating matrix-multiply operation and embedded in the computation unit 102. This is achieved by rearranging and reformatting image data and weight data into two matrices that can be raw matrix-multiplied. However, since there is no mechanism for efficiently sharing or reusing data in the scalar machine 100, data required to execute each scalar operation must be re-fetched from the register each time. The complexity of this operation becomes significantly larger as the amount of image data to be subjected to convolution operation increases.

A lot of data from the scalar machine 100 associated with additional steps to store and fetch intermediate results from registers 104, cache 106, and memory 108 to complete the arithmetic operation. The inability to reuse is only some of the shortcomings of existing systems, such as multiplier system 100.

Accordingly, what is needed are efficient transform methods and devices that transform a matrix into an input structure suitable for high performance-compute-processing systems and methods capable of performing matrix multiplication operations.

Reference is made to embodiments of the invention, and examples of embodiments may be shown in the accompanying drawings. These drawings are not limiting and are intended to be illustrative. Although the invention has been generally described in the context of these embodiments, it should be understood that it is not intended to limit the scope of the invention to these specific embodiments.

1 shows an example of a conventional matrix multiplication system.
2 is a flow diagram of an example process for mapping convolutions to a matrix multiplication circuit, in accordance with various embodiments of the present disclosure.
3 illustrates an exemplary matrix multiplication system using a data formatter to map convolutions to a multiplication circuit, according to various embodiments of the present disclosure.
4 is an example diagram illustrating a process for mapping convolutions to a matrix multiplication circuit, in accordance with various embodiments of the present disclosure.

In the following description, for purposes of explanation, certain details are presented to provide an understanding of the invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without these details. In addition, those skilled in the art realize that the embodiments of the present invention described below can be implemented in various ways, such as a method on a process, apparatus, system, device, or tangible computer-readable medium. There will be.

Components or modules shown in the drawings illustrate exemplary embodiments of the present invention and are intended to avoid obscuring the present invention. In addition, components throughout this description may be described as separate functional units, which may include sub-units, but those skilled in the art may understand that various components, or portions thereof, are individual components. It will be appreciated that they may be divided into or integrated together (including those integrated within a single system or component). It should be noted that the functions or operations discussed herein can be implemented as components. Components can be implemented in software, hardware, or a combination thereof.

Also, connections between components or systems in the drawings are not intended to be limited to direct connections. Rather, the data between these components can be modified, re-formatted by an intermediate component, or otherwise changed. Also, additional or fewer connections can be used. Also, the terms "coupled", "connected" or "communicatively coupled" include direct connections, indirect connections through one or more intermediate devices, and wireless connections. It should be understood as.

References herein to “one embodiment”, “preferred embodiment”, “embodiment”, or “embodiments” refer to the embodiments It is meant that the specific features, structures, features, or functions described are included in at least one embodiment of the invention, and may be one or more embodiments. In addition, the appearances of the above-mentioned phrases in various places in the specification are not necessarily all referring to the same embodiment or embodiments.

The use of specific terms in various places in this specification is for illustrative purposes and should not be construed as limiting. A service, function, or resource is not limited to a single service, function, or resource, and the use of these terms is a grouping of related services, functions, or resources that can be distributed or aggregated. (grouping). In addition, the use of memory, databases, information bases, data stores, tables, hardware, etc., can be used herein to refer to system components or components into which information may be entered or otherwise recorded. have.

Also, it should be noted that: (1) Certain steps can be selectively performed; (2) The steps may not be limited to the specific order presented herein; (3) Certain steps can be performed in different orders; (4) Certain steps can be performed simultaneously.

Also, those skilled in the art will know: (1) Certain steps can be selectively performed; (2) The steps may not be limited to the specific order presented herein; (3) Certain steps can be performed in different orders; (4) Certain steps can be performed simultaneously.

Although the embodiments herein are primarily discussed in the context of convolution, those skilled in the art can also deconvolution also be structured as a matrix-matrix type multiply operation, and thus, It will be understood that the principles of the present invention can be equally applied to deconvolution. Also, other types of mathematical operations can be implemented according to various embodiments of the present disclosure.

While the present specification will describe converting at least one input matrix to a vectorized input to a convolution matrix multiplication circuit, those skilled in the art may use the methods and structures described to convert a wide range of input data types from one format to another. You will know that it can be used. All of these different input data structures are within the scope of this specification. 2 is a flow diagram of an example process for mapping convolutions to a matrix multiplication circuit, in accordance with various embodiments of the present disclosure. The process 200 begins at step 202 when convolutional instructions including convolution parameters are received, for example, in a data formatting device. The convolution instructions or parameters can be, for example, a filter dimension, stride, address of input, or any number of outputs channels.

In step 204, based on the convolution parameters, inputs of the convolution operation are identified. In embodiments, the inputs are stored in one or more storage locations at different addresses that may represent a three-dimensional data matrix in an embodiment. In embodiments, the matrix is a first dimension corresponding to a number of pixels calculated in parallel for a given output channel, and a number of outputs channels operating in parallel. It has a two-dimensional (second dimension) corresponding to, and a third dimension (third dimension) corresponding to different colors (colors) for each pixel.

In step 206, overlapping/redundant input data is identified, for example, by a state machine. In an embodiment, the state machine uses filter parameters such as filter size and stride to determine reusable operands.

In step 208, local copies of overlapping input data are retrieved from the cache or buffer to reduce computation time and power consumption, for example re-fetching the same data from SRAM You can avoid what you have to do.

In step 210, for each convolution operation, a sequencer may, for example, allow the matrix processor to perform high-performance arithmetic operations (eg, large matrix multiplication). Based on the predetermined input format of the matrix processor to enable, it can be used to arrange the input according to the position in the matrix processor. In an embodiment, the sequencer arranges inputs for each cycle of the convolution operation. In certain embodiments, the sequencer maps data between two-dimensional matrices to vectors used to input data to the matrix processor. For example, a sequencer is based on its position in a two-dimensional matrix, an element-by-element (e.g., bit-by-) that effectively moves a data/computation object to a corresponding position in a vector. Bit-by-bit or byte-by-byte mapping may be performed. Once the inputs are arranged, the aligner in the embodiments can sort the data being arranged and feed it to the matrix processor according to a predefined sequence.

In step 212, based on the input format, the hardware elements of the matrix processor can be dynamically configured to perform matrix multiplication. In an embodiment, the matrix processor input accommodates a first vector comprising a computational object formatted according to a data input matrix, and a second vector comprising a computational object formatted according to a weighted input matrix.

Finally, in step 214, a matrix multiplication result can be used, for example, to convolve the input image into a filter to produce a convolution result. In embodiments, the convolution result is further enhanced to improve image output, for example, by using a non-linear function, normalization operation, and/or pooling operation. Can be processed.

Although the processes and systems disclosed herein are described in the context of convolutional methods and systems, it is intended to limit the scope of the present disclosure because the teachings of the present disclosure may equally apply to deconvolutional methods and systems. It should be noted that it is not intended.

3 illustrates an exemplary matrix multiplication system using a data formatter to map convolution to a multiplication circuit, in accordance with various embodiments of the present disclosure. System 300 includes memory device 302, data formatter 306, control logic 308, cache (or buffer) 312, logic circuit 314, and matrix processor 316, and sequencer 310. It includes. It is understood that one or more components, such as, for example, the data formatter 306 and sequencer 310 of FIG. 3 may be implemented as an integrated component that performs at least some function associated with each individual integrated component. Can.

The data formatter 306 can be implemented in an in-line formatter, and in embodiments, used to convert a data input matrix to a vector format that can be efficiently processed by the matrix processor 316. do.

The memory device 302 of FIG. 3 is any memory device known in the art and may include any type of data such as audio data, sensor data, and ultrasound data. In embodiments, memory 302 includes image data 304 stored in one or more locations and different addresses. In an embodiment, the address may represent a multi-dimensional data matrix.

In embodiments, control logic 308 may manage sequencer 310 and other components within matrix multiplication system 300. Control logic 308 may be embedded into data formatter 306 or sequencer 310, and in embodiments, data formatter 306 is expected locations within matrix multiplier 316. Data may be selected from memory 302 locations according to filter parameters (eg, stride), which may format the data to match. In embodiments, cache/buffer 312 is a local copy of data for reuse by convolution to avoid having to re-access and re-read data from memory 302. It is a local buffer that can be used to store.

Logic circuit 314 may include circuitry representing any number of input operation targets and data registers. In embodiments, the logic circuit 314 may include circuitry for inputting M weight operands and N image data calculation targets to the matrix processor 316. Weight data and image data 304 may be stored in various types of memory 302 (eg, SRAM).

The matrix processor 316 can perform any number of circuits and sub-circuit circuits (for example, simultaneous multiplication, accumulation, and shift operations) , Arithmetic logic units, registers, encoders, etc.). Each sub-circuit circuit may be a cell capable of performing an arithmetic operation. In embodiments, the matrix processor 316 receives and multiplies, for example, data matrices and pre-formatted weight matrices, multiplication for convolution operations. product) to generate multiply-and-add circuits.

The convolution operation takes a set of individual filters (i.e., a set of weights pre-formatted and stored in memory 302) for a number of input channels that may contain a different set of information. It can be applied to image data. This makes it possible to detect a wide variety of features in the input image.

Also, by analyzing a sequence of different features in a different order, macro features can be identified in the image. In embodiments, the convolution is a rectangular weight matrix to obtain the partial dot products that are summed (eg, by an adder) to produce a accumulated dot product (ie, integer) representing the output pixel in the output image. It includes multiplication of (rectangular weight matrix) and rectangular input matrix.

In embodiments, arithmetic operations are parallelized by using a plurality of rows and columns of matrix processor 316 to generate an NxN tile output. For example, a given row size of 96 and a corresponding column size of 96 facilitate output of 9216 pixels. The matrix processor 316 can accommodate any vector length on each axis and, in embodiments, is dimensioned with an MxN matrix processor comprising PxQ tiles arranged in a matrix format.

In embodiments, sequencer 310 receives convolution instructions, including any number of convolution parameters, for example from control logic 308. In an embodiment, the convolution command may include filter parameters (eg, filter size), a number of outputs channels, and the like. In an embodiment, sequencer 310 uses convolution parameters to identify the address or input of the input of the convolution operation, and receives or fetches these inputs from corresponding address locations in memory 302.

In an embodiment, the data formatter 306 converts two-dimensional or three-dimensional data containing image information into a single vector or string that can be expressed in rows or columns, thereby linearizing or vectorizing the image data ( vectorizing). In an embodiment, the formatter 306 maps image data for processing to the matrix processor 316 by mapping the image data 304 in an appropriate format according to the hardware requirements of the matrix processor 316, according to the convolution parameter. Preparing 304), the matrix processor 316 can perform matrix multiplication as part of the convolution calculation, for example, to generate output pixels.

In embodiments, the size of the vectorized image data is directly related to the number of inputs to the matrix processor 316, so that the computational object is aligned with the inputs to the matrix processor 316.

In embodiments, the data formatter 306 should identify overlapping (ie, identical or overlapping) inputs, eg, via a state machine, and be accessed more than once for a given convolution operation It can exist in more than one location. The state machine can be configured to identify the nested data as reusable data using filter parameters such as filter size and stride, so the matrix processor 316 needs to re-access and transfer data from memory 302 You can reuse the computational object without it. Instead, in embodiments, reusable data may be loaded, for example, from a local copy stored in cache 312, reducing computational effort, time and power consumption.

In embodiments, for example, for each convolution operation, the data sequencer 310 may be configured to perform a convolution operation, for example, to match a given input format of the matrix processor 316 when performing dot multiplication. The retrieved inputs can be arranged according to the positions expected by the matrix processor 316 in each cycle. The sequencer 310 can continue to track the state of the system 300 when generating addresses to record data, recording results, and performing convolution operations. In embodiments, sequencer 310 uses some or all of this information to determine which addresses in memory 302 obtain data and, for example, by matrix processor 316 in a subsequent convolution step. Decide how to handle it in a way that can be used properly. In an embodiment, the sequencer 310 is coupled to a data formatter 306 that can be used to sort image data retrieved and synchronized in a predefined order with the matrix processor 316 according to a given input format. )do.

In embodiments, based on the input format determined by various input parameters or configurations, the hardware elements of the matrix processor 316 may be any type of arithmetic operation (eg, Multiply Accumulate Multiply Add) )). In an embodiment, the input to the matrix processor 316 includes vectors formatted according to the data input matrix. In an embodiment, the input includes a second vector formatted according to the weighted input matrix, such that matrix multiplication can be used to convolution image data 304 for the neural network.

In embodiments, formatting is performed dynamically to accommodate the processing of matrices with different input sizes. In embodiments, reformatted matrices containing input channels are fed to the cache/buffer 312. In embodiments, cache/buffer 312 fetches data from memory 302 and creates a local copy of the data that can be reused by convolution without the need to re-access memory 302 to read data. To save.

Formatting functions performed by the CPU or GPU to convert the convolution operation into a matrix-multiply operation by rearranging the data in an alternative format suitable for high-speed matrix multiplication. Unlike the general software implementations of, the various hardware implementations of the present disclosure re-format data on the fly (e.g., 96 pieces of data per cycle). ) Make it executable. In fact, the present disclosure effectively maps convolution to matrix operations, as a relatively large number of elements in the matrix are processed in parallel. In embodiments, for 2N fetched inputs, N ² compute data may be obtained.

It is understood that system 300 can include additional components that are within the scope of the present disclosure. The additional components may be part of a DMA that retrieves data from memory 302 and stores data (eg, weights and results) in SRAM, for example, a hardware-accelerated pooling unit (hardware-accelerated) pooling unit), other post-processing elements known in the art, and the like.

4 is an example diagram illustrating a process for mapping convolutions to a matrix multiplication circuit, in accordance with various embodiments of the present disclosure. In diagram 400, each matrix 402A, 402B is shown as a highlighted section containing sub-matrix 406, 408. For example, sub-matrix 406 is for an input channel associated with a particular color. Matrix 418 represents one or more input matrices, such as weight data matrices. The matrix processor 316 is preferably a processor of the kind discussed with reference to FIG. 3.

In embodiments, the data matrix 402A, 402B is a three-dimensional array containing any number of rows, columns, and channels capable of holding input data, such as image data for three different colors. It forms part of the data matrix. Similarly, matrix 418 is a rectangular input weighting data matrix that can include any number of rows and columns to maintain weighting data for three different colors. Those skilled in the art will appreciate that the size of the input matrices and the number of input channels may vary for different applications.

4 also shows an array 410, 412 that can be obtained by linearizing each data sub-matrix 406, 408, in embodiments, to generate its linearized version. . Conversely, the weighted data matrix 418 can be vectorized to obtain an array 414. For example, in an embodiment, a 3x3 sub-matrix 406 for a first input channel associated with a first color, a 3x3 sub-matrix 406 for a second input channel associated with a second color, and a third color associated with a third color The third 3x3 sub-matrix 406 for the three input channels is assembled to form an array 410. Assuming a stride of 2, another 3x3 sub-matrix 408 within the data matrix 402B to form the array 412, etc., until all input columns of the matrix processor 316 can be filled with input data. The location of is identified. Similarly, input weight data can be reformatted to form additional output channels similar to array 414.

In embodiments, three 3x3 weighted data matrices 418, which may represent three separate input channels, may be reformatted into a vector containing a total of 27 elements (eg, with respect to FIG. 3). By the described formatter). For use in the convolution operation performed by the matrix processor 316, a 27-element array (eg, array 414) may be generated from the 27 elements.

Specifically, for example, to generate a partial dot product that can be accumulated by the matrix processor architecture 316 to produce a accumulated dot product 430, which can represent an output pixel in the output image, for example, a weight matrix Columns 414 corresponding to 3x3x3 elements of (weight matrix) 418 or each element of the array, and columns 410 or arrays corresponding to 3x3x3 elements of, for example, sub-matrix 406 By dot multiplying each element of, the data matrices 402A, 402B can be multiplied with the weighted data matrix 418, for example as part of a convolution operation. The next dot product can be obtained by dot-multiplying the row 414 and the vector corresponding to the column 412, whereby the values of these partial dot products are of the weight matrix 418 for the input data matrices 402A, 402B. Corresponding to the application, the accumulator output represents the total convolution. In embodiments, the matrix processor is 96×96 in size, which allows 9216 multiply accumulate operations to be performed in parallel, for example in a single clock cycle.

In the calculation of the output channels in embodiments, the matrix processor uses the same set of input data from the matrix 402A, 402B, except for a different set of weights from the matrix 418 (ie filters), the same output pixel. Since the input data can be read once, many output channels can be created at once by reading the input data. One or more input channels (eg, one for each color (eg, RGB)) may be used. For example, each convolution can use weights 418 representing three different matrices, one for each color.

Each output channel 436 can be created using different filters or weights 418 that represent different characteristics in the input data. The number of output channels may depend on the number of features. In an embodiment, the number of convolutions is equal to the product of the number of output channels 436 and the number of input channels, and each convolution may have N convolutions for each input channel.

It will be appreciated that as shown in FIG. 4, the input data matrix 402A, 402B, weighted data matrix 418, and arrays 410-414 can have different numbers of rows and columns. As accurately indicated, the number of input and output channels can be arbitrarily selected. In embodiments, if the weighted data matrix 418 is known, the array 414 may be created and stored in a vectorized format without using a formatter. In embodiments, dot-products may be performed concurrently to generate a one-shot matrix-matrix multiply operation.

Embodiments of the invention may be encoded on one or more non-transitory computer-readable media with instructions to one or more processors or processing units that cause steps to be performed. It should be noted that one or more non-transitory computer-readable media will include volatile and non-volatile memory. It should be noted that alternative implementations are possible, including hardware implementations or software/hardware implementations. Hardware-implemented functionality may be realized using ASIC(s), programmable arrays, digital signal processing circuitry, and the like. Accordingly, the term “means” in any claim is intended to cover both software and hardware implementations. Similarly, the term “computer-readable medium or media” as used herein refers to hardware and/or software having a program of instructions embodied in a computer-readable medium or media, or these Contains a combination of. With these implementation alternatives in mind, the drawings and accompanying description may require one of ordinary skill in the art to write program code (i.e. software) and/or manufacture circuits (i.e. hardware) to perform the required processing. It should be understood that it provides functional information.

It should be noted that embodiments of the present invention may further relate to computer products having a non-transitory type of computer-readable medium having computer code for performing various computer-implemented operations. The media and computer code may be specially designed and constructed for the purposes of the present invention, or may be of a kind known or available to those skilled in the art. Examples of tangible computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tapes; Optical media such as CD-ROMs and holographic devices; Magneto-optical media; And hardware devices specifically configured to store or execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices. Examples of computer code include machine code such as generated by a compiler, and a file containing higher level code executed by a computer using an interpreter. Embodiments of the invention may be implemented in whole or in part with machine-executable instructions that may be in program modules executed by a processing device. Examples of program modules include libraries, programs, routines, objects, components, and data structures. In a distributed computing environment, program modules may be physically located in settings that are local, remote, or both.

Those skilled in the art will recognize that a computing system or programming language is not critical to the practice of the present invention. Those skilled in the art will also recognize that the plurality of elements described above may be physically and/or functionally separated into sub-modules or combined together.

The components of the claims below can be arranged differently, including having multiple dependencies, configurations and combinations. For example, in embodiments, the subject matter of the various claims may be combined with other claims.

It will be understood by those skilled in the art that the foregoing examples and examples are illustrative and do not limit the scope of the invention. It is intended that all substitutions, enhancements, equivalents, combinations and improvements apparent to those skilled in the art through reading the specification and reviewing the drawings are within the spirit and scope of the invention.

Claims (21)

A method for mapping matrix data to an input of a matrix processor, the method comprising:
Receiving first matrix data processed by the matrix processor;
Identifying the length of the input vector associated with the matrix processor;
Mapping the first matrix data to the input vector using a first element-by-element sequence operation;
Identifying at least one element in the first matrix data that overlaps the second matrix data processed by the matrix processor;
Storing the at least one duplicate element in a cache; And
Mapping the second matrix data to the input vector using a second element-by-element sequence operation so that the at least one duplicate element is retrieved from the cache.
How to include.
According to claim 1,
The first matrix data and the second matrix data,
Associated with one or more convolution operations
Way.
According to claim 1,
The first element-by-element sequence operation,
Equivalent to the second element-by-element sequence operation
Way.
According to claim 1,
The cache,
Retrieving the at least one duplicate element from the cache is a local storage location requiring less computation time than retrieving data from SRAM.
Way.
According to claim 1,
Identifying at least one hardware configuration of the matrix processor in relation to identifying the length of the input vector.
How to further include.
The method of claim 5,
The at least one hardware configuration of the matrix processor,
Convolution parameters
How to include.
The method of claim 6,
The convolution parameter,
One or more addresses representing a three-dimensional data matrix
How to include.
The method of claim 7,
The convolution parameter,
At least one of filter size, number of weights, and stride
Way.
According to claim 1,
Mapping the first matrix data to the input vector,
Aligning the first matrix data with respect to the input vector based at least in part on a hardware configuration of the matrix processor.
How to include.
According to claim 1,
The elements of the input vector,
Identified for at least one of each cycle of each position and convolution operation in the matrix processor
Way.
According to claim 1,
The element-by-element sequence operation,
Performed once for each convolution operation
Way.
According to claim 1,
State machine using at least one of stride and filter sizes to determine the overlapping elements
How to include more.
According to claim 1,
Using the above method for mapping to convolve the input image.
How to further include.
According to claim 1,
The first dimension of the matrix processor,
Corresponding to the number of pixels computed in parallel for a given output channel
Way.
According to claim 1,
The second dimension of the matrix processor,
Corresponding to the number of output channels operating in parallel
Way.
A system for mapping convolutional data to a matrix multiplication circuit to improve computational speed,
A memory device for holding image data;
Control logic; And
A data formatter connected to the control logic and the memory device
Including,
The data formatter,
In response to receiving the convolution command, identifying first matrix data and second matrix data associated with the convolution operation;
Identifying the length of the input vector associated with the matrix processor;
Mapping the first matrix data to the input vector using element-by-element sequence operations;
Identifying at least one element in the first matrix data that overlaps with the second matrix data processed by the matrix processor;
Storing the at least one duplicate element in a cache; And
Mapping the second matrix data to the input vector using the element-by-element sequence operation so that the at least one duplicate element is retrieved from the cache.
How it is configured to perform.
The method of claim 16,
Logic circuit for generating the input vector
The system further comprising.
The method of claim 16,
The matrix processor,
In order to generate dot products for outputting convolution results, a plurality of sub-circuits performing dot-multiplications using the first matrix data and the second matrix data circuits)
System comprising a.
The method of claim 18,
The convolution result,
Which is the output matrix corresponding to the application of the filter to the area of the image.
system.
The method of claim 16,
Sequencer to perform the sequence operation
The system further comprising.
The method of claim 16,
The data formatter,
State machine using at least one of stride and filter sizes to determine the overlapping elements
System comprising a.

Images