JP7055809B2 - Vector reduction processor - Google Patents

Description

Background This specification relates to performing vector reduction in hardware.

In general, vector reduction is an operation performed on an element of an input vector to produce a scalar output, or an output vector with a smaller dimension than the input vector. For example, a total vector reduction operation can produce a scalar output equal to the sum of the elements of the input vector. In some examples, each vector reduction operation can be performed on multiple segments of the input vector. The segmentation vector reduction operation produces an output vector in which each element is a reduction of a segment of the input vector. For example, a segmented total vector reduction operation can generate an output vector in which each element is the sum of the elements of the segment of the input vector.

Overview In general, this specification describes a dedicated hardware circuit that computes vector reduction.

In general, one innovative aspect of the subject matter described herein can be realized in a vector reduction circuit configured to reduce a multi-element input vector. The vector reduction circuit comprises a plurality of cells. Each of the above plurality of cells other than the designated first cell that receives the specified first element of the input vector receives a specific element of the input vector and is temporarily reduced from another cell among the cells. Receive an element, execute a reduction operation using a specific element and a temporary reduction element, and give the result of executing a reduction operation using a specific element and a temporary reduction element as a new temporary reduction element. It is configured. The vector reduction circuit is an output circuit configured to provide a new temporary reduction element corresponding to the result of performing a reduction operation using the last element of the input vector to output the input vector as a reduction. To prepare for.

The implementation may optionally include one or more of the following features: In some implementations, the reduction operation is a maximum value reduction operation, and performing a reduction operation using a specific element and a temporary reduction element can be compared with a specific element and a temporary reduction element. It includes determining which of the specific elements and the temporary reduction element is the largest as a result of performing the reduction operation with the specific element and the temporary reduction element, at least based on comparison. The reduction operation is a maximum value reduction operation, and performing a reduction operation using a specific element and a temporary reduction element is a specific comparison between a specific element and a temporary reduction element, and at least based on the comparison. It includes determining the largest of a particular element and a temporary reduction element as a result of performing a reduction operation using the element and the temporary reduction element.

The implementation may also optionally include one or more of the following features: In some implementations, the reduction operation is a total reduction operation, and executing a reduction operation using a specific element and a temporary reduction element causes a reduction operation using a specific element and a temporary reduction element. Includes calculating the sum of a particular element and a temporary reduction element as a result of the execution. The reduction operation is a product reduction operation, and executing a reduction operation using a specific element and a temporary reduction element is a result of executing a reduction operation using a specific element and a temporary reduction element. Includes finding the product of and the temporary reduction element.

The implementation may also optionally include one or more of the following features: In some implementations, the reduction operation is a maximum index operation, and performing a reduction operation with a particular element and a temporary reduction element receives data indicating the index corresponding to the temporary reduction element. Comparing a specific element with a temporary reduction element, and at least based on the comparison, determining the largest of the specific element and the temporary reduction element, and at least based on the judgment, the specific element and the temporary reduction element. It involves identifying one of the indexes corresponding to a particular element or the index corresponding to a temporary reduction element as a result of performing the reduction operation with the reduction element.

The implementation may also optionally include one or more of the following features: In some implementations, the reduction operation is a minimum index operation, and performing a reduction operation with a particular element and a temporary reduction element receives data indicating the index corresponding to the temporary reduction element. Comparing a specific element with a temporary reduction element, and at least based on the comparison, determining the smallest of the specific element and the temporary reduction element, and at least based on the judgment, the specific element and the temporary reduction element. It involves identifying one of the indexes corresponding to a particular element or the index corresponding to a temporary reduction element as a result of performing the reduction operation with the reduction element.

The implementation may also optionally include one or more of the following features: In some implementations, a particular element of the input vector is received as the staggered input of the element. The input vector is identified as containing a plurality of segments consisting of a plurality of elements. A particular element of an input vector is the specified last element of a particular segment of the input vector, and each of a plurality of cells other than the designated first cell that receives the specified first element of the input vector. Gives the output circuit the result of performing a reduction operation with the specified last element and temporary reduction element of a particular segment of the input vector, assuming that a particular segment of the input vector has been scaled down. It is composed. Giving the result of performing a reduction operation using the specified last element and the temporary reduction element of a particular segment of the input vector as a new temporary reduction element can result in another cell of multiple cells. It involves giving a placeholder signal as a new temporary reduction element. The output circuit is configured to give multiple elements to output as a reduced version of the input vector, each of which performs a reduction operation using the last specified element of the segment of the input vector. It is a new temporary reduction element that corresponds to the result of execution. The output circuit is configured to give an output vector containing a plurality of elements in order to output the input vector as a reduced version.

The implementation may also optionally include one or more of the following features: In some implementations, the vector reduction circuit comprises a control register configured to receive a control vector consisting of a plurality of elements, and the control vector consisting of the elements specifies a plurality of segments consisting of the elements of the input vector. .. Giving the result of performing a reduction operation using a specific element and a temporary reduction element as a new temporary reduction element is to receive a specific element of the control vector corresponding to the specific element, and at least of the control vector. Based on a particular element, determine that a particular element is designated as the last element of a particular segment of the input vector, and at least based on the judgment, to the output circuit, the end of a particular segment of the input vector. It includes giving the result of performing a reduction operation using a specific element specified as an element of and a temporary reduction element as a reduction of a specific segment of the input vector.

The implementation may also optionally include one or more of the following features: In some implementations, giving the result of performing a reduction operation using a specific element and a temporary reduction element as a new temporary reduction element receives a specific element of the control vector corresponding to the specific element. And, at least based on a particular element of the control vector, that a particular element is designated as the last element of a particular segment of the input vector, and at least based on the judgment, of multiple cells. Includes giving a placeholder signal to another cell. Giving the result of performing a reduction operation using a specific element and a temporary reduction element as a new temporary reduction element is to receive a specific element of the control vector corresponding to the specific element, and at least of the control vector. Based on a particular element, determine that a particular element is not designated as the last element of a particular segment of the input vector, and at least based on the determination, the input vector for another cell of multiple cells. It includes giving the result of performing a reduction operation using a specific element that is not specified as the last element of a specific segment and a temporary reduction element as a new temporary reduction element.

The implementation may also optionally include one or more of the following features: In some implementations, the output circuit gives a reduced version of the input vector to the permutation circuit configured to compute the permutation of a vector or matrix. Each of the plurality of cells is configured to receive a control signal that specifies the type of reduction operation performed with the particular element and the temporary reduction element. The temporary reduction element corresponds to the result of performing a specified reduction operation on the element of the input vector that precedes the specific element of the input vector.

Certain embodiments of the subject matter described herein can be realized to obtain one or more of the following advantages: Vector reduction can be performed in hardware by a dedicated hardware circuit. Dedicated hardware circuits can perform vector reduction in fewer cycles than general purpose processors (eg GPUs or CPUs). In addition, by computing the vector reduction in hardware using a dedicated hardware circuit, the vector reduction can be performed off-chip or in software without sending the data back to the host computer, that is, at least part of the calculation. It is possible to do without. As a result, the processing delays that occur in off-chip or software vector reduction calculations that can be costly calculations that require more general purpose processor (eg GPU or CPU) cycles than dedicated hardware circuits are avoided.

By using hardware circuits specially designed to compute vector reductions, for example, more matrix convolution or other matrices-than systems that perform vector reductions in common matrix or vector processing hardware circuits. Than a system that is configured to perform vector computations, or, for example, a system that performs vector reductions in another dedicated hardware circuit that is configured to perform both vector reduction and matrix transpose computations. , It is possible to perform processing with high efficiency. By realizing vector reduction operations on a dedicated hardware circuit, other matrix-vector processing circuits can be processed efficiently without considering other matrix-vector calculation functions or efficiency, and other matrix or vector processing circuits can be used. Designs that comprehensively increase the efficiency of matrix-vector computations in hardware are possible by ensuring that they perform matrix or vector computations.

Details of one or more embodiments of the subject matter herein are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of this subject will become apparent from this description, drawings, and claims.

It is a figure which shows an example of a matrix-vector processing system. It is a figure which shows an example of a matrix-vector processing system including a vector reduction unit. It is a figure which shows an example of the architecture of the vector reduction unit in a matrix-vector processing system. It is a figure which shows an example of the architecture of the vector reduction unit in a matrix-vector processing system. It is a figure which shows an example of the architecture of the cell of the vector reduction unit in a matrix-vector processing system. It is a flow diagram of an example of the method of performing vector reduction in a matrix-vector processing system. It is a figure which shows an example of the vector reduction using the vector reduction unit in a matrix-vector processing system. It is a figure which shows an example of the vector reduction using the vector reduction unit in a matrix-vector processing system.

Similar reference numbers and names in various drawings indicate similar elements.
Detailed Description Vector reduction is performed on a multi-element input vector to produce an output that is a scalar, or a vector that is smaller than the input vector, that is, a low-dimensional multi-element vector. It is an operation for. For example, the total vector reduction operation produces a scalar output that is the sum of the elements of the input vector. As another example, the maximum or minimum vector reduction operation produces a scalar output, which is the largest of the elements of the input vector or the smallest of the elements of the input vector, respectively. Similarly, the maximum index or minimum index vector reduction operation produces an output indicating the index of the input vector of the maximum value of the input vector or the minimum value of the input vector, respectively. Other vector reduction operations are possible, for example, a product vector reduction operation that produces a scalar output that corresponds to the product of the elements of the input vector, and an average vector reduction operation that produces a scalar output that corresponds to the arithmetic average of the elements of the input vector. And so on.

In some implementations, the input vector may be segmented and each vector reduction operation may be performed on one or more of the segments of the input vector. An output vector is generated by performing a corresponding vector reduction operation on each of the plurality of segments of the input vector. Each element of the output vector corresponds to the result of performing a vector reduction operation on the corresponding segment of the input vector. For example, the output of a vector reduction operation performed on an input vector divided into three segments is an output vector consisting of three elements, and each element of this output vector is a vector for each segment of the input vector. This is the result of applying the reduction operation.

Vector reductions are often calculated in linear algebra and in data analysis for other real-world applications. For example, vector reduction can be calculated when training a neural network, for example as part of a machine learning process. In these examples, vector reduction can be used to backpropagate the gradient when training a neural network. Vector reduction can also be performed on the output of the layer of the neural network. The result of vector reduction can be given as an input to another layer of the neural network or as an output from the neural network. Other applications of vector reduction include multimedia processing in digital audio or video decoding. Signal processing, such as in the fields of analog and digital transmission, radar, and sonar, also uses frequent use of vector reduction techniques to specify, for example, the maximum or minimum in a received or transmitted signal.

In some implementations, the input matrix may be decomposed into vectors, or vector reduction may be applied individually to the vectors of the input matrix. For example, an image represented as a matrix, for example, in which the elements of the matrix correspond to the pixels of the image, can be decomposed into vectors. By applying a vector reduction operation such as the average vector reduction operation to the vector of the input matrix, it is possible to perform image smoothing or reduce the pixel size of the image, for example.

This specification describes a dedicated hardware circuit that executes a vector reduction operation on an input vector. This dedicated hardware circuit can generate a scalar or vector output that corresponds to a reduced input vector.

FIG. 1 shows a matrix-vector processing system 100 as an example. The matrix-vector processing system 100 is an example of a system realized as one or more computers in one or more places where the following systems, components, and techniques can be realized.

The matrix-vector processing system 100 is a system that executes vector or matrix calculation using a dedicated hardware circuit 110. The dedicated hardware circuit 110 is an integrated circuit for performing a vector or matrix calculation, including a vector reduction unit 120 configured to calculate the vector reduction in hardware. The dedicated hardware circuit 110 as an example will be described in more detail with reference to FIG.

The matrix-vector processing system 100 receives a request to perform a vector or matrix calculation on the dedicated hardware circuit 110, controls the dedicated hardware circuit 110 to perform the vector or matrix calculation, and performs the dedicated hardware. The result of the vector or matrix calculation generated by the wear circuit 110 is output. For example, the matrix-vector processing system 100 can receive a request to perform a vector reduction on an input vector, thereby calculating the vector reduction and outputting the calculation result of the vector reduction on the input matrix. The dedicated hardware circuit 110 can be controlled as described above. The dedicated hardware circuit 110 may be capable of performing other calculations in addition to vector reduction. For example, the dedicated hardware circuit 110 may also be a circuit or circuit for vector or matrix convolution, vector or matrix transposition, vector or matrix transpose, or other vector or matrix computation, for example to process layers of a neural network. May include other hardware components. Thus, the dedicated hardware circuit 110 can calculate the vector reduction as part of performing a larger calculation using the other components of the dedicated hardware circuit 110.

To implement matrix or vector computation on the dedicated hardware circuit 110, the matrix-vector processing system 100 includes a matrix-vector processing engine 150. The matrix-vector processing engine 150 can be implemented as one or more computer programs on one or more computers in one or more physical locations.

The matrix-vector processing engine 150 can generate instructions, give control signals, or derive data to control the dedicated hardware circuit 110 to perform a vector or matrix calculation on demand. For example, the matrix-vector processing system 100 may be required to apply a function to an input vector or matrix, and the matrix-vector processing engine 150 may receive the above on an input vector or matrix on a dedicated hardware circuit 110. Specific instructions or control signals can be determined to perform the calculations required to apply the function. The matrix-vector processing engine 150 can also derive data corresponding to an input vector or matrix that the dedicated hardware circuit 110 can use, for example, to perform the required calculations.

The matrix-vector processing engine 150 determines the calculation required to realize the vector or matrix request and controls the dedicated hardware circuit 110 to perform this required calculation. For example, the matrix-vector processing engine 150 may direct data for performing a vector or matrix calculation, such as an input vector or matrix, to a dedicated hardware circuit 110. The matrix-vector processing engine 150 is an instruction or control signal for controlling the dedicated hardware circuit 110 to perform necessary calculations on the data received from the matrix-vector processing engine 150 by the dedicated hardware circuit 110. It may be transmitted to the dedicated hardware circuit 110.

For example, the matrix-vector processing system 100 can receive a request to perform a vector or matrix calculation. The required calculation may be a relatively simple, eg, a request to perform a simple linear algebraic calculation, or a more complex function, eg, a function for backpropagating a gradient to train a neural network. be. The required computation may require one or more operations, such as one or more vector reductions. The request may also identify or include one or more vectors or matrices from which the calculation is to be performed, i.e., one or more input vectors or matrices to which the function applies. The matrix-vector processing engine 150 can accept this request and generate control signals or instructions to perform calculations on the input vector or matrix. The matrix-vector processing engine 150 can further direct the input vector or matrix to the dedicated hardware circuit 110.

For example, in order to calculate vector reduction, for example as part of a larger calculation, the matrix-vector processing engine 150 has obtained the input vector received or by dividing the input matrix into its vector components. The vector may be given to the dedicated hardware circuit 110. The matrix-vector processing engine 150 may also provide a control signal for initiating vector reduction on the vector reduction unit 120 or a parameter for performing vector reduction to the dedicated hardware circuit 110. The vector reduction unit 120 can receive one or more input vectors and a control signal given to the dedicated hardware circuit 110. The vector reduction unit 120 can execute the vector reduction in response to receiving the control signal, and can output the result of executing the vector reduction. The result generated by the vector reduction unit 120 may be used in other operations performed by the dedicated hardware circuit 110 to perform the requested calculation. The dedicated hardware circuit 110 can provide the result of the requested calculation, and the matrix-vector processing system 100 can return this result on demand.

FIG. 2 shows an example dedicated hardware circuit 200 capable of performing vector reduction. In some implementations, the circuit 200 may include additional components (not shown) for performing other vector or matrix calculations. Additional components for executing other vectors or matrices may also utilize one or more of the components shown in FIG.

Circuit 200 includes host interface 202. Host interface 202 can receive control signals, instructions, or arguments for performing vector or matrix calculations, including vector reduction. The argument may include, for example, one or more vectors for which vector reduction is to be performed. The instruction or control signal received by the host interface 202 can include instructions indicating where to store the received arguments so that the circuit 200 can perform vector reduction, or includes parameters related to the requested vector reduction. It may contain memory locations for obtaining or storing other information, such as the output of vector reductions. The control signal received by the host interface 202 can also control the circuit 200 to initiate the vector reduction calculation.

In some implementations, the host interface 202 can give instructions to sequencer 206, which translates the instructions into low-level control signals that control circuit 200 to perform vector reduction. For example, the control signal generated by sequencer 206 can manage the data flow in circuit 200, for example, where the input vector should be stored, or otherwise how the data should be routed through circuit 200. You can control what you should lead to. The sequencer 206 can receive an instruction to start the vector reduction calculation on the circuit 200, and can generate a control signal for controlling the vector reduction unit 212 to start the vector reduction calculation.

The sequencer 206 can send control signals to the memory 208 and the vector reduction unit 212. In some implementations, the sequencer 206 also sends control signals to the direct memory access engine 204. In some implementations, the sequencer 206 is a processor that produces control signals. The sequencer 206 can adjust the control signal to send it to the appropriate component of circuit 200 at the appropriate time. In some examples, the sequencer 206 receives a control signal from outside the circuit 200, eg, from the vector-matrix processing engine 150 of FIG. 1, from the host interface 202, so that the sequencer 206 generates the control signal. You may not be asked to do that. In such an example, the sequencer 206 may send the received control signal to the components of circuit 200 at the appropriate time. In addition, if a control signal is given to the circuit 200, the sequencer 206 may be an optional component of the circuit 200, i.e., an external component of the circuit 200, such as a matrix-vector processing engine 150, to provide the control signal. Given at the appropriate time, the circuit 200 may be controlled to perform operations including operations for performing vector reduction.

Host interface 202 can send an argument, eg, an input vector, to the direct memory access engine 204. The direct memory access engine 204 can store this argument in memory 208.

The memory 208 may be a dynamic memory, such as a dynamic random access memory (DRAM), or a static memory, such as a static random access memory (SRAM). In some implementations, memory 208 may be used as a memory buffer. The memory 208 may be on the circuit 200 or outside the circuit 200. It can store an argument, eg, a vector, input to circuit 200, or a parameter for performing vector reduction. The memory 208 can also store the output of the vector reduction unit 212, i.e., the result of performing vector reduction on the input vector. In some implementations, the direct memory access engine 204 can read from memory 208. For example, the direct memory access engine 204 can return the execution result of vector reduction for the input vector from the circuit 200 by reading from the memory 208.

The vector reduction unit 212 can access the arguments of memory 208. For example, after the direct memory access engine 204 stores the input vector in memory 208, the vector reduction unit 212 gives the input vector to the vector reduction unit 212 or makes it accessible so that the vector reduction unit 212 can reduce the vector relative to the input vector. You may be able to perform operations for this.

The vector reduction unit 212 is a circuit for calculating the vector reduction. In some embodiments, the vector reduction unit 212 should receive an argument, eg, an input vector, and information that specifies the type of vector reduction, eg, the vector reduction should be a total vector reduction or a maximum or minimum vector reduction. It is designed to calculate vector reductions based on information that specifies such things as. The information that specifies the type of vector reduction may include other information, such as information for performing segmented vector reduction. In some embodiments, the vector reduction unit 212 is configured to perform a predetermined vector reduction operation such that the vector reduction unit 212 requires only an argument or input vector to perform the vector reduction. May be done. For example, the vector reduction unit 212 may be configured to perform only total vector reduction operations or segment vector reduction on predetermined elements of the input vector. After receiving this information, the vector reduction unit 212 receives additional control signals or inputs so that if the vector reduction unit 212 receives the required inputs, ie, the input vector and the vector reduction parameters, the vector reduction is otherwise automated. Vector reduction may be performed without receiving. In such an implementation, once the vector reduction operation is started, the vector reduction unit 212 can perform the entire vector reduction based on the received information without requesting additional information from the off-chip. .. In some implementations, the vector reduction operation may be triggered in response to the vector reduction unit 212 receiving the required information, i.e. no start signal or other trigger is required. In other implementations, the start signal or other signal may trigger the vector reduction unit 212 to perform a vector reduction operation.

In general, to calculate vector reduction, the vector reduction unit 212 indicates the type of vector reduction operation to perform, eg, total vector reduction, maximum or minimum vector reduction, or maximum or minimum index vector reduction. Receive a control signal. For example, the vector reduction unit 212 can receive a control signal from the sequencer 206. The input vector is received from memory 208, for example, as a staggered data stream. The vector reduction unit 212 executes the vector reduction operation indicated by the control signal on the elements of the input vector. For example, the vector reduction unit 212 may perform a vector reduction operation upon receiving each element of the input vector. Alternatively, the vector reduction engine 212 may receive all elements of the input vector or part of the elements of the input vector, with respect to the elements of the input vector or part of the input vector. After receiving this, the vector reduction operation may be executed. In some examples, the result of the vector reduction operation does not depend on the order of the elements of the input vector to which the vector reduction operation is applied, for example in the case of total vector reduction. In such an example, the vector reduction unit 212 relatives to the elements of the input vector in a different order than the arrangement order of the elements in the input vector, or in a different order than the order in which the vector reduction unit 212 received these elements. It may be determined that the vector reduction operation is executed. After the vector reduction operation has been applied to all elements of the input vector, the vector reduction unit 212 can output the result of the vector reduction to, for example, memory 208 or to another component of the dedicated hardware circuit 200.

Since the input to the vector reduction unit 212 may be a staggered input, the vector reduction unit 212 may simultaneously perform vector reduction on a plurality of input vectors, for example in a pipeline fashion. That is, while the vector reduction unit 212 is executing the vector reduction operation on the first input vector, the vector reduction unit 212 may start executing the vector reduction operation on the second input vector. In addition, in some examples, a matrix may be given to circuit 200 as an argument. The input matrix may be decomposed into vectors, for example, by obtaining the row or column vectors that together form the input matrix. The vector reduction unit 212 can generate a vector of the vector reduction result by executing the vector reduction operation on the vector forming the input matrix. The resulting vector may be given, for example, to another component of circuit 200 as the output of the vector reduction unit 212.

FIG. 3 shows an example of the architecture of the vector reduction unit 300. The vector reduction unit 300 may be included in the dedicated hardware circuit of the matrix-vector processing system, for example, as the vector reduction unit 212 of the dedicated hardware circuit 200.

The value loader 310 of the vector reduction unit 300 receives data corresponding to one or more input vectors. Each value loader 310 corresponds to, for example, a lane of data received from memory 208 of the dedicated hardware circuit 200. In fact, each lane of data, and thus each value loader 310, corresponds to a different column or row of input vector or matrix. Therefore, although it is shown in FIG. 3 that it includes only the value loaders [0] to [3] and therefore enables the reduction of vectors of 4 × 1 or less, there are many vector reduction units 300 in other implementation forms. The value loader 310 of may be included. For example, a dedicated hardware circuit configured to operate on a 128x128 element matrix or 128x1 vector transmitted as 128 lanes of data may include a corresponding 128 value loader 310.

The value loader 310 can receive the data corresponding to the input vector in the staggered state. As used herein, staggered data is data transmitted with offsets in multiple lanes, so that the data stream in each lane is several cycles behind the data stream in another lane. .. For example, the data stream in the first lane corresponding to the value loader [0] may be one cycle ahead of the data stream in the second lane corresponding to the value loader [1]. This is different from, for example, flat data in which the data streams of each lane received by each of the value loaders 310 are aligned. For example, the value loader 310 receives a vector consisting of elements transmitted in a plurality of lanes in the same clock cycle.

In general, the vector reduction unit 300 and other components of the dedicated hardware circuit, including the vector reduction unit 300, perform more efficient calculations on the staggered data by performing certain operations in parallel. You will be able to do it. For example, as shown in more detail with respect to FIGS. 4 and 7A-7B, processing on staggered data allows the vector reduction unit 300 or other components for performing matrix transpose or matrix convolution to perform certain operations in parallel. You can do this, or you can optimize the processing of consecutive input vectors or matrices. For example, by processing the staggered data corresponding to a plurality of input vectors, the vector reduction unit 300 can execute the vector reduction operation for the plurality of input vectors in the same cycle. As shown in detail in FIG. 4, the vector reduction unit 300 can receive staggered data such that elements of different input vectors are given to the vector reduction unit 300 during the same cycle. The vector reduction unit 300 can then perform operations to calculate the reduction of different input vectors in one cycle.

The value loader 310 receives the data corresponding to the input vector and feeds this data to the input register 320. If the data received by a particular value loader 310 does not correspond to an element of the input vector or another element of the input vector, the value loader 310 can ignore or discard the data. In these cases, the input register 320 may not receive data from this value loader 310, or may receive a full, positive, or negative infinite value, or an element received from this value loader 310. May receive other data indicating that it does not contain elements of the input vector. For example, if the value loader 310 receives a 3x1 input vector, the value received by the value loader [3] may be ignored or discarded because it does not correspond to an element of the input vector. .. However, if the value received by the value loader [3] corresponds to a previous input vector, eg, a 4x1 input vector whose first element is received before the first element of the 3x1 input vector. , The value received by the value loader [3] can be given to the input register 320.

In some implementations, the vector reduction unit 300 is configured to perform the vector reduction operation only while the reduction unit 300 is processing the vector input data. For example, the vector reduction operation may be triggered when the first element of the input vector is received. This can improve the efficiency of the circuit, for example by reducing power consumption, and other data received by the input register 320 before the first element of the input vector that does not correspond to another element of the input vector. It can be guaranteed that the execution result of vector reduction for the input vector is not affected. For example, the value loader 310 receives staggered data, so if the vector reduction operation is triggered when the value loader [0] receives the first element of the input vector, any data received during the preceding cycle. It does not affect the reduction result of the input vector. The input register 320 receives and stores data corresponding to the input vector. For example, when the value loader 310 receives the elements of the input vector and sends these elements to the input register 320, these elements are stored in the input register. The input register 320 may include more than a number of elements of the value loader 310 in order to receive data from the value loader 310. Since the data received from the value loader 310 is staggered, the input register 320 does not contain all the elements of the input vector at any given time, but rather typically contains only one element of the input vector. In addition, since the data is staggered, the input register 320 may include elements of a plurality of input vectors at the same time if the vector reduction unit 300 receives consecutive input vectors.

The compute unit 330 of the vector reduction unit 300 receives or accesses an element stored in the input register 320. The compute unit 330 may include a plurality of cells, each corresponding to an element of the input register 320 or to a particular value loader 310. For example, if the vector reduction unit 300 includes four value loaders 310, the calculation unit 330 may include four cells to perform the vector reduction operation.

The calculation unit 330 also receives the vector reduction control signal 305 and the data from the control register 315. The vector reduction control signal 305 can specify a particular type of vector reduction operation to be performed by the compute unit 330. For example, the vector reduction control signal 305 may be a total vector reduction operation, a maximum value reduction operation, a minimum vector reduction operation, a maximum index vector reduction operation, a minimum index vector reduction operation, a product vector reduction operation, an average vector reduction operation, or another operation. You can specify a vector reduction operation. The vector reduction unit 300 can receive the vector reduction control signal 305 from the sequencer 206 of the dedicated hardware circuit 200, from the matrix-vector processing engine 150 of the matrix-vector processing system 100, or from another source.

The control register 315 can receive and store the control vector. The control vector specifies whether and how the compute unit 330 should perform segmentation vector reduction for the received input vector. As will be described in more detail with respect to FIG. 4, the control vector includes an element that can be given as a control signal to the cell of the calculation unit 330. These elements are whether each cell of calculation unit 330 should give the result of the vector reduction operation performed in this cell to another cell of calculation unit 330, or the vector executed in this cell. It can indicate whether the result of the reduction operation should be output as a reduction of the segment of the input vector. In other words, each element of the control vector can be given as a control signal to the corresponding cell, indicating whether or not this cell is processing the endpoints of the segments of the input vector.

When the result of a vector reduction operation is given from one cell to another, no segmentation vector reduction occurs in that cell. So, for example, control that controls a cell so that every cell, or every cell except the last cell, gives the result of a vector reduction operation performed within that cell to another cell of compute unit 330. When receiving a signal, the vector reduction operation indicated by the vector reduction control signal 305 is applied to all the elements of the input vector. Thus, the output of the compute unit 330, eg, the output of the last cell of the compute unit 330, is the result of a non-segmented vector reduction operation that reduces the entire input vector to produce one result.

When a particular cell of compute unit 330 does not give the result of a vector reduction operation to another cell, i.e., when no segmentation vector reduction operation occurs, null, i.e. zero, or a placeholder signal such as positive or negative infinity , This particular cell may be given to another cell of the compute unit 330. The result is the segmentation reduction that occurs in the elements of the input vector received by a particular cell. This is because the result of the vector reduction operation performed by a particular cell is output as the result of performing segmentation vector reduction on the elements of the input vector up to this element, including the elements of the input vector received by this particular cell. Because it is done. Since null or positive or negative infinity is given from one cell to another cell of the compute unit 330, the rest of the input vector by effectively initiating a new vector reduction calculation for the remaining elements of the input vector. Generate a second part of the segmentation vector reduction for the elements of.

The cell of the calculation unit 330 performs a reduction operation to generate the result of input vector reduction. Each cell can process a particular element of the input vector. To do so, the cell receives a temporary shrinking element, as described in more detail in FIG. The temporary reduction element is the result of a vector reduction operation performed by the preceding cell of the compute unit 330, or is a placeholder signal or initialization value, such as null or positive or negative infinity. This cell also receives a specific element of the input vector and performs a vector reduction operation specified by the vector reduction control signal 305 using the temporary reduction element and the specific element of the input vector. If no segmentation reduction operation is required, the cell gives the next cell of calculation unit 330 the execution result of the specified vector reduction operation. When a segmentation reduction operation is requested, the cell gives a placeholder, for example null or positive or negative infinity, to the next cell. This cell also outputs the execution result of the specified vector reduction operation corresponding to the execution result of the segmentation vector reduction for the elements of the input vector up to this element including the specific element of the input vector. If no segmented vector reduction operation is required, the result of performing the specified vector reduction operation using the last element of the input vector, which corresponds to the result of performing the vector reduction operation on all elements of the input vector. , May be provided for output.

The value output 340 receives the output of the vector reduction process executed by the calculation unit 330. The value output 340 can receive the outputs and write these outputs in a staggered manner. Since any of the cells of the calculation unit 330 can be configured to provide the result of the vector reduction, the vector reduction unit 300 includes value outputs [0] to [3] corresponding to the cells of the calculation unit 330. For example, if the second cell of compute unit 330 receives a signal from control register 315 indicating that this cell should perform segmentation vector reduction in that cell, then the second cell of compute unit 330 , The output can be given to the value output [1]. If no segmentation vector reduction is instructed, the result of the vector reduction can be given to the last cell of the compute unit 330 corresponding to the value output [3] for output. In some examples, only one value output 340 may be required, for example all the outputs from the cells of the compute unit 330 are given to the same value output 340. When the segmentation vector reduction operation is requested, the value output vector reduction unit 300 can output the resulting vector, and each element of this vector is a reduction of the segment of the input vector. The value output 340 can give the elements of the output vector to a register, eg the output register, and the result of the segmented input vector reduction is vector formatted for output, eg, as a staggered output vector output from the vector reduction unit 330. Can be given together at.

The value output 340 receives the vector reduction result from the compute unit 330 and is stored in the memory of the dedicated hardware circuit, eg, memory 208, in another component of the dedicated hardware circuit, eg, in a matrix or vector permute unit, or It can be output to an off-chip location, for example to another destination outside the matrix-vector processing engine 150 or the matrix-vector processing system 100.

FIG. 4 shows the architecture 400 as an example of the computational unit of the vector reduction unit. The architecture 400 includes, for example, a memory 410 similar to the memory 208 of FIG. 2, a vector reduction control signal 405 similar to the vector reduction control signal 305 of FIG. 3, and a control register 415 similar to the control register 315 of FIG. For example, the value output 440 similar to the value output 340 in FIG. 3 is included. The architecture also includes cell 430, which corresponds to the cell of compute unit 330 in FIG. The cells shown in FIG. 4 are serially connected, that is, cell [0] is connected to cell [1], cell [1] is connected to cell [2], and so on. In other implementations, cells 430 may be connected in any other order, for example, cell [1] receives data from cells other than cell [0] and feeds the data to cells other than cell [2]. You may.

Each of the cells 430 receives a staggered data lane from memory 410. In one example, cell 430, although not shown in FIG. 4, can receive data from a value input similar to the value loader 310 in FIG. For example, the chart 450 can show how data is written staggered in each of the four lanes corresponding to the four cells [0] to [3]. Therefore, in cycle 0, cell [0] can receive the first element of the input vector [0], and in cycle 1, cell [0] can receive the first element of the input vector [1]. [1] can receive the second element of the input vector [0], and so on.

Each cell 430 also receives a vector reduction control signal 405 that specifies a vector reduction operation to be performed. As shown in FIG. 4, the same vector reduction control signal 405 can be given to each of the cells 430. However, in other embodiments, different vector reduction control signals can be given to each or a subset of cells 430. By feeding different vector reduction control signals to different cells 430, the architecture 400 can allow different types of vector reduction operations to be performed simultaneously. For example, by giving the control signal to the cell 430, the cell 430 can be made to perform the total vector reduction for the first input vector and the maximum value vector reduction for the second input vector. By inputting the input vector in a staggered manner, the cell 430 may be able to execute these different vector reduction operations at the same time. Similarly, when performing a segmentation vector reduction operation, a first type of vector reduction operation, such as total vector reduction, is performed on the first segment of the input vector and on the second segment of the input vector. A second type of vector reduction operation, such as a maximum value vector reduction operation, may be performed. In some implementations, cell 430 is configured to perform one type of vector reduction operation, or to segment the vector reduction operation at the same element position for each input vector. In these implementations, the vector reduction control signal 405 or control register 415 may be any component of the architecture 400.

Further, each cell 430 receives an element of the control vector stored in the control register 415, which acts as a control signal for controlling the segmentation vector reduction in the architecture 400. The control register 415 can receive a vector of control signals, or can receive individual control signals stored in a particular element of control register 415. Each control signal indicates whether cell 430 should perform a segmented vector reduction operation in cell 430 or a non-segmented vector reduction operation in cell 430 by specifying a value to be interpreted by cell 430. be able to.

For example, a value zero received by cell 430 from control register 415 means that cell 430 must not perform a segmented vector reduction operation in cell 430, i.e. cell 430 is an input that should be treated as an endpoint of a segment of the input vector. It can be shown that it has not received a vector element. To perform a non-segmented vector reduction operation, cell 430 is an initialization value or placeholder signal, eg null or positive or negative infinity, or the result of a previous vector reduction operation from the previous cell 430. It can receive the corresponding temporary reduction element shown as <value> in FIG. Cell 430 also includes the index shown as <index> in FIG. 4, which indicates the particular cell 430 corresponding to the lane of staggered input data, eg, used to perform the maximum index or minimum index vector reduction operation. Can receive. In some implementations, the index may be a memory location, eg, a location in memory 208 where the input vector was or was stored. Cell 430 also receives an element of the input vector from memory 410. Cell 430 executes the vector reduction operation specified by the vector reduction control signal 405, and gives the result of the vector reduction operation and optionally the index obtained from the vector reduction operation to the next cell 430 of the architecture 400. .. Optionally, cell 430 can also give these results to the value output 440. The next cell 430 can receive the result of the vector reduction operation from the cell 430, and this result can be used in the next cell 430, for example, in the vector reduction operation to be executed in the next cycle.

Instead, if cell 430 receives a value of 1, cell 430 can perform a segmentation vector reduction operation and treat the received element of the input vector as the endpoint of the segment of the input vector. To do so, cell 430 outputs the result of the vector reduction operation performed in cell 430 as the result of performing the vector reduction operation on the segment of the input vector ending in the received element. To perform a segmented vector reduction operation, cell 430 can receive an initialization value or a temporary reduction element that corresponds to the result of the previous vector reduction operation from the previous cell 430. Cell 430 can also be indexed from the previous cell 430. Cell 430 also receives an element of the input vector from memory 410. Cell 430 performs the vector reduction operation specified by the vector reduction control signal 405. However, cell 430 does not give the result to the next cell 430 of architecture 400, but only to the value output 440 as a result of the segmentation vector reduction operation. Cell 430 sends null or positive or negative infinity to the next cell 430 as an initialization value or placeholder signal for the next vector reduction to start in the next cell 430.

For example, cell [1] can receive from cell [0] a temporary reduction element corresponding to the result of cell [0] performing the vector reduction operation specified by the vector reduction control signal. The result of the vector reduction operation performed by cell [0] may be the first element of the input vector [0]. For example, if the vector reduction control signal 405 specifies a total vector reduction operation, the result output from cell [0] to cell [1] will be the first element of the input vector [0]. Cell [1] may also receive a second element of input vector [0]. Then, the cell [1] can execute the vector reduction operation specified by the vector reduction control signal 405, for example, the total vector reduction operation, and generate a result. For example, if the specified vector reduction operation is a total vector reduction operation, the result is the sum of the first and second elements of the input vector [0].

Based on the control signal received from the [1] position of the control register 415, the cell [1] can give this result to the cell [2] and / or to the value output [1]. For example, if the control signal has a value of zero that specifies a non-segmented vector reduction operation for cell [1], cell [1] can give the result to cell [2], optionally this result. It can also be given to the value output [1] to output, for example, the intermediate result of the vector reduction operation. When the control signal has a value 1 that specifies the segmentation vector reduction operation for the cell [1], the cell [1] outputs the result to the value output [1], thereby outputting the result of the segmentation vector reduction operation. , For example, the sum of the first two elements of the input vector [0] is output. However, cell [1] does not output this result as a temporary reduction element to cell [2]. Rather, cell [1] gives the cell [2] null or positive or negative infinity, and cell [2] uses this received value as a temporary reduction factor. Therefore, when the cell [2] receives the third element of the input vector [0] from the memory 410, the vector reduction operation performed by the cell [2] is the second segment of the input vector [0] with respect to the second segment. It will be the first operation performed for the vector reduction operation. The result of the vector reduction operation, that is, the result when the segmentation vector reduction operation is executed, is output to the value output 440 and can be given to the dedicated hardware circuit 200 or other components of the matrix-vector processing system 100. can.

When the vector reduction operation indicated by the vector reduction control signal 405 specifies a minimum index vector reduction operation or a maximum index vector reduction operation, each of the cells 430 has a specific cell 430 corresponding to one lane of staggered input data. An index can be provided to indicate. In some examples, such an index may always be output from cell 430 when a vector reduction operation is performed, but is used unless the vector reduction control signal 405 specifies a maximum or minimum index vector reduction operation. It will not be possible data. If the vector reduction control signal 405 specifies one of these vector reduction operations, both the <value> output and the <index> output of cell 430 will be required to accurately determine the output. For example, when the maximum index vector reduction operation is specified, cell [0] has the value of the first element of the input vector [0] received by cell [0] as a temporary reduction element for cell [1]. It is possible to output an index [0] indicating that the first lane of the input data is an index having the element of the maximum value at present. Cell [1] receives this value from cell [0] and can compare this value from cell [0] with the value of the second element of the input vector [0]. If the value of the second element of the input vector [0] is greater than the value of the first element of the input vector [0], the cell [1] is in the cell [2] and the second element of the input vector [0]. The value can be output as a temporary reduction element and the index [1] can be output to cell [2] to indicate that the second lane of the input data is currently the index with the maximum value element. .. This process can be repeated in each cell 430 depending on whether the control vector specifies a segmented vector reduction operation or a non-segmentation vector reduction operation. After the vector reduction operation is complete, cell 430 outputs the index value to the value output 440 instead of outputting the temporary reduction element or value.

In some examples, the initialization element used by the system may be determined by the vector reduction operation specified by the vector reduction control signal 405. For example, if a total vector reduction operation is specified, the null value can be used as the initialization value. When the product vector reduction operation is specified, the unit value 1 can be used as the initialization value. Negative infinity can be used as the initialization element if a maximum value vector reduction operation or a maximum index vector reduction operation is specified. If a minimum vector reduction operation or a minimum index vector reduction operation is specified, positive infinity can be used as the initialization element. In some examples, the initialization value used for a particular vector reduction operation is indicated or given by a signal received from an external vector reduction unit 400, eg, as a second signal similar to the vector reduction control signal 405. .. In other implementations, cell 430 may be capable of determining initialization elements based on the vector reduction operation specified by the vector reduction control signal 405. For example, cell 430 can determine that null should be used as an initialization element based on the fact that cell 430 has received a signal specifying a total vector reduction operation.

FIG. 5 shows the architecture 500 as an example of the cells of the vector reduction unit. For example, each of cells 430 in FIG. 4 may have an architecture similar to architecture 500.

Architecture 500 as an example of this includes a temporary reduction element register 502 that receives a temporary reduction element. The temporary reduction element can be received from another cell, for example, from another cell in cell 430 of FIG. In some examples, the temporary reduction element register 502 can be initialized to, for example, null, positive infinity, or negative infinity. For example, cell [0] of the vector reduction unit 400 does not receive the result of the vector reduction operation executed by the previous cell, so that the temporary reduction element register 502 may be initialized to zero. Architecture 500 also includes an input vector element register 504 that receives the elements of the input vector. For example, the input vector element register 504 of the cell [0] of the vector reduction unit 400 can receive and store the first element of the input vector [0], and the input vector of the cell [1] of the vector reduction unit 400. The element register 504 can receive and store the second element of the input vector [0], and so on.

Temporary reduction elements can be given to the multiplexer 508 and elements of the input vector can be given to the multiplexer 510. Each multiplexer 508, 510 may be controlled by a vector reduction control signal 505, the vector reduction control signal 505 controlling the multiplexer 508, 510 to give each element to a particular computational circuit 512, 514, 516. Acts as a selection signal for.

For example, the vector reduction control signal 505 can indicate that a maximum value vector reduction operation or a minimum value vector reduction operation is scheduled to be performed, and in response to this signal 505, the multiplexer 508, 510 is a temporary reduction element. Input vector elements can be given to a comparison circuit 512 configured to find the minimum or maximum of these elements by comparing the elements. Similarly, if the vector reduction control signal 505 indicates that a total vector reduction operation is to be performed, the multiplexer 508, 510 receives the signal 505 and accordingly a temporary reduction element and an input vector element. It can be given to a totaling circuit 514 configured to sum the elements. If the vector reduction control signal 50 indicates that the product vector reduction operation is to be performed, the multiplexer 508, 510 is configured to calculate the product of the temporary reduction element and the input vector element. It can be given to the multiplication circuit 516. Other computational circuits, such as index identification circuits, circuits for calculating the arithmetic mean, or other computational circuits may also be included in the architecture 500.

In some implementations, cells may be configured to perform one type of vector reduction operation. In such an implementation, only one of one computational circuit, eg, comparison circuit 512, total circuit 514, or multiplication circuit 516, may be included. In addition, in such an implementation, the vector reduction control signal 505 for instructing the architecture 500 to perform a particular type of vector reduction operation would not be needed. Therefore, the vector reduction control signal 505 and the multiplexer 508, 510 may be arbitrary.

The result of the calculation executed by the calculation circuit 512, 514, 516 is output to the result register 520. The result register 520 also receives the control signal 515. The control signal 515 may be an element of the control register 415 of FIG. 4 corresponding to a particular cell having architecture 500. For example, when the architecture 500 corresponds to the cell [1] of the vector reduction unit 400, the control signal 515 may be the signal corresponding to the element at position [1] of the control register 415.

The result register 520 outputs the result to one or more components based on the control signal 515. The control signal 515 indicates that the segmentation vector reduction operation will not be performed, for example, if the control signal 515 is zero, the result register 520 receives the result received from the calculation circuits 512, 514, 516 as follows. Can be given to a cell, for example, so that it can be received in the temporary reduction element register of the next cell. Optionally, if the segmentation vector reduction operation is not planned to be performed, the result register 520 can also output the result to, for example, one of the value outputs 440 of FIG.

When the control signal 515 indicates that the segmentation vector reduction operation is to be performed, the result register 520 receives the result received from the calculation circuits 512, 514, 516, for example, the value output 440 of FIG. Can be given to the output. The result register is null, positive infinity, to segment the vector reduction of the input vector in the cell associated with architecture 500, either in the next cell, for example, in the temporary reduction element register of the next cell. Or a negative infinite value can be given.

Other cell architectures are also possible. In some implementations, the components of the computational circuits 512, 514, 516 may be arranged in series. For example, the comparison circuit 512 may receive a temporary reduction element and an input vector element. The output of the comparison circuit 512 may be given as the input of the total circuit 514, the output of the total circuit 514 may be given as the input of the multiplication circuit 516, or the output of the multiplication circuit 516 may be given to the result register 520. In such an implementation, the vector reduction control signal 505 activates or deactivates each component of the calculation circuits 512, 514, and 516, and only one of the vector reduction operations is performed on the data. It may be executed when passing through the components 512, 514, 516. Such an architecture would require components of computational circuits 512, 514, 516 to interpret the vector reduction control signal 505 as well, but would reduce the need for multiplexers 508 and 510. Other architectures are also within the scope of this specification.

FIG. 6 is an example process 600 for performing vector reduction on an input vector. In some examples, the input vector may be the vector of the input matrix. In general, the process 600 can be run by a system consisting of a dedicated hardware circuit, eg, a dedicated hardware circuit 110 with the vector reduction unit 120 of FIG. 1, consisting of one or more computers.

The system performs a series of operations (602-608) to generate a reduced version of the input vector. For example, a dedicated hardware circuit can contain multiple cells, each of which has a specified input vector specification, except for the specified first cell that receives the specified first element of the input vector. It is configured to calculate vector reduction by performing actions (602-608) on the elements of. In some implementations, the specified first cell that receives the specified first element of the input vector does not receive the output of the previous cell and therefore does not perform the operation (602-608). Thus, the designated first cell only gives the first designated element of the input vector to the next cell in the system. Each of the other cells receives the output of the previous cell and performs a series of operations (602-608) to generate its own output given to the next cell. The output of the last cell of the system, which corresponds to the last element of the input vector, is the result of performing the specified vector reduction operation.

The cell receives a specific element of the input vector (602). For example, a particular cell in a circuit can receive a particular element of the input vector. In some implementations, a first register, such as an input register, may be configured to receive elements of the input vector. By making the data stream that gives the element of the input vector to the input register staggered, the data corresponding to the input vector may be received in a plurality of lanes while being delayed. For example, the data may be staggered to receive the next element of the input vector in the input register for each cycle. A particular element of the input vector received by the input register is given to a particular cell of the circuit.

The cell receives a temporary reduction element from another cell of the plurality of cells (604). The temporary reduction element may correspond to the result of performing a reduction operation on an element of the input vector that precedes a particular element of the input vector. For example, a particular cell in a circuit may receive a temporary reduction element that corresponds to the result of performing a reduction operation on an element of the input vector that precedes the particular element of the received input vector. The temporary reduction element can be received from another cell of one or more cells of the circuit and may be, for example, the output of the previous cell. A cell is in the same clock cycle as this cell receives a particular element of the input vector, or during a clock cycle before or after this cell receives a particular element of the input vector. , Can receive temporary reduction elements. As an example, the circuit can perform a vector reduction operation on the input vector [0] of FIG. To do so, each of the elements of the input vector [0] is processed in the corresponding cell 430. That is, the first element of the input vector [0] is processed in the cell [0], the second element of the input vector [0] is processed in the cell [1], and so on. In such an example, the temporary reduction element received by cell [2] is the first and second element of the input vector, that is, the element of the input vector preceding the third element of the input vector received by cell [2]. Corresponds to the result of executing the reduction operation for.

The cell performs a reduction operation using a particular element and a temporary reduction element (606). For example, a particular cell in a circuit receives a particular element of the input vector and a temporary reduction element that corresponds to the result of performing the specified vector reduction operation on the element of the input vector that precedes this particular element. Later, the received data can be subjected to the vector reduction operation specified by the control signal. With reference to the example of FIG. 4 again, cell [2] receives a third element of the input vector [0] and the output of cell [1] as a temporary reduction element, and then with respect to these elements. , Can perform the specified vector reduction operation. For example, if the designated vector reduction operation is a total vector reduction operation, cell [2] may calculate the sum of the third element of the input vector [0] and the output of cell [1].

In some implementations, a particular cell of the system can receive a control signal that specifies a vector shrinking operation to be performed with a particular element of the input vector and a temporary reduction element. In some embodiments, the control signal may be the same for each of the plurality of cells, i.e., one control signal received by the circuit is given to each cell. In other embodiments, each cell of the circuit may receive a different control signal. This allows different cells of the circuit to perform different vector reduction operations at the same time. This is similar to the case where the segmentation vector reduction operation is performed on the input vector. In such an implementation, the circuit may receive the control signal given to the cell as a vector, for example in a register, and each element of this vector can be given to the corresponding cell as a control signal. The types of vector reduction operations are, for example, total vector reduction operation, maximum value vector reduction operation, minimum value vector reduction operation, maximum index vector reduction operation, minimum index vector reduction operation, average vector reduction operation, product vector reduction operation, or Other arbitrary vector reduction operations may be performed.

The cell gives the result of executing the specified reduction operation with a specific element and a temporary reduction element as a new temporary reduction element (608). For example, a particular cell in a circuit may give the result of performing a specified reduction operation with a particular element and a temporary reduction element to another cell of multiple cells as a new temporary reduction element. Can be done. With reference to the example of FIG. 4 again, after the cell [2] performs the specified vector reduction operation on the third element of the input vector [0] and the temporary reduction element received from the cell [1]. , The result of this specified vector reduction operation can be output to the cell [3] of the circuit. Instead, if the particular cell is the cell that performed the vector reduction operation on the last element of the input vector, the resulting temporary reduction element will be the result of performing the vector reduction on the input vector. It can be given as an equivalent output.

The system provides a new temporary reduction element corresponding to the result of performing the reduction operation using the last element of the input vector to output as a reduction of the input vector (610). For example, a vector reduction circuit may include an output circuit, eg, one or more value outputs, such as a value output 440, to give the result of performing a vector reduction operation with the last element of the input vector. Each cell of the circuit uses an element of the input vector and a temporary reduction element that corresponds to the result of performing the specified vector reduction operation on the element of the input vector that precedes this particular element of the input vector. Since the vector reduction operation is performed, the result of the specified vector reduction operation using the last element of the input vector will be the result of performing the specified vector reduction operation on all the elements of the input vector. Therefore, this result corresponds to a reduced input vector. For example, cell [3] of architecture 400 outputs the specified vector reduction operation to the fourth, i.e., the last element of the input vector [0] and the output of cell [2], i.e. the preceding cell. Can be done for temporary reduction elements. The result of the vector reduction operation executed by the cell [3] is the result of the vector reduction operation for the entire input vector. Therefore, this result can be given, for example, to the value output [3] for output as a reduced input vector. In some implementations, for example, if a vector reduction operation is performed on multiple segments of the input vector, the result of performing the vector reduction on the last element of the input vector can be given to another cell. .. In these implementations, the result of performing vector reduction on the last element of the input vector segment is output during the same clock cycle as the clock cycle that gives this result to the next cell as a new temporary reduction element. Or can be given for output in a clock cycle before or after the clock cycle when this result is given to the next cell as a new temporary reduction factor.

As if one or more cells in the circuit were sequentially receiving and feeding data, for example, starting with the first element of cell [0] and input vector [0] and ending with cell [3] and input vector [0]. Although described as if it ends with the element of, in other implementations, the cells may perform operations on the input vector in different orders. In these implementations, the cell may perform operations on different elements of the input vector, or the cell may / from another cell that is not the cell immediately preceding or immediately preceding this cell in a sequence. You may receive / give. For example, in the architecture 400 of FIG. 4, the output of cell [1] may be given to cell [3] as a temporary reduction element, or the output of cell [3] may be given to cell [2] as a temporary reduction element. .. In these implementations, the output of the last cell that processes the elements of the input vector would be the result of shrinking this input vector.

7A-7B show an example of performing vector reduction in a matrix-vector processor. In some implementations, the examples of FIGS. 7A-7B can be performed by the matrix-vector processing system 100 of FIG. 1, which comprises a dedicated hardware circuit 110 including a vector reduction unit 120. In the example shown in FIGS. 7A-7B, the total vector reduction operation is performed on the two input vectors. The first input vector is the vector [1234] and the second input vector is the vector [5678]. The total vector reduction operation performed on the first input vector is a segmentation vector reduction operation, and the result of the vector reduction operation on the first input vector is the elements [12] and [12] of the first input vector. 34] is a two-element vector corresponding to the sum. The total vector reduction operation performed on the second input vector is not a segmentation vector reduction operation, and the result of the vector reduction operation on the second input vector is the sum of the elements [5678] of the second input vector. It becomes a scalar equivalent to.

In each of the frames shown in FIGS. 7A-7B, memory 710, for example DRAM or SRAM that can be used to implement memory 208, may be accessed or read as staggered memory. Therefore, in each frame, the staggered memory is read and the input vector element is received by the input register 720. The input register 720 may correspond to, for example, the input register 320 of FIG. The input register 720 gives the elements of the received input vector to the reduction unit cell 730, for example cell 430 of FIG. Each of the frames of FIGS. 7A to 7B shows the reduction unit cell 730 before the vector reduction operation is executed in the reduction unit cell (front) 730a, and the vector reduction operation is executed in the reduction unit cell (rear) 730b. The reduced unit cell 730 after being made is shown. Further, the frames of FIGS. 7A to 7B indicate the control register 740, and the content of the control register 740 for controlling the vector reduction unit to execute the segmentation or non-segmentation vector reduction operation is shown in FIG. It corresponds to the contents of the control register 415. The reduced input vector is output to the output 750, which is the same as the value output 440 in FIG. 4, for example.

Briefly, in the frame (a) shown in FIG. 7A, the first element of the first input vector having the value 1 is read from the memory 710 and received by the input register 720. Cell 730 is initialized to null as shown in 730a, but in other implementations cell 730 may be initialized to positive infinity, negative infinity, or another value. The control register 740 has already received a vector consisting of elements for realizing the segmentation vector reduction operation of the first input vector. The vector consisting of this element is [0101] and is given to the control register 740, and each element of the vector consisting of the elements is a control signal of the corresponding cell 730. Since the [0] position of the control register 740 is zero, that is, it indicates that the unsegmented vector reduction operation is performed, cell 730 receives the first element of the first input vector. And the current value in cell 730 are summed together and the result is given to the next cell 730. Therefore, the first cell 730 performs the operation (Null) + 1 = 1 and sends 1 to the next cell 730 as shown in 730b of the frame (a). As shown in 730b, the contents of the first cell may be initialized to null.

In frame (b), the input register 720 receives a second element of the first input vector having the value 2 and a first element of the second input vector having the value 5. The reduction unit in the second cell 730 sums the currently stored value and the second element of the received first input vector, i.e., performs operation (1) + 2 = 3. Since the element of the control register 740 for the second cell 730 is 1, that is, it specifies the execution of the segmentation vector reduction operation, the second cell 730 outputs the resulting sum to the second output. Output to 750. In addition, the second cell 730 outputs zero to the third cell 730. In some examples, the second cell 730 may output a different value to the third cell 730, for example null, positive infinity or negative infinity. In some examples, the output 750 may be one output, for example cells 730 may all be connected to the same output 750.

In addition, the first cell 730 receives the first element of the second input vector, performs Null + 5 = 5, and outputs the result to the second cell 730 for storage. Therefore, the second cell 730 stores the value 5 in the frame (b) 730b. As shown, the first cell 730 may be reinitialized to null, or another value.

In frame (c), the input register 720 receives a third element of the first input vector and a second element of the second input vector. Further, since the segmentation vector reduction operation is executed for the first input vector, the control register 740 is updated to include an element for the non-segmentation vector reduction operation of the second input vector. Therefore, only the last element of the control register 740 is 1, and only one reduced output is generated for the second input vector. The third cell 730 stores the value zero as shown in 730a of the frame (c), receives the third element of the first input vector having the value 3, and executes the operation (0) + 3 = 3. And store the result in the last cell 730, as shown in 730b. The second cell 730 stores the value 5 in 730a, receives the second element of the second input vector having the value 6, performs operation (5) + 6 = 11, as shown in 730b. , The result is stored in the third cell 730.

In frame (d), the last element of the input register 720 receives the last element of the first input vector having the value 4 and the third element of the second input vector having the value 7. The last cell 730 receives the last element of the first input vector, calculates the sum of this and the value stored in the last cell 730, i.e. performs the operation (3) + 4 = 7 and 1 The result is output to the final output 750 based on the control register 740 that stores. The third cell 730 receives the third element of the second input vector, calculates the sum of this and the value stored in the third cell 730, that is, executes the operation (11) + 7 = 18. And the result is stored in the last cell 730 as shown in frame (d) 730b.

In frame (e), the input register 720 receives the last element of the second input vector having the value 8. This value is added to the value currently stored in the last cell 730, that is, operation (18) + 8 = 26 is performed. Since the last element of the control register 740 corresponding to the last cell 730 of the reduction unit is 1, the result is output to the last output 750. Therefore, the total vector reduction of both input vectors is output. The contents of cell 730 may be reinitialized to null or another value, since the element is read from cell 730 and is not replaced, for example, with the output of previous cell 730.

The embodiments and functional operations of the subject matter described herein include the structures and structural equivalents thereof disclosed herein in computer software or firmware tangibly realized in digital electronic circuits. It can be achieved in computer hardware or in combination of one or more of these. The embodiments of the subject matter described herein are as one or more computer programs, i.e., on a tangible non-temporary program carrier executed by a data processing device or for controlling the operation of the data processing device. It can be implemented as one or more modules of encoded computer program instructions. Alternatively or additionally, a program instruction is an artificially generated propagating signal, generated to encode information transmitted to a suitable receiver for execution by a data processor. For example, it can be encoded on an electric, optical, or electromagnetic signal generated by a machine. The computer storage medium may be a machine-readable storage medium, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of these.

The term "data processing device" includes all types of devices, devices, and machines for data processing, including, for example, programmable processors, computers, or multiple processors or computers. The device may include dedicated logic circuits such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits). The device also constitutes, in addition to hardware, code that creates an execution environment for the computer program, such as processor firmware, protocol stack, database management system, operating system, or a combination of one or more of these. May contain code to do.

A computer program (referred to or described as a program, software, software application, module, software module, script, or code) is optional, including a compiled or interpreter language, or a declarative or procedural language. It can be written in the programming language form of, and can be deployed in any form, including components, subroutines, or other units suitable for use in a computing environment, either as a stand-alone program or as a module. The computer program may or may not support the files in the file system. A program may hold other programs or data, eg, part of a file that holds one or more scripts stored in a markup language document, or in one file dedicated to that program, or in conjunction with it. It can be stored in multiple files, such as one or more modules, subprograms, or multiple files that store parts of the code. Computer programs can be deployed to run on one computer or multiple computers that are located in one location or distributed in multiple locations and interconnected by communication networks.

The processes and logical flows described herein may be performed by one or more programmable computers that execute one or more computer programs to act on the input data and perform functions by producing outputs. can. Processes and logic flows may be performed by dedicated logic circuits such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits), and the equipment may be implemented by dedicated logic circuits such as FPGAs (Field Programmable Gate Arrays) or ASICs. It may be realized as an integrated circuit for a specific application).

A computer suitable for executing a computer program may, for example, be based on a general purpose or dedicated microprocessor or both, or may be based on any other type of central processing unit. In general, central processing units receive instructions and data from read-only memory, random access memory, or both. Essential elements of a computer are a central processing unit for executing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer includes or receives data from one or more mass storage devices for storing data, such as magnetic, magneto-optical disks, or optical disks, or transfers data to the mass storage device. Or for both of these, it is operably coupled to the mass storage device. However, the computer does not have to have such a device. In addition, the computer may be another device, for example, a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, for example. It may be embedded in a universal serial bus (USB) flash device.

Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, which, for example, are semiconductor memory devices such as EPROM, EEPROM, and flash memory. Includes devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented with a dedicated logic circuit, or the processor and memory may be incorporated in the dedicated logic circuit.

For transmission in interaction with the user, embodiments of the subject described herein include a display device for displaying information to the user, such as a CRT (cathode tube) or LCD (liquid crystal display) monitor, and the user. Can be implemented on a computer with a keyboard and pointing device such as a mouse or trackball to allow the computer to send input. Other types of devices may be used to allow transmission in the interaction with the user. For example, the feedback given to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and the input from the user may be in any form including acoustic, audio, or tactile input. Can receive. In addition, the computer interacts with the user by sending a document to or receiving a document from the device used by the user, for example, on the user's client device in response to a request received from a web browser. It may be realized by sending a web page to the web browser of.

Embodiments of the subject matter described herein are, for example, a computing system that includes a back-end component as a data server, or a computing system that includes a middleware component such as an application server, or a front-end component, eg, described herein. A computing system that includes a client computer with a graphical user interface or web browser to enable the subject to interact with the user, or one of such backend, middleware, or frontend components. It can be realized in a calculation system including any combination of the above. The components of the system can be interconnected in the form or medium of digital data communication, such as a communication network. Examples of communication networks include local area networks (“LAN”) and wide area networks (“WAN”), such as the Internet.

Computational systems can include clients and servers. The client and server are usually separated from each other and generally communicate through a communication network. The client-server relationship is generated by a computer program that runs on each computer and has a client-server relationship.

Although this specification contains many specific details of implementation, they should not be construed as limiting the scope of any invention or claimable, but rather specific. It should be construed as an explanation of features that may be unique to a particular embodiment of the invention. Specific features described herein in terms of separate embodiments can also be combined and realized in one embodiment. Conversely, the various features described in terms of one embodiment can be realized separately in a plurality of embodiments or as any suitable subordinate combination. In addition, the above features may be described as functioning in a particular combination and initially claimed as such, but one or more features contained in the claimed combination may be this feature. It can be omitted from the combination, and the claimed combination may relate to a subordinate combination or a variant of the subordinate combination.

Similarly, the actions are shown in the drawings in a particular order, which requires or is shown that such actions are to be performed in the particular order or contiguous order shown. It should not be understood that all the actions that are in need to be performed to obtain the desired result. Multitasking and parallel processing may be advantageous in certain situations. In addition, the separation of various system modules and components in the above embodiments should not be understood to require such separation in all embodiments. It should be understood that the program components and systems described can generally be integrated into one software product or packaged into multiple software products.

Specific embodiments of the subject are as described above. Other embodiments are included in the scope of the following claims. For example, the operations described in the claims can be performed in different orders and still obtain the desired result. As an example, the process described in the accompanying drawings does not necessarily require that the desired result be obtained in a particular order or in a contiguous order as shown. Multitasking and parallel processing may be convenient in certain implementation examples.

Claims (20)

A vector reduction circuit configured to reduce an input vector composed of a plurality of elements, wherein the input vector is identified as including a plurality of segments composed of a plurality of elements, and the vector reduction circuit is a vector reduction circuit.
The plurality of cells include the plurality of cells, and the designated first cell among the plurality of cells receives the designated first element of the input vector, and the plurality of cells other than the designated first cell are received. One of them is
Receives a specific element other than the specified last element of the specific segment of the input vector
Receives a temporary reduction element from another cell of the plurality of cells
Performing a reduction operation using the specific element and the temporary reduction element,
It is configured to give the result of executing the reduction operation using the specific element and the temporary reduction element as a new temporary reduction element.
The other one of the plurality of cells other than the designated first cell is
Receives the specified last element of the particular segment of the input vector
Receives a temporary reduction element from another cell of the plurality of cells
Performs a reduction operation using the last specified element and the temporary reduction element.
The vector reduction circuit is configured to give the result of executing the reduction operation using the designated last element and the temporary reduction element as a reduction of the specific segment of the input vector. Furthermore,
Configured to give the result of performing the reduction operation using the designated last element of the particular segment of the input vector to output as a reduction of the particular segment of the input vector. Vector reduction circuit with output circuit. It is said that the result of performing the reduction operation using the specified last element of the specific segment of the input vector and the temporary reduction element is given as a reduction of the specific segment of the input vector. The vector reduction circuit according to claim 1 , wherein a placeholder signal is given to another cell among the plurality of cells. A vector reduction circuit configured to reduce an input vector composed of a plurality of elements, wherein the input vector is identified as including a plurality of segments composed of a plurality of elements, and the vector reduction circuit is a vector reduction circuit.
The plurality of cells include the plurality of cells, and the designated first cell among the plurality of cells receives the designated first element of the input vector, and the plurality of cells other than the designated first cell are received. Each of
Receives a specific element of the input vector
Receives a temporary reduction element from another cell of the plurality of cells
Performing a reduction operation using the specific element and the temporary reduction element,
The vector reduction circuit is further configured to give the result of executing the reduction operation using the specific element and the temporary reduction element as a new temporary reduction element.
It comprises an output circuit configured to provide a plurality of elements, each of which corresponds to the result of performing the reduction operation using the last specified element of a different segment of the input vector. A vector reduction circuit, which is a new temporary reduction element . The vector reduction circuit according to claim 3 , wherein the output circuit is configured to provide an output vector including the plurality of elements in order to output the input vector as a reduced version. A vector reduction circuit configured to reduce an input vector composed of a plurality of elements, wherein the input vector is identified as including a plurality of segments composed of a plurality of elements, and the vector reduction circuit is a vector reduction circuit.
A plurality of cells are provided, and the designated first cell among the plurality of cells receives the designated first element of the corresponding segment of the input vector, and is other than the designated first cell. Each of the plurality of cells
Receives a specific element of the input vector
Receives a temporary reduction element from another cell of the plurality of cells
Performing a reduction operation using the specific element and the temporary reduction element,
The vector reduction circuit is further configured to give the result of executing the reduction operation using the specific element and the temporary reduction element as a new temporary reduction element.
To output a new temporary reduction element corresponding to the result of performing the reduction operation using the specified last element of the specific segment of the input vector as a reduction of the specific segment of the input vector. The output circuit , which is configured to give to
A vector reduction circuit comprising a control register configured to receive a control vector composed of a plurality of elements, wherein the control vector composed of the plurality of elements designates a plurality of segments composed of the elements of the input vector . Giving the result of executing the reduction operation using the specific element and the temporary reduction element as a new temporary reduction element is possible.
Receiving a specific element of the control vector corresponding to the specific element
Determining that the particular element of the input vector is designated as the last element of a particular segment of the input vector, at least based on the particular element of the control vector.
At least based on the determination, the output circuit is subjected to the reduction operation using the specific element of the input vector designated as the last element of the specific segment of the input vector and the temporary reduction element. The vector reduction circuit according to claim 5 , wherein the result is given as a reduction of the specific segment of the input vector. Giving the result of executing the reduction operation using the specific element and the temporary reduction element as a new temporary reduction element is possible.
Receiving a particular element of the control vector corresponding to that particular element of the input vector
Determining that the particular element of the input vector is designated as the last element of a particular segment of the input vector, at least based on the particular element of the control vector.
The vector reduction circuit according to claim 5 , comprising giving a placeholder signal to another cell of the plurality of cells, at least based on the determination. Giving the result of executing the reduction operation using the specific element and the temporary reduction element as a new temporary reduction element is possible.
Receiving a particular element of the control vector corresponding to that particular element of the input vector
Determining that the particular element of the input vector is not designated as the last element of the particular segment of the input vector, at least based on the particular element of the control vector.
The reduction operation using the specific element not designated as the last element of the specific segment of the input vector and the temporary reduction element for another cell among the plurality of cells, at least based on the determination. The vector reduction circuit according to claim 5 , wherein the result of executing the above is given as a new temporary reduction element. The reduction operation is a maximum value reduction operation.
Performing a reduction operation using the specific element and the temporary reduction element
Comparing the specific element with the temporary reduction element,
At least based on the comparison, the determination of the largest of the specific element and the temporary reduction element is included as a result of executing the reduction operation using the specific element and the temporary reduction element. , The vector reduction circuit according to any one of claims 1 to 8 . The reduction operation is a minimum value reduction operation.
Performing a reduction operation using the specific element and the temporary reduction element
Comparing the specific element with the temporary reduction element,
It includes determining which of the specific element and the temporary reduction element is the smallest as a result of performing the reduction operation using the specific element and the temporary reduction element at least based on the comparison. , The vector reduction circuit according to any one of claims 1 to 8 . The reduction operation is a total reduction operation.
Performing a reduction operation using the specific element and the temporary reduction element
The invention according to any one of claims 1 to 8 , which includes obtaining the sum of the specific element and the temporary reduction element as a result of executing the reduction operation using the specific element and the temporary reduction element. Described vector reduction circuit. The reduction operation is a product reduction operation, and is
Performing a reduction operation using the specific element and the temporary reduction element
The invention according to any one of claims 1 to 8 , which comprises obtaining the product of the specific element and the temporary reduction element as a result of executing the reduction operation using the specific element and the temporary reduction element. Described vector reduction circuit. The reduction operation is a maximum index operation and is an operation.
Performing a reduction operation using the specific element and the temporary reduction element
Receiving data indicating the index corresponding to the temporary reduction element
Comparing the specific element with the temporary reduction element,
Determining the largest of the particular element and the temporary reduction element, at least based on the comparison.
As a result of executing the reduction operation using the specific element and the temporary reduction element based on at least the determination, one of the index corresponding to the specific element and the index corresponding to the temporary reduction element is selected. , The vector reduction circuit according to any one of claims 1 to 8 , comprising identifying. The reduction operation is a minimum index operation and is an operation.
Performing a reduction operation using the specific element and the temporary reduction element
Receiving data indicating the index corresponding to the temporary reduction element
Comparing the specific element with the temporary reduction element,
Determining the smallest of the particular element and the temporary reduction element, at least based on the comparison.
As a result of executing the reduction operation using the specific element and the temporary reduction element based on at least the determination, one of the index corresponding to the specific element and the index corresponding to the temporary reduction element is selected. , The vector reduction circuit according to any one of claims 1 to 8 , comprising identifying. The vector reduction circuit according to any one of claims 1 to 14 , which receives a specific element of the input vector as a staggered input of the element. The vector reduction circuit according to any one of claims 1 to 15 , wherein the output circuit gives a reduced version of the input vector to a replacement circuit configured to calculate a vector or matrix substitution. .. Any of claims 1 to 16 , wherein each of the plurality of cells is configured to receive a control signal designating the type of reduction operation performed with the particular element and the temporary reduction element. The vector reduction circuit according to claim 1 . The temporary reduction element corresponds to any one of claims 1 to 17 , which corresponds to the result of performing a specified reduction operation on the element of the input vector preceding the specific element of the input vector. Described vector reduction circuit. Each of the plurality of cells
An input vector element register configured to receive a particular element of the input vector,
A temporary reduction element register configured to receive a temporary reduction element from another cell among the plurality of cells.
A computational circuit configured to perform a reduction operation using a specific element of the input vector and a temporary reduction element.
Any one of claims 1-18 , including a result register configured to give the result of performing a reduction operation using the particular element of the input vector and the temporary reduction element as a new temporary reduction element. Vector reduction circuit described in. The vector reduction circuit according to claim 19 , wherein the calculation circuit includes at least one of a comparison circuit, a total circuit, and a multiplication circuit.

Images