TWI729939B - Method and processor for decompression of model parameters using functions based upon cumulative count distributions - Google Patents

Abstract

A predictive model utilizes a set of coefficients for processing received input data. To reduce memory usage storing the coefficients, a compression circuit compresses the set of coefficients prior to storage by generating a cumulative count distribution of the coefficient values, and identifying a distribution function approximating the cumulative count distribution. Function parameters for the determined function are stored in a memory and used by a decompression circuit to apply the function the compressed coefficients to determine the decompressed component values. Storing the function parameters may consume less memory in comparison to storing a look-up table for decompression, and may reduce an amount of memory look-ups required during decompression.

Description

Method and processor for decompression of model parameters using functions based on cumulative count distribution

The present invention relates to a method and processor for decompression, in particular a method and processor for decompression of model parameters using a function based on cumulative count distribution.

The present invention generally relates to the decompression of a model parameter, and specifically relates to the decompression of the model parameter used in a neural network.

Neural networks and other types of models can be used to process various types of data. For example, a neural network model can be trained to recognize whether certain types of objects exist in the received input image. Training and machine learning can be used to determine a set of coefficients to be used by the model to process input data, such as the weights between neurons of a neural network model.

A predictive model (for example, a neural network model) can be used in combination with a coefficient set of the model. The coefficient set can be stored in a memory and accessed for performing arithmetic operations on input data (for example, an image to be analyzed by the model).

To reduce memory usage, compress the coefficient set before saving. The stored compression factors will need to be decompressed before operating the input data. In some embodiments, the determined coefficient value is compressed based on a function. The function is generated based on the cumulative count distribution of one of the decompression coefficient values. For example, the count value of a model coefficient set can be approximated by a bimodal distribution, a Gaussian distribution, a Poisson distribution, or other types of distributions that can be defined by a function. The function parameters of the determined function can be stored in a memory and used by a decompression circuit to apply the function to the compression model coefficients for decompression. Compared with other decompression methods (for example, a look-up table), storing the function parameters can consume less memory and can also reduce the amount of memory search required during decompression.

In some embodiments, a method for decompressing model coefficient values is provided. The method includes receiving compression factor data associated with a model. In some embodiments, the value of the coefficient data is determined through a model training process, and a compression function based on a cumulative distribution of the value of the coefficient data is used to compress the coefficient data. The method further includes retrieving a set of function parameters associated with the compression function, the set of function parameters specifying at least one function type. The method further includes configuring a decompression circuit based on the retrieved function parameters. The method further includes decompressing the compression coefficient data based on the function parameter at the decompression circuit to generate a decompression coefficient value. The method further includes applying the model to the received input data by performing arithmetic operations on the received input data based on the compression factor values.

A predictive model (for example, a neural network model) can utilize a set of coefficients when processing the received input data. For example, for a neural network model, the coefficient set may correspond to the weights between different neurons of the neural network. The coefficient set can be stored in a memory and accessed for performing arithmetic operations on input data (for example, an image to be analyzed by a model).

To reduce memory usage, compress the coefficient set before saving. The stored compression factor will need to be decompressed before operating the input data. A lookup table can be used to map compression factor values to decompression factor values. However, the look-up table may require a large amount of memory for storage, especially when the coefficient range is large. In addition, in some embodiments, different types of compression may be performed on different coefficient subsets, and thus multiple lookup tables will need to be stored.

In some embodiments, the determined coefficient value is compressed based on a function. A function is generated based on the cumulative count distribution of one of the decompression coefficient values. For example, the count value of a model coefficient set can be approximated by a bimodal distribution, a Gaussian distribution, a Passon distribution, or another type of distribution that can be defined by a function. The function parameters of the determined function can be stored in a memory and used by a decompression circuit to apply the function to the compression model coefficients for decompression. Compared with storing a lookup table for decompression, storing function parameters can consume less memory. In addition, the storage space required to store the function parameters of a determined function has nothing to do with the range of coefficient values or the number of different possible coefficient values.

Using the decompression function also reduces the amount of memory lookup required during decompression. For example, the function parameters of a given function only need to be looked up once when the decompression circuit starts to decompress, and they are used to decompress a large number of coefficients compressed using the function. On the other hand, decompression using a lookup table usually requires a memory lookup for each coefficient to be decompressed.

FIG. 1 shows a schematic diagram of a system for storing and decompressing model coefficients used in a model according to some embodiments. A flow processor (TSP) 100 or other type of processor is configured to receive and process input data values 102 (for example, from an input image) based on a stored model to generate output data values (for example, the input image A classification, the identification of certain types of objects or characteristics in the input data and/or similar ones). The TSP 100 may be an integrated circuit (IC). In some embodiments, the input data value 102 may be an input value stored in the memory 108 and represents the result of an arithmetic operation performed elsewhere in the TSP 100.

The TSP 100 uses one or more arithmetic circuit units and one or more model coefficients to manipulate the input data value 102. The arithmetic circuit unit includes logic circuits that perform arithmetic operations on the input values 102 and model coefficients and generate output data values that represent a result of the arithmetic operations. For example, the arithmetic circuit unit can use the model coefficients to perform a matrix multiplication operation on the input value and generate an output data value representing the matrix product. The execution of a predictive model 118 (such as a neural network) can generally be implemented using successive stages of matrix multiplication. In other embodiments, the arithmetic operation of the arithmetic circuit unit may include a convolution operation, a one-point product operation, a Fast Fourier Transform (FFT) operation, and/or other arithmetic operations. The arithmetic circuit unit 106 may use single instruction multiple data (SIMD) processing to perform operations.

The TSP 100 includes a memory 108 that stores the compression model coefficients 112 used by the arithmetic unit to manipulate the input data value 102. The compression model coefficient 112 can be generated by the compiler 120 from the prediction model 118. The predictive model 118 may correspond to any type of model using a set of coefficients. In some embodiments, the coefficient set is determined through a machine learning or training procedure. For example, in some embodiments, the prediction model 118 is a convolutional neural network (CNN) or other types of neural network models.

Once the predictive model 118 has been constructed or fully trained, the model 118 can be compiled by a compiler 120 for use by the TSP 110 for processing the input data values 102. The compiler 120 analyzes the coefficient values of the prediction model 118 and selects one or more compression schemes for the coefficient values of the compression model. Then, the compression coefficient value is stored in the memory 108 as the compression model coefficient 112.

In order to be used by the arithmetic circuit unit to manipulate the input data value 102, the compressed model coefficient 112 associated with the model needs to be decompressed. A decompression circuit is configured to receive the compression model coefficients 112 from the memory 108 and output the decompression model coefficients that can be operated by the arithmetic unit.

In some embodiments, the compiler 120 selects a compression scheme for the coefficients of the predictive model 118 based on a function derived from a distribution of one of the coefficient values associated with the model. For example, in many cases, the distribution of the coefficient values of the model may have one of a bimodal distribution, a Gaussian distribution, or a Passon distribution. The compiler 120 determines a function type of the best fitting model coefficient distribution, and stores the parameters of the determined function as function parameters 114 in the memory 108. The function parameter 114 may indicate a function type associated with the distribution and the coefficient of the function and/or the value of other parameters associated with the function. In some embodiments, the type of the stored function parameter is based on a function type.

The decompression circuit supports several possible functions for decompressing the compression model coefficient 112. The decompression circuit determines the decompression model coefficient by applying a specific function defined by the function parameter 114 to the compression model coefficient 112 and the decompression model coefficient 112.

Using functions to perform decompression can reduce the amount of memory required to store the data used for decompression (for example, compared to a lookup table). In addition, the amount of memory required to store the function parameters can be independent of the range of coefficient values or the number of different possible coefficient values. Using the decompression function also reduces the amount of memory lookup required during decompression. For example, the function parameter 114 can represent a small amount of constant memory that is searched once at the beginning of decompression, and can be used to decompress a long string of data including many coefficients. On the other hand, decompression using a lookup table usually requires a memory lookup for each coefficient to be decompressed.

In some embodiments, the memory 108 may store the compression model coefficients of the prediction model 118 as a plurality of different coefficient sets (for example, a first compression model coefficient 112A set and a second compression model coefficient 112B set). May have been based on a different function (for example, a first function associated with the first function parameter 114A and a second function associated with the second function parameter 114B) and using arithmetic or Huffman coding compression 112 sets of coefficients for each compression model. In some embodiments, a different decompression circuit (e.g., decompression circuits 110A and 110B) can be used to decompress different sets of compression model coefficients compressed using different functions to generate different decompression model coefficients (e.g., decompression model coefficients). 116A and 116B) set. Multiple arithmetic units (e.g., arithmetic units 106A and 106B) can operate the output decompressed model parameters 116A and 116B to generate multiple sets of output data values (e.g., output data 104A and 104B).

In some embodiments, multiple functions can be used to decompress the compressed model coefficients. For example, when compressing model coefficients, the compiler can divide the coefficients into one or more subsets, and determine a function and parameter corresponding to the distribution of the coefficient values in each subset.

Figure 2 shows a block diagram of a set of compression model coefficients that can be decompressed using different functions. In some embodiments, the compression model coefficients can be transmitted from the memory to a decompression circuit via a plurality of bit channels (for example, bit channels 0 to n). The decompression circuit can use a different function (for example, function f_0 to function f_n) to decompress the incoming coefficient data on the bit channel. For example, the decompression circuit may receive multiple sets of function parameters corresponding to functions f_0 to f_n, and each function is used to decompress coefficient data received via a corresponding bit channel.

Although FIG. 2 illustrates a decompression circuit that applies a different function to each bit channel, it should be understood that in other embodiments, a common function may be applied to multiple bit channels. In addition, the function used by the decompression circuit to decompress the compression model coefficients can be configured to change over time. For example, at a time t=0, the decompression circuit can use the function f_0 to the function f_n to perform decompression. However, at a later time t=t ₁ , the decompression circuit may receive different sets of function parameters to change the function used to decompress one or more of the bit channels. In some embodiments, the compiler determines which functions are for which bit channels and when to compress/decompress the model coefficients when compressing the model coefficients to be stored in the memory of the TSP.

In some embodiments, when certain compression schemes are used for compression, certain coefficient values may occupy a larger number of bits when compressed compared to decompression. Therefore, the compiler can determine that there is no need to compress these coefficient values. During decompression, the decompression circuit can be configured to apply an identity function to these coefficient values. Alternatively, the decompression circuit can be bypassed.

FIG. 3A illustrates an exemplary graph showing a model coefficient distribution according to some embodiments. The graph 300 has an x-axis corresponding to the coefficient value and a y-axis corresponding to the count value. Although the x-axis of the graph 300 only shows integer coefficient values, it should be understood that the coefficient values of a model can be represented by integers, floating-point numbers, fixed-point numbers, and/or the like.

The graph 300 contains a first curve 302 showing the distribution of a coefficient value of a specific model. After generating a coefficient set of the model (for example, through a training procedure), the number of coefficients with each value in the set is counted. In many cases, the number of coefficients with each value will approximate a common distribution, such as a bimodal distribution, a Gaussian distribution, a Passon distribution, and/or the like. For example, as depicted by the first curve 302, the coefficient values of the specific model have a roughly bimodal distribution, where the largest number of coefficients has the value -2 or 2.

The graph 300 also shows a second curve 304, which indicates the cumulative distribution of one of the coefficient values of the model. The cumulative distribution curve 304 indicates that for each coefficient value represented on the x-axis of the graph 300, the total number of one of the coefficients is less than or equal to this value. Therefore, the cumulative distribution of a set of coefficients will increase monotonically, allowing the use of a function of the distribution to derive a unique coefficient value from a given count value.

The coefficient values of the model are compressed by the compiler based on a function of the cumulative count distribution of the fitting coefficients. In some embodiments, the compiler may first select a function type based on the cumulative count distribution, and determine the function parameters of the selected function type to achieve a best fit between the function type and the cumulative count distribution. For example, the third curve 306 shown in the graph 300 corresponds to a polynomial function that can be selected by the compiler to approximate the cumulative count distribution 304. As shown in FIG. 3, the polynomial function corresponding to the third curve 306 may be an eighth-order polynomial function. In some embodiments, the function may be based on an integral of a function of the approximate count distribution (curve 302) representing the coefficient value.

In some embodiments, the compiler uses arithmetic coding to compress coefficient values based on the determined function. For example, as shown in FIG. 3B, the count value of the function is fitted to a range between 0 and 1, where 0 is represented by the binary sequence 0000... and 1 is represented by the binary sequence 1111... This results in the more frequently occurring coefficients (for example, coefficients with higher count values) being represented by a short bit sequence, and the less frequently occurring coefficients being represented by a long bit sequence.

In some embodiments, each coefficient value may correspond to a value interval based on an interval between adjacent coefficient values. The interval of each coefficient value can be determined based on a rounding scheme, ceiling function, floor function, and/or the like. For example, in an embodiment where the coefficient value is an integer and a floor function is used to determine the value interval, the coefficient value 1 may correspond to the interval [1, 2), the coefficient value 2 may correspond to the interval [2, 3), etc. .

Each interval may correspond to a range of binary sequence values (such as those determined by using a function), wherein each coefficient value is coded with a bit sequence representing the range of the binary sequence value corresponding to the interval. Therefore, because coefficient values with high counts will generally correspond to a larger range of one of the binary sequence values, they can be compressed using a smaller number of bits.

For example, suppose that the coefficient values are integers, and the coefficient value 0 corresponds to the interval [-0.5, 0.5), and the coefficient value 2 corresponds to the interval [1.5, 2.5). As shown in FIG. 3B, based on the function 306, the bit sequence in the range 308 will be mapped to the coefficient value 0, and the bit sequence in the range 310 will be mapped to the coefficient value 2. Because the range 310 spans a larger range of bit sequences, the bit sequence of this range can usually be represented by a smaller number of common bits than the bit sequence of range 308. Therefore, the coefficient value 2 (which has a higher count compared to the coefficient value 0, as shown in FIG. 3A) is represented by a smaller number of bits when compressed than the coefficient value 0. For example, the range 308 spans the sequence 1000..., and the range 310 may span the binary sequence 1011... to 1110.... Therefore, the coefficient value 0 can be represented by the bit sequence 1000 (4 bits), and the coefficient value 2 can be represented by the bit sequence 110 (3 bits). It should be understood that, in some embodiments, the bit sequence representing a compression coefficient value may not represent all bit sequences within the range corresponding to the value, as long as the bit sequence does not represent corresponding to other coefficient values. The bit sequence of the range of the interval is sufficient.

FIG. 4A shows a block diagram of a decompression circuit according to some embodiments. The decompression circuit 400 may correspond to the decompression circuit shown in FIG. 1, and it is configured to receive the compression factor value 402 and output the decompression factor value 404. In some embodiments, the decompression circuit uses arithmetic coding techniques and a function associated with the coefficient value 402 to decompress the bits received from the compression coefficient value 402.

The decompression circuit receives a sequence of one or more bits of the compression coefficient value 402 at a sequence expander circuit 406, and the sequence expander circuit 406 generates a high bit sequence 408 and a low bit sequence 410 of the received bit sequence . As used herein, the high bit sequence 408 corresponds to the received bit sequence added with a plurality of binary "1" values, and the low bit sequence 410 corresponds to the received bit sequence added with a plurality of binary "0" values. Meta sequence. For example, for the received bit sequence "10", the high bit sequence will be "10111..." and the low bit sequence will be "10000...".

The decompression function circuit 414 determines a function to be used for decompression based on one or more received function parameters 412. For example, FIG. 4B illustrates an exemplary decompression function circuit including function calculation circuits corresponding to different function types according to some embodiments. The decompression function circuit 414 includes several function calculation circuits, and each of the function calculation circuits implements a different type of function for calculating an output value from an input value. For example, as shown in FIG. 4B, the function calculation circuit may include a first function calculation circuit 450a corresponding to a polynomial function, a second function calculation circuit 450b corresponding to a Gaussian distribution function, and a second function calculation circuit 450b corresponding to a Passon distribution function. A third function calculation circuit 450c.

The function parameter 412 may include a first function type parameter indicating a function type (for example, a polynomial function, a Gaussian distribution function, and/or the like) that can be used by the decompression function circuit 414 to determine a function calculation circuit to be used, and zero. One or more additional function coefficient parameters (for example, the coefficients of a polynomial function). As shown in FIG. 4B, different types of functions can be associated with different numbers of coefficients and/or different types of coefficients. For example, the function calculation circuit 450b may be configured to calculate a function for decompressing and fitting a coefficient value of a Gaussian distribution (for example, an integral and an inverse operation of an integral of a Gaussian distribution), and the function calculation circuit 450c may be Configure to calculate a function for decompressing a Passon-type distribution. In some embodiments, the decompression function circuit 414 may receive different sets of function parameters 412 based on the set or subset of compression coefficients to be decompressed. The decompression function circuit 414 applies the function to the high bit sequence 408 to determine a high coefficient value 416, and applies the function to the low bit sequence 410 to determine a low coefficient value 418.

In some embodiments, the decompression function circuit 414 uses a function to determine a corresponding value when processing a received bit sequence (for example, a high or low bit sequence), and identifies the corresponding value based on an interval of the corresponding value. A coefficient value. For example, if the corresponding value determined by the function corresponds to a value between two different coefficient values, the decompression function circuit 414 may be based on an interval selection scheme (for example, rounding, ceiling function, floor function, and/or similar ) Select a coefficient value.

The comparator and control circuit 420 receives the high coefficient value 416 and the low coefficient value 418 determined by the decompression function circuit 414, and determines whether the high coefficient value and the low coefficient value are the same. If the high coefficient value and the low coefficient value are the same, the received bit sequence is output as a decompressed output coefficient 404. Then, the decompression circuit 400 can start to receive a new bit sequence from the compression coefficient value 402.

On the other hand, if the high coefficient value 416 and the coefficient value 418 are not the same, the currently received bit sequence cannot be used to determine a decompressed output coefficient. The decompression circuit receives an extra bit from the compression coefficient value 402, and updates the high bit sequence 408 and the low bit sequence 410. In some embodiments, since either the upper bit sequence 408 or the lower bit sequence 410 will remain the same when receiving an extra bit, for each subsequent received bit, only a single extra extension bit needs to be recalculated Sequence (for example, if the received bit is "1", the lower bit sequence 410 is recalculated, or if the received bit is "0", the higher bit sequence 408 is recalculated). Similarly, the decompression function circuit 414 only needs to determine a coefficient value for the recalculated extension bit sequence, and does not need to recalculate both the high coefficient value and the low coefficient value for the high extension bit sequence and the low extension bit sequence. Then, the comparator 420 compares the updated coefficient values to determine whether a decompression coefficient value can be output or whether additional bits are needed.

Table 1 shows a simplified example of the compressed bit sequence mapped to the decompression coefficient value. For example, the decompression function circuit 414 may apply a function (as defined by the received function parameter 412) to a received bit sequence (e.g., 0011...), where the resulting value falls within a coefficient value (e.g., -2) Within the interval. Therefore, the decompression function circuit 414 will return the coefficient value "-2" in response to the received bit sequence "0011". Compressed bit sequence Decompression factor value 0000... -3 0001... -3 0010... -2 0011... -2 0100... -2 0101... -2 0110... -1 0111 -1 1000 0 1001 1 1010 1 1011 2 1100 2 1101 2 1110 2 1111 3 Table 1

As an illustrative example, suppose that the decompression circuit receives the bit sequence "0100111000000110". The decompression circuit 400 receives the first bit ("0") of the stream, so the sequence expander circuit 406 determines a high expansion bit sequence "0111..." and a low expansion bit sequence "0000...". The decompression function circuit 414 receives the high-expansion bit sequence and the low-expansion bit sequence, and determines the high coefficient value and the low coefficient value corresponding to "-1" and "-3", respectively. Because the high coefficient value and the low coefficient value do not match, the comparator and control circuit 420 cannot determine a single output coefficient value to be output. Therefore, the decompression circuit 400 receives a subsequent bit from the bit stream.

When the next bit of the bit stream is received, the current bit sequence at the decompression circuit 400 is "01". Because the high extension bit sequence is still "0111...", the sequence expander circuit 406 only needs to recalculate a low extension bit sequence for the current bit sequence ("0100..."). The decompression function circuit 414 also calculates an updated low coefficient ("-2") for the low extension bit sequence. Because the high coefficient value and the low coefficient value still do not match, the decompression circuit 400 receives another bit from the bit stream without outputting a decompression coefficient value.

After receiving the next bit stream bit, the current bit sequence is "010". The sequence expander circuit 406 determines an updated high-expansion bit sequence "0101...", and the decompression function circuit 414 determines that it corresponds to a coefficient value "-2". Because both the high coefficient value and the low coefficient value match, the decompression circuit 400 outputs “-2” as a decompression coefficient value 404. The decompression circuit can continue to receive the bits of the compressed bit sequence "0100111000000110" and output the corresponding coefficient value (for example, output "-1" for the bit sequence "011", and output "0" for the bit sequence "1000". The bit sequence "000" outputs "-3" and the bit sequence "110" outputs "2").

Although the above examples mainly discuss the use of arithmetic coding and decompression functions to compress and decompress model coefficient values, it should be understood that in other embodiments, different types of coding may be used. For example, in some embodiments, the Huffman code combination function can be used to compress and decompress model coefficient values.

In some embodiments, the coefficient set of a model may be divided into a plurality of subsets, and the coefficient count of each subset may conform to a different distribution. Therefore, each coefficient subset can be compressed and decompressed based on a different function (for example, as shown in FIG. 2). For example, different functions can be applied to the compression factor value stored in the memory of the TSP based on the bit channel and position of the compression factor. Interleaved input

In some embodiments, a plurality of decompression circuits can be used to decompress a bit stream containing compression factor data in parallel. For example, during a first clock cycle, each decompression circuit can process a first bit of a different compression factor. When a specific decompression circuit finishes decompressing a specific coefficient, it can move to a subsequent compression coefficient that is currently unprocessed.

For example, a bit stream of compression coefficient data may include x bits corresponding to a first coefficient and y bits corresponding to a second coefficient. During a first clock cycle, a first decompression circuit can process the first bit of the first coefficient, and a second decompression circuit can process the first bit of the second coefficient. If x<y, at the x+1th clock cycle, the first decompression circuit has finished processing the first coefficient and can start processing the first bit of a third coefficient, and the second decompression circuit The first bit of a fourth coefficient can be processed at the y+1th clock cycle.

For example, FIG. 5 shows a diagram of a plurality of decompression circuits for decompressing compression coefficient data in parallel according to some embodiments. The compression model coefficient 112 can generate a bit stream expressed as "aabbbbcccdd...", which includes 2 bits for writing a first coefficient "a" and 4 for writing a second coefficient "b" Bits, 3 bits for coding a third coefficient "c" and 2 bits for coding a fourth coefficient "d". A divider circuit 502 divides the bit stream between a first decompression circuit 110A and a second decompression circuit 110B. The divider 502 determines a position in the bit stream to start encoding each coefficient, and divides the bits of the bit stream between the decompression circuits 110A and 110B, so that each decompression circuit decompresses the bits of a different coefficient. For example, at a first clock cycle, the divider circuit 502 is configured to transmit the first bit of the coefficient "a" to the decompression circuit 110A and the first bit of the coefficient "b" to Decompression circuit 110B. Each of the decompression circuits 110A and 110B uses a function based on the stored function parameter 114 to process the received bits. During a third clock cycle, the decompression circuit 110A has finished processing the bits of the coefficient "a" and received the first bit of one of the next unprocessed coefficients (for example, the coefficient "c"), and the decompression circuit 110B Receive and process the third bit of the coefficient "b".

The decompression circuits 110A and 110B respectively output the decompression model coefficients 116A and 116B. In some embodiments, an interleaver circuit (not shown) of the interleaving decompression coefficients 116A and 116B may be used to form a decompression coefficient bitstream.

Because the compiler performs the initial compression of the model coefficients and therefore knows the bit length corresponding to each compression coefficient value, the compiler can store instructions specifying which decompression circuits operate which parts of the bit stream into memory, so that Each decompression circuit can receive a first bit of a subsequent compression coefficient after decompressing a previous coefficient. Procedure flow chart

FIG. 6 is a flowchart of a procedure for generating a set of compression model coefficients according to some embodiments. A machine learning program is used to construct 602 and/or train a predictive model, which generates a set of coefficients of the model. In some embodiments, the model may be a neural network model.

A compiler selects 604 a function for each of one or more subsets of the coefficient set based on the distribution of coefficient values within the subset. For example, the compiler generates a cumulative count distribution of one of the coefficient values of the subset, and identifies a function type of the distribution generated by the best fit. The function type may be based on a polynomial function, a Gaussian distribution function, a Passon distribution function, and/or the like. The compiler determines 606 the parameters of the selected function type to determine a function of the distribution of coefficient values of the best-fit subset (for example, the cumulative count distribution). The compiler compresses a subset of 608 coefficients based on the determined function type and function parameters.

The compression coefficient subset and the determined function parameters are stored 610 in a memory. The compression factor (after decompression) can be used by one or more arithmetic units to perform operations on the input data (for example, image data) according to the prediction model.

FIG. 7 is a flowchart of a procedure for decompressing compression model coefficients according to some embodiments. The decompression circuit receives 702 the data corresponding to the compression factor. In some embodiments, the input data is received as a bit stream, where each compression factor is represented by a variable length bit sequence.

The decompression circuit receives 704 one or more function parameters corresponding to a function to be used to decompress the received compression factor data. The function parameter may indicate a function type and one or more coefficients of the function (for example, when the function type is a polynomial, the function parameter may indicate the coefficient of the polynomial function). The decompression circuit is based on the received function parameter configuration 706 to be used by a decompression function circuit. For example, in some embodiments, the decompression circuit includes a plurality of decompression function circuits each corresponding to a different function type. In response to the received function parameter, the decompression circuit selection corresponds to a specific decompression function circuit of a function type indicated by the received parameter, and is configured based on one or more additional function parameters (for example, corresponding to function coefficient values) Decompress the function circuit.

The decompression circuit uses the decompression function circuit to decompress 708 the input data corresponding to the compression factor based on the configured function to output the decompression factor. The decompression factor can be provided to a TSP.

The TSP applies 710 the model to the input data by performing arithmetic operations on the received input data using the decompression coefficients received by the self-decompression circuit. The arithmetic operations may include matrix multiplication, dot product operations, FFT, and/or the like.

Fig. 8 is a flowchart of a procedure for decompressing the compression model coefficients using arithmetic decoding. The decompression circuit can receive the compression coefficient as a bit stream. Since each coefficient value can be represented by a variable-length bit sequence, the decompression circuit can evaluate each bit of the bit stream and determine whether a decompression coefficient value can be obtained from the currently received bit.

The decompression circuit receives 802 one bit of compression coefficient data. The decompression circuit generates 804 a high-expansion bit sequence and a low-expansion bit sequence using the current received bit sequence of the compression factor data by adding a sequence of high or low bits to the received sequence. The received bit sequence may correspond to a bit sequence received by the decompression circuit, which does not correspond to a decompression coefficient value that has been output by the decompression circuit.

The decompression circuit applies a decision function 806 to the high extension bit sequence and the low extension bit sequence to determine the value of the decompression coefficient. The determined function may correspond to a plurality of received function parameters corresponding to the compression coefficient value. In some embodiments, applying the function to the high-expansion bit sequence or the low-expansion bit sequence generates a value between two different possible coefficient values and associated with a specific coefficient value based on an interval scheme.

The decompression circuit determines 808 whether the decompression coefficient values of the high-bit sequence and the low-bit sequence are the same. If so, the current bit sequence is sufficient to determine a decompression coefficient value, and the decompression circuit output 810 corresponds to the decompression coefficient value of the currently received bit sequence. Then, the decompression circuit can receive the extra bits of the compression coefficient data as part of a new bit sequence to determine the subsequent decompression coefficient value.

On the other hand, if the decompression coefficient values of the high-bit sequence and the low-bit sequence are different, the current bit sequence is not enough to generate a decompression coefficient value, and the decompression circuit receives 812 extra bits of the compression coefficient data until it corresponds to The decompression coefficient values of the high-expansion bit sequence and the low-expansion bit sequence match. Additional configuration information

The foregoing description of the embodiments of the present invention has been presented for illustrative purposes; it is not intended to be exhaustive or to limit the present invention to the precise form disclosed. Those who are familiar with the relevant technology can understand that in view of the content disclosed above, many modifications and changes are possible.

Some parts of this description describe embodiments of the invention in terms of algorithms and symbolic representations of information operations. Usually, those who are familiar with data processing technology use these algorithm descriptions and representations to effectively convey the essence of their work to other people who are familiar with the technology. Although these operations are described functionally, computationally, or logically, they should be understood as being implemented by computer programs or equivalent circuits, microprogram codes, or the like. In addition, it has also proven that, without loss of generality, it is sometimes convenient to refer to such operational configurations as modules. The described operations and related modules can be embodied in software, firmware, hardware, or any combination thereof.

One or more hardware or software modules can be used alone or in combination with other devices to execute or implement any of the steps, operations, or procedures described herein. In one embodiment, a computer program product including a computer readable medium containing computer program code is used to implement a software module, the computer program code can be executed by a computer processor to perform any or all of the steps described , Operation or procedure.

The embodiments of the present invention may also relate to a device for performing the operations herein. This device may be specially constructed for the required purpose, and/or it may include a general-purpose computing device that is selectively activated or reconfigured by a computer program stored in the computer. This computer program can be stored in a non-transitory tangible computer-readable storage medium coupled to a computer system bus or any type of medium suitable for storing electronic instructions. In addition, any computing system referred to in this specification may include a single processor or may be an architecture designed with multiple processors to increase computing power.

The embodiments of the present invention may also relate to a product produced by one of the calculation procedures described herein. Such a product may include information derived from a computing process, where the information is stored in a non-transitory tangible computer-readable storage medium and may include any embodiment of a computer program product or other data combination described herein.

Finally, the language used in this specification has been chosen mainly for readability and guidance purposes, and it has not been chosen to describe or limit the subject matter of the invention. Therefore, the scope of the present invention is not intended to be limited by this detailed description, but is limited to the discussion of the scope of any patent application based on this application. Therefore, the disclosure of the embodiments is intended to illustrate rather than limit the scope of the present invention described in the scope of the following invention applications.

100: Tensor Stream Processor (TSP) 102: Enter data value 104: Output data value/output value 106: Arithmetic circuit unit 106A: Arithmetic unit 106B: Arithmetic unit 108: Memory 110: Decompression circuit 110A: Decompression circuit/first decompression circuit 110B: Decompression circuit/second decompression circuit 112: Compression model coefficient 112A: Compression model coefficient 112B: Compression model coefficient 114: Function parameters 114A: The first function parameter 114B: Second function parameter 116: Decompress model coefficients 116A: Decompression model coefficients/decompression model parameters 116B: Decompression model coefficients/decompression model parameters 118: Predictive Model 120: Compiler 300: chart 302: The first curve 304: second curve/cumulative distribution curve/cumulative count distribution 306: Third curve/function 308: range 310: Scope 400: Decompression circuit 402: Compression factor value 404: Decompression coefficient value/decompression output coefficient 406: Sequence Expander Circuit 408: High bit sequence 410: low bit sequence 412: function parameters 414: Decompression function circuit 416: High coefficient value 418: Low coefficient value 420: Comparator and control circuit 450: Function calculation circuit 450a: The first function calculation circuit 450b: Second function calculation circuit 450c: Function calculation circuit 502: divider circuit / divider 602: program 604: program 606: program 608: program 610: program 702: program 704: program 706: program 708: program 710: program 802: program 804: program 806: program 808: program 810: program 812: program

FIG. 1 shows a schematic diagram of a system for storing and decompressing model coefficients used in a model according to some embodiments.

FIG. 2 shows a block diagram of a set of compression model coefficients that can be decompressed using different functions according to some embodiments.

3A and 3B show exemplary graphs showing the distribution of model coefficients according to some embodiments.

FIG. 4A shows a block diagram of a decompression circuit according to some embodiments.

4B shows an exemplary decompression function circuit including function calculation circuits corresponding to different function types according to some embodiments.

FIG. 5 shows a diagram of a plurality of decompression circuits for decompressing compression coefficient data in parallel according to some embodiments.

FIG. 6 is a flowchart of a procedure for generating a compression model coefficient set according to some embodiments.

FIG. 7 is a flowchart of a procedure for decompressing the compression model coefficients according to some embodiments.

Fig. 8 is a flowchart of a procedure for decompressing the compression model coefficients using arithmetic decoding.

The figures depict embodiments of the invention for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structure and method illustrated in this article can be used without departing from the principles or advantages of the present invention described in this article.

702: receive

704: receive

706: configuration

708: Unzip

710: Application

Claims (20)

A processor includes: a memory, which stores: compression coefficient data corresponding to a coefficient set associated with a prediction model; a function parameter set associated with the compression coefficient data; a decompression circuit, which Comprising: a first function calculation circuit associated with a first function type; and a second function calculation circuit associated with a second function type; wherein the decompression circuit is configured to: receive from the memory The function parameter set; choose between the first function calculation circuit and the second function calculation circuit based on the received function parameter set; and use the selected function calculation circuit to decompress the compression coefficient data to generate a Decompress the set of coefficients. For example, the processor of claim 1, further comprising an arithmetic circuit unit to receive an input data set and the decompression coefficient set, and perform one or more arithmetic operations based on the input data set and the decompression coefficient set to generate an output Value set. For example, the processor of claim 1, wherein the function parameter set includes a first parameter indicating a function type and at least one additional parameter corresponding to a function coefficient, and wherein the decompression circuit is based on the function type indicating the first parameter The first parameter is in the first function calculation circuit and the Make a choice between the second function calculation circuit. Such as the processor of claim 1, wherein a compression function is used to compress the compression coefficient data from the coefficient set, the compression function is selected based on a cumulative distribution of a plurality of values of the coefficient set, and the function parameter set corresponds to all Select the compression function. Such as the processor of claim 1, wherein a model training procedure is used to determine the plural values of the coefficient set. For example, the processor of claim 1, wherein the function parameter set corresponds to a function type, and the function type is selected from a polynomial function, a bimodal distribution function, a Gaussian distribution function, or a Passon distribution function ( At least one of Poisson distribution function). Such as the processor of claim 1, wherein arithmetic coding or Huffman coding is used to compress the coefficient set to generate the compression coefficient data. Such as the processor of claim 1, wherein the decompression circuit is configured to apply the function parameter set to at least a part of the compression factor data corresponding to a first compression factor to determine a first decompression of the decompression factor set The coefficient value. Such as the processor of claim 8, wherein the decompression circuit is further configured to: receive one or more bits of the first compression factor from the compression factor data Row; generate a first extended bit sequence and a second extended bit sequence based on the sequence of received bits; apply the function parameter set to the first extended bit sequence and the second extended bit sequence to determine A first respective coefficient value and a second respective coefficient value; and in response to determining that the first coefficient value and the second coefficient value are the same, output the first coefficient value as the first decompression coefficient value. Such as the processor of claim 9, wherein the decompression circuit is further configured to: in response to determining that the first coefficient value and the second coefficient value are different, receive the compression appended to the sequence of one or more bits At least one extra bit of the coefficient data is used to generate an updated bit sequence; and based on the updated bit sequence, an updated first extended bit sequence and a second extended bit sequence are generated. A method for decompressing model coefficients, comprising: receiving compression coefficient data in a decompression circuit, wherein the compression coefficient data corresponds to a coefficient set associated with a prediction model, and the decompression circuit includes a first A first function calculation circuit associated with a function type and a second function calculation circuit associated with a second function type; a function parameter set associated with the compression factor data is received in the decompression circuit; based on the received The function parameter set is selected between the first function calculation circuit and the second function calculation circuit to decompress the compression factor data; and Use the selected function calculation circuit to decompress the compression coefficient data to generate a decompression coefficient set. For example, the method of claim 11 further includes: receiving an input data set and the decompression coefficient set in an arithmetic circuit unit; and executing one or more arithmetic in the arithmetic circuit unit based on the input data set and the decompression coefficient set Operate to generate a set of output values. Such as the method of claim 11, wherein the function parameter set includes a first parameter indicating a function type and at least one additional parameter corresponding to a function coefficient, and wherein the decompression circuit is based on the function type indicating the function type. The first parameter selects between the first function calculation circuit and the second function calculation circuit. Such as the method of claim 11, wherein a compression function is used to compress the compression coefficient data from the coefficient set, the compression function is selected based on a cumulative distribution of a plurality of values of the coefficient set, and the function parameter set corresponds to the selected The compression function. Such as the method of claim 11, wherein a plurality of values of the coefficient set are determined through a model training procedure. For example, the method of claim 11, wherein the function parameter set corresponds to a function type, and the function type is selected from a polynomial function, a bimodal distribution function, a Gaussian distribution function or a Poisson distribution function. distribution function) of At least one. Such as the method of claim 11, wherein arithmetic coding or Huffman coding is used to compress the coefficient set to generate the compression coefficient data. The method of claim 11, wherein using the selected function calculation circuit to decompress the compression factor data includes: applying the function parameter set to at least a part of the compression factor data corresponding to a first compression factor to determine the decompression factor Set one of the first decompression coefficient values. The method of claim 18, wherein using the selected function calculation circuit to decompress the compression factor data further comprises: receiving one or a sequence of bits of the first compression factor from the compression factor data; based on the received The sequence of bits generates a first extended bit sequence and a second extended bit sequence; the function parameter set is applied to the first extended bit sequence and the second extended bit sequence to determine the first respective coefficient value And a second respective coefficient value; and in response to determining that the first coefficient value and the second coefficient value are the same, output the first coefficient value as the first decompression coefficient value. A system for storing and decompressing model coefficients, which includes: a decompression circuit, which includes: a first function calculation circuit associated with a first function type; and A second function calculation circuit associated with a second function type; wherein the decompression circuit is configured to: a first function calculation circuit based on the received function parameter set associated with a compression factor data set And a second function calculation circuit; and use the selected function calculation circuit to decompress the compression coefficient data set to generate a decompression coefficient set; and an arithmetic circuit unit for receiving an input data set and The decompression coefficient set, and one or more arithmetic operations are performed based on the input data set and the decompression coefficient set to generate an output value set.

Images