KR100434391B1 - The architecture and the method to process image data in real-time for DSP and Microprocessor - Google Patents

Abstract

The present invention can effectively calculate sum of absolute differences (SAD) values in a vector calculation and motion estimation algorithm such as discrete cosine transform (DCT) of image data for image signal processing on a digital signal processor (DSP) and a microprocessor. To a calculation method and a circuit thereof.

Description

The architecture and the method to process image data in real-time for DSP and Microprocessor

The present invention can effectively calculate sum of absolute differences (SAD) values in vector computation and motion estimation algorithms such as discrete cosine transform (DCT) of image data for image signal processing on digital signal processors (DSPs) and microprocessors. To a calculation method and a circuit thereof.

Recently, as the processing and communication of multimedia data is required, the compression algorithm area takes up a large portion for processing a large amount of image data. The image compression process requires a very complex multi-processing process and is the most computational process in real time image processing. Therefore, the development of a new architecture is essential since the real-time processing of image compression is impossible with the existing DSP chips. Therefore, special purpose multimedia DSP chips have been developed.

DCT is the most widely used image conversion algorithm in image compression technology and is used before Huffman coding is performed in the Joint Photographic Experts Group (JPEG) and Moving Picture Experts Group (MPEG). The DCT for the 2D M × N image is shown in Equation 1.

here,

As shown in Equation 1, DCT requires multiplication of 2 × M × N times and addition of (M-1) (N-1) times in order to process one-dimensionally. Since the multiplier is very large hardware and slow computations, we use a lot of row column decomposition and Chen's algorithm to reduce the amount of computation. The matrix separation calculation results in a two-dimensional DCT by performing a one-dimensional DCT on a row basis and then performing the one-dimensional DCT on a column basis again. Chen's algorithm reduces unnecessary duplication by using the periodic characteristic of Cosine, and the DCT determinant Equation 2 can be expressed by Equation 3 and Equation 4. Looking at the coefficient matrix of Equation 2, it can be seen that the left and right symmetry according to the odd (odd) and even (even) row groups starting from the center column, and thus can be expressed by dividing Equation (3) and Equation (4). It can be seen that the number of operations is reduced compared to. Equations 5 and 6 show the IDCT results.

here, to be.

The most computational process among the encoding processes of the MPEG algorithm is a process of obtaining a motion vector. Therefore, the encoding process requires more computation than the decoding process. The motion vector can be obtained in various ways, and there is no standard for the method, but the most common method is BMA (Block Matching Algorithm), which is a macro of the image currently encoded in the picture to be compared as shown in Equation (7). The SAD value is calculated while moving the block, and the motion vector MV obtains the minimum value among the SAD values as shown in Equation (8). This process is also difficult to process in real time with a conventional DSP chip.

Conventional fixed-point DSP chips, DSP56100, DSP1610, ADSP2100, and TMS320C6x, are general purpose DSP processors that do not have a special operation structure to handle vector and SMA operations of BMA. We do a lot of operations by repeating the process of storing and storing the result in a register. Therefore, new high-performance multimedia DSP chips have been developed to process high-speed video and multimedia data in real time, which has a large amount of data and requires a large amount of computation for its processing.

Sun's Ultrasparc's multimedia-only Visual Instruction Set (VIS) instruction set includes a Pdist instruction that calculates the absolute value of the 8-bit difference between two 64-bit operands to support motion estimation algorithms. TriMedia also has an ume8ii command that does the same thing. The operation structure for processing this instruction consists of eight 8-bit subtractors and adders, and an operator that calculates the absolute value of the negative value after the subtraction operation. However, the efficiency of processing multimedia data (32 bits) is inferior because the size of registers used to store input data and results is 64 bits. Hewlett Packard's PA-RISC MAX-2 instructions also support the calculation of the absolute value of the difference between two data in the motion estimation algorithm. However, since each operation is performed on each input data, a large amount of operations are required to process a plurality of data, thereby increasing the amount of computation.

PMADDWD, which is supported by Intel's Pentium processor's MMX instruction set, is more efficient than conventional DSP processors in performing vector operations (two 16 × 16 multipliers, 32-bit adders, and 64-bits for data storage). 3 registers) are designed to reduce the number of operation cycles and to perform high-speed operation. However, the number of cycles required for the operation is increased in that two result values are added again.

In addition, the conventional multimedia DSP processor uses a method of improving the use efficiency of registers by storing data in one register by packing data by byte or word. Figure 1 shows a vector operation operation using the MMX instruction (PMADDWD) of the Pentium processor of Intel Corporation. Each packing data is stored in 64-bit registers 0 (S [63:48], T [47:32], U [31:16], and V [15: 0]) through the packing instruction PACKSS. These data and other data to perform vector operations are likewise packed word by word to register 1 (W [63:48], X [47:32], Y [31:16], Z [15: 0]). Are stored in. The data of register 0 and register 1 are connected to the input of the 16x16 multiplier for each word and perform a multiplication operation. Each of the four result data is connected to the input of the 32-bit adder and added, and the two result data are stored in the upper 32-bit register and the lower 32-bit register area of the 64-bit register to complete the instruction operation. In order to perform the final vector operation, a 32-bit adder is required, and the two operation results stored in the 64-bit register are inputted to the 32-bit addition operator again, and the operation is stored in the register to complete the vector operation. There is a problem that needs to add two more city commands.

In addition, in calculating the SAD value in the motion estimation algorithm, Sun's Ultrasparc Pdist instruction and multimedia DSP processor ume8ii, which is a multimedia DSP processor, perform the operation as shown in FIG. . As mentioned above, for the efficiency of registers, eight data are packed by byte and then each is a1 [64:56], a2 [55:48], a3 [47:40], a4 [39:32], Each byte of a5 [31:24], a6 [23:16], a7 [15: 8], and a8 [7: 0] is stored in 64-bit register 0. The stored data is packed in register 1 and inputted to the 8 × 8 subtraction operator for each byte and stored data (b1, b2, b3, b4, b5, b6, b7, b8) and subjected to subtraction. The negative result is the absolute value (abs: absolute), and the eight positive result values are added to the 8 × 8 adder to perform the addition operation, and these four result data are weighted sum. The SAD operation is completed by storing the data in a register. In the prior art, since a 64-bit register is used to store data, a register having a larger number of bits than the output data is used, thereby reducing space utilization and increasing hardware by using a large register.

As such, Sun's Ultrasparc, a DSP processor that supports conventional multimedia operations, or TriMedia, a multimedia DSP processor, can store input data and result data before and after processing a large amount of data when processing a motion estimation algorithm, as shown in FIG. This requires the use of very large 64-bit registers, which results in reduced register usage and increased hardware size.

Also, as shown in FIG. 1, the MMX instruction of Intel Corporation that supports vector operations needs to add two result data packed and stored in one register to perform the addition operation as shown in FIG. There was a problem that the vector operation must be completed by using more instruction cycles.

Accordingly, in order to solve the above problems, the present invention provides an operation method for efficiently calculating SAD values for vector operation and motion estimation of data in image signal processing in a DSP processor, and a circuit for executing the operation method. Therefore, it is effective for digital filter and DCT operation processing, and it is to reduce the computational cycle to reduce hardware burden and to process data in real time.

In addition, another object of the present invention is to provide an arithmetic circuit capable of increasing utilization by reducing the frequency of use of a register by providing a packing network structure during data processing and storage.

In order to achieve the above object, the present invention provides a first register for storing four input data by byte, and a second register for storing another input four data by byte; Four 8x8 multipliers for performing simultaneous multiplication of data stored in the registers byte by byte; Two 16-bit adders for adding the four multiplication operations of the multipliers by two; One 32-bit adder for performing addition of the operation values of the two adders once again; An operation circuit including a 40-bit third register in consideration of an overflow of the adder to store the result value of the 32-bit adder, and an operation method for processing image data using the circuit.

In addition, the present invention provides a 32-bit first register for storing four input data by byte, a 32-bit second register for storing four other input data by byte, and outputs each byte by byte from the two registers. Four subtractors for subtracting two data; Four abs calculators for calculating the result value subtracted by the four subtractors with two's complement, and first and second adders for adding the result values output from the four abs calculators two by two, Another arithmetic circuit including a third adder for re-adding the result added by the second adder, and a 32-bit third register for storing the 16-bit result added by the third adder and the circuit It is characterized by a calculation method for image data processing using.

1 is a diagram illustrating a flow of PMADDWD instruction operation of a Pentium MMX processor of Intel Corporation.

Figure 2 is a view showing the flow of Ultrasparc Pdist command operation of the prior art Sun.

Figure 3a illustrates arithmetic circuitry for executing PMADB instructions in accordance with the present invention.

Figure 3B illustrates arithmetic circuitry for executing PMADB16 instructions in accordance with the present invention.

4A illustrates arithmetic circuitry for executing PSADB instructions in accordance with the present invention.

4B illustrates arithmetic circuitry for executing PSADB16 instructions in accordance with the present invention.

5 is a view showing an abs operator used in the present invention.

Figure 6a illustrates a Pack16 mode in accordance with the present invention.

Figure 6b is a view showing a pack mode according to the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

3A and 3B show an arithmetic circuit for executing a vector operation according to the present invention, and the arithmetic execution method of the arithmetic circuit is called a PMADB instruction and a PMADB16 instruction. 3A shows an arithmetic circuit for executing the PMADB instruction, the arithmetic circuit comprising: a first register for storing four input data bytes by byte; A second register for storing another four pieces of data input for each byte; Four 8x8 multipliers for simultaneously multiplying data stored in the registers byte by byte; A first pipeline stage for reducing a delay path of the resultant data of the multiplier; Two 16-bit adders for adding the four multiplication operations of the multipliers by two; A second pipe line stage for reducing a delay path of the resultant data of the adder; One 32-bit adder for performing the addition operation of the two adders once again; In order to store the result value of the 32-bit adder, a 40-bit third register is configured in consideration of the overflow of the adder, and an operation circuit for executing the PMADB16 instruction executes the PMADB instruction as shown in FIG. 3B. It consists of a circuit including the component of FIG. 6 mentioned later in the calculating circuit for below.

An operation method using an operation circuit for executing the PMADB instruction for processing the vector operation is performed by byte (A [31:24], B [23:16], C [15: 8], D [7) in the first register. 4 data stored as: 0]) is stored in the second register (E [31:24], F [23:16], G [15: 8], H [7: 0]) and 4 bytes per byte. Input to 8 8 multipliers (A [31:24] × E [31:24], B [23:16] × F [23:16], C [15: 8] × G [15: 8] ], D [7: 0] × H [7: 0]). Four 16-bit data, which are the result of each operation of the multiplier, are inputted and added to two 16-bit adders, respectively, and the two 16-bit data, which is the result, are inputted to the 32-bit adder again to perform an add operation and 40 bits. The vector operation is completed by being stored in the third register of.

When only 16 bits of the calculation result are valid, the added data stores a value in a part of the register where all the data is stored through the PMADB16 command using the Pack network as shown in FIG. 3B. Can be used for other purposes to increase the utilization of register files and memory. Many of these operations are pipelined to process four data in one cycle. By utilizing the pipelined structure, the computational structure of the present invention enables high-speed data processing.

4A and 4B show an arithmetic circuit for executing an SAD operation, and the arithmetic method by the arithmetic circuit is called a PSADB (Sum of Absolute Difference Byte) instruction and a PSADB16 (SAD with packing) instruction, respectively. 4A shows an arithmetic circuit for executing the PSADB instruction, a 32-bit first register storing four input data bytes by byte, a 32-bit second register storing four other input data bytes by byte; Four subtractors for subtracting each of two pieces of data output for each byte from the two registers; A first pipeline stage for reducing a delay path of the resultant data subtracted by the four subtractors; Four abs calculators for calculating the result values passing through the first pipeline stage with two's complement; A second pipeline stage for reducing a delay path of the result data of the abs operator; First and second addition strengths for adding two result values of the abs calculator that have passed through the second pipeline stage; A third pipeline stage for reducing a delay path of resultant data added by the two adders; And a third adder for re-adding the result values of the first and second adders having passed through the third pipeline stage, and a 32-bit third register for storing the 16-bit result value added by the third adder. And an arithmetic circuit for executing the PSADB16 instruction is composed of a circuit including the components of FIG. 6 in the arithmetic circuit shown in FIG. 4A as shown in FIG. 4B.

In the calculation method using the PSADB operation circuit for processing the SAD operation, four data packed in the first register is subtracted from the four data packed in the second register and eight bits four subtracted from each byte to obtain an absolute sum of data. To the operation (A [31:24] -E [31:24], B [23:16] -F [23:16], C [15: 8] -G [15: 8], D [7 : 0] -H [7: 0]). The sign bit, which is the most significant bit ([31]) for the four data that are the result of the operation of the subtractor, is converted to a positive value (sign bit is '0') for a negative value (sign bit is '1'). In order to take two's complement, each of them is input to four abs operators.

As shown in FIG. 5, the abs operator is composed of one 32-bit NOT gate, one 2-to-1 multiplexer, one 32-bit adder, and one 3-state buffer to perform NOT operation on data, thereby performing data operation on all bits. 2's complement is completed by toggling ('0' → '1', '1' → '0') and applying a '1' signal to the carry input of the next adder. The two's complement operation is used to control the one's complement operation of the data by using the sine bit of the data as a select signal of the multiplexer, and the three's complement input to the '1' signal input of the carry input unit of the adder. -The state buffer controls the operation of each operator by using it as the enable control signal of the state buffer.

Four data, which are the result values of the abs operator, are inputted to the first adder and the second adder of 8 bits each to perform addition, and the two result values calculated by the adders are finally added through the 16-bit adder. It is added and stored in the lower 16 bits ([15: 0]) of the 32-bit register.

As with PMADB16, the PSADB16 instruction stores values in some of the registers according to the results of the data operation, thus increasing the utilization of register files and memory by using fewer registers and using the remaining registers for other purposes.

6A and 6B show a circuit for performing an operation of a pack network used to improve the efficiency of using a register according to the present invention, and operating in a pack mode using one 16-bit multiplexer. Will be

The inputs of the data storage registers of Fig. 6 are packed data [15: 0] and unpacked data [31: 0]. Pack networks are divided into three modes. No-pack mode with no operation on the pack network, 16-bit shift left shift of past calculation results, and 16-bit pack mode for storing the current result in the lower 16 bits, and 32-bit operation Pack mode saves the result to 16 bits. The no-pack mode is a mode in which no operations are performed on the pack network when performing general instructions. In pack 16 mode, if only 16 bits are valid, the previous 16 bits are shifted to the upper 16 bits and the current 16 bits are stored in the lower 16 bits. Pack mode is used when the median value of a 16-bit operation is extended to 32 bits, and the data must be packed with 16 bits to operate again with 16-bit data.

6A and 6B show operations of the pack 16 mode and the pack mode, and use a multiplexer to control the data output of the data storage register as a selection signal according to the pack mode. The operation result can be predicted in the form of the result value according to each instruction, and since the part necessary for the next operation can be known, the pack mode according to the data type can be determined. In other words, the processor decodes the instruction to generate a selection signal according to its characteristics, and thus the pack mode is determined. This packing network at the output stage saves two result data in the register used for data storage as described above, thereby reducing the number of registers required for data storage and reducing the number of registers used for other purposes. Increase

The multiplier used in the present invention is composed of a boot multiplier that reduces the size of hardware and the operation execution time by cutting the number of partial products in half for high speed processing. In addition, by using a pipeline structure for each stage, it reduces the processing delay time of data and performs high speed operation.

The present invention converts the data into a desired form by using the pack network at the same time as the input and output terminals during the calculation, and stores the calculated result values by left-shifting in units of bytes and words. As a result, registers can be used efficiently and register file complexity can be eliminated by eliminating byte addressing, and multiple operations can be processed simultaneously for fewer clock cycles.

In addition, the vector operation and SAD operation architecture of the present invention is efficient for 32-bit multimedia data processing by using 32-bit registers, and enables high-speed processing of data by reducing unnecessary cycles. Used in the output stage, the result data is sometimes packed and stored, thereby reducing the register usage space of the output stage in half. In addition, the remaining space can be used to store other data to increase the efficiency of register use, and as a result, using a small number of registers has the effect of allowing the remaining registers to be used for other purposes.

That is, the present invention processes multiple data and operations simultaneously to reduce the computational burden of the processor, improve the processing speed of the data, and improve the efficiency of registers by reducing the number of two computation cycles compared to the conventional multimedia DSP processor when performing vector computations. Can increase.

Claims (9)

delete A first register for storing four input data bytes by byte; A second register for storing another four pieces of data input for each byte; Four 8x8 multipliers for simultaneously performing data multiplication by bytes stored in the registers; A first pipeline stage for reducing a delay path of the resultant data of the multiplier; Two 16-bit adders for adding four multiplication values of the multipliers that have passed through the first pipeline stage by two; A second pipeline stage for reducing a delay path of the resultant data of the adder; One 32-bit adder for performing addition of the addition operation values of the two adders once passed through the second pipeline stage; A multiplexer for performing operations of 16 bits of packed operation result and 32 bits of unpacked operation result; A result storage register for storing the operation result value; And a 40-bit third register in consideration of the overflow of the adder in order to store the result value of the 32-bit adder. delete A 32-bit first register that stores four input data bytes by byte, A 32-bit second register for storing the other four input data for each byte; Four subtractors for subtracting each of two pieces of data output for each byte from the two registers; A first pipeline stage for reducing a delay path of resultant data of the subtractors; Four abs (absolute) calculators for calculating the result value subtracted from the four subtractors having passed through the first pipeline stage with two's complement; And First and second adders for adding two result values output from the four abs calculators; A second pipeline stage for reducing a delay path of the resultant data of the adders; A third adder for re-adding result values added by the first and second adders passing through the second pipeline stage; And a 32-bit third register for storing the 16-bit result added by the third adder. The method of claim 4, wherein And a multiplexer for performing operations of a 16-bit packed operation result and a 32-bit unpacked operation result between the third adder and the 32-bit register, and a result storage register for storing the operation result. Computation circuit for real-time operation data processing in the DSP processor and the microprocessor, characterized in that to increase the use efficiency of the register. The method of claim 2, A computing circuit for real-time arithmetic data processing in a DSP processor and a microprocessor, characterized in that it is configured in the same way as the circuit of claim 2 even if the number of bits of input / output data of the circuit changes. The method of claim 2, 4. A computing circuit for real-time arithmetic data processing in a DSP processor and a microprocessor, characterized in that it is configured identically to the circuit of claim 2 even if the size of the datapath in the circuit changes. The method of claim 2, Even if the size of the operand of the data is changed when the program is written to perform the operation, the same as the circuit of claim 2 is configured for real-time associative data operation data processing in the DSP processor and the microprocessor Operation circuit. Dividing the eight input data into two groups of four and storing the data in bytes; Performing subtraction by extracting the data stored in the two groups for each byte; Calculating the subtracted result with two's complement; Performing two additions of the four result values calculated by the two's complement and outputting two addition result values; And adding the result values of the two additions once again to output and store one addition result value.