JP4516020B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to a technique effective when applied to MPEG image compression / decompression.

As a technique studied by the present inventor, for example, the following technique can be considered in MPEG (Moving Picture Experts Group) image compression / decompression.
In moving image compression standards such as ISO / IEC14496-2 (MPEG4), ISO / IEC13818-2 (MPEG2), and ISO / IEC11172-2 (MPEG1), a digitized image is divided into blocks, and a motion vector is detected for each block. Discrete cosine transform (DCT), quantization, and AC / DC prediction are performed, and Huffman coding is performed to compress the image data.
Among them, the motion vector detection has the smallest difference from the current frame from the macroblock range of the previous or subsequent image frame with respect to the 16 × 16 pixel macroblock (shift block) in the image frame. A position of 16 × 16 pixels is detected. Then, using the position vector and the difference (frame difference), the subsequent DCT, quantization, AC / DC prediction, and Huffman coding are performed, thereby enabling high video compression.
Further, the decompression process of the compressed data is realized by generating a compensation image from a procedure reverse to the above-described compression, that is, Huffman decoding, AC / DC prediction, inverse quantization, inverse DCT, and motion vector information.
In addition, image processing including moving image compression often involves a relatively simple calculation algorithm, and data parallelism for the same instruction is large. Therefore, a SIMD (Single Instruction Multiple Data stream) type parallel calculation method is often suitable for speeding up image processing.
As an implementation example of the SIMD type parallel computing architecture, for example, there is a general-purpose neuro computer. This computer architecture comprises a plurality of processors and one control system, and the control system operates by broadcasting common instructions and data to all processors. Each processor includes a local memory and an arithmetic unit (multiplier, ALU, shifter, etc.). The control system includes a rewritable program memory and a global memory. A broadcast data bus that transmits data from the control system to all the processors, and a common bus in which any one processor specified by the address signal transmits data to the control system via the tristate buffer, between the control system and the processor. Data exchange is performed.
The data transmitted from the control system to all the processors via the broadcast data bus is either the data on the memory in the control system or the data received by the control system from any one processor via the common bus. You can choose.
In addition, it is possible to execute a command from the control system by designating only one processor designated by an address signal. As a result, the control system can initialize the local memory of each processor, and 1 to N (N is an arbitrary natural number) data transmission between the control system and the processor.
In a general-purpose neurocomputer, each of a plurality of processors controlled by a single command broadcast from a control system is a mimic unit of a neuron (neural cell). The operation of the neural network is imitated by calculating weight value data in the local memory of each processor for input data broadcast to all the processors via the control system. In other words, by rewriting the control system program, it becomes possible to perform general-purpose parallel computation of a neuro-algorithm represented by backpropagation (error back-propagation), other neuro-algorithms, or other algorithms.
When the general-purpose neurocomputer is applied to motion vector detection in image processing, a plurality of processors can be used as calculation units for shift blocks. That is, the motion vector detection range shift block (16 × 16 pixels plural) on the image one frame before is initialized in the local memory of the processor (plurality). The motion vector detection image (16 × 16 pixel macroblock) of the current frame is broadcast from the control system to all processors. Each processor calculates a frame difference between the broadcast data and the data in the local memory. The control system can detect the image position of the shift block set in the smallest processor as a motion vector position by comparing the frame differences of all the processors.

By the way, as a result of the study of the MPEG image compression / decompression technique as described above, the following has been clarified.
The amount of calculation processing for motion vector detection and DCT / inverse DCT in MPEG image compression / decompression is very large. In the simplest “motion-compensated interframe coding” in motion vector detection, ± 15 pixels around a macroblock one frame before is set as a detection range, and a shift block as a comparison with the macroblock of the current frame is 961. There are. Therefore, the amount of calculation for VGA (pixel size: 640 × 480 pixels) is 278 megatimes or more, and 8 gigaseconds / second is required when converted with processing of 30 frames per second. Also in DCT and inverse DCT, 0.8 giga times / second is required in terms of processing of 30 frames per second for VGA size.
When the above processing is performed by a single arithmetic unit of a semiconductor integrated circuit device, an operating frequency of 10 gigahertz is required, and it is very difficult to mount it directly on a portable home appliance driven with low power consumption such as a digital video camera. It is.
In addition, when motion detection is performed using the architecture of the general-purpose neurocomputer, comparison image information for a macroblock is stored in the local memory of each processor, the macroblock information is broadcast, and the information in the local memory is If the difference calculation is performed, parallel operation can be realized in each processor.
In the DCT / inverse DCT processing, difference information in units of blocks is stored in the local memory of each processor, and the parallel operation can be realized by broadcasting the DCT / inverse DCT coefficients.
However, when the above motion detection and DCT processing are realized by only one SIMD type parallel computer, it becomes necessary to move the difference block information result of the processor that detected the motion to another processor for the DCT processing. Processing performance decreases. In addition, the motion detection requires an absolute difference calculator in the processor, and DCT requires a multiplier. Therefore, when motion detection and DCT are processed by one SIMD type parallel computer, it is necessary to configure both the absolute difference calculator and the multiplier in the processor, which increases the overall gate size.
SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus capable of operating at a high speed with a small circuit configuration in image processing such as MPEG image compression / decompression.
The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.
Of the inventions disclosed in the present application, the outline of typical ones will be briefly described as follows.
That is, in the image processing apparatus according to the present invention, a plurality of processors including at least a difference calculator and a local memory operate with a single command from the first control unit, and transmit data from the first control unit to all the processors. Connecting a first SIMD type computer having a broadcast data bus to one or more second SIMD type computers in which a plurality of processors including at least a multiplier operate with a single instruction from a second control unit The first SIMD computer performs motion detection processing in image processing, and one or more second SIMD computers perform DCT, inverse DCT, quantization, or inverse quantization processing in image processing. It is.
In the above configuration, the calculation result of the first SIMD type computer is transmitted in parallel to the processor in the second SIMD type computer via the buffer, and the processor performs processing in parallel in units of image blocks. Do.
In addition, by adding header information (block shift vector information, I / P / B frame information, etc.) indicating block attributes for each block-based data transmission, each processor of the second SIMD computer can Each header information is discriminated and a process suitable for the block can be efficiently performed.
Of the inventions disclosed in the present application, effects obtained by typical ones will be briefly described as follows.
(1) In MPEG video compression, since motion detection processing and processing such as DCT / quantization are processed by separate SIMD type computers, it can be configured with a minimum required arithmetic unit and local memory for each processing, Image compression processing can be realized with the number of processors corresponding to the performance required for each processing.
(2) Since motion detection and DCT / quantization can be performed by pipeline operation for each process, the performance can be improved.
(3) With a configuration of a SIMD computer having a relatively small number of processors, image processing such as moving picture compression / decompression represented by MPEG can be performed in real time, so that, for example, low power consumption of a digital video camera or the like With a semiconductor integrated circuit (electronic component) that can be mounted on a driving portable home appliance, it is possible to realize an image processing function having a higher pixel density than conventional ones.

FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram showing an internal configuration of the SIMD computer 100 included in the image processing apparatus according to the embodiment of the present invention.
FIG. 3 is an explanatory diagram showing a motion detection processing procedure in the SIMD computer 100 in the image processing apparatus according to the embodiment of the present invention.
FIG. 4 is an explanatory diagram showing a DCT / quantization processing procedure in the SIMD computer 200 in the image processing apparatus according to the embodiment of the present invention.
FIG. 5 is an explanatory diagram showing an inverse DCT / inverse quantization process procedure in the SIMD computer 300 in the image processing apparatus according to the embodiment of the present invention.
FIG. 6 is a block diagram showing a configuration of a viewer system to which the image processing apparatus according to the embodiment of the present invention is applied.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that in all the drawings for explaining the embodiments, the same members are denoted by the same reference numerals, and the repeated explanation thereof is omitted.
First, an example of the configuration of an image processing apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus according to the present embodiment is, for example, an image compression system, and includes an SIMD computer 100, an SIMD computer 200, an SIMD computer 300, buffers 401 to 406, 501 to 506, 601 to 606, and the like. Yes.
The SIMD type computer 100 includes a processor array 130 including a plurality of processors 101 to 116 including an arithmetic unit (for example, a differential arithmetic unit) and a local memory, a control unit 140, and the like.
The SIMD type computer 200 includes a processor array 230 including a plurality of processors 201 to 206 including an arithmetic unit (for example, a multiplier) and a local memory, a control unit 240, and the like.
The SIMD type computer 300 includes a processor array 330 including a plurality of processors 301 to 306 including an arithmetic unit (for example, a multiplier) and a local memory, a control unit 340, and the like.
The SIMD computer 100 and the SIMD computer 200 are electrically connected via a plurality of buffers 401 to 406, and the output of the control unit 140 in the SIMD computer 100 is input to the plurality of buffers 401 to 406 in parallel. The outputs of the buffers 401 to 406 are input in parallel to the plurality of processors 201 to 206 in the SIMD computer 200. The SIMD type computer 200 and the SIMD type computer 300 are electrically connected via a plurality of buffers 501 to 506, and outputs of the plurality of processors 201 to 206 in the SIMD type computer 200 are parallel to the plurality of buffers 501 to 506. The outputs of the buffers 501 to 506 are input in parallel to the plurality of processors 301 to 306 in the SIMD computer 300. Outputs of the plurality of processors 301 to 306 in the SIMD type computer 300 are input to the plurality of buffers 601 to 606. The outputs of the buffers 501 to 506 are transmitted to AC / DC prediction and Huffman processing, and the outputs of the buffers 601 to 606 are transmitted to compensation image generation processing.
In the SIMD type computer 100, the processors 101 to 116 in the processor array 130 and the control unit 140 are electrically connected by an instruction bus 150, a broadcast data bus 160, a processor data output common bus 170, and the like. In the SIMD computer 200, the processors 201 to 206 and the control unit 240 are electrically connected by an instruction bus 250 or the like. In the SIMD computer 300, the processors 301 to 306 and the control unit 340 are electrically connected by an instruction bus 350 or the like.
In FIG. 1, the SIMD type computers 100 to 300 are configured in three stages, but may be configured in two stages or four or more stages. Further, the number of processors 101 to 116 in the SIMD computer 100 is 16, but any number is possible. Further, six processors 201 to 206 and 301 to 306 and buffers 401 to 406, 501 to 506, and 601 to 606 in the SIMD type computers 200 and 300 are arranged in parallel, but any number of processors may be used.
FIG. 2 shows a detailed configuration of the SIMD type computer 100. The SIMD computer 100 includes a plurality of memory units 121 to 129 in addition to the control unit 140 and the processor array 130. The local memory and the memory units 121 to 129 in the processors 101 to 116 are constituted by a RAM (memory).
In the processor array 130, the processors 101 to 116 are arranged in a matrix, and the local memory in each of the processors 101 to 116 is connected to other processors in the vertical and horizontal directions so that arithmetic data can be shifted in the vertical and horizontal directions. Yes. Further, the local memories of the processors 104, 108, 112, 113, 114, 115, 116 located at the end of the processor array 130 are connected to the memory units 121 to 129 arranged around the processor array 130, and the memory unit 121. The calculation data can be shifted with respect to .about.129. The control unit 140 and the arithmetic units in all the processors 101 to 116 are connected via the instruction bus 150 and the broadcast data bus 160, so that instructions and data are output from the control unit 140 to all the processors 101 to 116. Yes. The outputs of the arithmetic units in all the processors 101 to 116 are connected to the control unit 140 via the tristate buffer and the processor data output common bus 170, and the arithmetic data of the arithmetic units in the processors 101 to 116 are sent to the control unit 140. It is output.
Each memory unit 121 to 129 is connected to another adjacent memory unit so that data can be shifted between the memory units. The memory units 121 to 129 and the control unit 140 are connected via a memory common bus 180.
Further, the control unit 140 is connected to an external control (main CPU) and an external memory (image data).
Next, the operation of the image processing apparatus according to the present embodiment will be described with reference to FIG. First, the SIMD computer 100 performs motion detection processing in image processing. Then, the SIMD computer 100 outputs the difference information and motion vector information for each block, which is the result of the motion detection process, to the buffers 401 to 406 in units of blocks. After outputting the difference information and the motion vector information to the buffers 401 to 406, the SIMD computer 100 performs a motion detection process for the next macroblock.
In the SIMD computer 200, the processors 201 to 206 perform DCT operations in image processing in parallel. At this time, the processors 201 to 206 take in the difference information for each block in the buffers 401 to 406 and perform DCT calculation. Next, based on the calculation result, the SIMD computer 200 performs quantization processing in parallel by the processors 201 to 206. After the DCT operation and the quantization process for each block, the processors 201 to 206 output the processing results to the buffers 501 to 506 in parallel with the motion vector information. The motion vector information and quantized data of each block in the buffers 501 to 506 are subjected to AC / DC processing and Huffman processing, and are output as compressed data.
Further, the motion vector information and the quantized data of each block in the buffers 501 to 506 are output to the SIMD computer 300 for generating a compensation image. In the SIMD computer 300, the processors 301 to 306 perform inverse quantization processing on each block in parallel. Next, based on the processing result, the processors 301 to 306 perform inverse DCT operations in parallel. After the inverse quantization process and inverse DCT operation of each block, the processors 301 to 306 output the processing results to the buffers 601 to 606 in parallel with the motion vector information. The motion vector information of each block in the buffers 601 to 606 and the data after inverse DCT calculation are used for the compensation image generation process.
In the SIMD type computers 100, 200, and 300, each process (motion detection, DCT operation, quantization, inverse quantization, and inverse DCT operation) is performed via the buffers 401 to 406, 501 to 506, and 601 to 606. Pipeline parallel processing is possible, and the overall processing is speeded up.
In this embodiment, the SIMD computer 200 performs DCT operation and quantization processing, and the SIMD computer 300 performs inverse quantization processing and inverse DCT operation. However, one SIMD computer performs DCT operation, The quantization process, the inverse quantization process, and the inverse DCT calculation may be performed, or each process may be shared and executed by three or more SIMD type computers.
In addition, in the buffers 401 to 406, 501 to 506, and 601 to 606, not only the calculation processing result and the block vector information but also information for each block, for example, whether or not the difference processing with the comparison image has been performed is written. Each processor can judge the information and perform different arithmetic processing.
Next, the procedure of the motion detection process in the SIMD computer 100 will be described with reference to FIG. FIG. 3A shows the motion detection processing order in units of macroblocks for the entire image, and FIG. 3B shows the processing flow for each macroblock.
As shown in FIG. 3A, the entire image (current image) is divided into macroblocks (16 × 16 pixels) and processed for each macroblock.
As shown in FIG. 3B, the macroblock is composed of luminance (Y0, Y1, Y2, Y3) and color difference (U, V). Y0, Y1, Y2, Y3, U, and V are each composed of 8 × 8 color elements.
After dividing into macro blocks, motion detection processing is performed for each macro block. The motion detection process is performed by detecting a difference from the comparison image.
Therefore, the information after the motion detection process with the comparison image is the difference value information (Y0 ′, Y1 ′, Y2 ′, Y3 ′, U ′, V ′) for each block Y0, Y1, Y2, Y3, U, V. And motion vector information.
The block information is output to the buffers 401 to 406.
Next, referring to FIG. 4, the procedure of DCT calculation and quantization processing in the SIMD computer 200 will be described.
The difference value information (Y0 ′, Y1 ′, Y2 ′, Y3 ′, U ′, V ′) and motion vector information of each block Y0, Y1, Y2, Y3, U, V is sent from the buffers 401-406 to the processors 201-201. The data is input to 206 in parallel, processed in parallel, and the processing result is output to the buffers 501 to 506 in parallel.
Next, the procedure of inverse quantization processing and inverse DCT calculation in the SIMD computer 300 will be described with reference to FIG.
The processing results and motion vector information of each block Y0, Y1, Y2, Y3, U, V are input in parallel from the buffers 501 to 506 to the processors 301 to 306, and are processed in parallel, and the processing results are buffered 601 to 606. Are output in parallel.
Next, an application example of the image processing apparatus according to the present embodiment will be described with reference to FIG. FIG. 6 shows an example in which the image processing apparatus according to the present embodiment is applied to a viewer system. This system includes, for example, the image processing apparatus 700 of the present embodiment, an AC / DC prediction Huffman 701, an image memory 702, a display circuit 703, a monitor 704, a ROM 705, a RAM 706, a CPU 707, an IF (interface) circuit 708, and the like. ing.
The image processing apparatus 700 is connected to an image memory 702 and an AC / DC prediction Huffman 701, the image memory 702 is connected to a display circuit 703, and the display circuit 703 is connected to a monitor 704. In addition, the AC / DC prediction Huffman 701, the ROM 705, the RAM 706, the CPU 707, and the IF circuit 708 are connected via a bus. The IF circuit 708 is connected to the memory card 709.
This system is a system for displaying MPEG images taken by a digital movie camera or the like on a monitor or TV.
Since this system only performs MPEG decompression processing, the inverse quantization and inverse DCT are processed with the configuration of only the SIMD computer 300 in the image processing apparatus of the embodiment shown in FIG. Since motion detection, DCT, quantization, inverse quantization, and inverse DCT processing are processed separately by each SIMD computer, an image processing apparatus can be configured with a SIMD computer of only necessary portions, and the size can be reduced. Therefore, low power consumption can be achieved.
Therefore, according to the image processing apparatus of the above embodiment, in MPEG video compression, processing such as motion detection and DCT / quantization is processed by separate SIMD type computers. The image compression processing can be realized with the number of processors corresponding to the performance required for each processing. In addition, since motion detection, DCT / quantization, and the like can be performed by pipeline operation for each process, performance can be improved.
In addition, with the configuration of a SIMD computer with a relatively small number of processors, image processing such as moving image compression / decompression represented by MPEG can be performed in real time, so that image processing with a higher pixel density than in the past is possible. The function can be realized by a semiconductor integrated circuit device (electronic component) that can be mounted on a portable home appliance driven with low power consumption such as a digital video camera.
As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the invention is not limited to the embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.
For example, in the above embodiment, MPEG video compression / decompression has been described. However, the present invention is not limited to this, and other image compression / decompression can also be applied.
In the above description, the case where the invention mainly made by the present inventor is applied to the image processing that is the technical field to which the present invention is applied has been described. However, the present invention is not limited to this. For example, other image processing and audio processing are performed. The present invention can also be applied to all electronic devices that process calculation algorithms including matrix operations such as the beginning.

  As described above, the image processing apparatus according to the present invention is suitable for use in electronic equipment that performs moving picture compression / decompression, such as a digital video camera, a video deck, and an information terminal. Further, the present invention can be applied to all electronic devices that process calculation algorithms including matrix operations including other image processing and audio processing.

Claims (3)

A plurality of first processors including a difference calculator and a local memory, a first control unit that controls the first processor, and data transmission from the first control unit to all the first processors A first SIMD computer having a broadcast data bus,
One or a plurality of second SIMD type computers having a plurality of second processors including a multiplier and a second control unit for controlling the second processor;
The first SIMD type computer is configured to transfer the operation result of the first SIMD type computer to the plurality of second processors in the second SIMD type computer in parallel, and each of them performs pipeline parallel processing. And a plurality of buffers for connecting the second SIMD type computer ,
In the first SIMD type computer, a plurality of the first processors operate in parallel by a single instruction from the first control unit, and perform motion detection processing in image processing.
In the second SIMD type computer, a plurality of the second processors operate in parallel by a single instruction from the second control unit, and discrete cosine transform, inverse discrete cosine transform, quantization or inverse in image processing is performed. An image processing apparatus that performs quantization processing. A plurality of first processors including a difference calculator and a local memory, a first control unit that controls the first processor, and data transmission from the first control unit to all the first processors A first SIMD computer having a broadcast data bus,
A plurality of second SIMD type computers having a plurality of second processors including a multiplier and a second control unit for controlling the second processor;
The operation result of the second SIMD type computer at the front stage is transferred in parallel to the second processor in the second SIMD type computer at the rear stage , and the first SIMD type computer and the plurality of second SIMD types are transferred. A plurality of buffers for connecting the first SIMD computer and the plurality of second SIMD computers in a configuration in which each type computer performs pipeline parallel processing;
In the first SIMD type computer, a plurality of the first processors operate in parallel by a single instruction from the first control unit, and perform motion detection processing in image processing.
In the second SIMD type computer, a plurality of the second processors operate in parallel by a single instruction from the second control unit, and discrete cosine transform, inverse discrete cosine transform, quantization or inverse in image processing is performed. An image processing apparatus that performs quantization processing. The image processing apparatus according to any one of claims 1 and 2 ,
The data transferred between the first SIMD type computer and one or a plurality of the second SIMD type computers is added with header information indicating the attribute of the block for each block-based data transfer. An image processing apparatus.

Images