JP4498848B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that encodes and decodes an image signal for multimedia signal processing.

  A conventional image processing apparatus has a RISC (Reduced Instruction Set Computer) core in order to avoid an increase in data transfer load due to a single bus as disclosed in, for example, Japanese Patent Laid-Open No. 2003-204556. And an independent bus for connecting the video decoder and the post processor.

JP 2003-204556 A

A conventional image processing apparatus having an independent bus connecting a RISC core, a video decoder, and a post processor has the following problems.
1. MPEG-2 standard (Moving Pictures Expert Group) decoding processing of ISO (International Organization for Standardization), limited to MPEG-2 and ITU (International Telecommunication Union) H.261 image processing for equivalent processing Also, it cannot support encoding processing.
2. Processing performed in each block in the video decoder includes VLD (Variable Length Decoder), DCT / IDCT (discrete Cosine Transform / Inverse discrete Cosine Transform), MC (Motion Compensation; Even if each processing load is biased, other blocks cannot be processed. For example, when the processing time of MC is long, processing of the next macroblock cannot be performed by the VLD or DCT / IDCT processing block.
3. There is only one bus for accessing the memory, and it cannot cope with an increase in data transfer load caused by memory access by the RISC core, video decoder, and post processor.
The present invention has been made to solve the above-described problems. An image processing apparatus compliant with a plurality of image processing standards such as MPEG-2 and MPEG-4 can be used with an optimal hardware scale and operating frequency. It aims to be realized.

The image processing apparatus of the present invention
A plurality of parallel processing processors that respectively read and store programs in which different processing operations are defined before the operation starts, and perform multimedia codec decoding processing of the stored program definitions in parallel;
Is connected to each of the plurality of parallel processing processor, a multi-port memory to simultaneously access available-recording or reading of data from different parallel processor,
Read-program the processing operation is defined before the operation is started, a variable length codec processor for performing in accordance with a program defining a variable length coding process or variable length decoding processing of data to and from the parallel processor,
A calculation result from each parallel processing processor is stored and can be mutually accessed from each parallel processing processor, and a reset register for resetting the parallel processing processor and the variable-length code decoding processor, and the parallel processing processor and variable-length code decoding processing It has an operation control register for stopping the operation of the processor or starting an encoding operation or a decoding operation. Each parallel processing processor and the multi-port memory have input / output terminals different from each other and a register block provided independently.

In the image processing apparatus of the present invention, a plurality of image processing standards can be accommodated by performing image processing with a programmable processor, and the same H / W can be processed into a plurality of processes by changing a program loaded into the processor. IDCT, IQ (Inverse Quantization), motion compensation, etc.].
Further, by using a multi-port memory, it is possible to cope with independent memory access of each processor, and each processor can access data without delay.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an image processing apparatus according to the present invention.
In FIG. 1, each of 11 to 14 has a plurality of arithmetic units, the arithmetic units operate in parallel, and the operation is defined by a program, such as DCT, IDCT, motion search, motion compensation, and filter processing. Parallel processor that performs multimedia encoding / decoding processing, 2 is variable-length encoding processing, variable-length decoding processor that processes variable-length decoding processing with program-defined operation, 3 is simultaneous data from multiple computing units Multiport memory that enables access, 4 is a register block that stores intermediate operation results from each arithmetic unit provided in the parallel processing processors 11 to 14, and is mutually accessible from each arithmetic unit. Reset registers that reset the parallel processing processors 11 to 14 and the variable-length code decoding processor 2 by writing various values, and the parallel processing processors 11 to 14 and the variable length An operation control register for stopping the operation of the codec decoding processor 2 or starting an encoding operation or a decoding operation is also provided.
5 is a control processor for overall control, 6 is an image interface signal input / output terminal for inputting / outputting image data, 7 is a bit stream buffer for temporarily storing a bit stream, and 8 is a bit stream interface signal input / output terminal. It is.

Next, the operation will be described.
In the image processing apparatus of the present invention, an initialization operation is first performed.
The control processor 5 writes a significant value to the reset register assigned to the register block 4, and this is read by the parallel processing processors 11 to 14 and the variable-length code decoding processor 2, so that the parallel processing processors 11 to 14 and The variable-length code decoding processor 2 is reset. The parallel processing processors 11 to 14 and the variable-length code decoding processor 2 reset their program counters to 0 by being reset. In the register block 4, there are registers for controlling the operations of the parallel processing processors 11 to 14 and the variable-length code decoding processor 2. With this register setting, the operations of the parallel processors 11 to 14 and the variable-length code processor 2 are stopped during the initialization operation.

  Next, under the control of the control processor 5, the parallel processors 11 to 14 and the variable-length code processor 2 read and download a program for image processing from an external storage device. That is, under the control of the control processor 5, the program is written into the program memory inside the parallel processing processors 11 to 14 and the variable length code decoding processor 2 via the register block 4. Finally, according to the setting of the operation control register of the register block 4, the parallel processing processors 11 to 14 and the variable-length code decoding processor 2 are activated to start the encoding operation or the decoding operation.

Next, the encoding operation will be described taking MPEG-4 as an example.
In the encoding operation, for example, the parallel processing processor 11 performs motion prediction processing, the parallel processing processor 12 performs DCT / Q (Quantization) / AC-DC prediction processing, and the parallel processing processor 13 includes an image format conversion filter. It is assumed that the parallel processing processor 14 is programmed with AC-DC difference (AC-DC predictive decoding processing) / IQ / IDCT processing and motion compensation processing by an initialization operation.

  When an image signal for encoding is input from the image interface signal input / output terminal 6, the parallel processor 13 performs image format conversion filter processing. For example, when the input image is an image of 720x480 and the encoded image is 352x288, a process of compressing the length and width is performed. The image data compressed to the encoded image size by the parallel processor 13 is stored in the multiport memory 3. Next, a motion prediction process is performed by the parallel processor 11. Transfers the image to be encoded that has been subjected to the image format conversion filter processing by the parallel processor 13 from the multiport memory 3 and the reference image that is the previously encoded decoded image, and the position of the image with the smallest difference among the reference images Is output to the register block 4 as a motion component. At the same time, the difference between the reference image and the encoding target image is output to the multiport memory 3.

Next, the parallel processing processor 12 continues the encoding process. The difference value between the reference image and the encoding target image is input from the multiport memory 3 and DCT / Q / AC-DC prediction processing is performed. The result is stored in register block 4.
The variable-length code processor 2 performs Huffman coding using the encoding parameters stored in the register block 4, that is, the motion components that are the processing result of the parallel processing processor 12 and the processing result of the parallel processing processor 11, The bit stream is temporarily stored in the bit stream buffer 7 and output from the bit stream interface signal input / output terminal 8.

On the other hand, the parallel processor 14 performs a local decoding process to predict the next frame. AC-DC predictive decoding, IQ, IDCT processing is performed from various parameters stored in the register block 4, and the predicted image transferred from the multiport memory 3 based on the motion component stored in the register block 4 is also used. Add. The locally decoded image resulting from the addition is transferred to the multiport memory 3 as a reference image.
The entire encoding process is controlled by the control processor 5.

Next, the decoding operation will be described.
In the decoding operation, for example, the parallel processing processor 11 performs deblocking filter processing, the parallel processing processor 12 performs AC-DC predictive decoding, IQ processing, the parallel processing processor 13 performs image format conversion filter processing, and the parallel processing processor. 14 assumes that IDCT processing and motion compensation processing are programmed by initialization.

A bit stream is input to the variable length code decoding processor 2 from the bit stream interface signal input / output terminal 8 via the bit stream buffer 7. The variable-length code decoding processor 2 performs Huffman decoding and stores parameters necessary for decoding in the register block 4.
The parallel processing processor 12 inputs necessary parameters from the register block 4 and performs AC-DC predictive decoding and IQ processing. The result is stored in the multiport memory 3. The parallel processor 14 inputs the IQ processing result from the multiport memory 3 and performs IDCT processing. Thereafter, a reference image is input from the multiport memory 3 and added to the IDCT result. The decoded image thus formed is stored in the multiport memory 3.

The parallel processor 11 performs deblocking processing on the generated decoded image. At this time, necessary encoding parameters are input from the register block 4.
After the deblocking process, the parallel processor 13 performs a process of enlarging the encoded size image to the video output size. The reverse of the image format processing described in the encoding operation is performed.
The image data converted into the video output format is output from the image interface signal input / output terminal 6 to a display device (not shown).

It can be seen that the decoding process has a smaller processing load than the encoding process. In the encoding process, motion compensation, AC-DC predictive decoding, IQ, and IDCT processes were performed by one parallel processor 14, whereas in the decoding process, it was divided by two parallel processors 12 and 14. Is going. In such a case, the power consumption can be reduced by lowering the operating frequency of the parallel processors 12 and 14.
Further, if the image processing apparatus is limited to decoding processing, the parallel processing processor 14 may be caused to perform motion compensation, AC-DC predictive decoding, IQ, IDCT processing, and the parallel processing processor 12 may be deleted. This can reduce the circuit scale.
The frequency and circuit scale can be adjusted not only by the encoding / decoding process, but also by the difference in image size (CIF (352x288), QCIF (176x144), etc.) for encoding. .

  Although the explanation of the operation has been made with MPEG-4 as an example, it can be applied to MPEG-2, H.261, etc. by changing the program loaded at the time of initialization.

As described above, in the image processing apparatus of this embodiment, it is possible to cope with a plurality of image processing standards by performing image processing with a programmable processor, and the same H by changing a program loaded to the processor. / W can be assigned to multiple processes (IDCT, IQ, motion compensation, etc.).
In addition, each processor independently accesses the memory, but data access is possible without any delay due to the multiport memory.

Furthermore, since the number of processors can be varied according to the processing load, the amount of hardware and power consumption can be optimized.
Furthermore, each parallel processing processor may change the operable number of a plurality of arithmetic units built therein according to the processing load. With this configuration, the amount of hardware and power consumption can be optimized according to the processing load.

  When applied to a chip for a digital audio / video (digital AV) product such as a digital camera, the H / W circuit scale and power consumption can be optimized.

It is a block diagram which shows the image processing apparatus of this invention.

Explanation of symbols

  11 to 14: parallel processing processor, 2: variable-length code decoding processor, 3: multiport memory, 4: register block, 5: control processor, 6: image interface signal input / output terminal, 7: bitstream buffer, 8: Bit stream interface signal input / output pin.

Claims (3)

A plurality of parallel processing processors that respectively read and store programs in which different processing operations are defined before the operation starts, and perform multimedia codec decoding processing of the stored program definitions in parallel;
Is connected to each of the plurality of parallel processing processor, a multi-port memory to simultaneously access available-recording or reading of data from different parallel processor,
Read-program the processing operation is defined before the operation is started, a variable length codec processor for performing in accordance with a program defining a variable length coding process or variable length decoding processing of data to and from the parallel processor,
A calculation result from each parallel processing processor is stored and can be mutually accessed from each parallel processing processor, and a reset register for resetting the parallel processing processor and the variable-length code decoding processor, and the parallel processing processor and variable-length code decoding processing An operation control register for stopping the operation of the processor or starting an encoding operation or a decoding operation is provided, and each parallel processing processor and multi-port memory are provided with a register block provided separately and independently from the input / output terminals . An image processing apparatus.   The image processing apparatus according to claim 1, wherein the plurality of parallel processing processors are configured to vary the number of parallel processing processors operable in accordance with a processing load amount of the entire apparatus.   2. The parallel processing processor includes a plurality of arithmetic units capable of parallel processing, and is configured to vary the number of operable arithmetic units according to a processing load of the parallel processing processor. Image processing apparatus.

Images