JP2009170992A - Image processing apparatus and its method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of reducing loads for coefficient rearrangement and achieving high-speed processing, and to provide a method and a program for the image processing apparatus. <P>SOLUTION: The image processing apparatus 100 comprises a decoding unit 111 for decoding a quantized and encoded conversion coefficient and outputting a one-dimensional coefficient string, a coefficient scanning unit 112 for performing coefficient scanning of the conversion coefficient outputted from the decoding unit 111 from the one-dimensional coefficient string into a two-dimensional coefficient string, a dequantization unit 114 for dequantizing the conversion coefficient obtained by the coefficient scanning unit 112, and a conversion processing unit 115 capable of converting the one-dimensional coefficient string into the two-dimensional coefficient string with respect to dequantized coefficient data. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method for processing a digital image, and a program.

In recent years, image information has been digitized and handled. At that time, for the purpose of transmitting and storing information with high efficiency, orthogonal transform such as Discrete Cosine Transform (DCT) is used by utilizing redundancy unique to image information. Devices that conform to a system such as MPEG (Moving Picture Experts Group) that compresses by motion compensation are widely used for both information distribution in broadcasting stations and information reception in general households.
In particular, MPEG2 (ISO / IEC 13818-2) is defined as a general-purpose image coding system, and is a standard that covers both interlaced and progressively scanned images, standard resolution images, and high-definition images. And is now widely used in a wide range of consumer applications.

  FIG. 1 is a functional block diagram of a typical MPEG codec decoder.

  A typical MpegCodec decoder 1 shown in FIG. 1 is configured by combining IDCT (Inverse Discrete Cosine Transform) and prediction with motion compensation. The decoder 1 includes an IDCT processing unit 2, a prediction unit 3 with motion compensation, an adding unit 4, and a frame buffer 5.

The IDCT processing unit 2 includes a variable length decoding unit 21, an AC / DC prediction unit 22, an inverse quantization unit (IQ) unit 23, a coefficient scanning (ZigZag-Scan / Alternative Scan) unit 24, and an IDCT unit 25. .
The prediction unit 3 with motion compensation includes a motion vector decoding unit 31, a motion vector prediction unit 32, and a motion compensation unit 33.
Then, finally, the output of the IDCT processing unit 2 and the output of the prediction unit 3 with motion compensation are added together by the adding unit 4 to generate pixel data.

In a variable length encoding unit of an encoder (not shown), rearrangement into a quantized one-dimensional coefficient sequence and variable length encoding is performed to reduce redundancy.
Specifically, when the coefficient is arranged as a one-dimensional array in the variable length coding unit, a combination of run (RUN) information indicating the number of zeros and level (LEVEL) information indicating the size of the coefficient value. Generate run-length data.
Variable length coding is performed by assigning codewords having different code lengths to the combination of the RUN information and the coefficient value, thereby reducing the data capacity.
Therefore, the variable length decoding unit 21 of the decoder 1 generates zero coefficients for the number of decoded RUN information values, and generates a decoded coefficient sequence with the level (LEVEL) value as a coefficient value for the non-zero coefficients.

The DCT coefficients to be decoded are collectively variable-length decoded in units of 8 × 8, and after AC / DC prediction is applied by the AC / DC prediction unit 22, the inverse quantization unit 23 multiplies the quantization coefficient by the quantization coefficient. Quantized.
After inverse quantization, the coefficient scanning unit 24 refers to the coefficient scanning table and rearranges each coefficient to convert it to a frequency array that is input to the IDCT processing, and from the frequency domain by IDCT (inverse discrete cosine transform) of the IDCT unit 25. Pixel data is decoded by converting into the spatial domain.

In the above-described MPEG Codec decoder, a device has been devised to increase the decoding speed by omitting the IQ operation for the zero coefficient.
In this case, only non-zero coefficients are packed and stored in a predetermined area on the data memory, subjected to AC / DC prediction, IQ calculation, etc., and then rearranged into a two-dimensional data array in consideration of the zero coefficient (coefficient Scan).

With this method, it is possible to omit useless IQ calculation, but the time required for coefficient rearrangement increases.
In particular, since the MPEG Codec stream encoded at a high bit rate has a low ratio of non-zero coefficients, the gain due to ignoring the zero coefficients and reducing IQ processing is low, and conversely, the overhead due to memory access for array rearrangement is relatively low. Therefore, there is a disadvantage that the processing load increases at a high bit rate.

  An object of the present invention is to provide an image processing apparatus, a method thereof, and a program capable of reducing the effort of rearranging coefficients and realizing a high-speed processing.

  A first aspect of the present invention is an image processing apparatus that blocks an input image signal, inversely quantizes image information quantized in units of the block, performs inverse transform processing, and decodes the quantized code. A decoding unit that decodes the transformed transform coefficients and outputs a one-dimensional coefficient sequence; and a coefficient scanning unit that scans the transform coefficients output from the decoding unit from the one-dimensional coefficient sequence to the two-dimensional coefficient sequence An inverse quantization unit that inversely quantizes the transform coefficient obtained by the coefficient scanning unit, and a conversion process that can convert the inversely quantized coefficient data from a one-dimensional coefficient string to a two-dimensional coefficient string Part.

  Preferably, the coefficient scanning unit loads a coefficient into an internal memory of an arithmetic unit having an instruction set for calculating a plurality of elements in parallel, and stores the coefficient in advance in a two-dimensional array that can be directly input to the conversion processing unit. Has the function of rearranging.

  Preferably, a non-zero DCT coefficient value that is variable-length decoded by the decoding unit and output is output using a zigzag scan table and RUN information, and is converted from a one-dimensional coefficient sequence to an input sequence of the conversion processing unit. An address is generated for conversion into a frequency array of dimension coefficient sequences.

  Preferably, in a series of decoding processes from the decoding unit to the conversion processing unit, zero coefficient masking is performed on two-dimensional data including zero coefficients as at least preprocessing of inverse quantization (IQ) processing.

  Preferably, the zero coefficient mask related to the IQ process masks the IQ process when all the calculation target elements of the calculator for calculating a plurality of elements in parallel are zero when performing the IQ calculation.

  Preferably, the arithmetic unit is implemented as a hardware (H / W) dedicated instruction that is applied to the plurality of elements to be calculated in the IQ processing validity / invalidity determination.

  Preferably, the arithmetic unit loads only a quantization table corresponding to a coefficient group effective for IQ calculation according to the validity / invalidity determination of the IQ processing.

  Preferably, an AC / DC prediction unit that performs AC / DC prediction is provided after the coefficient scanning unit and before the conversion processing unit, and the arithmetic unit performs AC / DC prediction according to the validity / invalidity determination of the IQ processing. When all the target coefficient groups at the time of DC prediction are zero, a predetermined pixel value is moved from the adjacent block as it is at the time of AC / DC prediction.

  Preferably, the arithmetic unit directly loads a one-dimensional coefficient sequence, which is an output of the decoding unit, into the target internal memory, and moves a coefficient scan to the two-dimensional coefficient sequence between internal memories (mv). Execute by instruction.

  Preferably, the computing unit selectively switches between a first process performed at the output stage of the decoding unit and a second process performed at the input stage of the conversion processing unit. It has possible functions.

  Preferably, the arithmetic unit outputs the coefficient scanning process when the coefficient output from the decoding unit that outputs the one-dimensional coefficient sequence exceeds the preset threshold value, and performs the coefficient scanning process in the second process. Is switched to the first process.

  Preferably, the arithmetic unit holds the value obtained from the decoding unit in the internal memory of the arithmetic unit without processing until the processing to be adopted is determined to be the first processing or the second processing. .

  A second aspect of the present invention is an image processing method that blocks an input image signal, inversely quantizes image information quantized in units of the block, and performs inverse transform processing to decode the quantized code. A decoding step of decoding the transformed transform coefficient to obtain a one-dimensional coefficient sequence, a coefficient scanning step for scanning the transform coefficient obtained in the decoding step from a one-dimensional coefficient sequence to a two-dimensional coefficient sequence, An inverse quantization step for inversely quantizing the transform coefficient obtained in the coefficient scanning step; a transform processing step for performing a transform process from the one-dimensional coefficient string to the two-dimensional coefficient string on the inversely quantized coefficient data; Have.

  Preferably, the input image signal is blocked, the image information quantized in units of the block is inversely quantized, and inverse transform processing is performed to decode the image information. Decoding processing to obtain a one-dimensional coefficient sequence, and conversion coefficients obtained in the decoding step are obtained in the coefficient scanning processing in which the coefficient scanning from the one-dimensional coefficient sequence to the two-dimensional coefficient sequence is performed in the coefficient scanning step. Image processing including inverse quantization processing that inversely quantizes the conversion coefficient and conversion processing that performs conversion processing from the one-dimensional coefficient sequence to the two-dimensional coefficient sequence for the inversely quantized coefficient data This is a program to be executed.

According to the present invention, the quantized and encoded transform coefficients are decoded in the decoding unit, and the resulting one-dimensional coefficient sequence is input to the coefficient scanning unit.
In the coefficient scanning unit, coefficient scanning (coefficient rearrangement) from a one-dimensional coefficient sequence to a two-dimensional coefficient sequence is performed and output to the inverse quantization unit.
In the inverse quantization unit, the transform coefficient decoded by the decoding unit is inversely quantized and output to the transform processing unit.
In the change processing unit, conversion processing from the one-dimensional coefficient sequence to the two-dimensional coefficient sequence is performed on the dequantized coefficient data.

  According to the present invention, efficient parallel processing in a plurality of processing devices can be realized.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

An embodiment of the present invention provides a decoding unit (variable-length decoding unit, arithmetic coding unit, etc.) that outputs a one-dimensional coefficient sequence for an arithmetic unit having an instruction set (SIMD calculation, etc.) for calculating a plurality of elements in parallel Further, the present invention is applied to a decoding device having coefficient scanning (ZigZagScan), inverse quantization (IQ), and conversion processing from a one-dimensional coefficient sequence to a two-dimensional coefficient sequence (inverse discrete cosine transform processing, inverse wavelet transform, etc.).
In the embodiment of the present invention, unlike a typical decoding device, basically, a one-dimensional output sequence of a decoding unit (variable length decoding unit, arithmetic code unit, etc.) that outputs a one-dimensional coefficient sequence. The conversion from the coefficient sequence to the two-dimensional coefficient sequence is performed immediately after the variable length decoding.
That is, in the present embodiment, a two-dimensional format that can be directly input to a conversion processing unit (such as an IDCT unit) from a one-dimensional coefficient sequence to a two-dimensional coefficient sequence at the stage of loading coefficients into the internal memory of the arithmetic unit By rearranging in the array in advance, a configuration that reduces redundant rearrangement processing at an intermediate stage is realized.
In other words, in this embodiment, after variable length decoding (VLC), an IQ code is obtained by dividing a DCT block into a plurality of sub DCTs and assigning a quantization table corresponding to a processor element for each sub DCT block. Can be efficiently performed by SIMD.

  A specific configuration and function of an MPEG codec decoder as an image processing apparatus according to an embodiment of the present invention will be described below.

<First Embodiment>
FIG. 2 is a functional block diagram showing a configuration of an MPEG codec (Codec) decoder as an image processing apparatus according to an embodiment of the present invention.

As shown in FIG. 2, the present decoder (image processing apparatus) 100 is configured by combining IDCT (Inverse Discrete Cosine Transform) and prediction with motion compensation.
The decoder 100 includes an IDCT processing unit 110, a prediction unit 120 with motion compensation, an adding unit 130, a frame buffer memory 140, and a computing unit (processor) 200 having a SIMD computing function.

The IDCT processing unit 110 includes a variable length decoding unit 111, a coefficient scanning (ZigZag-Scan / Alternative Scan) unit 112, an AC / DC prediction unit 113, an inverse quantization unit (IQ) unit 114, and an IDCT unit 115. .
Further, the motion compensated prediction unit 120 includes a motion vector decoding unit 121 and a motion compensation prediction unit 122.
Finally, pixel data is generated by adding the output of the IDCT processing unit 110 and the output of the motion compensated prediction unit 120 by the adding unit 130, and stored in the frame buffer memory 140.

  The decoder (image processing apparatus) 100 according to the present embodiment targets a configuration including an arithmetic unit that targets image decoding by software and calculates a plurality of elements in parallel. Further, the present invention is applied to a decoding process from the variable length decoding unit to the IDCT unit.

  The variable length decoding unit 111 receives data encoded by an encoding device (not shown), performs variable length decoding processing, and outputs quantized data obtained as a result of the processing to the coefficient scanning unit 112.

In a variable length encoding unit of an encoder (not shown), rearrangement into a quantized one-dimensional coefficient sequence and variable length encoding are performed to reduce redundancy.
Specifically, when the coefficient is arranged as a one-dimensional array in the variable length coding unit, a combination of run (RUN) information indicating the number of zeros and level (LEVEL) information indicating the size of the coefficient value. Generate run-length data.
Variable length coding is performed by assigning codewords having different code lengths to the combination of the RUN information and the coefficient value, thereby reducing the data capacity.
Therefore, the variable length decoding unit 111 of the decoder 100 generates zero coefficients corresponding to the number of decoded RUN information values, and for non-zero coefficients, the decoding coefficient sequence is set with the value of the DCT coefficient (level: LEVEL) as the coefficient value. Generate.

  The coefficient scanning unit 112 uses the zigzag scan table and the RUN information for the non-zero DCT coefficient (= LEVEL) value output after the variable length decoding in the variable length decoding unit 111, and converts the IDCT from the one-dimensional coefficient sequence. An address is generated for conversion to a frequency array of a two-dimensional coefficient sequence as an input sequence.

  The AD / DC prediction unit 113 performs AD / DC prediction on the output of the coefficient scanning unit 112 and outputs the result to the inverse quantization unit 114.

  The inverse quantization unit 114 dequantizes the quantized data obtained by the variable length decoding unit 101 for each macroblock (MB), for example, in units of blocks of 8 pixels × 8 lines, and obtains the obtained DCT (Discrete Cosine Transform) : Discrete cosine transform) output coefficient data to the IDCT unit 115.

  The IDCT unit 115 performs IDCT processing on the supplied DCT coefficient data from the inverse quantization unit 102, and outputs the obtained pixel data to the addition unit 130.

  The motion vector decoding unit 121 decodes the motion vector based on the data from the variable length decoding unit 111, and controls the operation of the motion compensation prediction unit 122 based on the result.

The motion compensated prediction unit 122 is controlled in its operation by the motion vector decoding unit 121 and does not supply any data to the addition unit 130 when the addition unit 130 is processing an I picture at that time.
When the adder 130 is processing a P picture at this time, the motion compensation prediction unit 122 accesses the frame buffer memory 140 to read out image data corresponding to a past frame, and performs predetermined arithmetic processing on the read image data. The calculation data obtained by performing the above is supplied to the adding unit 130.
In addition, when the adder 130 is currently processing a B picture, the motion compensation prediction unit 122 accesses the frame buffer memory 140 to read out image data corresponding to past and future frames. Calculation data obtained by performing predetermined calculation processing is supplied to the adder 130.

Incidentally, the frame buffer memory 140 is configured to hold image data corresponding to an I picture and a P picture among the decoded image data sequentially output from the adding unit 130.
The adder unit 130 is configured to output the pixel data from the IDCT unit 115 as decoded image data as it is when an image processed at that time is an I picture.
In addition, the addition unit 130 adds the pixel data from the IDCT unit 115 and the operation data from the motion compensation prediction unit 133 when the processing at that time is a P picture or a B picture, The decoded image data is obtained and output.

Also, the decoder (image processing apparatus) 100 according to the present embodiment performs inverse quantization (IQ) or AC on two-dimensional data including zero coefficients in the decoding process from the variable length decoding unit 111 to the IDCT unit 115. It has a function of performing processing such as IQ and AC / DC prediction at high speed by performing zero coefficient masking as preprocessing for processing such as / DC prediction.
The zero coefficient mask related to IQ processing masks IQ processing when all the calculation target elements of a computing unit that calculates a plurality of elements in parallel are zero when performing IQ calculation.
The IQ processing validity / invalidity determination has a function of speeding up by being implemented as a hardware (H / W) dedicated instruction applied simultaneously to a plurality of elements to be calculated.
In addition, by loading only the quantization table corresponding to the IQ calculation effective coefficient group according to the IQ mask determination, there is a function of eliminating redundant memory-register load and speeding up the IQ processing.
If the target coefficient group at the time of AC / DC prediction is all zero according to IQ mask determination, a predetermined pixel value is moved (mv) from an adjacent block as it is at the time of AC / DC prediction (zero coefficient and Are not performed).

  Further, in the decoding process of the present embodiment, the one-dimensional coefficient sequence that is the output of the variable-length decoding unit 111 is directly loaded into an internal memory that is a target of an arithmetic unit having an instruction set for calculating a plurality of elements in parallel, By implementing conversion to a two-dimensional coefficient sequence (coefficient scanning: ZigZagScan) with an inter-internal memory mv instruction, it has a function of speeding up.

  Hereinafter, the decoding process according to the present embodiment will be further described in detail.

  FIG. 3 is a functional block diagram of a processor (arithmetic unit) having a SIMD arithmetic instruction system, which is an example of an instruction set for calculating a plurality of elements in parallel according to the present embodiment.

3 includes a coefficient (Coeff) data buffer (processor internal memory) 201, a zigzag scanner conversion table 202, an MV instruction unit 203, a Q matrix (matrix) 204, an intermediate value data buffer 205, a sub-macro unit IQ mask determination and An IQ command unit 206, an IDCT command unit 207, and an intermediate value buffer 208 are included.
In FIG. 3, reference numeral 112a denotes an external coefficient (Coeff) data buffer.

FIG. 4 is a diagram schematically showing a decoding process according to the present embodiment.
FIG. 5 is a diagram illustrating a decoding process of an existing decoder as a comparison with FIG.
FIG. 6 is a diagram illustrating a decoding process when IQ processing is masked in the present embodiment.
FIG. 7 is a diagram showing a decoding process after IQ operation in the present embodiment.

  The coefficient scanning unit 112 uses the zigzag scan table and the RUN information for the non-zero DCT coefficient (= LEVEL) value that is variable-length decoded and output by the variable-length decoding unit 111 to input IDCT from a one-dimensional coefficient sequence. An address is generated for conversion to a frequency array of a two-dimensional coefficient sequence as a sequence.

In the present embodiment, unlike the typical MpegCodec decoder shown in FIG. 1, conversion from a one-dimensional to a two-dimensional coefficient sequence is performed immediately after variable-length decoding.
That is, as shown in FIG. 4, at the stage of loading the coefficients into the register of the SIMD arithmetic processor 200, by rearranging them in a two-dimensional array in a format that can be directly input to the IDCT, Reduce the sorting process.

  In the configuration of the typical MpegCodec decoder shown in FIG. 5, AC / DC prediction / IQ processing is performed on a one-dimensional array in which only non-zero coefficients are stored, and zero coefficients are included in the calculation target array. By performing the conversion to the dimensional array immediately before the IDCT processing, redundant operations for the zero coefficient are omitted.

On the other hand, in the configuration of the decoder 100 according to the present embodiment, as shown in FIG. 4, coefficient scanning and IQ processing are performed in a state where a plurality of coefficient elements are mapped to a processor 200 capable of performing SIMD calculation in parallel. To do.
For this reason, DCT coefficients are converted from one dimension to two dimensions immediately after variable length decoding.
When this method is used, conversion to a two-dimensional array in which zero coefficients are included in the array is performed before IQ or AC / DC prediction, so that there is a possibility that redundant calculations on the zero coefficients may occur.
Therefore, by performing a zero coefficient mask by a hardware (H / W) instruction as a pre-process of each process, a redundant operation mask is performed at a higher speed than in the existing method.

As described above, the conversion address generation is performed by software processing, but the actual coefficient rearrangement processing is executed by using the mv instruction between internal memories in the arithmetic unit capable of performing multi-element parallel calculation.
After the conversion address is generated, as shown in FIG. 4, the non-zero DCT coefficient sequence stored as a one-dimensional array is allocated to the internal memory 201 in the processor 200 which is an arithmetic unit.
Prior to this processing, the internal memory in the computing unit has been cleared to zero. Therefore, the zero coefficient load can be omitted.
When a non-zero DCT coefficient sequence is loaded into the internal memory 201 in the processor 200 which is an arithmetic unit, the coefficient sequence is linearly transferred with a predetermined stride width because it is a one-dimensional array. In the example of FIG. 4, the stride width is set to 4.
Also, the conversion address array generated by the coefficient scanning unit 112 is assigned to an internal memory in the processor 200 which is also an arithmetic unit, and is linearly transferred with a predetermined stride width. This operation is repeated for one block (8 × 8).

Next, the corresponding non-zero coefficient is moved (coefficient scanning) according to the conversion address loaded in the internal memory in the processor 200 which is an arithmetic unit.
At the time of this movement, as shown in FIG. 4, it is implemented in parallel with a plurality of elements using an mv instruction between internal memories of the processor 200 which is an arithmetic unit capable of calculating a plurality of elements in parallel.
However, since the coefficient scanning table is a coordinate conversion table set for 8 × 8 elements, the mv instruction of an arithmetic unit capable of performing multi-element parallel calculation in this configuration has a destination address outside the range of the calculation target area (however, Cases within the 8x8 range are also supported.

  After rearrangement (coefficient scanning) in the parallel calculation instruction of a plurality of elements, AC / DC prediction and IQ operation are performed by applying the parallel calculation instruction 206 to the plurality of elements.

  Next, when the target coefficient group is all zero according to the IQ mask determination at the time of AC / DC prediction, a predetermined pixel value is moved (mv) from the adjacent block as it is at the time of AC / DC prediction. In this case, calculation such as addition with the zero coefficient is not performed.

When performing the IQ calculation, as shown in FIG. 6, if all the multiple coefficient elements that are the calculation target of the parallel calculation for the multiple elements are all zero, the IQ process is masked.
This valid / invalid determination is implemented as the H / W dedicated instruction 206.
At this time, only when the parallel operation element size for the plurality of elements of the processor 200 is 8 × 8, the IQ processing mask determination instruction using the coded block pattern and the H / W dedicated instruction has equivalent information.
However, in general, there are many cases where there is a gain in processing speed when using an IQ processing mask determination instruction based on an H / W instruction.

At the time of IQ calculation, as shown in FIG. 6, a predetermined matrix table 204 is loaded into the register 210 in advance, and as soon as the AC / DC prediction is completed, the quantization coefficient and DCT coefficient of the loaded Q matrix table. Is performed in parallel by a plurality of elements by SIMD arithmetic processing.
In addition, when loading the Q matrix table 204, only the Q matrix table 204 corresponding to a plurality of elements not subjected to IQ operation masking is loaded, thereby eliminating unnecessary Q matrix table loading.

After IQ calculation, as shown in FIG. 7, since the coefficients on the internal memory of the processor 200, which is an arithmetic unit, are arranged as a two-dimensional frequency array, the 8 × 8 IDCT calculation is performed as it is without converting the arrangement address, and the pixel data is converted. Generate.
In the IDCT operation, Coded Block Pattern is used as the mask for normal operation. However, if the processing speed is faster when the IQ processing mask determination instruction used in IQ operation is used, the invalid determination is made without using Coded Block Pattern. An alternative is to apply the instructions to multiple elements simultaneously.

  After the pixel data generated by the IDCT is stored in the frame buffer memory 140, the internal memory of the processor 200, which is a used arithmetic unit, is cleared to zero by a dedicated H / W instruction.

The first embodiment has been described above.
Next, a processing example when almost no non-zero coefficient is included in one block will be described as a second embodiment.

Second Embodiment
The present method (first method; first processing) described in the first embodiment is particularly advantageous at a high bit rate. However, it is an existing method in the case where almost no non-zero coefficient is included in one block. There is a case where the method (second method; second processing) is more advantageous.
Therefore, in the second embodiment, by using a predetermined threshold during the decoding process, the first method (first process) and the second method (second process) are dynamically switched. It is possible to make a configuration that always selects the optimum data arrangement.

FIG. 8 is a diagram for explaining the configuration of the second embodiment.
Hereinafter, description will be given in association with FIG.
The second embodiment has a configuration based on the first embodiment described above. Here, only the difference from the first embodiment will be described.

When VLD processing of DCT coefficients in the variable length decoding unit 111 outputs a number of coefficients exceeding the threshold Vth of 8 × 8 units (ST1, ST2), the second method to the first method (first embodiment) Switching to the SIMD calculation described in (1).
Until the VLD output DCT coefficient reaches the threshold value Vth, the value obtained from the VLD is held in an internal register (global register) of the processor 200, which is a predetermined arithmetic unit, without being processed (ST3).

  By deferring the processing after variable length decoding until the number of coefficients exceeds a certain threshold value Vth, it is possible to adaptively switch processing while minimizing unnecessary memory access (ST2 to ST5). .

  The threshold value Vth is determined in consideration of the number of global registers of the CPU, the number of elements of SIMD operation, and the balance of delays in SIMD operation.

  As described above, according to the present embodiment, in the configuration of the decoder 100 according to the present embodiment, coefficient scanning and IQ processing are performed in a state where a plurality of coefficient elements are mapped to the processor 200 that can perform SIMD calculation in parallel. In order to implement, a one-dimensional to two-dimensional transformation of DCT coefficients is performed immediately after variable length decoding. When this method is used, conversion to a two-dimensional array in which the zero coefficient is included in the array is performed before IQ or AC / DC prediction, so that there is a possibility that redundant calculation for the zero coefficient may occur. From the above, by performing a zero coefficient mask as a pre-processing of each processing by a hardware (H / W) instruction, a redundant operation mask is performed at a higher speed than the existing method, and the following effects can be obtained. it can.

In a system having a computing element capable of performing multi-element parallel computation, it is possible to efficiently mask IQ processing by using non-zero coefficient bias when an IDCT coefficient is arranged in a frequency component, so that the processing speed is high. It becomes.
In other words, since non-zero coefficients are usually concentrated in the low-frequency component region during IDCT, there is some non-zero data bias even after rearrangement into two-dimensional data. Rather than allocating only non-zero coefficients together as in (), parallel processing is performed at once by rearranging the data into two-dimensional data in the previous process and applying arithmetic processing to multiple elements in parallel. The merit for is greater.

The time and effort required to draw the quantization coefficient for the coefficient in the table is reduced, and the processing speed is increased accordingly.
In other words, since the existing method sequentially performs IQ processing on a one-dimensional array of non-zero coefficients, the two-dimensional arrangement address of each non-zero coefficient is calculated and the corresponding quantization coefficient value is referred to from the quantization table. In this embodiment, the IQ processing is performed in a state where the IDCT coefficients are two-dimensionally arranged. Therefore, it is not necessary to calculate the quantization table reference address for each coefficient.
Therefore, it is possible to perform a vector operation directly on the two-dimensionally arranged IDCT coefficients and the quantization table.

  Further, the processing speed can be increased by applying the arithmetic processing in parallel to the coefficient scanning.

In addition, since arithmetic processing is performed in parallel in a plurality of elements, decoding processing can be performed with a substantially constant processing amount regardless of the non-zero coefficient ratio (= bit rate) of each DCT.
As an example of the merit, the load balance of each processor is stabilized when each process is assigned to a plurality of processors and the entire decoding system is pipeline controlled. As a result, the performance of the entire decoder is stabilized and the processing speed is increased.

Further, the method described above in detail can be formed as a program corresponding to the above-described procedure and executed by a computer such as a CPU.
Such a program can be configured to be accessed by a recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a floppy (registered trademark) disk, or the like, and to execute the program by a computer in which the recording medium is set.

1 is a functional block diagram of a typical MPEG codec (Codec) decoder. FIG. It is a functional block diagram which shows the structure of the MPEGCodec decoder as an image processing apparatus which concerns on embodiment of this invention. It is a functional block diagram of a processor (arithmetic unit) having a SIMD arithmetic instruction system which is an example of an instruction set for calculating a plurality of elements in parallel according to the present embodiment. It is a figure which shows typically the decoding process which concerns on this embodiment. FIG. 5 is a diagram illustrating a decoding process of an existing decoder as a comparison with FIG. 4. In this embodiment, it is a figure which shows the decoding process in case IQ processing is masked. In this embodiment, it is a figure which shows the decoding process after IQ calculation. It is a figure for demonstrating 2nd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus (decoder), 110 ... IDCT processing part, 111 ... Variable length decoding part, 112 ... Coefficient scanning part, 113 ... AC / DC prediction part, 114 ... Inverse quantization unit, 115 ... IDCT unit, 120 ... prediction unit with motion compensation, 121 ... motion vector decoding unit, 122 ... motion compensation prediction unit, 130 ... addition unit, 140 ... Frame memory.

Claims (14)

An image processing apparatus that blocks an input image signal, inversely quantizes image information quantized in units of the block, performs inverse transform processing, and decodes the image information.
A decoding unit that decodes the quantized and encoded transform coefficients and outputs a one-dimensional coefficient sequence;
A coefficient scanning unit that performs coefficient scanning from a one-dimensional coefficient sequence to a two-dimensional coefficient sequence, and transform coefficients output from the decoding unit;
An inverse quantization unit that inversely quantizes the transform coefficient obtained by the coefficient scanning unit;
A conversion processing unit capable of converting from a one-dimensional coefficient sequence to a two-dimensional coefficient sequence on the dequantized coefficient data;
An image processing apparatus. The coefficient scanning unit is
2. A function of rearranging the coefficients in an internal memory of an arithmetic unit having an instruction set for calculating a plurality of elements in parallel in a two-dimensional array that can be directly input to the conversion processing unit. The image processing apparatus described. The coefficient scanning unit is
Using a zigzag scan table and RUN information for a non-zero DCT coefficient value output after variable length decoding by the decoding unit, a two-dimensional coefficient sequence serving as an input sequence of the conversion processing unit from a one-dimensional coefficient sequence The image processing apparatus according to claim 2, wherein address generation for conversion to a frequency array is performed. 4. The image according to claim 3, wherein, in a series of decoding processes from the decoding unit to the conversion processing unit, zero coefficient masking is performed as a preprocessing of at least inverse quantization (IQ) processing on two-dimensional data including zero coefficients. Processing equipment. The image processing apparatus according to claim 4, wherein the zero coefficient mask for the IQ processing masks the IQ processing when all the calculation target elements of the arithmetic unit for calculating a plurality of elements in parallel are zero when performing IQ calculation. . The computing unit is
The image processing apparatus according to claim 5, wherein the IQ processing validity / invalidity determination is implemented as a hardware (H / W) dedicated instruction applied to a plurality of elements to be calculated. The computing unit is
The image processing apparatus according to claim 6, wherein only the quantization table corresponding to the coefficient group effective for IQ calculation is loaded according to the validity / invalidity determination of the IQ process. An AC / DC prediction unit that performs AC / DC prediction at a stage after the coefficient scanning unit and before the conversion processing unit;
The computing unit is
The predetermined pixel value is moved from an adjacent block as it is at the time of AC / DC prediction when all the target coefficient groups at the time of AC / DC prediction are zero according to the validity / invalidity determination of the IQ processing. Image processing device. The computing unit is
The one-dimensional coefficient sequence that is an output of the decoding unit is directly loaded into the target internal memory, and the coefficient scan to the two-dimensional coefficient sequence is executed by an inter-memory move (mv) instruction. Image processing device. The computing unit is
The function capable of selectively switching between a first process in which the coefficient scanning process of the coefficient scanning unit is performed at the output stage of the decoding unit and a second process performed in the input stage of the conversion processing unit. Image processing apparatus. The computing unit is
When the coefficient output from the decoding unit that outputs the one-dimensional coefficient sequence outputs a number of coefficients exceeding a preset threshold, the coefficient scanning process is switched from the second process to the first process. Item 15. The image processing apparatus according to Item 10. The computing unit is
The image processing apparatus according to claim 11, wherein the value obtained from the decoding unit is stored in the internal memory of the computing unit without being processed until the process to be adopted is determined to be the first process or the second process. An image processing method that blocks an input image signal, dequantizes image information quantized in units of the block, performs inverse transform processing, and decodes the image information.
Decoding a quantized and encoded transform coefficient to obtain a one-dimensional coefficient sequence;
A coefficient scanning step for performing a coefficient scanning from the one-dimensional coefficient string to the two-dimensional coefficient string for the transform coefficient obtained in the decoding step;
An inverse quantization step for inversely quantizing the transform coefficient obtained in the coefficient scanning step;
A conversion processing step for performing conversion processing from the one-dimensional coefficient sequence to the two-dimensional coefficient sequence on the dequantized coefficient data;
An image processing method. An image process that blocks an input image signal, dequantizes image information quantized in units of the block, performs an inverse transform process, and decodes the image information.
A decoding process for decoding a quantized and encoded transform coefficient to obtain a one-dimensional coefficient sequence;
A coefficient scanning process for performing the coefficient scanning from the one-dimensional coefficient string to the two-dimensional coefficient string for the transform coefficient obtained in the decoding step;
An inverse quantization process for inversely quantizing the transform coefficient obtained in the coefficient scanning step;
A program that causes a computer to execute image processing including conversion processing that performs conversion processing from a one-dimensional coefficient sequence to a two-dimensional coefficient sequence on inversely quantized coefficient data.

Images