JP2005218055A - Apparatus, method and program for processing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively utilize parallel processing performance of a processor and to prevent an adverse effect on processing of adjacent pixels, when a parallel processing instruction processes an image including fewer pixels than concurrently processable pixels. <P>SOLUTION: An SIMD parallel processor includes a conversion unit 105 for arranging a plurality of image data so that image data of respective components alternate for each N/M (where M is an integer) pixels, when the parallel processing instruction of the processor SIMD is to concurrently process the plurality of pixel data including the pixels less than the concurrently processable pixels N; a read control unit 106 for reading the plurality of pixel data which have been arranged and converted by 2N pixels with N pixels as a unit; and a write control unit 107 for extracting the pixel data of the N/M pixels to be processed from the pixel data of the 2N pixels read by executing a byte shuffle instruction of the SIMD processor and compensating the shift of the pixel data of the extracted N/M pixels. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program for performing image processing on image data by a SIMD (Single Instruction Multiple Data) type parallel processing processor that performs read / write processing on a memory at a predetermined address position. It is.

  In recent years, image encoding and decoding of moving image data such as MPEG-1, MPEG-2, and MPEG-4 are generally performed by software. Here, the motion compensation process divides a decoded image into rectangular blocks called macroblocks, and from the reference image decoded in the past for each macroblock, a motion vector (how much each macroblock in the video is in which direction and how much This is a process of copying a macroblock at a position shifted by (information on whether it is moving) (see, for example, Patent Document 1).

  In a general MPEG decoding process, image data is stored in a frame sequential format in which a luminance component (Y) and a color difference component (Cb / Cr) are arranged as a group for each frame or field. The macroblock size is 16 × 16 pixels for the luminance component and 8 × 8 pixels for the color difference component.

  Here, an instruction set called SIMD (Single Instruction Multiple Data) is prepared for an Intel processor. The SIMD instruction included in this instruction set is an instruction that performs processing common to a plurality of data with one instruction. When similar processing is performed on a plurality of pixels as in MPEG, simultaneous processing is performed using this SIMD instruction. By doing so, it is possible to speed up the processing.

  The Intel processor includes an SIMD instruction using a 64-bit register called MMX / SSE (Streaming SIMD Extension) and a SIMD instruction using a 128-bit register called SSE2. Since 1-pixel data is expressed by 8 bits, SSE2 is used for the luminance component, and 16 pixels in the horizontal direction of the macroblock can be processed simultaneously. For color difference components, MMX / SSE can be used to simultaneously process 8 pixels of the block.

  On the other hand, there is a possibility that a RISC processor having only a 128-bit SIMD instruction may be mounted as a CPU mounted on a next-generation game machine, a home server, a digital television, or the like. For example, the AltiVec instruction set in the Power PC processor is representative of such a processor. The SIMD instruction that can be used in such a RISC processor targets individual data obtained by dividing 128 bits, that is, 16 bytes into a plurality of words (8 bits, 16 bits, 32 bits). In the future, the amount of data that can be simultaneously processed by the processor tends to increase, and there is a possibility that a RISC processor equipped with a SIMD instruction capable of simultaneously processing 256 bits (32 bytes) may appear.

JP 11-215509 A

  However, when the motion compensation process in the MPEG decoding process is executed by the RISC processor having such a 128-bit SIMD instruction set, the SIMD instruction can execute only a process of 128 bits. For this reason, when such a RISC processor is used, the motion compensation processing of the luminance component whose macroblock is 16 × 16 pixels can be performed using 128 bits effectively, but the macroblock has 8 × 8 pixels. In the case of the motion compensation processing of the color difference component, only 64 bits out of 128 bits are used, and the remaining 64 bits are not used effectively. That is, the area of the chrominance component (Cb / Cr) macroblock for performing motion compensation is ½ of the luminance component macroblock, but when the 128-bit SIMD instruction set is used, the chrominance component is SIMD. There is a problem that the parallelism of the instructions cannot be fully utilized, and as a result, the processing load on the same level as the motion compensation processing of the luminance component is applied, and the efficiency of the parallel processing by the RISC processor cannot be improved.

  Further, in a general RISC processor, memory addressing can be performed only from addresses aligned in the processing unit of the processor, and is different from an Intel processor that can perform addressing freely. Therefore, when data is read from the memory, the read address is 16 × n (n is an integer).

  Therefore, when MPEG decoding is performed by such a RISC processor, the number of pixels in the horizontal direction of the color difference component is a multiple of 16 at the MPEG-2 SDTV resolution (generally horizontal 720 × vertical 480 pixels). Therefore, unlike the case where motion compensation is performed using a processor that can acquire data from a free address such as an Intel CPU, the next line is used when processing the 8 pixels at the right end of the 16-byte alignment. There is a problem of adversely affecting the leftmost pixel.

  The present invention has been made in view of the above points, and effectively uses the parallel processing performance of a processor when performing image processing with a number of pixels smaller than the number of pixels that can be processed simultaneously by a parallel processing instruction. In addition, an object is to provide an image processing apparatus, an image processing method, and an image processing program that can prevent adverse effects on processing of adjacent pixels.

  In order to solve the above-described problems and achieve the object, the present invention processes SIMD (Single Instruction Multiple Data) parallel processing that processes a plurality of data with one instruction and performs read / write processing on a memory at a predetermined address position. An image processing method for performing image processing on image data by a processing processor, wherein a plurality of pixels that are different color components of the image data and are smaller than the number N of pixels that can be simultaneously processed by the parallel processing command of the SIMD type processor In the case of processing data simultaneously, the plurality of pixel data are arranged and converted by the conversion step for arranging and converting the pixel data of each component alternately for each N / M (M: integer) pixel. Read multiple pixel data for 2N pixels in units of N pixels A read step and a byte shuffle instruction provided by the SIMD type processor to extract pixel data of N / M pixels to be image-processed from pixel data of 2N pixels read by the read step; And an image processing step for performing image processing on the extracted pixel data of N / M pixels.

  The present invention also stores a SIMD (Single Instruction Multiple Data) type parallel processing processor that processes a plurality of data with a single instruction and performs read / write processing on an image memory at a predetermined address position, and stores the program. A memory accessible by the type parallel processor, and an image memory storing the image data and accessible by the SIMD type parallel processor, wherein the program includes the SIMD type parallel processor. When simultaneously processing a plurality of pixel data having different color components and smaller than the number N of pixels that can be processed simultaneously by the parallel processing instruction of the SIMD type processor, the plurality of pixel data is converted into N / M (M: integer). The pixel data of each component alternates for each pixel A conversion unit that performs layout conversion in such a manner, a reading unit that reads a plurality of pixel data that has been converted by the conversion unit for 2N pixels in units of N pixels, and a byte shuffle instruction provided by the SIMD processor Then, the pixel data of N / M pixels to be image-processed is extracted from the pixel data of 2N pixels read out by the reading step, and the image processing is performed on the extracted pixel data of N / M pixels. It functions as a processing means.

  The present invention also provides image processing for performing image processing on image data by a SIMD (Single Instruction Multiple Data) type parallel processing processor that processes a plurality of data with instructions and performs read / write processing on a memory at a predetermined address position. The plurality of pixel data when the program simultaneously processes a plurality of pixel data having different color components of the image data and having a parallel processing instruction of the SIMD processor smaller than the number N of pixels that can be processed simultaneously. For each N / M (M: integer) pixel, the conversion procedure for converting the pixel data of each component alternately, and a plurality of pixel data subjected to the conversion by the conversion step are converted into 2N in units of N pixels. A readout procedure for reading out pixels, and the SIMD type The N / M pixel extracted from the pixel data of the 2N pixels read out by the read step by executing the byte shuffle instruction provided by the image processor is extracted. And an image processing program for causing a computer to execute an image processing procedure for performing image processing on the pixel data.

  According to the present invention, when a plurality of pieces of pixel data that are different color components of image data and are less than the number N of pixels that can be processed simultaneously by a parallel processing instruction of the SIMD processor, / M (M: integer) The pixel is converted so that the pixel data of each component is alternated, and a plurality of pixel data subjected to the conversion are read for 2N pixels in units of N pixels, and a byte shuffle instruction is executed. Then, pixel data of N / M pixels to be image-processed is extracted from the read pixel data of 2N pixels, and the image processing is performed on the extracted pixel data of N / M pixels. Image processing of component image data can be performed simultaneously without generating a useless area, and motion compensation processing can be performed effectively using parallel processing of SIMD type processors. There is an effect that that.

  In the image decoding apparatus according to the present embodiment, a plurality of pieces of pixel data are arranged and converted so that the pixel data of each component is alternated for each N / M (M: integer) pixel. Thus, even when a processor that cannot access the memory is used, it is possible to perform image processing without adversely affecting the pixel data in other regions when processing the pixel data on the right end.

  Exemplary embodiments of an image processing apparatus, an image processing method, and an image processing program according to the present invention are explained in detail below with reference to the accompanying drawings. In the present embodiment, the image processing apparatus, the image processing method, and the image processing program of the present invention are applied to an image decoding method, an image decoding apparatus, and an image decoding program that perform MPEG decoding.

  FIG. 1 is a block diagram showing a functional configuration of an image decoding apparatus according to an embodiment of the present invention. As shown in FIG. 1, the image decoding apparatus mainly includes a stream buffer 120, a stream buffer control unit 130, a decoder control unit 110, a decoder 100, and an image memory 140.

  As shown in FIG. 1, the decoder 100 includes a variable length decoding unit 101, an inverse quantization unit 102, an inverse scan unit 103, an inverse DCT unit 104, and a motion compensation unit 105.

  The stream buffer 120 is a DRAM (Dynamic Random Access Memory) that stores an encoded stream (MPEG2 bit stream) input from the outside of the apparatus via a recording medium such as a DVD, a network such as the Internet, or a broadcast wave such as a satellite. Memory.

  The stream buffer control unit 130 is a processing unit that equalizes the variation in the input time of the encoded stream based on an instruction from the decoder control unit 110 and inputs the encoded stream to the decoder 100.

  The decoder control unit 110 is a processing unit that outputs an encoded stream from the stream buffer 120 to the decoder 100 via the stream buffer control unit 130.

  The decoder 100 is generally a processing unit that decodes the encoded stream input from the stream buffer control unit 130 based on an instruction from the decoder control unit 110 and outputs the decoded stream to the image memory 140. This decoding process is performed through processing by the processing units of the variable length decoding unit 101, the inverse quantization unit 102, the inverse scan unit 103, the inverse DCT unit 104, and the motion compensation unit 108. Hereinafter, processing of each processing unit of the decoder 100 will be described.

  The variable length decoding unit 101 is a processing unit that decodes variable length encoded data included in an encoded stream to restore quantized DCT coefficients. Specifically, the coded stream input from the stream buffer control unit 130 is separated into macroblocks according to instructions from the decoder control unit 110, the quantized DCT coefficients of each macroblock are decoded, and the decoded quantized DCT coefficients are decoded. Is output to the inverse quantization unit 102.

  The variable length decoding unit 101 also decodes parameters such as the prediction mode and the prediction vector, and outputs the decoded prediction mode and prediction vector to the motion compensation unit 105.

  The inverse quantization unit 102 is a processing unit that inversely quantizes the quantized DCT coefficient input from the variable length decoding unit 101 to decode the DCT coefficient, and outputs the decoded DCT coefficient to the inverse scan unit 103.

  The inverse scan unit 103 is a processing unit that inversely scans the DCT coefficient input from the inverse quantization unit 102 and outputs the inversely scanned DCT coefficient to the inverse DCT unit 104. Note that FIG. 1 shows a configuration in which the inverse scan process is performed after the inverse quantization process is performed, but it is not always necessary to follow this order, and the inverse quantization process is performed after the inverse scan process is performed. The structure to perform may be sufficient.

  The inverse DCT unit 104 performs inverse DCT transform on the DCT coefficient input from the inverse scan unit 103 to decode image data (data having pixel values) that is a real image component before encoding, and moves the decoded image data. It is a processing unit that outputs to the compensation unit 108.

  The motion compensation unit 108 performs motion compensation based on the image data input from the inverse DCT unit 104 and the prediction mode and prediction vector input from the variable length decoding unit 101, and the image data subjected to motion compensation. Is a processing unit for writing the image data into the image memory 140, and is configured from a conversion unit 105, a writing control unit 106, and a reading control unit 107 in terms of functional concept. Here, the conversion unit 105 constitutes the conversion means in the present invention, the write control unit 106 constitutes the image processing means in the present invention, and the read control unit 107 constitutes the read means in the present invention.

  The conversion unit 105 in the motion compensation unit 108 arranges the pixel data of the color difference components Cb and Cr in the frame memory (Cb / Cr) 144 alternately for every 8 pixels of each component.

  The read control unit 107 in the motion compensation unit 108 reads a plurality of image data from the image memory. Specifically, the readout control unit 107 reads out the pixel data of the luminance component Y for 16 pixels from the frame memory (Y) 141 of the image memory 140, and the color difference components Cb and Cr are read from the frame memory (Cb / Cr) 144. A process of reading out 16 pixels of pixel data arranged alternately every 8 pixels is performed.

  The write control unit 106 in the motion compensation unit 108 executes a byte shuffle command from the pixel data for 16 pixels of the luminance component read from the read control unit 107, and extracts pixels necessary for the motion compensation process. Further, the writing control unit 106 executes a byte shuffle command from the pixel data for 16 pixels of the color difference component Cb / Cr read from the reading control unit 107, and extracts pixels necessary for the motion compensation process.

  The image memory 140 stores image data to be subjected to motion compensation processing. In the image memory 140, a frame memory (Y) 141, which is an area for storing pixel data of the luminance component Y, an area for storing image data in which the pixel data of the color difference component Cb and the pixel data of the color difference component Cr are alternately arranged Frame memory (Cb / Cr) 144 is secured. The frame memory (y) 141 and the frame memory (Cb / Cr) 144 are each one in the figure, but actually, in order to perform motion compensation processing, the frame memory of the reference image and the decoded image A frame memory (y) 141 and a frame memory (Cb / Cr) 144 are secured as frame memories.

  The image decoding apparatus according to the present embodiment includes a RISC SIMD (Single Instruction Multiple Data) parallel processing processor that performs read / write processing on a memory with 16-byte alignment, a recording device such as a hard disk device (HDD), an image memory 140, This is a configuration using a moving image playback device or computer having a storage device such as a RAM (Random Access Memory) and a ROM (Read Only Memory).

  The image decoding program executed by the image decoding apparatus according to the present embodiment is provided in a form incorporated in a ROM when the image decoding apparatus according to the present embodiment is a moving image reproduction apparatus. When the image decoding apparatus according to the present embodiment is configured using a computer, the image decoding program is a file in an installable format or an executable format, such as a CD-ROM, a flexible disk (FD), The program is recorded on a computer-readable recording medium such as a CD-R or a DVD (Digital Versatile Disk).

  The image decoding program executed by the image decoding apparatus according to the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. . The image decoding program executed by the image decoding apparatus according to the present embodiment may be provided or distributed via a network such as the Internet.

  The image decoding program executed by the image decoding apparatus according to the present embodiment is loaded on the main storage device by being read from the ROM or the storage medium and executed, and the motion compensation unit 108 and the variable length decoding unit described above are loaded. 101, an inverse quantization unit 102, an inverse scan unit 103, and an inverse DCT unit are generated on the main storage device. Further, a part or all of the motion compensation unit 108, the variable length decoding unit 101, the inverse quantization unit 102, the inverse scan unit 103, and the inverse DCT unit 104 may be configured by hardware.

  Next, various types of the image decoding apparatus according to the present embodiment configured as described above will be described. The image decoding apparatus according to the present embodiment includes a variable length decoding process by the variable length decoding unit 101 of the decoder 100, an inverse quantization process by the inverse quantization unit 102, an inverse scan process by the inverse scan unit 103, and an inverse DCT. In response to the inverse DCT conversion processing by the unit 104, the motion compensation processing by the motion compensation unit 108 is executed.

  In the image decoding apparatus according to the present embodiment, motion compensation processing is executed by the motion compensation unit 108. In the present embodiment, the conversion unit 105 sets the format of the image data in the frame memory so that the color difference components Cb / Cr are alternately arranged every 8 pixels (8 bytes). FIG. 2 is an explanatory diagram showing a normal frame sequential format. As shown in FIG. 3, in the frame sequential format, pixels of luminance component (y) and color difference component (Cb / Cr) are arranged as a group for each frame or field. On the other hand, FIG. 3 is an explanatory diagram showing a format of image data used in the image decoding apparatus according to the present embodiment. As shown in FIG. 3, the format of the luminance component of the frame memory (Y) 142 is the same as the frame sequential shown in FIG. 2, but the frame memory (Cb / Cr) 144 contains the pixels of the color difference components Cb and Cr. Alternatingly arranged every 8 bytes. Usually, the decoding process of the color difference component in MPEG is performed for each macroblock of 8 × 8 pixels. For this reason, there is no processing overhead due to storing the color difference component pixel data in the format shown in FIG.

  In the image decoding apparatus according to the present embodiment, first, motion compensation processing is performed on the luminance signal (Y), and then motion compensation processing is performed on the color difference signal (Cb / Cr). The motion compensation process for the luminance signal (Y) will be described later. First, the motion compensation process for the color difference signal (Cb / Cr) will be described. FIG. 4 is a flowchart showing the procedure of the motion compensation process for the color difference signal (Cb / Cr).

  First, the read control unit 107 calculates the address radr at the upper left corner of the reference block of the frame memory (Cb / Cr) 144 of the reference image as shown in Equation 1 (step S401).

  Here, (x, y) represents the coordinates of the upper left corner of the decoded block, (mvx, mvy) represents the motion vector, and width represents the width of the luminance component image (720 pixels in SDTV). In addition, mvx ′ is obtained from the horizontal direction component mvx of the motion vector by Equation (2).

  Further, offset (ref, c) indicates the address of the upper left corner of the frame memory of the color difference component (Color) of the reference image. Here, offset (ref, c) is always a 16-byte aligned address. Since the color difference component Cb / Cr is processed in units of 8 pixels, x is always a multiple of 16. Normally, the motion vector has an accuracy of 0.5 pixel unit, but here it is assumed to be 1 pixel accuracy for simplicity. For this reason, the address radr calculated by the above equation is generally not 16-byte aligned.

  Next, the read control unit 105 calculates the address waddr at the upper left end of the decoded block of the decoded image frame memory (Cb / Cr) 144 as shown in Equation 3 (step S402).

  Here, offset (cur, c) indicates the address of the upper left corner of the frame memory (Cb / Cr) 144 of the color difference component of the decoded image (CURrent). The calculated waddr is always aligned to 16 bytes.

  Next, the read control unit 105 reads 16 bytes (128 bits) of image data from an address ((radr / 16) * 16) in which the lower 4 bits of the upper left end address radr of the reference block are 0, and stores them in the register R0. Load (step S403).

  Here, since the address radr calculated in step S401 is not the address at the position aligned by 16 bytes, the address in which the lower 4 bits of the address radr are set to 0 is loaded into the register R0 of the image data. The address of the 16 byte aligned position immediately before the address radr is obtained.

  Next, the read control unit 105 transfers 16 bytes (128 bits) from an adjacent address ((radr / 16) * 16 + 16) 16 bytes apart from an address in which the lower 4 bits of the upper left end address radr of the reference block are 0. Image data is read and loaded into the register R1 (step S404).

  Then, the write control unit 106 reads the (radr & 0x07) th byte shuffle control word from the control table and loads it into the register R2 (step S405). Here, the byte shuffle control word defines pixels to be extracted from the pixels loaded in R0 and R1 for motion compensation processing in the byte shuffle instruction provided by the SIMD type instruction set executed in the next step. is there. This byte shuffle control word includes 32 pixels of pixel data (8 pixels) loaded into the register R0 and pixel data (8 pixels) loaded into the register R1 for each address difference between the address radr of the reference block and the 16-byte alignment position. The positions in the state where the 16-pixel registers R0 and R1 extracted from are connected to the control table are registered in advance.

  Here, the address difference between the address radr of the reference block and the position of the 16-byte alignment is calculated by radr & 0x07. In this embodiment, the control table may be held in the image decoding program in addition to being held in the memory.

  FIG. 5 is an explanatory diagram showing the contents of the control table. The value of radr & 0x07 is an address obtained by setting the lower 4 bits of radr to 0, that is, the address difference between the address radr of the reference block and the position of 16 byte alignment.

  The 16 numerical values of the control word indicate the number of bytes from the head position when the registers R0 and R1 are concatenated.

  Next, the write control unit 106 executes a byte shuffle instruction (for example, bshuffle instruction), and extracts 16 pixels necessary for motion compensation from the pixels stored in the registers R0 and R1 according to the control word of the register R2. The execution result, that is, the extracted 16 pixels are stored in the register R3 (step S406).

  Here, the byte shuffle instruction will be described. The byte shuffle instruction is an instruction for extracting an arbitrary byte of two registers for each byte from two registers in a RISC processor having a SIMD instruction. This byte shuffle instruction is an instruction provided to read data from or write data to an arbitrary address in the memory when the memory can be accessed only at an address at a 16-byte alignment position. As such a byte shuffle instruction, for example, a bshuffle instruction provided in the VIS instruction set that is a SIMD instruction implemented in the Ultra SPARC-III of Sun Microsystems (R), or a vperm instruction in the AltVec instruction set in the PowerPC architecture. The same command applies to.

  FIG. 6 is an explanatory diagram for explaining the operation of the byte shuffle instruction. For example, as shown in FIG. 6, a value of 1 byte of each of A0 to AF is stored in the register RA (16 bytes), and a value of 1 byte of each of B0 to BF is stored in the register RB (16 bytes). Think about the case. Here, considering a state in which the registers RA and RB are connected, numbers from 00 to 1f are assigned for each byte from the top. Data specifying such a number in the control word is output as the execution result of the byte shuffle instruction.

  In the example of FIG. 6, since 0x0f000030e0018061a051a051a061302080c is set in the control word, when a byte shuffle instruction is executed, “0f”, “00”, “03”, “0e”, “00”, “00” of the registers RA and RB are executed. The data at the positions of “18”, “06”, “1a”, “05”, “1a”, “1b”, “06”, “12”, “03”, “08”, “0c” are shown in FIG. This is extracted after the execution of the byte shuffle instruction.

  Here, the above process will be described with a specific example. FIG. 7 shows the values of the registers R0 and R1 after executing steps 403 and S404 when the lower 4 bits of radr are 4, that is, when radr is shifted by 4 bytes from the 16-byte alignment address, and step S406. It is explanatory drawing which shows an example of the value of the register | resistor R3 after this byte shuffle instruction execution.

  When the lower 4 bits of radr are 4, the control word becomes the sequence “04 05 06 07 10 11 12 13 0c 0d 0e 0f 18 19 1a 1b” where radr & 0x07 is 4 from FIG. Therefore, as shown in FIG. 7, when a byte shuffle instruction is executed, the pixels R0 to 04, 05, 06, and 07 and the pixels R1 to 10, 11, 12, and 13 are stored as Rb pixels in R3. Extracted. Further, as pixels of Cr, pixels of registers R0 to 0c, 0d, 0e, and 0f and pixels of registers R1 to 18, 19, 1a, and 1b are extracted.

  FIG. 8 shows the values of the registers R0 and R1 after executing step 503 and S504 when the lower 4 bits of radr are 6, that is, when radr is shifted by 6 bytes from the 16-byte alignment address, and It is explanatory drawing which shows an example of the value of the register | resistor R3 after the byte shuffle instruction execution of step S506.

  When the lower 4 bits of radr are 6, the control word becomes the column “06 07 10 11 12 13 1415 0e 0f 18 19 1a 1b 1c 1d” with radr & 0x07 as shown in FIG. Therefore, as shown in FIG. 8, when a byte shuffle instruction is executed, the pixels R0 to 06 and 07 and the pixels R11 to 10, 11, 12, 13, 14, and 15 are registered as Cb pixels in R3. Extracted. Further, as the pixels of Cr, the pixels of registers R0 to 0e and 0f and the pixels of registers R1 to 18, 19, 1a, 1b, 1c and 1d are extracted. As a result, Cb / Cr components necessary for motion compensation can be acquired simultaneously.

  After execution of the byte shuffle instruction, the write control unit 106 writes the contents of the register R3 from the address waddr at the upper left corner of the decoded block (step S407). Then, the read control unit 105 acquires the head address of the next line of the reference block and stores it in radr (step S408), and further acquires the head address of the next line of the decoded block and stores it in waddr (step S408). S409). Then, the processing from step S403 to S409 is repeated eight times, and the above processing is performed for all the pixels of the macroblock, that is, 8 × 8 pixels. This completes the motion compensation process for the color difference component.

  Next, the luminance signal motion compensation processing executed before the color difference signal motion compensation processing will be described. FIG. 9 is a flowchart showing a procedure of luminance component motion compensation processing. As shown in FIG. 2, the luminance component of the image data is stored in the frame memory (Y) in the frame sequential format as in the conventional case.

  First, the read control unit 107 calculates the upper left address radr of the reference block of the reference image frame memory (Y) 141 as in the following equation (step S901).

  Here, (x, y) represents the coordinates of the upper left corner of the decoded block, (mvx, mvy) represents the motion vector, and width represents the width of the luminance component image (720 pixels in SDTV).

  Further, offset (ref, Y) indicates the address of the upper left corner of the frame memory of the luminance component of the reference image (REFerence). Note that offset (ref, Y) is always a 16-byte aligned address.

  Next, the read control unit 105 calculates the address waddr at the upper left end of the decoded block in the frame memory (Y) 141 of the decoded image as shown in Equation 5 (step S902).

  Here, offset (cur, Y) indicates the address of the upper left corner of the frame memory (Y) 141 of the luminance component of the decoded image (CURrent). The calculated waddr is always aligned to 16 bytes.

  Next, the read control unit 105 reads 16 bytes (128 bits) of image data from an address in which the lower 4 bits of the upper left end address radr of the reference block are 0, and loads the image data into the register R0 (step S903).

  Next, the read control unit 105 reads 16 bytes (128 bits) of image data from an adjacent address at a distance of 16 bytes from the address in which the lower 4 bits of the upper left end address radr of the reference block are 0, and loads the image data into the register R1 (Step S904).

  Then, the write control unit 106 reads the byte shuffle control word from the control table and loads it into the register R2 (step S905). The control table used in the luminance component motion compensation process is different from the control table used in the color difference component motion compensation process shown in FIG.

  Next, the write control unit 106 executes a byte shuffle instruction (for example, bshuffle instruction), and extracts 16 pixels necessary for motion compensation from the pixels stored in the registers R0 and R1 according to the control word of the register R2. The execution result, that is, the 16 extracted pixels are stored in the register R3 (step S906).

  After execution of the byte shuffle instruction, the write control unit 106 writes the contents of the register R3 from the address waddr at the upper left corner of the decoded block (step S907). Then, the read control unit 105 acquires the head address of the next line of the reference block and stores it in radr (step S908), and further acquires the head address of the next line of the decoded block and stores it in waddr (step S908). S909). Then, the processing from step S903 to S909 is repeated 16 times, and the above processing is performed for all the pixels of the macroblock, that is, 16 × 16 pixels. Thus, the motion compensation process for the luminance component is completed.

  Note that the processing related to the byte shuffle instruction in steps S904 to S906 is performed in the same manner as the motion compensation processing for the color difference component.

  As described above, in the image decoding apparatus according to the present embodiment, the readout control unit 106 of the motion compensation processing unit 108 uses 16 pixels as a unit from the image data in which the color difference components Cb and Cr are alternately converted every 8 pixels. The data is loaded into the registers R0 and R1, and the write control unit 107 extracts 16-pixel data to be subjected to motion compensation processing according to the byte shuffle specification according to a predetermined byte shuffle control word. It is possible to arrange the color difference components Cb and Cr in one register having a capacity of 8 pixels and mix them, and to perform motion compensation processing for Cb and Cr at the same time without generating a wasteful area of the register. Since the motion compensation processing for the color difference component is completely common between Cb and Cr, the processing necessary for the subsequent motion compensation can be performed in half the time when Cb and Cr are processed independently. The motion compensation processing can be performed by effectively using the parallel processing of the SIMD type processor. As a result, the number of processing loops for the color difference component is smaller than the number of motion loops for the luminance component compared to the conventional image decoding apparatus in which improvement in processing efficiency cannot be expected due to the same number of processing loops for motion compensation for the luminance component and the color difference component. This is half the processing loop of the compensation processing, and the processing speed can be improved by about 25% as a whole of the motion compensation processing.

  Further, in the image decoding apparatus of the present embodiment, the pixel data of the color difference components Cb and Cr in the frame sequential format are alternately arranged every 16 pixels and mixed in one line. The total number of pixels in one line is a multiple of 16, and as a result, even in SIMD type processors that can only access the image memory with 16-byte alignment, pixels in other regions are processed when processing the rightmost pixel data. Motion compensation processing can be performed without adversely affecting data.

  In the image decoding apparatus according to the present embodiment, the image data in the frame sequential format is alternately arranged for each color difference component. However, the image data in the dot sequential format in which the color difference component is arranged for each pixel is also described above. Applied motion compensation processing.

  In the image decoding apparatus of the present embodiment, the image processing of the present invention is applied to motion compensation processing for MPEG color difference components. However, if image processing is performed on image data of different components at the same time, The present invention can also be applied to other image processing.

  As described above, the image processing apparatus, the image processing method, and the image processing program according to the present invention are useful for image processing of image data, and in particular, for an image decoding apparatus that performs motion compensation processing on color difference components Cb and Cr. Are suitable.

It is a block diagram which shows the functional structure of the image decoding apparatus concerning embodiment of this invention. It is explanatory drawing which shows a field sequential format. It is explanatory drawing which shows the format of the image data used with the image decoding apparatus of this Embodiment. It is a flowchart which shows the procedure of the motion compensation process about a color difference signal (Cb / Cr). It is explanatory drawing which shows the content of a control table. It is explanatory drawing for demonstrating operation | movement of a byte shuffle instruction. When the lower 4 bits of radr are 4, it is explanatory drawing which shows an example of the value of register R0, R1 after performing step 503, S504, and the value of register R3 after the byte shuffle instruction execution of step S506. It is explanatory drawing which shows an example of the value of register | resistor R0, R1 after execution of step 403, S404 when the lower 4 bits of radr are 6, and the value of the register | resistor R3 after the byte shuffle instruction execution of step S406. It is a flowchart which shows the procedure of the motion compensation process of a luminance component.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Decoder 101 Variable length decoding part 102 Inverse quantization part 103 Inverse scanning part 104 Inverse DCT part 105 Conversion part 106 Read control part 107 Write control part 108 Motion compensation part 110 Decoder control part 120 Stream buffer 130 Stream buffer control part 140 Image Memory 141 Frame memory (Y)
144 Frame memory (Cb / Cr)

Claims (11)

An image processing method for performing image processing on image data by a single instruction multiple data (SIMD) type parallel processing processor that processes a plurality of data with one instruction and performs read / write processing on a memory at a predetermined address position,
Every N / M (M: integer) pixels when simultaneously processing a plurality of pixel data having different color components of the image data and smaller than the number N of pixels that can be processed simultaneously by the parallel processing command of the SIMD processor A conversion step for converting the arrangement so that the pixel data of each component are alternately arranged;
A reading step of reading a plurality of pixel data arranged and converted in the conversion step by 2N pixels in units of N pixels;
By executing a byte shuffle instruction provided by the SIMD type processor, pixel data of N / M pixels to be image-processed is extracted from the pixel data of 2N pixels read by the reading step, and the extracted N / M An image processing step for performing image processing on pixel data of M pixels;
An image processing method comprising:   When simultaneously processing a plurality of pieces of pixel data that are different color difference components of the image data and are smaller than the number N of pixels, the plurality of pieces of pixel data are processed for each N / M (M: integer) pixel. The image processing method according to claim 1, wherein the arrangement conversion is performed so that the two are alternately arranged.   The image processing step executes the byte shuffle instruction based on control information determined in advance for each address difference between the predetermined address position and the pixel address, and is read out by the reading step. 3. The pixel data of N / M pixels to be subjected to image processing is extracted from pixel data of 2N pixels, and image processing is performed on the extracted pixel data of N / M pixels. The image processing method as described.   In the image processing step, pixel data of N / M pixels to be subjected to motion compensation processing is extracted from the pixel data of 2N pixels read out by the reading step by executing the byte shuffle instruction, and the extracted N / M pixels are extracted. The image processing method according to claim 1, wherein motion compensation processing is performed on pixel data of M pixels. In the conversion step, when a plurality of pixel data which are different color difference components of the image data and are smaller than the number of pixels N are processed simultaneously, the plurality of pixel data is processed for every N / 2 (M: integer) pixels. Change the arrangement so that the pixel data of each component alternate,
In the image processing step, pixel data of N / 2 pixels to be subjected to motion compensation processing is extracted from the pixel data of 2N pixels read out by the reading step by executing the byte shuffle instruction, and the extracted N / 5. The image processing method according to claim 4, wherein motion compensation processing is performed on pixel data of two pixels. A SIMD (Single Instruction Multiple Data) type parallel processing processor that processes a plurality of data with a single instruction and performs read / write processing on the image memory at a predetermined address position;
A memory for storing a program and accessible by the SIMD type parallel processing processor;
An image memory for storing image data and accessible by the SIMD type parallel processing processor,
The program includes the SIMD type parallel processing processor,
When processing a plurality of pieces of pixel data which are different color components of the image data and are less than the number N of pixels which can be processed simultaneously by the SIMD processor parallel processing instructions, the plurality of pieces of pixel data are converted to N / M ( M: integer) conversion means for converting the arrangement so that pixel data of each component is alternated for each pixel;
Reading means for reading a plurality of pixel data arranged and converted by the converting means for 2N pixels in units of N pixels;
By executing a byte shuffle instruction provided by the SIMD type processor, pixel data of N / M pixels to be image-processed is extracted from pixel data of 2N pixels read by the reading unit, and the extracted N / M An image processing apparatus that functions as an image processing unit that performs image processing on pixel data of M pixels.   The conversion means processes the plurality of pixel data for each N / M (M: integer) pixels when simultaneously processing a plurality of pixel data that are different color difference components of the image data and are smaller than the number N of pixels. The image processing apparatus according to claim 6, wherein the arrangement conversion is performed so that pixel data of each component is alternated.   The image processing means executes the byte shuffle instruction based on control information predetermined for each address difference between the predetermined address position and the pixel address, and is read by the reading means 8. The pixel data of N / M pixels to be image processed is extracted from pixel data of 2N pixels, and image processing is performed on the extracted pixel data of N / M pixels. The image processing apparatus described.   The image processing means extracts the pixel data of N / M pixels subject to motion compensation processing from the pixel data of 2N pixels read out by the reading means by executing the byte shuffle instruction, and extracts the extracted N / M 9. The image processing apparatus according to claim 6, wherein motion compensation processing is performed on pixel data of M pixels. The conversion means processes the plurality of pixel data for each N / 2 (M: integer) pixels when simultaneously processing a plurality of pixel data that are different color difference components of the image data and are smaller than the number N of pixels. Change the arrangement so that the pixel data of each component alternate,
The image processing means extracts the pixel data of N / 2 pixels subject to motion compensation processing from the pixel data of 2N pixels read by the reading means by executing the byte shuffle instruction, and extracts the extracted N / N The image processing apparatus according to claim 9, wherein motion compensation processing is performed on pixel data of two pixels. An image processing program that performs image processing on image data by a SIMD (Single Instruction Multiple Data) type parallel processing processor that processes a plurality of data with one instruction and performs read / write processing on a memory at a predetermined address position,
When processing a plurality of pieces of pixel data which are different color components of the image data and are less than the number N of pixels which can be processed simultaneously by the SIMD processor parallel processing instructions, the plurality of pieces of pixel data are converted to N / M ( M: integer) a conversion procedure for converting the arrangement so that pixel data of each component is alternated for each pixel;
A reading procedure for reading out a plurality of pixel data arranged and converted by the conversion procedure for 2N pixels in units of N pixels,
The N / M pixel data to be image-processed is extracted from the 2N pixel pixel data read out by the readout procedure by executing the byte shuffle instruction provided by the SIMD type processor, and the extracted N / M An image processing procedure for performing image processing on pixel data of M pixels;
An image processing program for causing a computer to execute.

Images