JP2008011211A - Image processor and processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To compress pixel values of respective pixels independently of adjacent pixels without using differences in pixel values between adjacent pixels when a decoded image of a digital moving picture compressed stream is stored in a frame memory, and to expand the compressed data independently of adjacent pixels. <P>SOLUTION: A writing unit 21 finds mean values of pixel values of pixels included in respective blocks, block by block. An information compression unit 22 finds differences between the mean values and pixel values, cuts multiple low-order bits of the difference values to generate the compressed data consisting of bits less than the original pixel values, and stores the compressed data in frame memories 24a to 24d. An information expanding unit 25 adds mean values, used to generate compressed data read out of the frame memories 24a to 24d, to the compressed data to reproduce pixel values of respective pixels. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method, and more particularly, to an image processing apparatus and an image processing method for encoding and decoding moving image data according to a digital moving image compression method such as MPEG (Moving Picture Experts Group) 2. .

  In general, image processing apparatuses such as a moving picture coding apparatus and a moving picture decoding apparatus that employ a digital moving picture compression method such as MPEG2 are provided with a frame memory for holding frame information. In recent years, as the screen size of display devices has increased, the capacity of the frame memory and the flow rate (bandwidth) of information in the frame memory per unit time have increased, and these have become bottlenecks in development. Yes.

  In order to eliminate this bottleneck, proposals have been made to compress the amount of information stored in the frame memory. For example, a technique for compressing the amount of information by performing Huffman coding on a difference in pixel values between adjacent pixels is known (see, for example, Patent Document 1). Also, a technique for compressing the amount of information by quantizing a difference in pixel value between adjacent pixels with a quantization coefficient is known (for example, see Patent Document 2).

  FIG. 5 is a block diagram showing a configuration of an image processing apparatus that compresses and decompresses the above-described conventional information amount. As shown in FIG. 5, in the conventional image processing apparatus, the writing unit 1 decodes the digital moving image compressed stream into pixel values of each pixel. Then, the information compression unit 2 obtains a difference between pixel values between adjacent pixels, and the data is compressed by Huffman encoding or quantizing the difference value as described above. The compressed data is written to each frame memory 4a, 4b, 4c, 4d in the memory 4 via the memory bus 3.

  On the other hand, compressed data is read from the frame memories 4a, 4b, 4c, and 4d. The read data is reconstructed by the information decompression unit 5 using the pixel value difference between adjacent pixels, thereby being reproduced to the pixel value of each pixel. The reproduced pixel value is output by the reading unit 6 to a display device (not shown). In FIG. 5, the information amount (hatched portion) of each of the first to fourth frame memories 4a, 4b, 4c, and 4d is different because the complexity of the picture is different for each frame memory. .

  FIG. 6 is an explanatory diagram showing the concept of conventional data compression processing. As shown in FIG. 6, in the conventional data compression process, the pixel values 7a, 7b, 7c, and 7d of adjacent pixels are used in the raster scan order, that is, the leftmost pixel in the frame image. When the pixel is scanned horizontally to the rightmost pixel, the compressed data 9a, 9b, and 9c are sequentially generated so as to scan from the left to the right in the same manner for the pixel row one step below.

  FIG. 7 is an explanatory diagram showing the concept of conventional data decompression processing. As shown in FIG. 7, in the conventional data decompression processing, each pixel value 7a, 9b, 9c, 9d is decompressed by using the difference values 8a, 8b, 8c, 8d in the raster scan order. 7b, 7c, and 7d are reproduced.

Japanese Patent Laid-Open No. 11-136686 JP-A-10-243402

  However, as described above, in the conventional technique, since data compression is performed using a difference in pixel value between adjacent pixels, the data is sequentially displayed in units of pixels from left to right in the horizontal direction in the frame image. It is necessary to perform compression processing. The same applies to the data decompression process.

  FIG. 8 is an explanatory diagram showing a state in which a part of the area is read by the conventional data compression / decompression process. In particular, if a difference in pixel values between adjacent pixels is Huffman encoded, an arbitrary difference value in the frame memory cannot be extracted. Therefore, as shown in FIG. 8, in the conventional technique, when a pixel in a small rectangular area 10 in the frame memory 4c is read out and enlarged and displayed, the frame image has an area 11 on the left side of the rectangular area 10. Since it is necessary to sequentially read from the top of the frame including the pixels (not the display target), there is a problem that the reading efficiency is poor.

  In order to eliminate the above-described problems caused by the conventional technology, the present invention compresses the pixel value of each pixel independently from the adjacent pixel without using the difference in pixel value between adjacent pixels, and the compression. An object of the present invention is to provide an image processing apparatus and an image processing method capable of expanding the processed data independently from adjacent pixels.

  In order to solve the above-described problems and achieve the object, an image processing apparatus and an image processing method according to the present invention obtain a pixel value of each pixel by decoding a digital moving image compressed stream, and for each processing unit, The representative value of the pixel value of the pixel included in the processing unit is obtained, the bit number of the pixel value of each pixel in the processing unit is compressed using the representative value for each processing unit, and the compressed pixel is compressed. A value is stored in a frame memory. The compressed value of each pixel read from the frame memory is expanded using the representative value of the processing unit including the pixel, and the expanded value is output as the pixel value of each pixel. To do.

  In the present invention, the compressed value of each pixel may be obtained by obtaining a difference between the pixel value of each pixel and the representative value, and cutting out the lower-order multiple bits of the difference. Moreover, in this invention, you may use the average value of the pixel value of the pixel contained in a processing unit as a representative value. Further, in the present invention, the representative value of each processing unit may be stored in the representative value memory. In this case, the representative value memory may have a capacity capable of storing representative values for the maximum number of processing units that can be stored in the frame memory.

  According to the present invention, the pixel value of each pixel is compressed for each processing unit by using the representative value of the processing unit including the pixel, so that the compression is performed without using the pixel value difference between adjacent pixels. Can do. Further, since the compressed value of each pixel is expanded using the representative value of the processing unit including that pixel, it is possible to expand without using the difference in pixel value between adjacent pixels.

  According to the image processing device and the image processing method of the present invention, the pixel value of each pixel is compressed independently from the adjacent pixel without using the difference in pixel value between adjacent pixels, and the compression is performed. The data can be expanded independently from adjacent pixels.

  Exemplary embodiments of an image processing apparatus and an image processing method according to the present invention are explained in detail below with reference to the accompanying drawings.

(Configuration of image processing apparatus)
FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. FIG. 2 is an explanatory diagram showing the configuration of the frame memory and the representative value memory that constitute the image processing apparatus shown in FIG. As shown in FIG. 1, the image processing apparatus includes a writing unit 21, an information compression unit 22, a memory bus 23, a memory 24 such as an SDRAM (Synchronous DRAM), an information decompression unit 25, a reading unit 26, and a representative value memory 27. ing. The memory 24 is provided with a plurality of frame memories 24a, 24b, 24c, and 24d.

  The writing unit 21 decodes a digital moving image compressed stream such as MPEG2 to obtain a pixel value of each pixel. In addition, the writing unit 21 obtains representative values of pixel values of all pixels included in each processing unit for each processing unit of one frame including a plurality of processing units. The representative value of each processing unit is stored in the representative value memory 27. For example, as a representative value, although not particularly limited, an average value of pixel values of all pixels included in a processing unit can be used.

  Here, as shown in FIG. 2, in a digital moving image compression method such as MPEG2, one frame is composed of small units called a plurality of blocks 31. Therefore, in the present embodiment, although not particularly limited, the block 31 can be used as the processing unit. The number of pixels 32 included in each block 31 is, for example, 64 (= 8 pixels × 8 pixels).

  Therefore, when the block 31 is a processing unit and an average value is used as a representative value, the writing unit 21 calculates an average of the pixel values of 64 pixels included in each block 31 and uses the value as a representative value of the block 31. Value. Conventionally, since decoding processing of a digital moving image compressed stream is performed in units of blocks, an average of pixel values in a block can be easily calculated. Hereinafter, a case where the block 31 is used as a processing unit and an average value is mainly used as a representative value of the block 31 will be described.

For each block, the information compression unit 22 compresses the number of bits of the pixel value of each pixel in the block using the representative value of the block, that is, reduces the number of bits. When the average value is used as the representative value, the difference between the K (for example, 8) bit average value and the K bit pixel value is calculated as in the following equation (1).
Average value in block (K bits) -Each pixel value in block (K bits) (1)

  Thus, the difference between the average value of the 64 pixel values in the block and the average value of each pixel value in the block is considered that the difference between each pixel value and the average value is not so large. Because it is. Further, the information compression unit 22 calculates the lower order M (1 <M <K, for example, from the difference value of [K + 1] bits (1 bit represents a positive / negative sign) obtained by the calculation of the equation (1). M cuts out 6) bits and writes them into the frame memories 24a, 24b, 24c and 24d via the memory bus 23 as compressed data of M bits.

  The information decompression unit 25 reads compressed data (M bits) corresponding to the pixel to be read from the frame memories 24a, 24b, 24c, and 24d via the memory bus 23. The information decompressing unit 25 reads the representative value of the block to which the pixel to be read belongs from the representative value memory 27. Then, the information decompression unit 25 performs an operation reverse to the operation at the time of data compression in the information compression unit 22 on the compressed data using the representative value, reproduces a K-bit pixel value, and reads it out 26.

  When the information compression unit 22 compresses the data by performing the calculation of the equation (1), the information expansion unit 25 adds the M-bit compressed data and the K-bit average value. The reading unit 26 transfers the pixel value sent from the information decompression unit 25 to a display device (not shown). The representative value memory 27 has a capacity capable of storing all the average values of the respective blocks in the frame.

(Image processing procedure)
Next, the flow of processing from writing to reading of image data to the frame memory will be described. FIG. 3 is a flowchart showing an image processing procedure according to the embodiment of the present invention. As shown in FIG. 3, first, the writing unit 21 decodes the pixel value (K bits) of each pixel from a digital moving image compressed stream such as MPEG2. The writing unit 21 obtains a representative value (K bits) of the pixel value for each block (step S1).

  The average value of each block is stored in the representative value memory 27. Next, the information compression unit 22 performs the calculation of the expression (1) for each block, and the lower M bits of the calculation result are cut out to obtain compressed data (step S2). The compressed data is written in the frame memories 24a, 24b, 24c, and 24d (step S3).

  Next, the information decompression unit 25 reads compressed data (M bits) corresponding to the pixel to be read from the frame memories 24a, 24b, 24c, and 24d (step S4). The information decompression unit 25 reads the average value (K bits) of the block to which the pixel to be read belongs from the representative value memory 27, and decompresses the compressed data using the representative value (step S5). The pixel value reproduced thereby is transferred from the information decompression unit 25 to the reading unit 26, and further transferred to a display device (not shown) (step S6).

  By compressing the data as described above, pixel data of K (for example, 8) bits can be compressed to M (for example, 6) bits and stored in the frame memories 24a, 24b, 24c, and 24d. In addition, since compression is performed without using a difference in pixel value between adjacent pixels, any pixel in the frame memories 24a, 24b, 24c, and 24d can be directly accessed. Therefore, for example, even when a part of the rectangular areas in the frame memories 24a, 24b, 24c, and 24d is enlarged and displayed, only the pixels in the rectangular area to be displayed are efficiently obtained from the frame memories 24a, 24b, 24c, and 24d. Can be read out automatically.

  The present invention can be applied to each of luminance (Y) and color difference (Cb / Cr). Further, the present invention can be applied to a unit such as a macro block or a slice including a plurality of macro blocks in addition to a block as a processing unit. If the processing unit is increased, the capacity of the representative value memory 27 can be reduced. In addition to the average value, a value close to the pixel value of each pixel in the block can be used as the representative value of the block.

  Further, the data compression rate in the information compression unit 22 may be adaptively changed according to the design. In this case, the user may set the compression rate to be low when a complicated pattern is to be processed, and set the compression rate to be high if the pattern is not complicated. Alternatively, the writing unit 21 or the information compression unit 22 may detect the complexity of the pattern to be processed and adjust the compression rate as appropriate according to the complexity.

  Furthermore, the correspondence relationship between K-bit data and M-bit data may be tabulated in advance, and the K-bit data may be quantized into M-bit data by referring to this table. The present invention is not limited to MPEG2, but can be applied to MPEG4, DV (Digital Video), H264 established by ITU-T, or other digital moving image compression methods.

(Configuration of example applied to MPEG2 HL decoder)
Next, an example in which the present invention is applied to an MPEG2 HL decoder will be described. FIG. 4 is a block diagram showing the configuration of an MPEG2 HL decoder to which the present invention is applied. FIG. 4 shows a system that performs processing until an externally input stream (PS / TS format) is output to the outside as a video signal.

  As shown in FIG. 4, the HL decoder includes an HL decoder unit 41, an information compression unit 42, a memory bus 43, a memory 44 such as an SDRAM having a plurality of frame memories 44a, 44b, 44c and 44d, an information decompression unit 45, a display A unit 46, an average value memory 47, a system separation unit 48, and a control unit 49. The HL decoder unit 41, the display unit 46, and the average value memory 47 correspond to the writing unit 21, the reading unit 26, and the representative value memory 27 of the image processing apparatus shown in FIG.

  The system separation unit 48 separates a stream (PS / TS format) input from the outside into a stream such as VES / AES and stores it in the memory 44 via the memory bus 43. The HL decoder unit 41 decodes the VES stream stored in the memory 44 and writes the decoded image in the memory 44. The HL decoder 41 refers to pixel data from the decoded I picture and the decoded P picture in order to perform motion compensation during decoding.

  In addition, the HL decoder unit 41 transfers the pixel value (8 bits) of the B picture decoded image to the information compression unit 42. In the process of decoding the B picture, the HL decoder unit 41 calculates the average value (8 bits) of the pixel values for each block and transfers it to the information compression unit 42 and the average value memory 47.

  The information compression unit 42 obtains the difference between the average value of the block (8 bits) and the pixel value (8 bits) of the B picture decoded image according to the above equation (1), and for example, the lower 6 bits of the signed 9-bit value Cut out. The extracted 6-bit compressed B-picture data is written to the third frame memory 44c or the fourth frame memory 44d via the memory bus 43.

  The average value memory 47 has a capacity capable of storing all the average values of all the blocks constituting the third frame memory 44c and the fourth frame memory 44d. The information decompression unit 45 reads the compressed data (6 bits) of the B picture designated by the control unit 49 from the third frame memory 44c or the fourth frame memory 44d via the memory bus 43.

  Further, the information decompression unit 45 reads the average value (8 bits) of the block to which the B picture pixel instructed by the control unit 49 belongs from the average value memory 47. Then, the information decompression unit 45 calculates the pixel value (8 bits) of the B picture decoded image by adding the obtained compressed data (6 bits) and the average value (8 bits), and displays it on the display unit 46. Forward to.

  The display unit 46 reads each pixel value (8 bits) of the I picture decoded image and the P picture decoded image of the frame instructed by the control unit 49 from the first frame memory 44a or the second frame memory 44b. The display unit 46 converts the pixel value of the I picture decoded image (8 bits), the pixel value of the P picture decoded image (8 bits), and the pixel value of the B picture decoded image (8 bits) into a video signal, Output to a display device (not shown).

  The control unit 49 controls the operation of each block described above. Similar to the B picture, the information compression unit 42 compresses each pixel value of the I picture or P picture and writes it to the frame memories 44a and 44b, and the information decompression unit 45 stores the compressed data of the I picture or P picture. You may make it expand | extend.

(Supplementary note 1) Decoding the digital moving image compressed stream to obtain the pixel value of each pixel, and for each processing unit, writing means for obtaining a representative value of the pixel value of the pixel included in the processing unit; The information compression means for compressing the number of bits of the pixel value of each pixel in the processing unit using the representative value obtained by the writing means, and the compressed value of each pixel compressed by the information compression means A frame memory to be stored; an information decompression unit that decompresses a compressed value of each pixel read from the frame memory using the representative value of a processing unit including the pixel; and an decompression decompressed by the information decompression unit An image processing apparatus comprising: a reading unit that outputs a value as a pixel value of each pixel.

(Additional remark 2) The said information compression means calculates | requires the difference of the pixel value of each pixel, and the said representative value, and makes the value of the low-order multiple bits of this difference the value which compressed the pixel value of each pixel, The image processing apparatus according to appendix 1.

(Additional remark 3) The said representative value is an average value of the pixel value of the pixel contained in a process unit, The image processing apparatus of Additional remark 1 or 2 characterized by the above-mentioned.

(Supplementary note 4) The image processing apparatus according to any one of Supplementary notes 1 to 3, further comprising: a representative value memory that stores the representative values for the maximum number of processing units that can be stored in the frame memory. .

(Supplementary Note 5) A representative value calculating step of decoding a digital moving image compressed stream to obtain a pixel value of each pixel and obtaining a representative value of a pixel value of a pixel included in the processing unit for each processing unit; For each unit, an information compression step for compressing the number of bits of the pixel value of each pixel in the processing unit using the representative value obtained in the representative value calculation step, and each pixel compressed in the information compression step A compressed value storing step for storing the compressed value of the pixel in the frame memory, and an information decompressing step for reading the compressed value of each pixel from the frame memory and decompressing the compressed value using the representative value of the processing unit including the pixel And a reading step of outputting the decompressed value decompressed in the information decompressing step as a pixel value of each pixel.

(Additional remark 6) The said information compression process calculates | requires the difference of the pixel value of each pixel, and the said representative value, and makes the value of the low-order multiple bits of this difference the value which compressed the pixel value of each pixel, The image processing method according to appendix 5.

(Supplementary note 7) The image processing method according to supplementary note 5 or 6, wherein the representative value is an average value of pixel values of pixels included in a processing unit.

  As described above, the image processing apparatus and the image processing method according to the present invention are useful for an apparatus and a method for encoding and decoding moving image data according to a digital moving image compression method, and in particular, MPEG2, MPEG4, DV. Alternatively, it is suitable for an apparatus and method for encoding and decoding moving image data according to a digital moving image compression method such as H264.

It is a block diagram which shows the structure of the image processing apparatus concerning embodiment of this invention. FIG. 2 is an explanatory diagram illustrating a configuration of a frame memory and a representative value memory included in the image processing apparatus illustrated in FIG. 1. It is a flowchart which shows the image processing procedure concerning embodiment of this invention. It is a block diagram which shows the structure of the MPEG2 HL decoder to which this invention is applied. It is a block diagram which shows the structure of the conventional image processing apparatus. It is explanatory drawing which shows the concept of the conventional data compression process. It is explanatory drawing which shows the concept of the conventional data expansion | extension process. It is explanatory drawing which shows a mode that a one part area | region is read by the conventional data compression and expansion | extension processing.

Explanation of symbols

DESCRIPTION OF SYMBOLS 21 Writing part 22 Information compression part 24a, 24b, 24c, 24d Frame memory 25 Information decompression part 26 Reading part 27 Representative value memory

Claims (5)

Writing means for decoding a digital moving image compressed stream to obtain a pixel value of each pixel, and for each processing unit, obtaining a representative value of the pixel value of the pixel included in the processing unit;
Information compression means for compressing the number of bits of the pixel value of each pixel in the processing unit using the representative value obtained by the writing means for each processing unit;
A frame memory for storing a compressed value of each pixel compressed by the information compression means;
Information decompression means for decompressing the compressed value of each pixel read from the frame memory using the representative value of the processing unit including the pixel;
A readout means for outputting the decompressed value decompressed by the information decompressing means as a pixel value of each pixel;
An image processing apparatus comprising:   2. The information compression unit obtains a difference between a pixel value of each pixel and the representative value, and sets a value of lower multiple bits of the difference as a value obtained by compressing the pixel value of each pixel. An image processing apparatus according to 1.   The image processing apparatus according to claim 1, wherein the representative value is an average value of pixel values of pixels included in a processing unit.   The image processing apparatus according to claim 1, further comprising a representative value memory that stores the representative values for the maximum number of processing units that can be stored in the frame memory. A representative value calculation step of obtaining a pixel value of each pixel by decoding the digital moving image compressed stream and obtaining a representative value of the pixel value of the pixel included in the processing unit for each processing unit;
An information compression step for compressing the number of bits of the pixel value of each pixel in the processing unit using the representative value obtained in the representative value calculation step for each processing unit;
A compressed value storing step of storing a compressed value of each pixel compressed in the information compressing step in a frame memory;
An information decompression step of reading a compressed value of each pixel from the frame memory, and decompressing the compressed value using the representative value of a processing unit including the pixel;
A readout step of outputting the decompressed value decompressed in the information decompressing step as a pixel value of each pixel;
An image processing method comprising:

Images