JP2010141771A - Image processing apparatus and method - Google Patents

Abstract

An image processing apparatus capable of reducing a memory bus band in an image processing apparatus in which access to an arbitrary position in a screen occurs.
When an image write request is issued, the image is reduced by thinning or the like, and the reduced image is stored in a memory. Further, the reduced image is enlarged by linear interpolation or the like to obtain difference data from the original image. The difference data is compressed in units of lines and rectangles and stored in the memory. An image read request is performed in units of lines or rectangles. The difference data in units of lines or rectangles including the lines or rectangles with read requests and the compression including lines or rectangles in requests are read out and enlarged. Expand and add to obtain the requested image data.
[Selection] Figure 4

Description

  The present invention relates to an image processing apparatus and method.

  With the widespread use of high-definition broadcasting and large-capacity optical discs, the images handled by image processing LSIs are becoming larger and their functions are becoming increasingly sophisticated. As screens and functions increase, the memory capacity and memory bus bandwidth used by image processing LSIs continue to increase.

  As a conventional technique for processing large-screen image data with a small-capacity memory, there is a technique (Patent Document 1) that stores a reduced image and compressed difference data in a memory. The reduced image is created by thinning out the original image, and the difference data is obtained by taking the difference between the image obtained by enlarging the reduced image again and the original image. If a reduced image is generated from the original image and enlarged again, the amount of information is lower than that of the original image, resulting in a reduction in image quality. Therefore, information that has dropped in the reduced image is stored as difference data, and when the image is restored, the reduced data is combined with the difference data to prevent the information amount of the image from dropping. I am doing so. When storing the difference data, the storage area is saved by compressing the difference data into compressed difference data. This conventional technology assumes that image data for one screen is processed in a fixed order for use in printers, etc., and is suitable for applications that access any position in the screen. Absent.

For example, in the MPEG decoder, there is a possibility that a rectangular area at an arbitrary position in the screen is read for reference picture access. However, in this conventional technology, such access is not assumed and efficient. Access is not possible. Especially for compressed differential data, if the data for the screen is combined and variable-length encoded, it is difficult to specify the data at the specified position in the screen. We have to go. Therefore, in the case of reading at an arbitrary position such as reference image access, unnecessary compressed differential data that is not actually used is read, resulting in an increase in the memory access amount. For the reasons described above, from the viewpoint of the memory bus bandwidth, the prior art cannot be applied to an apparatus that accesses an arbitrary position in a screen such as an MPEG decoder.
JP-A-5-227444

  An object of the present invention is to provide an image processing apparatus capable of reducing a memory bus band in an image processing apparatus in which access to an arbitrary position in a screen occurs.

  The image processing apparatus of the present invention generates a reduced image data by reducing the image data by reducing the image data, stores the reduced image data in the memory, generates an enlarged image data by expanding the reduced image data, and the image data The difference data from the enlarged image data is compressed in a plurality of different units, an image data compression unit that generates a plurality of compressed difference data and stores it in the memory, and reads out the compressed difference data in a unit indicated by the request request And an image data decompression unit that reads the reduced image data and generates decompressed image data based on the compressed difference data and the reduced image data.

According to the present invention, in an image processing apparatus in which access to an arbitrary position in a screen occurs,
An image processing apparatus that can reduce the memory bus bandwidth can be provided.

  The embodiment of the present invention provides a technique and configuration for reducing the use bandwidth of a memory bus in a device that requires a high memory bus bandwidth, such as a decoder and an encoder device such as MPEG and H.264.

  In particular, in the present embodiment, when writing to the memory, the image processing apparatus temporarily reduces the image data, enlarges the reduced image data again, and creates difference data from the original image. Thereafter, the created difference data is compressed in units of a plurality of different in-screen areas, and the reduced image and the plurality of compressed difference data are stored in the memory. At the time of reading, the reduced image and one of the compressed difference data are read out, enlarged and developed, and then added to restore the original image.

  In the image processing apparatus, when writing to the memory, the image data is temporarily reduced, the reduced image data is enlarged again, and difference data from the original image is created. Thereafter, the created difference data is compressed in units of a plurality of different in-screen areas, and the reduced image and the plurality of compressed difference data are stored in the memory. At the time of reading, a method for reading only a reduced image to create an enlarged image and a reduced image and any one of the compressed difference data are read out, enlarged and developed, and added to restore the original image.

  One of the in-screen area units for compressing the difference data is a line unit, and one is a rectangular unit obtained by dividing the screen by a specified rectangular size. The compressed difference data is stored in a designated memory area for each unit.

FIG. 1 shows an image stored in a memory of reduced images and compressed differential data.
FIG. 1 shows an example in which a reduced image is generated by thinning out the length 1/2 and width 1/2, and the difference data has a format compressed in units of lines and compressed in units of 16 × 16 rectangles.

  FIG. 1A shows an original image, and circles indicate pixels. By thinning one line out of the original image in FIG. 1A in two lines in the vertical and horizontal directions, a reduced image compressed in half in the vertical and horizontal directions is obtained. The reduced image is shown in FIG. In the reduced image, only the pixels represented by the thick circles in FIG. 1A remain. The reduced image is stored in the storage area as a reduced image as shown in the left part of FIG. On the other hand, as shown in the middle or right of FIG. 1C, the compressed difference data is divided into line units or rectangular units of a predetermined size and compressed for each line or each rectangle. . The compressed line data or the compressed rectangular data is given the address of the head position in the storage area, and stored in line units or rectangular units. Therefore, when it is desired to read specific line data or specific rectangular data, it is stored in the storage area so that it can be easily read by designating the address of the head position.

FIG. 2 shows an example of the order of differential data compression processing.
FIG. 2A shows a processing order in the case where compressed differential data is generated in units of lines. Data for 1920 pixels is read and compressed from the row 0 of the differential data in the horizontal direction. When line 0 ends, processing is sequentially performed from line 0 in the vertical direction, such as line 1. In line-by-line compression, the line number is used as an identifier. By associating the identifier with the head address of the storage area, the line to be read can be read by specifying the head address. Note that it is not always necessary to set all the data of one line as one unit, and a line area obtained by dividing one line into a plurality of areas may be set as one unit. FIG. 2B shows the processing order when generating compressed differential data in units of rectangles. As shown in FIG. 2B, the difference data composed of 1920 pixels in the horizontal direction and 1088 pixels in the vertical direction is divided into rectangles of 16 pixels × 16 pixels, and an identifier is attached to each rectangle. If it is a rectangle at the upper left corner, (0, 0) is used as an identifier, and as it goes in the row direction, the coordinate value in the row direction is increased to (1, 0),. As it goes, the coordinate value in the column direction is increased to (0, 1),... (0, 67). Then, compression is performed in units of blocks of 16 pixels × 16 pixels and stored in the storage area. By associating the identifier with the head address of the storage area, the rectangular area to be read can be read by designating the head address. Note that the size of the rectangle is not necessarily 16 pixels × 16 pixels.

  In addition, the compression difference data size is calculated and recorded for each in-screen area unit (for each line, for each rectangle) in which the difference data is compressed. As a recording means, the size of the compressed differential data may be written at the head of the compressed differential data, or may be recorded as a separate table. The size does not necessarily have to be a strict value, and may be recorded in units of the memory access size at the time of reading.

  When the reading request is rectangular reading, the difference data compressed in units of rectangles including the rectangular area in the reduced image corresponding to the rectangular area of the original image and the area corresponding to the rectangular area of the original image is read. Then, after each expansion and expansion, the data other than the requested rectangular area is discarded, and the original image is restored by adding the expanded image data from the reduced image in the requested rectangular area and the expanded data from the compressed difference data. .

  When the read request is line read, the line in the reduced image corresponding to the line of the original image and the difference data compressed in line units corresponding to the line of the original image are read. After each enlargement and expansion, the data other than the requested area is discarded, and the original image is restored by adding the enlarged image data from the requested reduced image and the expanded data from the compressed difference data.

  In this embodiment, the image processing apparatus is provided with a built-in memory for storing differential data read in the past. If the area indicated by the second and subsequent read requests includes the area of differential data stored in the internal memory, the corresponding compressed differential data is not read and the internal memory differential data is used. To restore the original image. The built-in memory is provided with a built-in memory for storing difference data in units of rectangles and a built-in memory for storing difference data in units of lines. A rectangular built-in memory is used for rectangular reading, and a line built-in memory is used for line reading.

  Consider a memory controller in which a module for rectangular access to image data and a module for line access are connected. In response to a request from the module accessing the rectangle, the reduced image and the compressed difference data for rectangular access are read out, and in response to a request from the module accessing the line, the reduced image and the compressed difference data for line access are read out. As a result, it is possible to reduce the data access amount per access to the memory.

FIG. 3 is a diagram for explaining how the amount of data access to the memory is reduced.
For example, consider the case of reading 256 pixels (256x1 when reading a line, 16x16 when reading a rectangle) in the screen with the original image size. As shown in FIGS. 3 (a) and 3 (b), the reduced image creation method is vertical 1/1 (no thinning), horizontal 1/4 thinning, and the creation method of the enlarged image from the reduced image is linear interpolation. The difference data is compressed in 256 pixel units (256x1 area for lines, 16x16 area for rectangles) of the screen area of the original image size, and the compression rate of the difference data is 1/8 of the original image. Here, since the reduction method is thinning, the difference value between the value of the pixel written in the reduced image and the value of the corresponding pixel in the original image is always zero, so that the corresponding pixel is written as difference data. Absent. Therefore, the difference data holds 3/4 pixels of the original image. The data access amount to the memory is compared by the following three methods. In the following table and calculation, the data access amount required for one access is shown by the number of pixels.
Method 1: Store original image directly in memory Method 2: Store only reduced differential data and compressed differential data for lines in memory Method 3: Store compressed differential data for both reduced images and lines / rectangular in memory

Access volume calculation
Write
Method 1 16x16 (original image) = 256
Method 2 4x16 (reduced image) + 256x3 / 4x1 / 8 (compressed difference data) = 88
Method 3 4x16 (reduced image) + 256x3 / 4x1 / 8x2 (2 types of compressed differential data) = 112

  In method 1, since the original image is stored in the memory as it is, a capacity of 256 pixels is required for one access. On the other hand, in the method 2, the reduced image is written with the reduced image thinned by 1/4 in the horizontal direction and the compressed difference data. The compressed difference data is accessed every 256 pixels × 1 line pixel. The compressed difference data has 3/4 pixels of the original image, and the compression rate is 1/8, so that it is as described above. Method 3 is the same as Method 2 for reduced images. The compressed difference data is written in units of 16 pixels × 16 pixels and in units of 256 pixels × 1 line. At this time, the compression difference data has 3/4 pixels of the original image and has a compression rate of 1/8. In the method 3, since compression is performed in units of lines and in units of rectangles, 2 is applied.

Line Read
Method 1 256x1 (original image) = 256
Method 2 65x1 (reduced image) + 256x3 / 4x1 / 8x1 (compressed differential data 1 area) = 89 (Min)
66x1 (reduced image) + 256x3 / 4x1 / 8x2 (compressed differential data 2 area) = 114 (Max)
Method 3 65x1 (reduced image) + 256x3 / 4x1 / 8 (compressed differential data 1 area) = 89 (Min)
66x1 (reduced image) + 256x3 / 4x1 / 8x2 (compressed differential data 2 area) = 114 (Max)

  In method 1, one line of 256 pixels is read as it is. In method 2, since 256 pixels are read in the row direction, it is necessary for the reduced image to read four corresponding lines in the reduced image that has become 1/4 in the horizontal direction. At this time, if the pixel of the reduced image is represented by ● and the pixel that has been linearly complemented is represented by ○, there is a case where ● ○○○ ● ○○○. In this case, in order to obtain the left circle by linear interpolation, another pixel ● of the reduced image is required at the right end. Further, there is a case where it becomes XXXXXXXX. In this case, pixels ● of the reduced image are required at both ends. Therefore, it is necessary to read pixels of the reduced image of 16 pixels × 4 + 1 (or +2) = 65 (or 66) depending on which part of the line is read. For compressed differential data, there are cases where the read line requires only one access unit, and may straddle two access unit lines. For this reason, it is necessary to read a line of two access units at the maximum. In method 3, compression difference data is held in units of rectangles, but is also held in units of lines. Therefore, when reading is performed in units of lines, data held in units of lines is read. Therefore, it is the same as in the case of method 2.

Rectangle Read
Method 1 16x16 (original image) = 256
Method 2 5x16 (reduced image) + 256x3 / 4x1 / 8x16 (compressed differential data 16 areas) = 464 (Min)
6x16 (reduced image) + 256x3 / 4x1 / 8x32 (compressed differential data 32 areas) = 864 (Max)

  In method 2, when reading 16 × 16 blocks, the reduced image is read out only (4 + 1) × 16 rows depending on the position of the read block due to the linear interpolation as in the case of method 2 in the case of line reading. Or (4 + 2) × 16 rows are read out. Further, there are cases where the compressed differential data read line requires only one access unit in the horizontal direction and may cross over two access unit lines. In the case of reading 16 rows, if the rectangle crosses two access units, it is necessary to read 2 units × 16 rows, that is, 32 access units. In other words, although the compression unit is a line unit, the processing unit is 16 pixels × 16 pixels of a macroblock, and therefore, it is assumed that access is performed by 16 rows.

Method 3 5x16 (reduced image) + 256x3 / 4x1 / 8 (compressed differential data 1 area) = 104 (Min)
6x16 (reduced image) + 256x3 / 4x1 / 8x4 (compressed difference data 4 areas) = 192 (Max)

   In the method 3, the reduced image is the same as the method 2, but since the compressed differential data is held in the rectangular unit, the compressed differential data in the rectangular unit is read in the case of the rectangular read. For compressed differential data, the amount of read data differs depending on whether the read rectangular area is included in one holding unit block, spans two unit blocks, or spans four unit blocks.

 From the comparison result of the access amount, it is possible to reduce the memory access amount in both rectangular access and line access by using the method 3 of the present embodiment.

  Depending on the application of the image data, such as when the display size is sufficiently smaller than the original image size, there may be no problem in image quality even if the original image is not completely restored. In such an application, it is possible to further reduce the memory access amount by reading out only the reduced image and omitting the access to the compressed differential data. Depending on the request from the request issuer, there is a method of transmitting the reduced image data after enlarging it, or a method of transmitting the reduced image as it is.

  Since the compressed differential data has a variable size after compression in units of compression, there is a possibility that useless memory access may be required if the size is unknown. Therefore, if the size is calculated and stored after the compression processing and before storage, the necessary size can be known and unnecessary memory access can be avoided.

  When the difference data is compressed with a variable length code and stored in the memory, the storage position of the difference data corresponding to the designated area in the screen requested by the request issuer cannot be completely specified. In the case of a normal variable length code, since it is necessary to perform expansion processing from the beginning of the encoding unit, it is necessary to read out data in a region that is not actually used and perform expansion processing. According to the read method of the request issuer, the compressed differential data including the necessary area is read, and after decompression, the unnecessary part is discarded, so that the original image of only the necessary area can be restored.

  In general, image data processing is often performed in a predetermined order within a screen, and a continuous read request from a certain module often accesses a nearby region in the screen. The compressed differential data read in one request is data collected in a unit to be compressed, and may include an unnecessary part that is not used in the request, but is often necessary data in the next request. Therefore, if this difference data is stored in the built-in memory and there is a request for the difference data area in the built-in memory in the next request, access to the outside memory is not performed and the corresponding data is stored. The amount of memory access can be reduced.

  A plurality of built-in memories for storing the difference data may be provided for each read request source. For example, in an MPEG decoder or the like, it is efficient to have a differential data storage built-in memory for reading a reference image and a differential data storage built-in memory for display reading.

  As a method for storing the difference data in the built-in memory, the difference data after expansion or the compressed difference data before expansion may be used. If it is stored in the differential data format after decompression, it can be processed without performing the decompression process in the next access, and if it is stored in the format of compressed differential data before decompression, the corresponding amount of internal memory The capacity of can be reduced.

FIG. 4 is a block diagram of an MPEG decoder LSI to which the first embodiment of the present invention is applied.
The LSI 9 is composed of three blocks, an MPEG decoder unit 10, a display control unit 11, and a memory controller unit 12, and is connected to an external memory (SDRAM) 13 for storing a decoding result image and the like. The MPEG decoder unit 10 is a block that decodes an input MPEG stream and generates a decoded image. The resulting decoded image is written to the SDRAM 13 via the memory controller unit 12. During the decoding process, a past decoded image is read as a reference image and used as necessary. The display control unit 11 is a block that reads out the decoding result image generated by the MPEG decoder unit via the memory controller unit 12 and outputs it to the display device. The memory controller unit 12 is a block that manages access to the SDRAM 13 in response to requests from the MPEG decoder unit 10 and the display control unit 11.

  The memory controller unit 12 is provided with a request control unit 15 that receives requests from the MPEG decoder unit 10 and the display control unit 11. When receiving the request, the request control unit 15 sends a request to the request conversion unit 19 to generate control signals to the image data compression unit 16, the rectangular image data expansion unit 17, and the line image data expansion unit 18.

  When a write request is issued from the MPEG decoder unit 10, the image data compression unit 16 receives the write data in the image storage memory 20 and inputs the data to the image reduction circuit 21. The image reduction circuit 21 reduces the image and writes the reduced image to the SDRAM 13 via the SDRAM access control unit 37. The reduced image is enlarged by the image enlargement circuit 22 and the difference from the original image is taken by the subtractor 23. The difference data is stored in the difference data storage memory 24 and input to the rectangular difference data compression circuit 25 and the line difference data compression circuit 26. In the rectangular difference data compression circuit 25 and the line difference data compression circuit 26, the difference data is converted into compression difference data in units of rectangles and compression difference data in units of lines, and is transferred to the SDRAM 13 via the SDRAM access control unit 37. Written.

When a rectangular read request is issued from the MPEG decoder unit 10, the rectangular image data expansion unit 17 reads reduced image data and rectangular compression difference data from the SDRAM 13 via the SDRAM access control unit 37. The reduced image data is stored in the reduced image storage memory 27 and then enlarged in the rectangular image enlargement circuit 29. The rectangular compressed differential data is stored in the differential data storage memory 28 and then expanded in the rectangular differential data expansion circuit 30. The enlarged reduced image and the developed rectangular difference data are added by the adder 35 and sent to the MPEG decoder unit 10 as rectangular read data.

  When a line read request is issued from the display control unit 11, the line image data expansion unit 18 reads the reduced image and the line compression difference data from the SDRAM 13 via the SDRAM access control unit 37. The reduced image data is stored in the reduced image storage memory 31 and then enlarged in the line image enlargement circuit 33. The line compressed differential data is stored in the differential data storage memory 32 and then expanded by the line differential data expansion circuit 34. The enlarged reduced image and the developed line compression difference data are added by the adder 36 and passed to the display control unit 11 as line read data.

In addition, the rectangular read request is issued from the MPEG decoder unit 10 and the line read request is issued from the display control unit 11. This is done by referring to the reference image in units of rectangles when displaying by the MPEG. This is because when an image is generated, the image is referred to in units of lines. However, in general, the decoder may perform line read access, or the display control unit may perform rectangular read access.
FIG. 4 shows the configuration of the decoder, but the configuration of the memory controller unit 12 is the same in the encoder.

FIG. 5 shows a processing flow of the memory controller according to the first embodiment of the present invention.
First, in step S10, a request from each block is received. In step S11, it is determined whether there are a plurality of requests. If a plurality of requests are received, in step S12, the request is arbitrated and one request to be processed is selected. In step S13, the contents of the selected request are analyzed, and any one of a write request, a rectangular read request, and a line read request is determined, and processing for each is performed.

  In step S14, it is determined whether or not the request is a write request. If the request is not a write request, the process proceeds to step S26. In the case of a write request, in step S15, first, write data (original image data) is acquired from the request request source. Next, in step S16, a reduced image of the original image is generated, and in step S17, an enlarged image of the reduced image is generated. Thereafter, in step S18, a difference is taken between the original image and the generated enlarged image after reduction, and difference data is created. In step S19 and step S20, the difference data is subjected to two types of compression: rectangular compression and line compression. In step S21, the generated reduced image, rectangular compression difference data, and line compression difference data are respectively written in the SDRAM. However, since the compression difference data is not necessarily completed with only one write request data depending on the unit to be compressed, writing is performed when the data of the unit to be compressed is collected. That is, in step S22, it is determined whether or not data for the rectangular difference data writing unit has been accumulated. If accumulated, the rectangular compressed difference data is written to the SDRAM in step S23. In step S24, it is determined whether or not data has accumulated for the unit for writing differential data for lines. If so, the compressed differential data for lines is written to SDRAM in step S25, and the process ends.

In step S26, it is determined whether the request is a rectangular read request. If it is not a rectangular read request, the process proceeds to step S33. In the case of the rectangular read request, in step S27 and step S28, the reduced image area and the rectangular compressed difference data area to be read from the SDRAM are determined from the request contents, and necessary data is read from the SDRAM. The reduced image is one rectangular area, but the compression difference data for the rectangle may be spread over a plurality of areas, and may be divided into a plurality of requests. Next, in steps S29 and S30, the read reduced image is enlarged, and the rectangular compression difference data is expanded, and in step S31, both are added to restore the original image. At this time, since the read rectangular compression difference data may include data outside the requested rectangular area from the request source, it is necessary to cut out and use only the necessary area after expansion. Step S32: The original image restored last is transmitted to the request request source.

  In the case of a line read request, the reduced image is read from the SDRAM in step S33, and the line compression difference data is read from the SDRAM in step S34. In step S35, the reduced image is enlarged, and in step S36, the line compression difference data is expanded. In step S37, the enlarged image and the developed difference data are added to restore the original image. The restored original image is transmitted to the request source.

  Regarding the read of compressed difference data, the read size cannot be determined unless the data size after compression is known. However, as a method of avoiding unnecessary memory access, the data size is calculated at the time of writing, and the size at the beginning of the read data is set. There are a method of writing the data, and a method of separately holding a table in which the data size after compression is written.

  Also in the case of a line read request, the reduced data area and the compressed data area for lines to be read from the SDRAM are determined from the request contents, and the necessary data is read from the SDRAM. A reduced image is a one-line area, but the line compression difference data may be spread over a plurality of areas, and may be divided into a plurality of requests. Next, the read reduced image is enlarged, the line compression difference data is expanded, and both are added to restore the original image. At this time, since the read line compression difference data may include data outside the requested line area from the request source, it is necessary to cut out and use only the necessary area after expansion. The last restored original image is transmitted to the request request source.

  As a reduced image generation method, subsampling, averaging processing, reduction filter processing, and the like are conceivable. In addition, as a method for generating an enlarged image, pixel copy, averaging processing, enlargement filter processing, and the like can be considered, and any of them may be used. In addition, as for the processing method of the rectangular boundary, there are a method of performing enlargement processing with data within the rectangular size of one request at the time of writing and a method of using peripheral pixels outside the rectangular size. Since an MPEG decoder normally performs write processing in units of macroblocks (16 pixels × 16 pixels), there is one method for performing reduced image generation processing and enlarged image generation processing using only data within the rectangle.

FIG. 6 shows an example of a method of handling a rectangular boundary area when generating an enlarged image.
The reduced image generation method is horizontal subsampling (no vertical reduction), and the enlarged image generation method is linear interpolation. When generating an enlarged image by linear interpolation, a pixel at horizontal position 16 is required to generate pixels at horizontal positions 13, 14, and 15. This is a pixel that is not included in the corresponding macroblock. For this reason, it is treated as a copy of horizontal position 12 here. That is, in linear interpolation, the pixel values at the horizontal positions 13, 14, and 15 are equal to the pixel value at the horizontal position 12. Another method is to use the pixel value of the next macroblock.

  In this method, when creating an enlarged image, it is necessary to wait for the data input of the next macroblock, so it is not possible to generate difference data at horizontal positions 13, 14, and 15 by processing the corresponding macroblock. is required.

FIG. 7 is a diagram showing an image restoration image at the time of rectangular read access.
As shown in FIG. 7A, a rectangular read request area is a macroblock (0, 0,
0) and (1, 0). First, a reduced image including a request area and differentially compressed data are read, and enlarged and expanded, respectively. Then, only the part of the request area is cut out and added to obtain a restored image. Here, the request area part is cut out after obtaining the restored image or before or after the restoration image, but either may be used. However, it is preferable to cut out before the restoration image is obtained by the addition processing because the processing amount of the addition processing is reduced.

FIG. 8 shows a processing flow of a request generated by the memory controller for one request from the access request source.
In this flow, the reduced image generation method is vertical 1/1 (no reduction), horizontal 1/4 thinning, the enlarged image generation method is linear interpolation, and the rectangular compression difference data area unit is 16 pixels x 16 pixel area The unit of the line compression difference data area is a unit of 256 pixels in the line direction. The processing of this flow is executed by the request conversion unit 19 in the block diagram of FIG. 4, but data processing such as image reduction / enlargement processing and differential data compression processing is performed by the image data compression unit 16, respectively. The rectangular image data developing unit 17 and the line image data developing unit 18 perform processing.

  In step S40, it is determined whether or not the request is a write request. Assuming MPEG decoder write processing, it is assumed that a write request is always made in units of macroblocks (16 × 16 pixels), and the request issuance order is also in macroblock processing order. When the upper left coordinates (16n, 16m) and the rectangular size (16, 16) are received as the write request parameters in step S41, the coordinates and size in the reduced image space are set to 1/4 in the horizontal direction. Therefore, in step S42, the upper left coordinates (4n, 16m) and the rectangular size (4, 16) are acquired. The memory controller performs a reduced image write process using the coordinates and size of the reduced image space as new parameters. Next, since the rectangular compressed difference data is written in units of 16x16 pixels, basically one area is written in one macroblock, but when creating difference data, the right adjacent macroblock Since the pixel value is required, the write processing is not performed on the leftmost macroblock on the screen, but is performed with a shift of 1 macroblock. Therefore, in step S43, it is determined whether or not the current macroblock is the leftmost macroblock on the screen. If it is not the leftmost macroblock, the rectangular compression difference data (n−1, m) is written to the SDRAM in step S44. In the macro block at the right end of the screen, write processing for two areas is performed. Therefore, in step S45, it is determined whether or not the current macroblock is the rightmost macroblock on the screen. In the case of the rightmost macroblock, rectangular compression difference data (n, m) is written to the SDRAM in step S46.

  Next, since the compressed differential data for lines is compressed in units of 256 pixels in the line direction, the compressed differential data for 16 lines is written for every 16 macroblocks. Here, since the pixel of the right adjacent macroblock is required for enlarged image generation when creating the differential data, the writing process is performed with a shift of 1 macroblock (for example, the writing process of the differential data of 0 to 15 macroblocks (Done during block 16 processing). Therefore, in step S47, it is determined whether the current macroblock is the rightmost macroblock or n is a multiple of 16. If the determination in step S47 is Yes, the process enters the loop of step S48, and the line compression difference data (n / 16, 16m + i) is obtained in step S49 while i changes from 0 to 15 in increments of 1. Write to SDRAM.

In step S50, it is determined whether or not the request is a rectangular read request. In the case of a rectangular read request, it is assumed that there is a request with an arbitrary coordinate and size. In step S51, when the upper left coordinates (X0, Y0) and the rectangular size (SX, SY) are received as the parameters of the rectangular read request, the coordinates and size in the reduced image space are calculated. The necessary image area in the reduced image space is calculated from the upper left coordinates and lower right coordinates of the rectangular area. In step S52, the lower right coordinates (X1, Y1) are calculated as X1 = X0 + SX-1 and Y1 = Y0 + SY-1. Step S
In 53, the upper left coordinates (X0 ′, Y0 ′) and lower right coordinates (X1 ′, Y1 ′) of the rectangle in the reduced image are calculated. The calculation formula is as follows.
X0 '= X0 / 4
Y0 '= Y0
X1 '= (X1 + 3) / 4
Y1 '= Y1

Next, in step S54, the rectangular area coordinates (RX0, RY0) of the difference data of the compression unit including the upper left pixel are calculated. The calculation formula is as follows.
RX0 = X0 / 16
RY0 = Y0 / 16

Next, in step S55, the rectangular area coordinates (RX1, RY1) of the difference data in the compression unit including the lower right pixel are calculated. The calculation formula is as follows.
RX1 = X1 / 16
RY1 = Y1 / 16

  In step S56, a reduced image having upper left coordinates (X0 ', Y0') and size (X1'-X0 '+ 1, Y1'-Y0' + 1) is read. Step S57 is a loop for changing i from RX0 to RX1 in increments of 1. Step S58 is a loop that changes j from RY0 to RY1 in increments of 1. In the loop of steps S57 and S58, the rectangular compression difference data (i, j) is read in step S59.

  In the original image, if the left end coordinate of the rectangle is not a multiple of 4, it is necessary from the pixel on the left side of the left end coordinate in the reduced image for linear interpolation. Similarly, if the right end coordinate is not a multiple of 4, it is reduced. In the image, it is necessary from the pixel on the right side of the right end coordinate. Therefore, the request parameters in the reduced image space are the upper left coordinates (X0 / 4, Y0) and the size ((X0 + SX + 2) / 4−X0 / 4 + 1, SY). Which rectangular area data is read from the rectangular compression difference data is calculated from the rectangular area including the upper left pixel and the rectangular area including the lower right pixel, and the read processing of the necessary number of rectangular compression difference data is performed. I do.

  In the case of a line read request, it is assumed that there is a request with an arbitrary coordinate and size. In step S60, the coordinates and size in the reduced image space when the left end coordinates (X0, Y0) and size SX are received as parameters of the line read request are calculated. If the required image area in the reduced image space is calculated from the left and right end coordinates in the same way as the rectangular lead, the request parameters on the reduced image are the left end coordinates (X0 / 4, Y0), size (X0 + SX +2) / 4-X0 / 4 + 1. The line area data to be read with respect to the line compression difference data is calculated from the line area including the left end pixel and the line area including the right end pixel, and the read processing of the required number of line compression difference data is performed. Do.

In step S61, the right end coordinate (X1, Y1) on the original image is calculated based on the following equation.
X1 = X0 + SX-1
Y1 = Y0

In step S62, the line left end coordinates (X0 ′, Y0 ′) and the line right end coordinates (X1 ′, Y1 ′) on the reduced image are calculated according to the following equations.
X0 '= X0 / 4
Y0 '= Y0
X1 '= (X1 + 3) / 4
Y1 '= Y1

In step S63, the coordinates (LX0, LY0) of the line area of the compression unit including the leftmost pixel are calculated based on the following formula.
LX0 = X0 / 256
LY0 = Y0

In step S64, the coordinates (LX1, LY1) of the line area of the compression unit including the right end pixel are calculated according to the following formula.
LX1 = X1 / 256
LY1 = Y1

  In step S65, a reduced image having the left end coordinates (X0 ′, Y0 ′) and size X1′−X0 ′ + 1 is read. Step S66 is a loop that changes k from LX0 to LX1 in increments of 1. In the loop of step S66, line compression difference data (k, LY0) is read in step S67.

  By applying a memory controller with the above functions, it is possible to reduce the amount of memory access to SDRAM.

  A second embodiment in which a mode for creating an enlarged image using only a reduced image at the time of read access will be described below. The block configuration is basically the same as in the above embodiment, but in the second embodiment, the request issuer uses both the reduced image and the difference data when issuing the request in the rectangular read request and the line read request. Or a method for passing as a parameter whether to use only a reduced image. For example, if the display image is large, the display control unit issues a request for a mode that uses both the reduced image and the difference data. If the display image is small, the display control unit issues a request for a mode for only the reduced image. The MPEG decoder unit issues a request for a mode that uses both reduced image and difference data when processing a P picture that will be used as a reference picture in the future, but error accumulation may occur when processing a B picture that is not used as a reference picture in the future. Since there is no request, a request for a reduced image only mode is issued.

FIG. 9 shows a processing flow of the memory controller of the second embodiment.
In FIG. 9, the same steps as those in FIG.
The rectangular write request, the rectangular read request with difference data, and the line read request are the same flow as in the first embodiment. For rectangular read requests and line read requests without differential data (only reduced images), it is not necessary to read the differential data from SDRAM, so after reading the reduced images from SDRAM, an enlarged image is generated and data is sent to the request issuer. Send.

  In FIG. 9, if it is determined in step S26 that the request is a rectangular read request, it is determined in step S70 whether or not to include the difference data. If difference data is included in step S70, the same processing as in FIG. 5 is performed after step S27. If the difference data is not included, the reduced image is read from the SDRAM in step S71, the reduced image is enlarged in step S72, and the read data is transmitted to the request source in step S73.

  Also, if it is determined in step S26 that the request is not a rectangular read request, it is determined in step S74 whether or not the reading includes difference data. If it is determined in step S74 that the reading includes difference data, the same processing as in FIG. 5 is performed after step S33. If it is determined that the reading does not include the difference data, the reduced image is read from the SDRAM in step S75, the reduced image is enlarged in step S76, and the read data is transmitted to the request source in step S77.

  By using a memory controller equipped with the above functions, it is possible to further reduce the memory access amount to the DRAM within a range in which the error is not conspicuous in accordance with the display image size or the like.

FIG. 10 is a block diagram of an MPEG encoder LSI to which the third embodiment of the present invention is applied.
This LSI is composed of four blocks, an image capturing unit 40, an MPEG encoder unit 41, a display control unit 42, and a memory controller unit 43. An external memory (SDRAM) 44 for storing images necessary for encoding processing is provided. Connected. The image capturing unit 40 captures an input image and writes the image to the SDRAM 44 via the memory controller unit 43. Image input is performed in raster scan order, and writing processing to the SDRAM 44 is in line units.

  The MPEG encoder unit 41 is a block that reads the captured input image through the memory controller unit 43, performs an encoding process according to MPEG, and generates an MPEG stream. During the encoding process, a past local decoded image is read as a reference image and used as necessary.

  The display control unit 42 is a block that reads a local decoded image generated by the MPEG encoder unit 41 or an input image captured by the image capturing unit via the memory controller unit 43 and outputs the read image to the display device.

The memory controller unit 43 is a block that manages access to the SDRAM 44 in response to requests from the image capturing unit 40, the MPEG encoder unit 41, and the display control unit 42.
The third embodiment is different from the first embodiment in that the circuit is shared and the past difference data is stored in the buffer and reused. The image reduction and enlargement method is the same as that in the first embodiment.

  The memory controller unit 43 is provided with a request control unit 45 that receives requests from the image capturing unit 40, the MPEG encoder unit 41, and the display control unit 42. When receiving the request, the request control unit 45 sends a request to the request conversion unit 46 to generate a control signal to the image data compression unit 47 and the image data expansion unit 48.

  When a line write request or a rectangular write request is issued from the image capturing unit 40 or the MPEG encoder unit 41, either the line write data or the rectangular write data is selected by the selector 50 of the image data compression unit 47. The image storage memory 51 receives line write data or rectangular write data and inputs the data to the image reduction circuit 52. The image reduction circuit 52 reduces the image and writes the reduced image in the SDRAM 44 via the SDRAM access control unit 65. The reduced image is enlarged by the image enlargement circuit 53, and the difference from the original image is taken by the subtractor 54. The difference data is stored in the difference data storage memory 55 and input to the difference data compression circuit 56. In the difference data compression circuit 56, regardless of whether the request is a line write request or a rectangular write request, the difference data is converted into both line-by-line compressed difference data and rectangular-unit compressed difference data, and the SDRAM access control unit 65 It is written in the SDRAM 44.

  When a rectangular read request is issued from the MPEG encoder unit 41, the image data expansion unit 48 reads the reduced image data and the rectangular compressed difference data from the SDRAM 44 via the SDRAM access control unit 65. The reduced image data is stored in the reduced image storage memory 57 and then enlarged in the image enlargement circuit 59. The image enlargement circuit 59 is hardware that supports both lines and rectangles, and performs either processing under the control of the request conversion unit 46. The rectangular compressed differential data is stored in the differential data storage memory 58 and then expanded in the differential data expansion circuit 60. The differential data expansion circuit 60 is hardware that supports both lines and rectangles, and performs either process under the control of the request conversion unit 46. The developed rectangular difference data is stored in the past difference data buffer 61 that is used for both the rectangle and the line, and is input to the selector 62. Either the expanded rectangular difference data or the past difference data stored in the past difference data buffer 61 is selected by the selector 62, and is added to the enlarged reduced image by the adder 35 to obtain the rectangular read data. The data is sent to the MPEG encoder 41 via the switch 64.

The case where the line read request is issued from the display control unit 42 is the same as the case where the rectangular read request is issued from the MPEG encoder unit 41.
In the third embodiment, the difference data compression circuit 56, the reduced image storage memory 57, the difference data storage memory 58, the image enlargement circuit 59, and the difference data expansion circuit 60 are hardware capable of processing both rectangular and line. It is configured as wear.

  Although FIG. 10 is a configuration diagram of the encoder, the configuration of the memory controller unit 43 is the same even in the case of the decoder.

FIG. 11 shows a processing flow of the memory controller according to the third embodiment of the present invention.
In FIG. 11, the same steps as those in FIG. 5 are denoted by the same reference numerals, and only a brief description will be given.
First, in step S10, a request from each functional block is received. When a plurality of requests are received (step S11), request arbitration is performed (step S12), and one request to be processed is selected. The contents of the selected request are analyzed (step S13), and one of a rectangular write request, a line write request, a rectangular read request, and a line read request is determined, and processing is performed on each of them.

  Since the processing flow for the line write request and the rectangular write request is the same, in FIG. 11, a single flow is shown as the case of the write request. First, write data (original image data) is acquired from the request request source (step S15). Next, a reduced image of the original image is generated (step S16), and an enlarged image of the reduced image is generated (step S17). Thereafter, a difference is taken between the original image and the generated enlarged image after reduction to create difference data (step S18). The difference data is subjected to two types of compression: rectangular compression and line compression (steps S19 and S20). The generated reduced image, rectangular compression difference data, and line compression difference data are each written to the SDRAM (step S21). However, with respect to the compressed differential data, the compression is not necessarily completed with only one write request data depending on the unit to be compressed, and thus writing is performed when the data of the unit to be compressed is collected (steps S22 to S25). ). In the third embodiment, circuits such as each storage memory, an image reduction / enlargement circuit, and a differential data compression circuit are shared for lines and rectangles. The differential data compression circuit for line and rectangle is used in a time division manner for one write request.

  The basic processing flow for a line read request and a rectangular read request is the same. First, the reduced image area and the rectangular or line compression difference data area to be read from the SDRAM are determined from the request contents, and necessary data is read from the SDRAM (steps S27 and S33). The reduced image needs to read one area of a rectangle or a line, but the compressed difference data for a rectangle or line has a function of storing past difference data in a buffer in the third embodiment. Read access does not always occur. When necessary difference data is stored in the past difference data buffer (steps S80 and S84), only the reduced image is read and enlarged without performing the difference data read access to the SDRAM, and the past difference data buffer is read out. The original image is restored by adding the corresponding difference data. Then, the restored image is transmitted to the request request source. If the necessary difference data is not stored (steps S80 and S84), the reduced image and the compressed difference data are read (steps S81 and S85), and are enlarged and expanded (steps S82 and S86). Is added to restore the original image (steps S31 and S37), and the restored image is transmitted to the request request source (steps S32 and S38). The developed difference data is stored in the past difference data buffer for future read requests (steps S83 and S87). In the third embodiment, circuits such as each storage memory, an image reduction / enlargement circuit, and a difference data expansion circuit are shared for lines and rectangles. Also, the difference data for lines and rectangles are stored in separate areas in the past difference data buffer so that past difference data can be used for each line, rectangle, and request. It has become.

  By using a memory controller equipped with the above functions, the difference data read in the past can be reused to reduce the amount of difference data read, thus reducing the amount of memory access to SDRAM. Is possible.

  As a fourth embodiment of the present invention, an example in which another reduction / enlargement method is used for the configuration of the third embodiment will be described. The block configuration is basically the same as in the third embodiment, but in the fourth embodiment, the reduced image generation method is thinning out the vertical 1/2 and horizontal 1/2, and the enlarged image generation method is linear interpolation. The rectangular compression difference data area unit is 16 pixels × 16 pixel area, and the line compression difference data area unit is 256 pixels in the line direction.

FIG. 12 shows a thinning method from the original image to the reduced image in the fourth embodiment.
FIG. 12A is a diagram showing the pixels to be thinned out and the pixels that remain in the original image, and FIG. 12B shows the state of the pixels of the reduced image after reduction. In the fourth embodiment, in addition to thinning in the horizontal direction, thinning in the vertical direction is performed, and enlargement processing is performed by linear interpolation. For this reason, in the line read of the odd lines of the original image, data of two lines adjacent vertically are necessary. Normally, the image data to the display device is output in order from the upper line on the screen to the lower line, so the line read access amount of the reduced image can be saved by storing the past line in the buffer. Can be reduced. In the fourth embodiment, it is assumed that read access is performed in order from the upper end (line 0) of the screen.

FIG. 13 shows a procedure of line read request processing according to the fourth embodiment.
First, a line read request for line n is received (step S90). If line n is the first line on the screen (line 0 in the original image) (step S91), the first line (line 0) in the reduced image is read (step S91). Step S92). Since line 0 of the original image is a line corresponding to line 0 of the reduced image, line 0 of the original image space is generated by enlargement processing (step S93), and the reduced image line 0 is stored in the buffer (step S94). ). In the fourth embodiment, this buffer secures an area for holding a past line in the reduced image storage memory separately from the buffer area for one request, and holds line data. Shall. In step S102, the line n (the currently processed line) is expanded in the horizontal direction, and the subsequent processes of reading, expanding, and adding the compressed difference data (steps S103 to S108) are the same as in the third embodiment. (Similar to the processing from steps S80 to S32 and steps S84 to S38 in FIG. 11). Next, when a read request for line 1 is received, the flow proceeds to the flow of odd lines (step S95). In the case of an odd-numbered line, the corresponding line is not stored as a reduced image, so it is necessary to create data by linear interpolation from vertically adjacent lines. Since the upper adjacent line (in this case, line 0 of the reduced image) has already been stored in the past line buffer area, the read processing to SDRAM is not performed, and the lower adjacent line (in this case, line 1 of the reduced image) is changed. Read to generate an odd line of the original image (in this case, line 1 of the original image). The newly read lower adjacent line is stored in the past line buffer area, and is used when processing the next line. When the line read request for line 2 is received, the flow moves to the even line flow. For even lines other than the first line, the corresponding reduced space line is already stored in the past line buffer area during the previous odd line processing, so read access to the SDRAM is not performed and only the data in the buffer is used. It is possible to restore the corresponding line.

  That is, if it is determined in step S95 that the current line is an odd line, the line (n + 1) / 2 of the reduced image is read in step S96, and the previous line buffer is read in step S97. Get line (n-1) / 2. In step S98, the original image line n is generated from the reduced image lines (n-1) / 2 and (n + 1) / 2. In step S99, the line (n + 1) / 2 is stored in the past line buffer, and the process proceeds to step S102. If it is determined in step S95 that the current line is not an odd line, line n / 2 is acquired from the past line buffer in step S100, and the original image line n is reduced from the reduced image line n / 2 in step S102. And proceeds to step S102.

  By holding the past line data in the past line buffer area in the flow as described above, it is possible to reduce the amount of reduced image read at the time of an odd line read request when there is vertical thinning.

  By using the embodiment of the present invention, it is possible to reduce the memory bus bandwidth that is actually used in an apparatus that requires a high memory bus bandwidth, such as large-screen MPEG decoding / encoding. In the embodiment of the present invention, the memory bus bandwidth is reduced in both the MPEG read, the rectangular read access used for encoding processing, and the line read access used for display control, so the memory bus bandwidth reduction effect is high. Application to various devices is possible. When the memory bus band to be used is halved by applying the embodiment of the present invention, the memory frequency can be halved or the number of memories can be halved. If the same memory bus bandwidth is used, a larger screen can be processed. If the memory bus bandwidth is halved, it is possible to realize processing with twice the screen size or twice the frame rate.

In addition to the above embodiment, the following supplementary notes are disclosed.
(Appendix 1)
Memory,
Reduced image data is generated to generate reduced image data, and is stored in the memory, and the reduced image data is enlarged to generate enlarged image data, and a plurality of different difference data between the image data and the enlarged image data are set. An image data compression unit that generates a plurality of compressed differential data and stores them in the memory;
An image data decompression unit that reads out the compressed difference data according to the content indicated by the request request, reads out the reduced image data, and generates image data decompressed based on the compressed difference data and the reduced image data; ,
An image processing apparatus comprising:
(Appendix 2)
The image processing apparatus according to appendix 1, wherein the plurality of units include a rectangular unit and a line unit.
(Appendix 3)
The image processing apparatus according to appendix 1, wherein a method of generating the reduced image data from the image data is based on thinning.
(Appendix 4)
The image processing apparatus according to attachment 1, wherein the method for enlarging the reduced image is based on linear interpolation.
(Appendix 5)
A buffer for storing the reduced image data and the compressed differential data read out before;
When the reduced image data or the compressed difference data to be read is already stored in the buffer, the image data decompression unit does not access the memory and stores the reduced image data or the compressed difference data to be read. The image processing apparatus according to appendix 1, wherein the image processing apparatus is read from the buffer.
(Appendix 6)
The image processing apparatus according to appendix 1, wherein the compression method of the compressed difference data is based on variable length coding.
(Appendix 7)
The image processing apparatus according to appendix 1, wherein the compressed differential data is stored in the memory in a designated area in units of compression.
(Appendix 8)
The image processing apparatus according to appendix 1, further comprising means for calculating and recording a data size in the unit of compression in storing the compressed differential data in the memory (appendix 9)
The image processing apparatus according to claim 1, wherein the compressed differential data is stored in the memory in a unit different from the unit of compression.
(Appendix 10)
The image processing apparatus according to appendix 9, wherein the unit different from the unit of compression is a macroblock unit.
(Appendix 11)
The compressed difference data is read out, the reduced image data is read out, and expanded image data is generated based on the compressed difference data and the reduced image data, or only the reduced image data is read out, and the reduced image data is read out The image processing apparatus according to appendix 1, further comprising an image data expansion unit that can select whether to generate image data expanded based on the image data.
(Appendix 12)
Reduce the image data to generate reduced image data,
Enlarging the reduced image data to generate enlarged image data,
Compressing difference data between the image data and the enlarged image data in a first unit to generate first compressed difference data;
Compressing difference data between the image data and the enlarged image data in a second unit to generate second compressed difference data;
Based on the request request, select either the first compressed differential data or the second compressed differential data,
An image processing method, wherein decompressed image data is generated based on the selected compressed difference data and the reduced image data.

It is a figure which shows the memory storage image of a reduction image and compression difference data. It is a figure which shows the example of a difference data compression process order. It is a figure explaining the mode of the reduction of the data access amount to a memory. 1 is a block configuration diagram of an MPEG decoder LSI to which a first embodiment of the present invention is applied. FIG. It is a figure which shows the processing flow of the memory controller of the 1st Embodiment of this invention. It is a figure which shows an example of the handling method of the rectangular border area at the time of enlarged image production | generation. It is a figure which shows the image restoration image at the time of a rectangular read access. It is a figure which shows the processing flow of the request which a memory controller produces | generates with respect to one request from an access request source. It is a figure which shows the processing flow of the memory controller of 2nd Embodiment. FIG. 9 is a block configuration diagram of an MPEG encoder LSI to which a third embodiment of the present invention is applied. It is a figure which shows the processing flow of the memory controller of the 3rd Embodiment of this invention. It is a figure which shows the thinning-out method from the original image to a reduction image in 4th Embodiment. It is a figure which shows the procedure of the line read request process of 4th Embodiment.

Explanation of symbols

10, 41 MPEG decoder unit 11, 42 Display control unit 12, 43 Memory controller unit 13, 44 SDRAM
15, 45 Request control unit 16, 47 Image data compression unit 17 Rectangular image data development unit 18 Line image data development unit 19, 46 Request conversion unit 20, 51 Image memory 21, 52 Image reduction circuit 22, 53 Image enlargement circuit 23, 54 Subtractors 24 and 55 Difference data storage memory 25 Difference data compression circuit (for rectangle)
26 Differential data compression circuit (for line)
27, 31, 57 Reduced image storage memory 28, 32, 58 Difference data storage memory 29 Image enlargement circuit (for rectangle)
30 Differential data expansion circuit (for rectangle)
33 Image enlargement circuit (for line)
34 Differential data expansion circuit (for line)
35, 36, 63 Adder 37 SDRAM access control unit 40 Image capture unit 48 Image data development unit 50 Selector 56 Differential data compression circuit (for rectangle / line)
59 Image enlargement circuit (for rectangle / line)
60 Differential data expansion circuit (for rectangle / line)
61 Past difference data buffer (rectangular / line)
62 selector 64 switch

Claims (5)

Memory,
Reduced image data is generated to generate reduced image data, and is stored in the memory, and the reduced image data is enlarged to generate enlarged image data, and a plurality of different difference data between the image data and the enlarged image data are set. An image data compression unit that generates a plurality of compressed differential data and stores them in the memory;
An image data decompression unit that reads out the compressed difference data in units indicated by a request request, reads out the reduced image data, and generates image data decompressed based on the compressed difference data and the reduced image data;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the plurality of units include a rectangular unit and a line unit. A buffer for storing the reduced image data and the compressed differential data read out before;
When the reduced image data or the compressed difference data to be read is already stored in the buffer, the image data decompression unit does not access the memory and stores the reduced image data or the compressed difference data to be read. The image processing apparatus according to claim 1, wherein the image processing apparatus is read from the buffer.   The image processing apparatus according to claim 1, wherein the compression difference data is stored in the memory in a designated area in the compression unit. Reduce the image data to generate reduced image data,
Enlarging the reduced image data to generate enlarged image data,
Compressing difference data between the image data and the enlarged image data in a first unit to generate first compressed difference data;
Compressing difference data between the image data and the enlarged image data in a second unit to generate second compressed difference data;
Based on the request request, select either the first compressed differential data or the second compressed differential data,
An image processing method, wherein decompressed image data is generated based on the selected compressed difference data and the reduced image data.