JP2005269620A - Image compressing/expanding method, image compressing apparatus, image expanding apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve a data compression ratio in a data compressing/expanding method utilizing a down sampling process. <P>SOLUTION: An image data is divided into many DS areas in a DS area dividing portion 130, and an unimportant DS area is converted into a reduced data consisting of a block size by the down sampling at a down sampling portion 131. Rearranging portion 132 arranges reduced data of 2 or more in one DS area, inserts a file data into other parts in the DS area, and reduces a size of the entire image by eliminating the DS area where the reduced data is not arranged. The image data reduced like this is input in a JPEG encoder 14, and a DCT processing and a quantization processing or the like each block are conducted. The image data is restored to its original arrangement at a restoration time after it is expansion-processed by a JPEG decoder, the reduced data is interpolation-processed, and the image data is restored. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image compression / decompression method, an image compression device, and an image expansion device. More specifically, the present invention relates to an improvement in an image compression / decompression method such as JPEG in which orthogonal transformation and quantization processing are performed for each block area into which image data is divided. About.

  As a compression / decompression method of image data composed of still images, the JPEG (Joint Photographic Expert Group) standard standardized by CCITT (International Telegraph and Telephone Consultative Committee) and ISO (International Standards Organazasion) is widely known. In this JPEG standard, a compression method of image data and an expansion method thereof are defined by dividing a frame image into a plurality of blocks with 8 × 8 pixels as one block and converting spatial coordinates into frequency coordinates. Yes.

  A data compressor (hereinafter referred to as a JPEG compressor) according to the JPEG standard divides input image data into a number of blocks, and each block performs DCT (Discrete Cosine Transform) processing and quantization processing. Done. In this quantization process, a value obtained by multiplying the data defined for each DCT coefficient by the quantization table by the quantization factor Q is used as the quantization step width. The amount of data is irreversibly reduced by quantizing the DCT coefficient obtained by the DCT processing with the quantization step width. Thereafter, entropy encoding processing using run length processing, difference processing, Huffman encoding processing, and the like is performed, and compressed image data is generated. This encoding process is a process for reversibly reducing the amount of data.

  On the other hand, a data decompressor according to the JPEG standard (hereinafter referred to as a JPEG decompressor) performs the reverse process of the JPEG compressor and restores image data. That is, the input compressed image data is decoded and dequantized using the same quantization table and quantization factor Q. Thereafter, the inverse DCT processing unit performs inverse DCT transform, and the image data is restored by combining the blocks.

  In the above-described JPEG compressor, when the data compression rate is to be increased, it is necessary to change the quantization table or the quantization factor Q to increase the quantization step width. However, if a large amount of data is reduced in the quantization process, which is an irreversible process, the quality of the restored image data is significantly degraded. Moreover, since this quality deterioration occurs over the entire screen, there is a problem that even if there are important and non-important areas in the image, the image quality is uniformly deteriorated in both areas. .

  In order to solve such a problem, a compression processing method has been conventionally proposed in which the quality after restoration is different for each region in an image (for example, Patent Document 1). Patent Document 1 discloses a data compression processor including a mask circuit that masks DCT coefficients before quantization processing. In this data compression processor, the mask used in the mask circuit is different for each region, so that the important region is encoded with high image quality, and the non-important region is encoded with low image quality.

However, in this data compression processor, since it is necessary to perform mask processing during the DCT processing and quantization processing which are a series of processing, it is possible to use a general-purpose JPEG compression processor such as a JPEG chipset. There is a problem that the data compression processor becomes expensive. Further, since the mask process is a process in a block, there is a limit to the amount of data that can be reduced.
Japanese Patent Laid-Open No. 06-054310

  As one proposal for solving the above-mentioned problems, the inventors of the present application have filed a prior patent application (Japanese Patent Application No. 2003-43367). This prior application discloses a technique in which image data is divided into a plurality of small regions and downsampling processing is performed on some of the small regions as preprocessing of JPEG compression processing. This down-sampling process is a process for reducing the image data in the divided small area by down-sampling to match the block and inserting fill data in the remaining part in the area. Using this method, the amount of data can be reduced by using a general-purpose JPEG compression processor while varying the quality for each region in the image.

  The present invention has been made in view of the above circumstances, and an object thereof is to improve a data compression rate in a data compression / decompression method using downsampling processing. In addition, a general-purpose data compressor and data decompressor can be used, and the object is to improve the data compression rate in an image compression / decompression method that reduces the amount of data by changing the quality for each region in an image. . In particular, an object is to achieve a higher compression rate by using a JPEG compressor and a JPEG decompressor.

  The image compression / decompression method according to the first aspect of the present invention includes an image compression method and an image expansion method. This image compression method includes a downsampling area dividing step, a downsampling step, a rearrangement step, and a data compression step. The image expansion method includes a data expansion step, an arrangement restoration step, and an interpolation step.

  The downsampling area dividing step divides the image data into downsampling areas having n times the block size (n is an integer of 2 or more). In the downsampling step, downsampling is performed on at least a part of the downsampling region to reduce the number of pixels, and reduced data having m times the block size (m is a natural number smaller than n) is generated. In the rearrangement step, two or more pieces of reduced data are arranged in one down-sampling area, fill data is inserted into the pixels left in the down-sampling area, and a down-sampling area where reduced data is not arranged is removed. . In the data compression step, the image data on which the rearrangement step has been performed is divided into block regions having the block size, and compression processing including orthogonal transformation and quantization processing for each block region is performed to generate compressed image data. To do. With such a configuration, the image data can be spatially compressed as preprocessing of the data compression step by downsampling and rearranging a part of the image region.

  In the data decompression step, decompression processing including inverse quantization processing and inverse orthogonal transform for each block area is performed on the compressed image data to restore the image data. The arrangement restoration step restores the arrangement of the reduced data included in the restored image data. In the interpolation step, pixel data is interpolated for each reduced data included in the restored image data to restore image data including a downsampling area. With such a configuration, the image data that has been spatially compressed can be restored by post-processing of the data decompression step.

  An image compression apparatus according to a second aspect of the present invention includes a downsampling area dividing unit that divides image data into downsampling areas each having an integral multiple of the block size, and performs downsampling on at least a part of the downsampling areas to obtain the number of pixels Downsampling means for generating reduced data having a block size, two or more reduced data are arranged in one downsampling area, and fill data is inserted into the remaining pixels in the downsampling area In addition, a rearrangement unit that removes a downsampling area where reduced data is not arranged, and image data generated by the rearrangement unit are divided into block areas having the block size, and orthogonal transformation and quantization for each block area Perform compression processing including processing It constructed a data compressing means for generating compressed image data Te.

  In addition to the above-described configuration, the image compression apparatus according to the third aspect of the present invention is configured such that the rearrangement unit arranges the reduced data at a predetermined position in the down-sampling area, leaving an identification area having a block size. Is done. With such a configuration, fill data is inserted at a predetermined position in the downsampling area where the reduced data is arranged. Therefore, at the time of image expansion, it is possible to identify a downsampling area where reduced data is arranged based on the fill data.

  In addition to the above configuration, the image compression apparatus according to the fourth aspect of the present invention is configured such that the rearrangement unit adds a down-sampling area composed of fill data to the image data, and shapes the image data into a rectangular shape. Is done. With such a configuration, general-purpose means such as a JPEG compressor can be used as the data compression means.

  An image decompressing apparatus according to a fifth aspect of the present invention rearranges reduced data consisting of an integral multiple of a block size obtained by down-sampling a part of image data together with fill data, and further, for each block area having the block size. An image decompressing device that decompresses compressed image data that has undergone compression processing including orthogonal transformation and quantization processing of the image, and for decompressing the compressed image data, including inverse quantization processing and inverse orthogonal transformation for each block region The data decompression means for restoring the image data, the arrangement restoration means for restoring the arrangement of the reduced data based on the fill data contained in the restored image data, and the restoration data included in the restored image data Interpolation means for interpolating pixel data and restoring image data before downsampling for each reduced data .

  In the image compression apparatus according to the sixth aspect of the present invention, in addition to the above configuration, the rearrangement unit causes the average value of the pixel data in the block area in which the fill data in the downsampling area is not inserted to be a predetermined distance from the fill data. Is not configured, the pixel data in the block area is changed. With such a configuration, it is possible to prevent the image data after the decompression process from matching with the fill data and being erroneously determined as fill data.

  In the image compression apparatus according to the seventh aspect of the present invention, in addition to the above configuration, the fill data is configured as a median value of pixel data. With such a configuration, it is possible to easily prevent erroneous determination of fill data when level shift is performed in the data compression means.

  In the image compression apparatus according to the eighth aspect of the present invention, in addition to the above configuration, the predetermined distance is determined based on a quantization step width for a DC coefficient in the data compression means. With such a configuration, even when the quantization table or the quantization factor can be changed, it is possible to prevent erroneous discrimination of fill data.

  In the image compression apparatus according to the ninth aspect of the present invention, in addition to the above configuration, the downsampling area dividing unit divides the image data into two or more downsampling areas having different sizes, and the downsampling unit includes the downsampling area. Depending on the size, downsampling with different reduction rates is performed to generate reduced data of the same size. With such a configuration, downsampling with different reduction rates can be used in combination, and the data compression rate can be further improved.

  In the image compression apparatus according to a tenth aspect of the present invention, in addition to the above-described configuration, the down-sampling means performs down-sampling for at least a part of the down-sampling region to generate intermediate reduced data having a block size, The intermediate reduced data generated by the downsampling is collected to generate a new downsampling area, and the new downsampling area is downsampled again to generate reduced data having a block size. The With such a configuration, downsampling with different reduction ratios can be easily combined and used with such a configuration.

  An image compression apparatus according to an eleventh aspect of the present invention performs compression processing of image data composed of luminance data and color difference data, and includes downsampling area dividing means, luminance downsampling means, luminance data rearranging means, and color difference downgrading means. A sampling unit, a color difference data rearrangement unit, and a data compression unit are provided. The down-sampling area dividing means divides the luminance data constituting the image data into luminance down-sampling areas consisting of integer multiples of the block size, and the color difference down-sampling areas consisting of integer multiples of the block size of the color difference data constituting the image data Divide into

  The downsampled luminance data is processed by luminance downsampling means and luminance data rearrangement means. The luminance downsampling means downsamples the luminance data for at least a part of the luminance downsampling areas to reduce the number of data, and generates reduced luminance data composed of an integral multiple of the block size. The luminance data rearrangement unit arranges two or more pieces of reduced luminance data in one luminance downsampling area, and inserts fill data into the remaining pixels in the luminance downsampling area. At that time, the reduced luminance data is arranged at a predetermined position in the luminance down-sampling area, leaving an identification area having a block size.

  On the other hand, the downsampled color difference data is processed by the color difference downsampling means and the color difference data rearranging means. The chrominance down-sampling means reduces the number of data by down-sampling the chrominance data for the chrominance down-sampling region determined based on the luminance down-sampling region where the down-sampling is performed, and is a reduction that is an integral multiple of the block size Generate color difference data. The color difference data rearrangement unit arranges two or more reduced color difference data in one color difference downsampling area.

  With such a configuration, it is not necessary to arrange an identification area in the color difference downsampling area where the reduced color difference data is rearranged, and the compression efficiency of image data composed of luminance data and color difference data can be improved.

  An image compression apparatus according to a twelfth aspect of the invention includes, in addition to the above configuration, color difference data dividing means for dividing a color difference down-sampling area that has not been down-sampled into the same size as reduced color difference data and generating divided color difference data. And the color difference data rearranging means is configured to rearrange the reduced color difference data and the divided color difference data.

  With such a configuration, not only reduced data but also non-reduced data can be rearranged for color difference data, so that the compression efficiency of image data composed of luminance data and color difference data can be further improved.

  According to the present invention, the data compression rate in the data compression / decompression method using downsampling processing can be improved. In addition, general-purpose data compressors and data decompressors can be used, and the data compression rate in the image compression / expansion method that reduces the data amount by changing the quality for each region in the image can be improved. In particular, it is possible to achieve higher compression ratios using JPEG compressors and JPEG decompressors.

Embodiment 1 FIG.
<Image transmission system>
FIG. 1 is a block diagram showing a configuration example of an image compression / decompression system according to Embodiment 1 of the present invention, and shows an example of an image transmission system. This image transmission system includes a transmission side unit Ut and a reception side unit Ur connected by a communication network 100, and can compress image data and transmit the compressed image data from the transmission side unit Ut to the reception side unit Ur. Here, the image data of the image input apparatus 101 connected to the transmission side unit Ut is transmitted to the image output apparatus 102 connected to the transmission side unit Ur.

  The communication network 100 includes a wired or wireless communication line for transmitting digital data, and includes an exchange, a repeater, and the like as necessary. For example, packet communication networks such as Ethernet (registered trademark), the Internet, ATM (Asynchronous Transfer Mode), and other digital networks can be used.

  The image input device 101 is a device that provides image data, and includes, for example, an imaging device such as a camera, an image reading device such as a scanner, and a data storage device such as an HDD (Hard Disc Drive). In the present embodiment, it is assumed that the image input device 101 generates image data of a still image having an RGB format and outputs it to the transmission side unit Ut. Note that a still image in this specification means an image composed of a large number of pixels arranged in a two-dimensional manner, and includes each frame image constituting a moving image, a difference image between frames, and the like.

  The image output device 102 is a device that uses image data output from the receiving unit Ur, and includes, for example, a display device such as an LCD, an image forming device such as a printer, and a data storage device such as an HDD (Hard Disc Drive). Become.

<Sender unit>
FIG. 2 is a block diagram illustrating a configuration example of the transmission side unit Ut of FIG. The transmission side unit Ut includes a YUV conversion unit 10, an image compression unit 11, and a data transmission unit 12. The image data generated by the image input apparatus 101 is first converted into image data in the YUV format by the YUV conversion unit 10 and then compressed by the image compression unit 11 to become compressed data with a reduced data amount. The compressed data is sent to the communication network 100 by the data transmission unit 12.

  The YUV conversion unit 10 is a format conversion unit that converts image data in RGB format into image data in a predetermined YUV format (for example, YUV410, 411, 420, 422, 444). Note that the YUV conversion unit 10 is omitted when YUV format image data is input from the image input device 101 or when JPEG compression is performed with the RGB format image data as it is.

  The image compression unit 11 includes an image reduction converter 13 and a JPEG encoder 14. The image reduction converter 13 divides the input image into a plurality of small areas, reduces the resolution of at least some of the small areas, and further reduces the size of the entire image by rearranging these small areas. . This reduction in resolution is performed for each of the small regions according to the required image quality. On the other hand, the JPEG encoder 14 performs compression processing on the image data according to the JPEG standard to generate compressed data with a further reduced data amount.

<Receiving unit>
FIG. 3 is a block diagram showing a configuration example of the receiving side unit Ur of FIG. The reception side unit Ur includes an RGB conversion unit 22, an image expansion unit 21, and a data reception unit 20. The compressed data sent from the transmitting unit Ut to the communication network 100 is received by the data receiving unit 20. The received compressed data is decompressed by the image decompressing unit 21, and image data in the YUV format is restored. The image data is converted into the RGB format by the RGB conversion unit 22 and output to the image output device 102.

  The image expansion unit 21 includes a JPEG decoder 23 and an image enlargement converter 24. The compressed data from the data receiving unit 20 is decompressed by the JPEG standard in the JPEG decoder 23, and the image data before JPEG compression is restored. The expanded image data is enlarged in image size by rearrangement and higher resolution in the image enlargement converter 24, and the image data before the lower resolution is restored.

  The RGB conversion unit 22 converts the YUV format image data output from the image enlargement converter 24 into the RGB format and outputs the converted data to the image output device 102. Note that the RGB converter 22 is omitted when YUV format image data is input to the image output apparatus 102 or when JPEG compression is performed with the RGB format image data as it is.

<Image compression unit>
FIG. 4 is a block diagram showing a configuration example of the image compression unit 11 in FIG. 2, and shows detailed configuration examples of the image reduction converter 13 and the JPEG encoder 14. The image reduction converter 13 includes a downsampling region dividing unit (DS region dividing unit) 130, a downsampling unit 131, and a rearrangement unit 132.

[DS area division unit]
The DS area dividing unit 130 divides the YUV converted image data into a plurality of DS areas. Each DS area can be an area having an arbitrary number of pixels and an arbitrary shape. However, in consideration of compression efficiency, a block used in the JPEG encoder 14 is a basic unit, and an aggregate of the blocks is used. It is desirable to use the DS region.

  Since the block size of JPEG is 8 × 8 pixels, 16 × 16 pixels, 32 × 32 pixels, 24 × 16 pixels, 8 × 32 pixels, etc. are conceivable as DS region sizes that are integer multiples thereof. However, when a block size identification area, which will be described later, is inserted into each DS area, the DS area size needs to be at least three times the block size in terms of area ratio in order to reduce the entire image area by rearrangement. There is.

  In addition, each DS region may have a different size or a different shape, but it is desirable to have the same size and the same shape in consideration of the ease of rearrangement processing described later. A rectangular shape (particularly a square shape) is more desirable. Hereinafter, in the present embodiment, an example in which the entire image area is divided into DS areas of the same size composed of 16 × 16 pixels will be described.

[Downsampling section]
The down-sampling unit 131 performs down-sampling of each pixel data constituting the DS area for at least a part of the DS area, and converts it into reduced data in which the number of pixels is reduced. The size of the reduced data is preferably set to an integral multiple of the block size in consideration of the compression rate in the JPEG encoder, and here it is assumed to match the block size (8 × 8 pixels).

  The DS area to be subjected to the downsampling process is determined according to the image quality required for each DS area. For example, when there are important areas that require high image quality and non-important areas that are not so in the entire image area, the down-sampling process is performed only for the DS areas that are non-important areas. For example, input data (non-reduced data) is output as it is for a DS region including an important region, and reduced data obtained by downsampling processing is output for a DS region that does not include an important region.

  The determination of the important area and the non-important area, that is, the determination of the DS area to be downsampled is determined in advance or designated by the operator. Further, an important area and an unimportant area may be automatically determined based on an output signal of a sensor (not shown), or may be automatically determined based on image data. For example, based on a comparison result between frames of moving images, it is possible to determine a region with motion as an important region. It is also possible to determine a flat image area with relatively little change as an unimportant area.

  The downsampling process is realized by a thinning process for extracting pixels in the DS region, a filter process using a low-pass filter, or the like. At this time, the downsampling process may be realized only by the thinning process, but by using the filter process and the thinning process in combination, a highly accurate downsampling process with less distortion can be realized. For example, in processing using a Gaussian filter as a low-pass filter, a one-dimensional Gaussian filter is applied in the horizontal direction of the image data, and as a result, the same filter is applied in the vertical direction to the obtained data. By performing the thinning process on the image data filtered by the Gaussian filter in this way, downsampling can be performed with high accuracy.

  5A and 5B are diagrams showing an example of DS area division and downsampling in the image compression unit 11. (A) in the figure shows a case where the entire image area is divided into a DS area A1 of 16 × 16 pixels. BL in the figure is an 8 × 8 pixel JPEG block. The DS area dividing unit 130 matches the division reference position (upper left of the screen area in the drawing) with the case of block division in JPEG so that the DS area A1 is an aggregate of blocks BL. The down-sampling unit 131 down-samples the DS area A1 which is an unimportant area so that the number of pixels in the DS area is halved in both the vertical and horizontal directions, and converts it into reduced data of 8 × 8 pixels. Similarly, (b) in the drawing shows a case where the entire image area is divided into a DS area A2 of 32 × 32 pixels. The DS area A2, which is a non-important area divided by the DS area dividing section 130, is converted by the down-sampling section 131 into 8 × 8 pixel reduced data obtained by down-sampling each of the vertical and horizontal directions to ¼.

  FIG. 6 is a block diagram illustrating a detailed configuration example of the downsampling unit 131. The down-sampling unit 131 is an adaptive down-sampling unit that performs down-sampling based on each pixel data in the DS region, and maintains a high image quality while maintaining a high image quality in a DS region that includes contours and fine changes. In the DS region composed of images, the data amount is reduced by reducing the resolution.

This adaptive down-sampling unit includes a low-pass filter 30, a thinning-out processing unit 31, an output selection unit 32, a subtractor 33, and a down-sampling determination unit (DS determination unit) 34, and the 16 × 16 pixels constituting the DS region. Pixel data f ₁₆ (x, y) is input. The pixel data f ₁₆ (x, y) is the luminance data Y or the color difference data U, V at the coordinates (x, y) in the DS region.

すなわち、ローパスフィルタ３０及び間引き処理部３１においてダウンサンプリング処理が行われ、縮小データが生成される。出力選択部３２は、非縮小データであるピクセルデータf₁₆(x,y)、又は、縮小データであるピクセルデータf₈(x,y)の一方を選択し、出力ピクセルデータf_o(x,y)として再配置部１３２へ出力している。 High-frequency components are removed from the input pixel data f ₁₆ (x, y) by the low-pass filter 30 to become pixel data f _L (x, y) composed of low-frequency components. This pixel data f _L (x, y) is further thinned out every other bit in the thinning-out processing unit 31 to become pixel data f ₈ (x, y) of 8 × 8 pixels expressed by the following equation.

That is, downsampling processing is performed in the low-pass filter 30 and the thinning-out processing unit 31, and reduced data is generated. The output selection unit 32 selects either pixel data f ₁₆ (x, y) that is non-reduced data or pixel data f ₈ (x, y) that is reduced data, and outputs pixel data f _o (x, y). This is output to the rearrangement unit 132 as y).

The DS determination unit 34 determines whether to perform downsampling based on the high-frequency component included in the DS region, and controls the output selection unit 32 based on the determination result. The DS determination unit 34 includes an MSH (Means Square High Frequency component) calculation unit 35 and a threshold value comparison unit 36, and receives pixel data f _H (x, y) including a high frequency component obtained by the subtractor 33. Yes. The MSH calculation unit 35 obtains the square sum (MSH) of the pixel data f _H (x, y) according to the following equation.

The threshold comparison unit 36 compares the MSH obtained by the above formula with a predetermined determination threshold Th, and when the MSH is smaller than the determination threshold Th, the pixel data f ₈ (x, y) is output, and if it is equal to or greater than the determination threshold Th, the input pixel data f ₁₆ (x, y) is output as it is.

[Relocation section]
The rearrangement unit 132 rearranges the output data of the downsampling unit 131 to generate image data in which the size of the entire image is reduced, and outputs the image data to the JPEG encoder 14. The data output from the downsampling unit 131 includes non-reduced data composed of 16 × 16 pixels and reduced data composed of 8 × 8 pixels. Therefore, if two or more pieces of reduced data are collected and rearranged in the same DS area, the number of DS areas can be reduced, and the size of the entire image can be reduced.

  In the downsampling unit 131, when downsampling is performed on each continuous DS region and reduced data is continuously output, the rearrangement unit 132 sequentially stores these reduced data in the same DS region. Place it. If the reduced data is not continuous, or if the number of continuous data is small, after the reduced data is arranged in the DS area, the fill data is inserted into the remaining portion. Fill data is predetermined arbitrary pixel data. An area where the fill data is inserted is called a fill area.

  In addition, in the DS area where the reduced data is arranged, a fill area composed of 8 × 8 pixels is always provided separately from the empty area. The fill area is an identification area that makes it possible to identify that the DS area is an area where reduced data is arranged, and a position in the DS area is determined in advance. Since the arrangement order of the reduced data in the DS area is determined in advance and the empty area is set as the fill area, the receiving unit Ur restores the arrangement of the reduced data based on the fill data. It is not necessary to separately transmit information on the arrangement of the reduced data from the transmission side unit Ut to the reception side unit Ur. In addition, by setting the empty area and the identification area as a fill area in which the same data is inserted, the compression rate when the JPEG encoder 14 performs compression processing can be increased.

  Note that the JPEG encoder 14 cannot process image data having a shape other than a rectangle. For this reason, the rearrangement unit 132 adds a DS area consisting only of fill data to the end of the image data from which the DS area has been reduced, and shapes the image data to have a rectangular shape.

7A to 7C are diagrams illustrating an example of the operation of the image reduction converter 13.
The processing by the DS region dividing unit 130, the downsampling unit 131, and the rearrangement unit 132 is started from the upper left of the image data, and is sequentially shifted from left to right in the horizontal direction. When the processing is completed up to the right end of the image data, the processing target is shifted downward, and the same processing is repeated for the region located immediately below the processed region.

  (A) in the figure shows the state of the DS area division. The DS area dividing unit 130 divides the entire area of the image into 36 DS areas (# 1 to # 36) of 16 × 16 pixels with respect to the 96 × 96 pixel image data input from the image input device 101. Then, the image data of each DS region is sequentially output to the downsampling unit 131. Of these, # 22, # 28, and # 34 are important regions, and the other DS regions are non-important regions.

  The state of downsampling is shown in (b) in the figure. The downsampling unit 131 determines whether each DS region sequentially output from the DS region dividing unit 130 is an important region or an unimportant region. As a result, when it is determined that the region is an important region, pixel data of 16 × 16 pixels is output without performing downsampling. On the other hand, if it is determined that the region is an unimportant region, downsampling is performed to reduce the number of vertical and horizontal pixels to ½, and reduced data composed of 8 × 8 pixels is output.

  (C) in the figure shows the state of data rearrangement. When the reduced data is output from the downsampling unit 131, the rearrangement unit 132 arranges the data in the order of upper left, upper right, and lower left in the DS region. Also, fill data is inserted into the lower right 8 × 8 pixels in the DS area to form a fill area (identification area). In the drawing, # 1 to # 3 are arranged in the first DS area, and # 4 to # 6 are arranged in the next DS area. In the same way, three consecutive reduced data # 1 to # 21 are arranged in the DS area in units of three.

  Thereafter, when the non-reduced data # 22 is output from the downsampling unit 131, the rearrangement unit 132 arranges the data # 22 in a new DS area. Thereafter, continuous reduced data # 23 to # 27 are output again, and are arranged in the DS region three by three. Next, when the non-reduced data # 28 is output from the downsampling unit 131, only two reduced data (# 26, 27) are arranged in the immediately preceding DS region. For this reason, the rearrangement unit 132 inserts fill data into the lower left empty area in the DS area to form a fill area.

  Next, when the reduced data # 29 to # 33 and the non-reduced data # 34 are sequentially output, they are arranged in the same manner. At the time when the last reduced data # 36 is arranged, there is an empty area (lower left) in the DS area, so fill data is also inserted into the empty area. Finally, three DS regions consisting only of fill data are added to shape the entire image into a rectangular shape, and the rearrangement unit 132 ends the process.

  In the rearrangement unit 132, when the image data is reduced, the horizontal size (number of pixels) of the reduced image data is matched with the original image size, and only the vertical size is reduced. Yes. In this way, if the size of the image data related to the processing direction in the rearrangement unit 132 is determined based on the original image data, the image enlargement converter 24 correctly restores the information even if there is no information about the original image size. be able to. Therefore, it is not necessary to separately transmit information on the image size from the transmission side unit Ut to the reception side unit Ur.

  Further, since the image data in the YUV format is composed of one luminance data Y and two color difference data U and V, the DS area dividing unit 130, the downsampling unit 131, and the rearrangement unit 132 have these three data. Each of the above-described processes is performed.

  Further, the data output from the downsampling unit 131 to the rearrangement unit 132 may be only reduced data and non-reduced data, or fill data is inserted into an empty area in the DS area where downsampling has been performed. Data (that is, data having the same area size as the original image data) may be used.

<JPEG encoder>
The JPEG encoder 14 includes a block dividing unit 140, a DCT processing unit 141, a quantization processing unit 142, an encoding unit 143, a quantization table T1, and a code table T2 (see FIG. 4). The image data output from the image compression converter is divided into a plurality of blocks of 8 × 8 pixels in the block dividing unit 140. The DCT processing unit 141 performs a discrete cosine transform for each divided block to obtain a DCT coefficient. Each DCT coefficient obtained in this way is quantized by the quantization processing unit 142 using the quantization table T1.

  FIG. 8 is a diagram showing an example of the quantization table T1, in which data for defining the quantization step width is shown in a matrix for each frequency component in the horizontal direction and the vertical direction. Usually, different quantization tables are used for the quantization processing of the luminance data and the color difference data. In the figure, (a) shows a quantization table for luminance data, and (b) shows a quantum table for color difference data. An example of a conversion table is shown. Each of these data tables consists of data that expands the quantization step width as the frequency increases, and this tendency is that the quantization table for color difference data is better than the quantization table for luminance data. Is also prominent.

  When quantizing the DCT coefficient, the quantization processing unit 142 reads data corresponding to the DCT coefficient from the quantization table T1, and further multiplies the read data by a quantization factor (quantization coefficient) Q. The value is used as the quantization step width. This quantization factor Q is an arbitrary value for adjusting the compression rate and the image quality, and is given in advance. If the quantization factor Q is increased, the quantization step width is increased and the data compression rate can be improved, but with this, block distortion occurs and the image quality is lowered.

  When the adaptive downsampling unit 131 shown in FIG. 6 is used, the data amount after JPEG compression can be reduced by increasing the determination threshold Th in the threshold comparison unit 36. For this reason, a desired compression rate can be obtained using a relatively small quantization factor Q. However, if the determination threshold Th is excessively increased, the contour and the region with a minute change are also deteriorated. For this reason, it is necessary to determine the combination of the determination threshold Th and the quantization factor Q so that the restored image data has good quality. That is, the determination threshold Th is determined based on the quantization factor Q, and the quantization factor Q is determined based on the determination threshold Th.

  The encoding unit 143 performs a run-length transform process within the block for the quantized AC coefficients, and performs the difference process between the blocks for the quantized DC coefficients, The encoding process using the entropy code is performed. The entropy code is a code system having a code length corresponding to the appearance probability, and the Huffman code is widely known. The code table T2 holds a code table of Huffman codes, and the encoding unit 143 performs the encoding process using the code table T2.

<Image expansion part>
FIG. 9 is a block diagram showing an example of the configuration of the image expansion unit 21 in FIG. 3, in which detailed configuration examples of the JPEG decoder 23 and the image enlargement converter 24 are shown.

[JPEG decoder]
The JPEG decoder 23 includes a decoding unit 230, an inverse quantization processing unit 231, an inverse DCT processing unit 232, a quantization table T1, and a code table T2. The JPEG decoder 23 decompresses the compressed data by performing processing reverse to that of the JPEG encoder 14. The image data before JPEG compression is restored. The quantization table T1 and the code table T2 need to use the same data table as the JPEG encoder 14, and may be added to the compressed data as necessary and transmitted from the transmission side unit Ut to the reception side unit Ur. it can.

  The decoding unit 230 decodes the Huffman code in the compressed data using the code table T2, and further decodes the differentially processed DC coefficient and the run-length converted AC coefficient. The decoded data is inversely quantized by the inverse quantization processing unit 231 using the quantization table T1, and the DCT coefficient for each block is restored. This DCT coefficient is subjected to inverse DCT processing in the inverse DCT processing unit 232, and image data before JPEG compression processing, that is, image data reduced by the image reduction converter 13 is restored.

[Image Enlargement Converter]
The image enlargement converter 24 includes a DS region dividing unit 240, an arrangement restoration unit 241, and an interpolation processing unit 242, and performs reverse processing to the case of the image reduction converter 13 to restore image data before the reduction process. . First, the DS area dividing unit 240 divides the image data output from the JPEG decoder 23 into a plurality of DS areas. The DS area has the same size (here, 16 × 16 pixels) as the DS area used by the DS area dividing unit 130 of the transmission side unit Ut.

  The arrangement restoring unit 241 determines the arrangement state of non-reduced data or reduced data in the DS area based on the fill area existing in each divided DS area, and arranges the reduced data based on the determination result. After the replacement, the data is output to the interpolation processing unit 242.

  (A) to (e) of FIG. 10 are diagrams showing all arrangement states in the DS region when the DS region is 16 × 16 pixels and the reduced data is 8 × 8 pixels. (A) is a case where no fill area exists in the DS area, and it can be determined that non-reduced data is arranged. This determination can be made only by determining that the lower right in the DS area is not the fill area. (B) to (d) are cases in which 1 to 3 pieces of reduced data are arranged in the DS area, and (e) is a case in which the DS area is composed only of the fill area, and all are 8 × 8 pixels. It can be determined by the number of fill areas consisting of.

Incidentally, the DS region of 16 × 16 pixels, to place any number of fill area consisting of 8 × 8 pixels, a combination of the arrangement pattern is considered in two ways ^4, the arrangement pattern of (a) and (e) Otherwise, there will be 14 ways. That is, there are four arrangement patterns for one fill area, six arrangements for two arrangement patterns, and four arrangements for three arrangement patterns. For this reason, the fill region arrangement method in the cases (b) to (d) is not limited to the example of the present embodiment, and it is needless to say that any of the above 14 methods can be selected.

  The interpolation processing unit 242 restores the reduced data output from the arrangement restoring unit 241 to the original size before downsampling (that is, the size of the DS region) by interpolating between pixels and increasing the number of pixels. For this interpolation process, an LP enlargement method, linear interpolation, quadratic interpolation, cubic interpolation, or the like can be used. The LP enlargement method is a method for restoring an image by estimating a high-frequency component lost during sampling by the downsampling unit. On the other hand, interpolation processing is not performed for non-reduced data. In this way, the image data before size reduction by the image reduction converter 13 is restored and output to the RGB conversion unit 20.

  Note that the compressed data of the image in the YUV format is composed of compressed data for one luminance data Y and two color difference data U and V, so the DS area dividing unit 240, the arrangement restoring unit 241, and the interpolation processing unit 242. Performs the above-described processing on these three pieces of compressed data.

  In the above embodiment, an example in which image data is divided into 16 × 16 pixel DS regions and a part of the region is downsampled to 8 × 8 pixels has been described. However, the present invention is not limited to such a number of pixels.

<32 × 32 DS area>
(A) to (c) of FIG. 11 are diagrams illustrating a state in which image data is divided into 32 × 32 pixel DS regions, downsampled to 8 × 8 pixel reduced data, and rearranged. . In (a), the entire image area is divided into a plurality of DS areas each consisting of 32 × 32 pixels. In (b), the DS region that does not include the important region is down-sampled by reducing the number of vertical and horizontal pixels to ¼, and converted to reduced data of 8 × 8 pixels. In (c), continuous reduced data is collected and rearranged in the same DS area, and the size of the entire image area is reduced.

  In FIG. 11, the size of the entire image area after the reduction is larger than in the case of FIG. 7, but in the case where the entire image area is further enlarged and there are many continuous non-important areas, FIG. It can be expected that the size of the entire image area is made smaller than the method 7.

Embodiment 2. FIG.
In the image compression / decompression system according to the first embodiment, the arrangement restoration unit 241 in the image enlargement converter 24 determines the arrangement of non-reduced data and reduced data based on the position of the fill area existing in the DS area. . For this reason, when a part of the non-reduced data or the reduced data matches the fill area, the arrangement restoration unit 241 may make an erroneous determination. In the present embodiment, an image compression / decompression system capable of preventing such erroneous determination will be described.

  In general, quantization error occurs in the image data compressed by the JPEG encoder 14. In other words, if an arbitrary value is adopted as the fill data, the fill data inserted into the image data in the image reduction converter 13 is then input to the image enlargement converter 24 through JPEG compression processing and JPEG expansion processing. At that point, it may have changed to different data. In this case, the arrangement restoration unit 241 cannot correctly determine the reduced data.

  As a first countermeasure for avoiding such erroneous discrimination, it is conceivable that the image enlargement converter 24 employs a discrimination method capable of discriminating fill data even when a quantization error occurs. That is, if the arrangement restoration unit 241 determines that pixel data within the range having a width based on the maximum value of the quantization error centered on the fill data is fill data, the fill data can be used even if a quantization error occurs. Data can be correctly determined.

  Further, as a second countermeasure for avoiding the erroneous determination, it is conceivable to adopt a value that is not affected by the quantization error as the fill data. That is, if any one of the quantization steps in the quantization processing unit 142 is adopted as fill data, the fill data will not change to a different value due to the quantization process, and the arrangement restoration unit 241 correctly determines the reduced data. can do.

  However, in the JPEG encoder 14, the quantization step width is given by the quantization table T1 and the quantization factor Q, and the quantization step is determined based on the quantization step width. For this reason, if the quantization table T1 and the quantization factor Q change, the maximum value of the quantization error, the quantization step, and the fill data must be changed. However, only the value 0, which is a reference value for obtaining the quantization step from the quantization step width, is always a quantization step regardless of the quantization step width. Therefore, it is desirable to adopt the second countermeasure and adopt the value 0 at the time of quantization processing as the fill data.

  In the JPEG encoder 14, for the convenience of calculation processing, the median value is subtracted from each pixel data, level-shifted, and converted into signed data before DCT processing is performed. Then, the original pixel data is restored by adding the median to the pixel data for which the inverse DCT processing has been completed in the JPEG decoder 23. Therefore, this median value may be adopted as fill data.

  In general, each pixel data is represented by 8 bits and takes a value of 0 to 255. Therefore, in the JPEG encoder 14, the median value 128 is subtracted from each pixel data and converted into signed data. That is, the DC coefficient 0 at the time of quantization processing corresponds to the average value 128 of the pixel data. Therefore, if the fill data is set to the median value 128 of the pixel data, the DCT coefficients at the time of the quantization process are all zero, and the fill data is not affected by the quantization error regardless of the quantization step width.

  Next, as described above, in the compression / decompression process according to the JPEG standard, the quantization table T1 (see FIG. 8) in which the quantization step width is increased as the frequency component is increased is used. For this reason, there is a possibility that the image data restored through the JPEG compression / decompression process includes blocks that are narrowed down to only the DC component in the compression process. In such a block, the values of all pixels match after restoration, and the value is an average value (but an integer) of pixel data before compression. Therefore, when the average value of the pixel data constituting the JPEG block is a value close to the fill data, all pixels in the JPEG block match the fill data through the JPEG compression / decompression process, and are distinguished from the fill area. You may not be able to. That is, there is a possibility that the arrangement restoration unit 241 may erroneously determine the fill area and the image enlargement converter 24 cannot correctly restore the original image data.

In order to prevent such misidentification of the fill area, the rearrangement unit 132 needs to process a JPEG block having an average value near 128 so that the average value is away from 128. Specifically, if the maximum fluctuation range of the average value in the block by JPEG compression / decompression processing is less than α, the average value for the block whose difference between the average value before JPEG compression and 128 is less than α is calculated. The pixel data should be changed so that becomes 128 + α or 128−α. For example, when the pixel data in the block is f (x, y) and the following equation (5) is satisfied, the average value of the image data is changed to 128−α in advance, and the following equation (6) is satisfied. In this case, it is necessary to change the average value of the image data to 128 + α in advance.

A method for determining α will be described. In the DCT processing by JPEG, the DC coefficient is obtained by the following equation (7). For this reason, if the average value in the JPEG block is changed by α, the DC coefficient is changed by 16α. For example, when α is added to each pixel data, the DC coefficient increases by 16α.

  As described above, the DC coefficient is quantized in the quantization processing unit 142 with the quantization step width determined by the quantization table T1 and the quantization factor Q. In the case of a DC coefficient, data at the upper left corner of the quantization table T1 shown in FIG. 8 is used. This data is normally 16, and 16Q is used as the quantization step width. In this case, the quantization error of the DC coefficient can be said to be less than 16Q, except for errors in calculation accuracy. Therefore, if 16α ≧ 16Q (that is, α ≧ Q) holds, the average value in the block after JPEG compression / decompression processing changes to 128 when the average value in the block is 128−α and 128 + α. Absent.

  If the influence on the image quality due to the change of the pixel data is taken into consideration, it is preferable to adopt a value as small as possible as α, and from this point of view, α = Q is desirable. However, if α <2Q, in principle, the restored image data is the same as when α = Q. Therefore, if an error in calculation accuracy is considered, Q <α <2Q. It is more desirable to determine α.

  Specifically, the rearrangement unit 132 converts each of the four JPEG blocks (8 × 8 pixels) constituting the 16 × 16 pixel non-reduced data and the reduced data including 8 × 8 pixels to the above formula ( Applies to 5) and (6). As a result, when the above equation (5) is satisfied, the same value is added to each pixel in the block so that the average value becomes 128 + α. When the above equation (6) is satisfied, the same value is subtracted from each pixel in the block so that the average value is 128−α.

Embodiment 3 FIG.
In the first embodiment, an example in which the entire image area is divided into DS areas having the same size has been described. On the other hand, in the present embodiment, a case will be described where the DS regions are divided into different sizes and the reduction ratios in downsampling are made different.

  FIGS. 12A to 12C are diagrams showing an example of the operation of the image compression / decompression system according to the third embodiment of the present invention.

  (A) in the figure shows how the DS area dividing unit 130 divides the DS area. The DS area dividing unit 130 divides the entire image area by mixing two or more DS areas having different sizes. Here, a large DS area composed of 32 × 32 pixels and a small DS area composed of 16 × 16 pixels are mixed. At this time, the important area is divided so as to be included in a small DS area. As for the non-important region, a portion far from the important region is divided into large DS regions, and a portion close to the important region is divided into small DS regions.

  Specifically, with respect to 96 × 96 pixel image data input from the image input device 101, the entire area of the image is divided into seven large DS areas (# 1 to # 3, # 4, # 7, # 10, # 13) and eight small DS regions (# 5, # 6, # 8, # 9, # 11, # 12, # 14, # 15), and the image data in each DS region is downsampled 131. Is output to. Of these, # 9, # 12, and # 15 are important regions, and all are divided into small DS regions.

  (B) in the figure shows a state of downsampling by the downsampling unit 131. The down-sampling unit 131 performs down-sampling on each DS area, which is an unimportant area, and converts it into reduced data composed of 8 × 8 pixels. That is, downsampling is performed at a reduction rate corresponding to the size of the DS area, and the data is converted into reduced data of the same size regardless of the size of the DS area.

  Specifically, for the large DS regions (# 1 to # 3, # 4, # 7, # 10, # 13) divided into 32 × 32 pixels by the DS region dividing unit 130, the number of vertical and horizontal pixels is respectively set. Downsampling is performed to reduce it to 1/4. On the other hand, for small DS regions (# 5, # 6, # 8, # 11, # 14) divided into 16 × 16 pixels, down-sampling is performed to reduce the number of vertical and horizontal pixels by half. . As a result, 16 × 16 pixel non-reduced data (# 9, # 12, # 15) and 8 × 8 pixel reduced data are output to the rearrangement unit 132.

  (C) in the figure shows a state of data rearrangement by the rearrangement unit 132. The rearrangement unit 132 rearranges the reduced data in a small DS area (16 × 16 pixels). At that time, in the downsampling unit 131, the reduced data down-sampled to 1/4 (1/4 reduced data) and the reduced data down-sampled to 1/2 (1/2 reduced data) are distinguished. However, rearrangement is performed so that 1/4 reduced data and 1/2 reduced data are not mixed in the same DS area.

  Also, in the case of arranging 1/4 reduced data and the case of arranging 1/2 reduced data, the arrangement of the fill area and reduced data in the DS area is made different so that both can be identified based on the fill area. ing. Specifically, when ½ reduced data is arranged in the DS area, the fill area is arranged in the lower right as in the first embodiment, and the reduced data is arranged in the order of upper left, upper right, and lower left. Place it. On the other hand, when 1/4 reduced data is arranged in the DS area, the fill area is arranged in the upper left, and the reduced data is arranged in the order of lower right, lower left, and upper right.

  Since the reduced data # 1 to # 3 are 1/4 reduced data, the fill area is arranged at the upper left in the DS area of 16 × 16 pixels, and the reduced data are arranged in a predetermined order from the lower right. Since the reduced data # 4 is 1/4 reduced data, it is similarly arranged in the next DS area. Since the subsequent reduced data # 5 and # 6 are ½ reduced data, they are not arranged in the same DS area as the reduced data # 4, but are arranged in a new DS area. Accordingly, fill data is inserted into the empty area of the DS area where the reduced data # 4 is arranged, the fill area is arranged at the lower right in the next DS area, and the reduced data # 5 and # 6 are predetermined from the upper left. Arranged in order. Since the next reduced data # 7 is 1/4 reduced data, it is arranged in a different DS area from the reduced data # 5 and # 6. Thereafter, similarly, non-reduced data and reduced data are rearranged in order.

  If (c) in the figure is compared with (c) in FIG. 11, in any case, down-sampling is not performed for the important area, and down-sampling is performed for the non-important area to convert it into 1/4 reduced data. Nevertheless, it can be seen that the size of the entire image area can be further reduced.

  (A) to (h) of FIG. 13 are diagrams showing all arrangement states in the DS area when the DS area is 16 × 16 pixels and the reduced data is 8 × 8 pixels. (A) is a case where no fill area exists in the DS area, and it can be determined that non-reduced data is arranged. This determination can be made only by determining that the upper left and lower right in the DS area are not the fill area. (B) to (d) show that 1 to 3 1/2 reduced data are arranged in the DS area, and (e) to (g) show 1 to 3 1/1 in the DS area. When 4 reduced data is arranged, (h) is a case where the DS area is composed only of the fill area, which can be determined by the arrangement of the fill area composed of 8 × 8 pixels. Note that the fill region arrangement method in the cases (b) to (g) is not limited to the example of the present embodiment, and can be arbitrarily selected from the 14 types described above.

Embodiment 4 FIG.
In the third embodiment, the case has been described in which the entire image region is divided into DS regions of different sizes and the reduction ratio in downsampling is made different. On the other hand, in the present embodiment, a case will be described in which the entire image region is divided into DS regions of the same size and hierarchical downsampling is performed.

  In the present embodiment, the downsampling unit 131 performs downsampling twice (or three times or more) for a part of DS regions not including the important region. That is, for a plurality of DS regions, a plurality of reduced data obtained by downsampling is collected, and downsampling is performed again on the aggregate of these reduced data. In this case, the DS region that has been down-sampled twice can obtain the same result as when down-sampling with a reduction ratio of 2 is performed, except for errors due to calculation accuracy and differences due to filter processing. That is, even if hierarchical downsampling is performed, almost the same result as in the third embodiment can be obtained.

  14 (a) to 15 (e) are diagrams showing an example of the operation of the image compression / decompression system according to the fourth embodiment of the present invention. (A) in the figure shows how the DS area dividing unit 130 divides the DS area. The DS area dividing unit 130 divides the entire image area into 16 × 16 pixel DS areas (# 1 to # 36) in exactly the same manner as in the first embodiment. Of these, # 22, # 28, and # 34 are important regions.

  (B) in the figure shows the state of the first downsampling by the downsampling unit 131. The down-sampling unit 131 performs down-sampling for each DS area which is an unimportant area, and converts it into 1/2 reduced data composed of 16 × 16 pixels. The operation so far is the same as that in the first embodiment.

  (C) in the figure shows how reduced data is grouped. The downsampling unit 131 generates a new DS region composed of 16 × 16 pixels by grouping four adjacent ½ reduced data. That is, adjacent reduced data # 1, # 2, # 7 and # 8 are collected to generate a new DS area #A, and reduced data # 3, # 4, # 9 and # 10 are collected to create a new DS area #B. Is generated. Similarly, new DS regions #A to #G are generated. At this time, the reduced data # 15, # 16, # 21, # 27 and # 33 which cannot be grouped are left as they are.

  (D) in the figure shows the second down-sampling state by the down-sampling unit 131. FIG. The second downsampling is performed only on the grouped new DS regions, and 1/4 reduced data consisting of 8 × 8 pixels is generated for these regions. When the second downsampling is completed, the same state as in FIG. 12B in the third embodiment is obtained.

  (E) in the figure shows a state of data rearrangement by the rearrangement unit 132. The rearrangement unit 132 separates the 1/4 reduced data down-sampled twice and the 1/2 reduced data down-sampled once, and reduces the 1/4 reduced data and 1/2 in the same DS region. Rearranged so that reduced data is not mixed. The arrangement method is the same as in the third embodiment.

Embodiment 5 FIG.
In the above embodiment, the image reduction converter 13 and the image enlargement converter 24 perform the same processing for the luminance data Y and the color difference data U and V, respectively, and the processing for each of these data has been described without distinction. On the other hand, in the present embodiment, a case will be described in which processing for luminance data Y is different from processing for color difference data U and Y.

  FIG. 16 is a block diagram showing a configuration example of a main part of the image compression apparatus according to the fifth embodiment of the present invention, and shows a detailed configuration of the image reduction converter 13 of FIG. The image reduction converter 13 includes a DS region dividing unit 130 common to YUV, three down-sampling units 131 and 131s for each YUV, and three rearrangement units 132 and 132s for each YUV.

  The DS area dividing unit 130 divides each of the luminance data Y and the color difference data U and V into the DS area, and the DS area dividing process similar to that in the first embodiment is performed. That is, for each YUV, DS area division is performed so that the same pixel data belongs to the same DS area.

  The downsampling unit 131 is an applied downsampling unit that performs downsampling of the luminance data Y. As in the case of the first embodiment, the downsampling unit 131 determines whether or not the target DS region is an important region. Downsampling is performed based on the result (see FIG. 6). On the other hand, the two down-sampling units 131s perform the down-sampling of the color difference data U and V, respectively. The determination of whether or not to perform the down-sampling depends on the determination result of the down-sampling unit 131 for luminance data.

  FIG. 17 is a block diagram illustrating a detailed configuration example of the downsampling unit 131s of FIG. The downsampling unit 131s does not have a subtractor 33 and a downsampling determination unit (DS determination unit) 34 for controlling the output selection unit 32 as compared with the adaptive downsampling unit 131 of FIG. Is different. Therefore, the output selection unit 32 is operated based on the determination result of the DS determination unit 34 in the downsampling unit 131 for the luminance data Y. That is, in the same DS region, if the luminance data Y is downsampled, the color difference data U and V are also downsampled. If the luminance data Y is not downsampled, the color difference data U and V are also downsampled. Sampling is not performed.

  The rearrangement unit 132 rearranges the reduced data with respect to the luminance data Y, and rearranges the reduced data as in the case of the first embodiment, and at a predetermined position in the DS area where the reduced data is arranged. An identification area made up of fill data is arranged. On the other hand, the rearrangement unit 132s rearranges the reduced data for the color difference data U and V. At this time, even in the DS area in which the reduced data is arranged, the rearrangement unit 132s includes fill data in the DS area. Do not place the identification area. Therefore, more reduced data can be arranged in the same DS area than the luminance data Y.

  FIG. 18 is a diagram showing an example of the operation of the image compression / decompression system according to Embodiment 5 of the present invention. This figure relates to the same image data as FIG. 7 (Embodiment 1), (a) in the figure shows the rearrangement of the luminance data Y, and (b) in the figure shows the rearrangement of the color difference data U and V. Each is shown.

  Since the rearrangement process related to the luminance data Y is the same process as the method described in the first embodiment, the description thereof is omitted here. Regarding the color difference data U and V, when the reduced data is rearranged in the DS area, four pieces of reduced data are arranged in one DS area. Therefore, when five or more pieces of reduced data are continuously input from the downsampling unit 131s, the fifth and subsequent pieces of reduced data are arranged in the next DS area. That is, the reduced data is arranged in each DS area in a packed manner until the maximum number that can be arranged in one DS area is reached. On the other hand, if the number of continuous reduced data is three or less, fill data is inserted into the empty area. In the figure, # 1 to # 4 are arranged in the first DS area, and # 5 to # 8 are arranged in the next DS area. In the same manner, four consecutive reduced data # 1 to # 21 are arranged in the DS region in succession. Other processes are the same as those in the case of the luminance data Y.

  In this case, since the re-arranged color difference data U and V do not have an identification area made up of fill data, the reduced data and the non-reduced data are reconstructed based on the color difference data U and V after the rearrangement. Cannot be identified. However, in the present embodiment, the color difference data U and V and the luminance data Y are made to coincide with the DS area where downsampling is performed, and the color difference data U and V after JPEG compression are the luminance data after JPEG compression. Along with Y, the data is transmitted from the transmission side unit Ut to the reception side unit Ur. Therefore, reduced data and non-reduced data can be identified for the color difference data U and V after rearrangement based on the luminance data Y after rearrangement, and the arrangement can be restored.

  FIG. 19 is a block diagram showing a configuration example of a main part of an image expansion apparatus according to Embodiment 5 of the present invention, and shows a detailed configuration of the image enlargement converter 24 of FIG. The image enlargement converter 24 includes a DS region dividing unit 240 common to YUV, three arrangement restoring units 241 and 241 s for each YUV, and an interpolation processing unit 242 common to YUV.

  The DS area dividing unit 240 divides each of the luminance data Y and the color difference data U and V output from the JPEG decoder 23 into the DS area, and the DS area dividing process similar to that in the first embodiment is performed. Has been done.

  The arrangement restoration unit 241 performs the arrangement restoration of the luminance data Y, and determines whether the target DS area is reduced data or non-reduced data, as in the case of the first embodiment. Based on the above, these data arrangements are restored. On the other hand, the two arrangement restoration units 241s perform the arrangement restoration of the color difference data U and V. The determination of whether the target DS area is reduced data or non-reduced data is for luminance data. The determination result of the arrangement restoration unit 241 is used.

  That is, the arrangement of the reduced data and the non-reduced data is uniquely determined if the order of appearance is determined. For this reason, the arrangement of the reduced data and the non-reduced data in the rearranged color difference data U and V can be known from the determination result of the arrangement restoration unit 241 for the luminance data Y. In this way, the arrangement restoration unit 241s performs the arrangement restoration.

Embodiment 6 FIG.
In the fifth embodiment, an example has been described in which the YUV conversion unit 10 performs conversion to the YUV444 format and the number of converted luminance data and color difference data is the same. On the other hand, in this embodiment, the case where the conversion to YUV422, YUV420, etc. is performed, and the number of data of the converted luminance data and color difference data is further described.

  In general, the human eye has a weak discriminating power for color components compared to a luminance component, and a weak discriminating power in the horizontal direction compared to the vertical direction. In formats such as YUV422 and YUV420, the number of color difference data is smaller than that of luminance data using such visual characteristics. That is, in the YUV conversion unit 10, the number of luminance data Y and the number of color-difference data U and V does not match the image data in which the RGB format is converted into a format such as YUV422 and YUV420. Accordingly, when the luminance data Y and the color difference data U and V are respectively divided into DS areas of the same size in the DS area dividing unit 130, the DS areas of the color difference data U and V and the DS area of the luminance data Y are as follows. It will not match. In the following description, the “color difference DS region” of the DS region of the color difference data U and V and the DS region of the luminance data Y are referred to as “luminance DS region” as necessary.

  FIG. 20 is a diagram illustrating an example of image data in the YUV422 format. In the case of the YUV422 format, the number of horizontal data of the color difference data U and V is half that of the luminance data Y, and each color difference data U and V is associated with two pieces of luminance data Y adjacent in the horizontal direction. . For this reason, the color difference DS region also corresponds to two luminance DS regions adjacent in the horizontal direction.

  (A) in the figure shows the state of the DS area division for the luminance data Y, and (b) in the figure shows the state of the DS area division for the luminance data Y. The image data in the YUV422 format composed of 96 × 96 pixels includes 96 × 96 luminance data Y and 48 × 96 color difference data U and V, respectively. Since the luminance data Y and the color difference data U and V are each divided into 16 × 16 DS regions by the DS region dividing unit 130, the luminance data Y is divided into 36 luminance DS regions (# 1 to # 1). The color difference data U and V are divided into 18 color difference DS regions (# 1 to # 18). In this case, the color difference DS area #n corresponds to the luminance DS areas # 2n-1 and # 2n (n is an integer of 1 to 18).

  The downsampling unit 131s for the color difference data U and V is based on the determination result in the downsampling unit 131 regarding the two luminance DS regions # 2n-1 and # 2n corresponding to the color difference DS region #n. It is determined whether or not downsampling is performed for the region #n. Specifically, when the two luminance DS regions # 2n-1 and # 2n are both important regions, it is determined that the color difference DS region #n is an important region and no downsampling is performed. On the other hand, if one or both of the two luminance DS regions # 2n-1 and # 2n are non-important regions, it is determined that the color difference DS region #n is a non-important region, and downsampling is performed. In FIG. 20, # 21, # 22, # 27, # 28, # 33, and # 34 in the luminance DS area are important areas, so # 11, # 14, and # 17 in the color difference DS area are important areas. However, when one of the two luminance DS regions # 2n-1 and # 2n is an important region and the other is an unimportant region, whether or not the downsampling is performed for the color difference DS region #n is determined in advance. It can be arbitrarily determined.

  FIG. 21 is a diagram showing an example of the operation of the image compression / decompression system according to the sixth embodiment of the present invention. FIG. 21A shows the state of data rearrangement for the luminance data Y shown in FIG. FIG. 21B shows the state of data rearrangement for the color difference data U and V shown in FIG.

  The rearrangement unit 132 is output from the downsampling unit 131 so that the luminance data Y input to the image reduction converter 13 and the luminance data Y output from the image reduction converter 13 match the number of data in the horizontal direction. The reduced data is rearranged. In FIG. 21A, rearrangement is performed so that the number of horizontal data is 96.

  In exactly the same manner, the rearrangement unit 132s matches the color difference data U and V input to the image reduction converter 13 with the number of data in the horizontal direction for the color difference data U and V output from the image reduction converter 13. Furthermore, the reduced data output from the downsampling unit 131s is rearranged. In FIG. 21B, rearrangement is performed so that the number of horizontal data is 48.

  In the present embodiment, the color difference DS area and the luminance DS area do not match, but the color difference DS area is associated with the luminance DS area according to the YUV format. Therefore, as in the case of the fifth embodiment, reduced data and non-reduced data can be identified for the color difference data U and V after rearrangement based on the identification information included in the luminance data Y after rearrangement. Can be restored.

  A JPEG compressor (such as a JPEG chip) is provided with a YUV conversion unit 10 that converts an RGB format into a YUV422 format before the JPEG encoder 14 and adopts an RGB format as a format of input image data. is there. When such a JPEG compressor is used as the image compression unit 11 in FIG. 2, image data in the YUV format output from the image reduction converter 13 may be temporarily converted into the RGB format and input to the JPEG compressor. .

  In this case, the YUV conversion unit 10 at the front stage of the image reduction converter can select an arbitrary format such as YUV444 or YUV422. When YUV422 is selected, the image reduction converter 13 is the same as in the above-described embodiment. On the other hand, when YUV444 is adopted, it is desirable to determine the size of the reduced data in the downsampling process in consideration of the subsequent re-conversion to YUV422 within the JPEG compressor. In other words, for color difference data, it is desirable to combine two adjacent DS regions in the horizontal direction to form a new DS region, and downsample the new DS region.

  However, if the DS area dividing unit 130 changes the size of the DS area relating to the luminance data Y and the color difference data U and V, and the color difference data U and V are divided into the new DS areas, The synthesis process is not necessary. For example, in the case of the YUV422 format, the DS area dividing unit 130 divides the luminance data Y into 16 × 16 pixel DS areas, and the color difference data is 32 × obtained by doubling the number of pixels in the horizontal direction. What is necessary is just to divide | segment into the DS area | region which consists of 16 pixels.

Embodiment 7 FIG.
In Embodiments 5 and 6 described above, the reduced data of the color difference data U and V is rearranged by being packed in one color difference DS area, while the case where such rearrangement is not performed for non-reduced data has been described. . In contrast, in the present embodiment, a case where non-reduced data is rearranged together with reduced data will be described.

  FIG. 22 is a block diagram showing a configuration example of the image reduction converter 13 according to the seventh embodiment of the present invention. This image reduction converter 13 is different from the case of FIG. 16 (Embodiment 5) in that it includes two non-reduction data division units 134.

  The non-reduced data dividing unit 134 divides the non-reduced data output from the downsampling unit 131s for the color difference data U and V into the same size as the reduced data, and outputs it to the rearrangement unit 132s. The rearrangement unit 132s rearranges these data in the same manner as in the case of the reduced data in the fifth or sixth embodiment without distinguishing between the reduced data and the non-reduced data. That is, until the maximum number that can be arranged in one color difference DS area is reached, reduced data or non-reduced data is packed and rearranged for each color difference DS area.

  23 and 24 are diagrams showing an example of the operation of the image compression / decompression system according to the seventh embodiment of the present invention. FIG. 23 is a diagram showing an example up to the non-reduced data dividing process in the image reduction converter 13 of FIG. An example of the color difference data U and V output from the DS area dividing unit 130 is shown in (a) in the figure. The color difference data U and V are the same as those in the case of FIG. 20 (Embodiment 6), and are divided into color difference DS areas # 1 to # 18 made up of 16 × 16 data in the DS area dividing unit 130. ing. Further, it is assumed that the color difference DS areas # 11, # 14, and # 17 are important areas, and the other color difference DS areas are non-important areas.

  (B) in the figure shows reduced data and non-reduced data output from the downsampling unit 131s. Non-important regions are down-sampled and converted to 8 × 8 reduced data. On the other hand, 16 × 16 pieces of data are output as non-reduced data without down-sampling for important regions.

  (C) in the figure shows reduced data and divided data output from the non-reduced data dividing unit 134. The non-reduced data dividing unit 134 outputs the reduced data input from the down-sampling unit 131s as it is, and outputs the non-reduced data by dividing it into four parts having the same size (8 × 8) as the reduced data. Here, non-reduced data # 11, # 14, and # 17 that have not been down-sampled are each divided into four, and 12 divided data # 11a to # 11d, # 14a to # 14d, and # 17a to # 17d. Has been generated.

  FIG. 24 is a diagram illustrating an example of the operation of the rearrangement unit 132s of FIG. The rearrangement unit 132s rearranges the reduced data and the divided data. At this time, as in the case of the fifth embodiment, four color difference DS regions can be rearranged in units, but in this embodiment, they are sequentially arranged in the horizontal direction. That is, starting from the upper left of the image data, the image data is sequentially arranged from left to right in the horizontal direction, and when the right end of the image data is reached, the image data is shifted downward and again sequentially arranged from the left to the right in the horizontal direction. The operation is repeated. In the case of arranging divided data, four divided data constituting one non-reduced data are arranged continuously.

  In this way, when the image reduction converter 13 rearranges the color difference data U and V in the horizontal direction, the image enlargement converter 24 restores the color difference data U and V before the rearrangement to the color difference DS area. There is no need to split. For this reason, the color difference data U and V output from the JPEG decoder 23 are input to the arrangement restoring unit 241 s without passing through the DS area dividing unit 240.

  Non-reduced data that has not been downsampled is larger in size than reduced data, but if non-reduced data is divided into divided data having the same size as the reduced data, reduced data and non-reduced data are mixed. Even in this case, these data can be rearranged with no gaps. Accordingly, the fill data only needs to be added at the bottom in order to shape the image data into a rectangular shape, and the size of the color difference data U and V after rearrangement can be further reduced.

  At the time of restoration in the image enlargement converter 24, the arrangement restoration unit 241 performs not only the arrangement restoration of the reduced data but also the arrangement restoration of the divided data. It goes without saying that the divided data arrangement restoration processing can also be performed based on the identification information included in the luminance data Y, similarly to the reduced data arrangement restoration processing.

  25 and 26 are explanatory diagrams for comparing and explaining the image reduction converters according to the sixth and seventh embodiments. FIG. 25 shows an example of image data in which an important area and an unimportant area repeatedly appear in the horizontal direction. This image data is 128 × 96 pixel image data adopting the YUV422 format. The luminance DS area and the color difference DS area both have a data size of 16 × 16, the luminance data Y is divided into 48 luminance DS areas # 1 to # 48, and the color difference data U and V are 24 color difference DS areas. It is divided into # 1 to # 24. In the image data, an important area and an unimportant area repeatedly appear in the horizontal direction, so that a significant difference occurs in the processing results in the image reduction converter 13 of the sixth and seventh embodiments.

  FIG. 26 shows output data when the image data of FIG. 25 is input to the image reduction converter 13 of the sixth and seventh embodiments. (A) in the figure shows a state after rearrangement according to the sixth embodiment, and (b) shows a state after rearrangement according to the seventh embodiment.

  As described above, in the image reduction converter 13 of the sixth embodiment, an identification area must be inserted into the luminance data Y, but it is not necessary to insert an identification area into the color difference data U and V. For this reason, if the sizes of the luminance data Y and the color difference data U and V after rearrangement are compared, the color difference data U and V are usually smaller. However, in the image data as shown in FIG. 25, since the important area and the non-important area repeatedly appear in the horizontal direction, the luminance data Y and the color difference data U and V in the horizontal direction are similar to the image data in the YUV422 format. When the number of data is different, the above relationship regarding the size after rearrangement may be reversed.

  In (a) in the figure, the luminance data Y is reduced, but the color difference data U and V are not reduced. In order to perform compression processing in the JPEG encoder 14, the vertical lengths of the luminance data Y and the color difference data U and V need to be matched. Fill data is also added to the part surrounded by. As a result, the image data is not reduced at all by the image reduction converter 13.

  On the other hand, in (b) in the figure, both the luminance data Y and the color difference data U and V are reduced, and the image reduction converter 13 reduces the image data. In Embodiment 7, the non-reduced data of the color difference data U and V is divided and efficiently rearranged together with the reduced data. For this reason, even if the important area and the non-important area repeatedly appear, the compression rate does not reverse as in the case of the sixth embodiment.

  The image compression unit 11 and the image expansion unit 21 in each of the above embodiments can be provided as a computer program that can be executed on a computer such as a personal computer. Further, the JPEG encoder 14 and the JPEG decoder 23 may be configured by general-purpose hardware capable of high-speed processing such as a JPEG chip and a PC add-on board, and the image reduction converter 13 and the image enlargement converter 24 may be configured by a computer program. These computer programs are provided by being stored in, for example, an optical storage medium such as a CD-ROM, a magnetic storage medium such as a flexible disk, or a semiconductor storage medium such as an IC memory. It can also be provided via a telecommunication line such as the Internet or a LAN (Local Area Network).

1 is a block diagram illustrating a configuration example of an image compression / decompression system according to Embodiment 1 of the present invention. It is the block diagram which showed one structural example of the transmission side unit Ut of FIG. It is the block diagram which showed one structural example of the receiving side unit Ur of FIG. FIG. 3 is a block diagram illustrating a configuration example of the image compression unit 11 in FIG. 2, in which detailed configuration examples of the image reduction converter 13 and the JPEG encoder 14 are illustrated. It is the figure which showed an example of DS area division and downsampling in the image compression part. 3 is a block diagram illustrating a detailed configuration example of a downsampling unit 131. FIG. FIG. 10 is a diagram illustrating an example of the operation of the image reduction converter 13. It is the figure which showed an example of quantization table T1. FIG. 4 is a block diagram showing a configuration example of the image expansion unit 21 in FIG. 3, in which detailed configuration examples of the JPEG decoder 23 and the image enlargement converter 24 are shown. It is the figure which showed all the arrangement | positioning states in DS area | region in case DS area | region consists of 16x16 pixels and reduction data consists of 8x8 pixels. It is the figure which showed the mode in the case of dividing | segmenting image data into 32 * 32 pixel DS area | region, down-sampling to 8x8 pixel reduction data, and rearranging. It is the figure which showed an example of operation | movement of the image compression / decompression system by Embodiment 3 of this invention. It is the figure which showed all the arrangement | positioning states in DS area | region in case DS area | region is 16x16 pixel and reduction data are 8x8 pixel. It is the figure which showed an example of operation | movement of the image compression / decompression system by Embodiment 4 of this invention. FIG. 15 is a diagram illustrating an example of the operation of the image compression / decompression system following FIG. 14. A detailed configuration of the image reduction converter 13 according to the fifth embodiment of the present invention is shown. It is the block diagram which showed the detailed structural example of the downsampling part 131s of FIG. It is the figure which showed an example of the operation | movement of the image compression / decompression system by Embodiment 5 of this invention. A detailed configuration of the image enlargement converter 24 according to the fifth embodiment of the present invention is shown. It is the figure which showed an example of the image data of a YUV422 format. It is the figure which showed an example of operation | movement of the image compression / decompression system by Embodiment 6 of this invention. It is the block diagram which showed one structural example of the image reduction converter 13 by Embodiment 7 of this invention. It is the figure which showed an example to the division process of the non-reduced data in the image reduction converter 13 of FIG. It is the figure which showed an example of operation | movement of the rearrangement part 132s of FIG. It is the figure which showed an example of the image data in which an important area | region and an unimportant area | region appear repeatedly in the horizontal direction. The output data when the image data of FIG. 25 is input to the image reduction converter 13 of the sixth and seventh embodiments is shown.

Explanation of symbols

13 Image reduction converter 14 JPEG encoder 23 JPEG decoder 24 Image enlargement converter 130 DS region division unit 131, 131s Downsampling unit 132, 132s Relocation unit 134 Non-reduction data division unit 140 Block division unit 141 DCT processing unit 142 Quantization processing unit 143 Encoding unit T1 Quantization table T2 Code table 230 Decoding unit 231 Inverse quantization processing unit 232 Inverse DCT processing unit 240 DS region division unit 241, 241s Arrangement restoration unit 242 Interpolation processing unit

Claims (12)

A downsampling area dividing step for dividing the image data into downsampling areas consisting of an integral multiple of the block size;
A downsampling step for reducing the number of pixels by performing downsampling for at least a part of the downsampling region, and generating reduced data consisting of an integral multiple of the block size;
A rearrangement step of disposing two or more pieces of reduced data in one downsampling region, inserting fill data into the remaining pixels in the downsampling region, and removing a downsampling region in which the reduced data is not placed;
A data compression step for dividing the image data on which the rearrangement step has been performed into block regions having the block size, and performing compression processing including orthogonal transformation and quantization processing for each block region to generate compressed image data; ,
A data decompression step for restoring image data by performing decompression processing including inverse quantization processing and inverse orthogonal transform for each block area on the compressed image data;
An arrangement restoration step for restoring the arrangement of the reduced data included in the restored image data;
An image compression / decompression method comprising: an interpolation step of interpolating pixel data and restoring image data including a downsampling area for each reduced data included in the restored image data. Down-sampling area dividing means for dividing the image data into down-sampling areas consisting of an integral multiple of the block size;
Downsampling means for reducing the number of pixels by performing downsampling for at least a part of the downsampling region, and generating reduced data consisting of an integral multiple of the block size;
Relocation means for disposing two or more pieces of reduced data in one downsampling region, inserting fill data into the remaining pixels in the downsampling region, and removing a downsampling region in which the reduced data is not arranged;
Data compression means for dividing the image data generated by the rearrangement means into block areas having the block size and generating compressed image data by performing compression processing including orthogonal transformation and quantization processing for each block area; An image compression apparatus comprising:   3. The image compression apparatus according to claim 2, wherein the rearrangement unit arranges the reduced data at a predetermined position in the downsampling area, leaving an identification area having a block size.   3. The image compression apparatus according to claim 2, wherein the rearrangement unit adds a downsampling area composed of fill data to the image data after rearrangement, and shapes the image data into a rectangular shape. Compression processing including rearrangement of reduced data consisting of an integral multiple of the block size obtained by down-sampling part of the image data together with fill data, and further including orthogonal transformation and quantization processing for each block area of the block size An image decompression device for decompressing compressed image data subjected to
Data decompression means for restoring image data by performing decompression processing including inverse quantization processing and inverse orthogonal transformation for each block area on the compressed image data;
An arrangement restoration means for restoring the arrangement of the reduced data based on the fill data included in the restored image data;
An image expansion apparatus comprising interpolation means for interpolating pixel data and restoring image data before downsampling for each reduced data included in the restored image data.   The rearrangement unit changes the pixel data in the block area when the average value of the pixel data in the block area in which the fill data is not inserted in the downsampling area does not have a predetermined distance from the fill data. The image compression apparatus according to claim 2.   The image compression apparatus according to claim 6, wherein the fill data is a median value of pixel data.   8. The image compression apparatus according to claim 7, wherein the predetermined distance is determined based on a quantization step width for a DC coefficient in the data compression unit. The downsampling area dividing means divides the image data into two or more downsampling areas having different sizes,
The image compression apparatus according to claim 2, wherein the downsampling unit performs downsampling with different reduction ratios according to the size of the downsampling area to generate reduced data of the same size. The down-sampling means performs down-sampling for at least a part of the down-sampling region to generate intermediate reduced data having a block size,
Collecting the intermediate reduced data generated by the downsampling, generating a new downsampling area, downsampling the new downsampling area again, and generating reduced data having a block size. The image compression apparatus according to claim 2. Downsampling area division that divides luminance data constituting image data into luminance downsampling areas consisting of integer multiples of the block size and divides the color difference data constituting the image data into color difference downsampling areas consisting of integer multiples of the block size Means,
Luminance downsampling means for downsampling luminance data to reduce the number of data and generating reduced luminance data consisting of an integral multiple of the block size for at least some luminance downsampling regions;
Luminance data rearrangement means for arranging two or more pieces of reduced luminance data in one luminance downsampling area and inserting fill data into pixels left in the luminance downsampling area;
For the color difference downsampling area determined based on the downsampled luminance downsampling area, the color difference is generated by reducing the number of data by downsampling the color difference data and generating reduced color difference data consisting of an integral multiple of the block size. Down-sampling means;
Color difference data rearranging means for arranging two or more reduced color difference data in one color difference downsampling area;
The luminance data and color difference data generated by the luminance data rearrangement unit and the color difference data rearrangement unit are divided into block areas having the block size, and compression processing including orthogonal transformation and quantization processing for each block area is performed. Data compression means for generating compressed image data by performing,
The luminance data rearranging means arranges the reduced luminance data at a predetermined position in the luminance downsampling area, leaving an identification area having a block size. A non-reduced data dividing unit that divides a color difference down-sampling area that has not been down-sampled into the same size as the reduced color difference data and generates divided color difference data,
12. The image compression apparatus according to claim 11, wherein the color difference data rearrangement unit rearranges the reduced color difference data and the divided color difference data.

Images