JP4245576B2 - Image compression / decompression method, image compression apparatus, and image expansion apparatus - Google Patents

Description

  The present invention relates to an image compression / decompression method, an image compression apparatus, and an image expansion apparatus. More specifically, the present invention relates to image compression such as MPEG in which motion compensation prediction is performed for each motion compensation area composed of a plurality of block areas into which image data is divided. It relates to the improvement of the stretching method.

  An MPEG (Moving Picture Experts Group) standard is known as a method for compressing and expanding a moving image composed of a plurality of still images. The MPEG standard is an image processing standard standardized by ISO (International Standards Organazasion) or the like, and defines a compression / decompression method of image data performed by motion compensation prediction based on inter-frame differences.

  A data compressor according to the MPEG standard (hereinafter referred to as an MPEG compressor) divides input image data into a large number of block areas for each frame, with 8 × 8 pixels as one block, and from four adjacent block areas. Motion compensation prediction is performed for each motion compensation region, and encoding processing is performed for each block region. In this motion compensation prediction, the motion for each motion compensation region is predicted based on the difference between the current frame and the previous and subsequent frames. In the encoding process, the difference value between the frames is calculated based on the motion prediction result. It becomes.

  DCT (Discrete Cosine Transform) is an orthogonal transformation that transforms spatial coordinates relating to pixel data into frequency coordinates. In the quantization process, the quantization factor Q is converted into data defined for each DCT coefficient by a quantization table. A value multiplied by is used as the quantization step width. The amount of data is irreversibly reduced by quantizing the DCT coefficient obtained by the DCT processing with the quantization step width.

  On the other hand, in a data decompressor according to the MPEG standard (hereinafter referred to as an MPEG decompressor), the input compressed image data is decoded for each block area, and motion compensation prediction is performed for each motion compensation area. Is restored.

  In the above-described MPEG compressor, when the data compression rate is to be increased, it is necessary to change the quantization table or the quantization factor Q to increase the quantization step width. However, if a large amount of data is reduced in the quantization process, which is an irreversible process, the quality of the restored image data is significantly degraded. Moreover, since this quality degradation occurs over the entire image, there is a problem that even if there are important and non-important regions in the image, the image quality is uniformly degraded in both regions. .

  Therefore, as preprocessing of the MPEG compression process, it is conceivable to divide the image data into a plurality of small areas and perform a downsampling process for reducing the number of pixels in some small areas. Specifically, the image data is divided into small regions each having an integral multiple of the block size, and reduced data matching the block size is generated by downsampling. Then, the reduced data and the small area where the downsampling has not been performed are sequentially arranged from the top of the image data to generate reduced image data. When the reduced data is arranged, the reduced data is arranged in order in the small area, and the fill data is inserted into the remaining part in the small area. This fill data is pixel data that is inserted to identify whether downsampling has been performed when restoring image data, and is always inserted in small areas other than the small areas that have not been downsampled. Is done.

  If this method is used, the process of reducing the data amount for a part of small areas is performed as a pre-process of the data compression process, so that the data compression rate can be improved while using a general-purpose image compressor. In particular, when there are an important area and an unimportant area in the image, if the data amount is reduced only for the unimportant area, the data compression rate can be improved without degrading the image quality in the important area.

  However, when an image compressor such as an MPEG compressor that performs motion compensation prediction based on inter-frame differences is used, fill data is rewritten by motion prediction for each motion compensation region, and image data cannot be restored. It was. That is, in motion prediction, a peripheral region including the motion compensation region in the preceding and succeeding frames with respect to the motion compensation region of the current frame becomes a subject of motion prediction, and motion is identified based on a difference from this peripheral region. As a result, a difference value between an area different from the motion compensation area of the current frame may be encoded. In such a case, the fill data arranged in the small area is changed and processed in the same manner as the small area that has not been down-sampled, so that the compressed image data can be restored appropriately. It becomes difficult.

  In addition, when the important area is composed of a plurality of adjacent small areas, when generating the reduced image data, the important area is divided and rearranged for each small area. There was also a problem that it was not done.

  The present invention has been made in view of the above circumstances, and it is intended to improve the data compression rate in an image compression / decompression method using an image compressor such as an MPEG compressor that performs motion compensation prediction based on interframe differences. Objective. In particular, an object of the present invention is to provide an image compression / decompression method that can use a general-purpose image compressor and image decompressor and can improve the data compression rate without degrading the image quality in an important region in the image. To do. It is another object of the present invention to provide an image compression / decompression method capable of appropriately performing motion prediction for important regions in an image.

  The image compression / decompression method according to the first aspect of the present invention includes an image compression method and an image expansion method. This image compression method includes a downsampling area dividing step, a non-reduced area designating step, a downsampling step, a rearrangement step, and a data compression step. The image expansion method includes a data expansion step, an arrangement restoration step, and an interpolation step.

  The down-sampling area dividing step divides the image data into two or more down-sampling areas having n times the block size (n is an integer of 2 or more). In the non-reduced area designation step, an adjacent area consisting of the downsampling area is designated as a non-reduced area. In the down-sampling step, down-sampling is performed for each down-sampling area except the non-reduced area to reduce the number of pixels, and reduced data consisting of m times the block size (m is an integer of 1 or more smaller than n) is generated. To do. The rearrangement step arranges the non-reduced area and each reduced data, and generates reduced image data including the size and position information of the non-reduced area as header information. The data compression step divides the reduced image data into block areas each having a block size, performs motion compensation prediction for each motion compensation area including a plurality of adjacent block areas, performs encoding processing for each block area, and generates a compressed image. Generate data. With such a configuration, the image data can be spatially compressed as preprocessing of the data compression step by down-sampling a part of the image area, rearranging the non-reduced area and the reduced data. In addition, if the non-reduced area is designated as the important area, when the reduced image data is generated, the non-reduced area is rearranged without being divided, so that motion prediction for the non-reduced area is appropriately performed. Can do.

  In the data decompression step, the image data is restored based on the decoding process for each block area and the motion compensation prediction for each motion compensation area for the compressed image data. The arrangement restoration step restores the non-reduced area and the arrangement of each reduced data included in the restored image data based on the header information. In the interpolation step, pixel data is interpolated for each reduced data included in the restored image data, and restored to image data including a downsampling area. With such a configuration, since the arrangement of the non-reduced area and the reduced data is restored based on the header information, even if the size or position of the non-reduced area is changed, the spatially compressed image data Can be properly restored by post-processing of the data decompression step.

  According to a second aspect of the present invention, there is provided an image compression apparatus comprising a downsampling area dividing means for dividing image data into two or more downsampling areas having an integral multiple of a block size, and an adjacent area consisting of the downsampling areas as non-reduced areas. Non-reduced area designating means for specifying, down-sampling means for reducing the number of pixels by performing down-sampling for each down-sampling area excluding the non-reduced area, and generating reduced data consisting of an integral multiple of the block size; A rearrangement unit that arranges the non-reduced area and the respective reduced data, and generates reduced image data including size and position information of the non-reduced area as header information, and a block area including the reduced image data of the block size Divided into multiple block areas adjacent to each other It performs motion compensation prediction for each 償領 area, and a data compressing means for generating compressed image data by performing encoding processing for each block area.

  In addition to the above configuration, the image compression apparatus according to the third aspect of the present invention is configured such that the down-sampling area dividing means divides the image data into down-sampling areas consisting of an integral multiple of the motion compensation area. In this image compression apparatus, the image data is divided into down-sampling areas that are k times the motion compensation area as a motion prediction processing unit (k is an integer equal to or greater than 1), and the adjacent areas including such down-sampling areas are divided. Designated as non-reduced area. According to such a configuration, since the non-reduced region is always an integer multiple of the motion compensation region, it is possible to appropriately perform motion prediction for the non-reduced region.

  In addition to the above-described configuration, the image compression apparatus according to the fourth aspect of the present invention is configured such that the non-reduction area designating unit designates a rectangular adjacent area consisting of an integral multiple of the motion compensation area as the non-reduction area. . According to such a configuration, since the non-reduced area is always an integer multiple of the motion compensation area, a general-purpose image compressor such as an MPEG compressor is used as a data compression unit, and motion prediction for the non-reduced area is performed. Can be performed appropriately.

  In addition to the above configuration, the image compression apparatus according to the fifth aspect of the present invention is configured such that the rearrangement unit arranges the non-reduced area at the head of the reduced image data. According to such a configuration, the position of the non-reduced area in the reduced image data is defined in advance, so that the reduced image data can be correctly restored. That is, when the arrangement of the non-reduced area and the reduced data is restored, the non-reduced area and the reduced data for the reduced image data are based on the size information of the non-reduced area included as header information in the reduced image data. Therefore, the arrangement of the non-reduced area and the reduced data can be correctly restored.

  In addition to the above configuration, the image compression apparatus according to the sixth aspect of the present invention is configured such that the rearrangement unit generates reduced image data having the same width as the horizontal width of the non-reduced region. According to such a configuration, with respect to the rectangular non-reduced area arranged at the head in the reduced image data, the side in contact with the reduced data is only one side in the horizontal direction. It is possible to effectively suppress the difference value from the reduced data being encoded.

  In the image compression apparatus according to a seventh aspect of the present invention, in addition to the above-described configuration, the rearrangement unit adds a block area including fill data to the image data, shapes the image data into a rectangular shape, and generates reduced image data. Configured to do. According to such a configuration, a general-purpose image compressor such as an MPEG compressor can be used as the data compression means.

  An image expansion apparatus according to an eighth aspect of the present invention rearranges reduced data consisting of an integral multiple of a block size obtained by down-sampling a part of image data together with a non-reduced area, and further from a plurality of adjacent block areas. An image decompressing apparatus that decompresses compressed image data that has been subjected to encoding processing based on motion compensation prediction for each motion compensation region, and that performs decoding processing on each compressed block data block region having the block size. And a data decompression unit that performs motion compensation prediction for each motion compensation region and restores the image data, and includes the non-reduced region and the arrangement of each reduced data included in the restored image data in header information in the image data. And an arrangement restoration means for restoring based on the size and position information of the non-reduced area included in the image data, and included in the restored image data Interpolating the pixel data for each reduced data, and includes an interpolation means for restoring the composed image data by a plurality of down-sampling areas consisting of an integral multiple of the block size.

  According to the present invention, it is possible to improve a data compression rate in an image compression / decompression method using an image compressor such as an MPEG compressor that performs motion compensation prediction based on an inter-frame difference. In particular, general-purpose image compressors and image decompressors can be used, and the data compression rate can be improved without degrading the image quality in important areas in the image. In addition, it is possible to appropriately perform motion prediction for important regions in the image.

Embodiment 1 FIG.
<Image transmission system>
FIG. 1 is a block diagram showing a configuration example of an image compression / decompression system according to Embodiment 1 of the present invention, and shows an example of an image transmission system. This image transmission system includes a transmission side unit Ut and a reception side unit Ur connected by a communication network 100, and can compress image data and transmit the compressed image data from the transmission side unit Ut to the reception side unit Ur. Here, the image data of the image input apparatus 101 connected to the transmission side unit Ut is transmitted to the image output apparatus 102 connected to the reception side unit Ur.

  The communication network 100 includes a wired or wireless communication line for transmitting digital data, and includes an exchange, a repeater, and the like as necessary. For example, packet communication networks such as Ethernet (registered trademark), the Internet, ATM (Asynchronous Transfer Mode), and other digital networks can be used.

  The image input device 101 is a device that provides image data, and includes, for example, an imaging device such as a camera, an image reading device such as a scanner, and a data storage device such as an HDD (Hard Disc Drive). In the present embodiment, it is assumed that the image input device 101 generates image data of a still image having an RGB format and outputs it to the transmission side unit Ut. Note that a still image in this specification means an image composed of a large number of pixels arranged in a two-dimensional manner, and includes each frame image constituting a moving image, a difference image between frames, and the like.

  The image output device 102 uses image data output from the receiving unit Ur, and includes, for example, a display device such as an LCD, an image forming device such as a printer, and a data storage device such as an HDD.

<Sender unit>
FIG. 2 is a block diagram illustrating a configuration example of the transmission side unit Ut of FIG. The transmission side unit Ut includes a YUV conversion unit 10, an image compression unit 11, and a data transmission unit 12. The image data generated by the image input apparatus 101 is first converted into image data in the YUV format by the YUV conversion unit 10 and then compressed by the image compression unit 11 to become compressed data with a reduced data amount. The compressed data is sent to the communication network 100 by the data transmission unit 12.

  The YUV conversion unit 10 is a format conversion unit that converts image data in RGB format into image data in a predetermined YUV format (for example, YUV410, 411, 420, 422, 444). Note that the YUV conversion unit 10 is omitted when YUV format image data is input from the image input device 101 or when JPEG compression is performed with the RGB format image data as it is.

  The image compression unit 11 includes an image reduction converter 13 and a JPEG encoder 14. The image reduction converter 13 divides the input image into a large number of small areas, reduces the resolution of each small area except for some specified areas, and further reduces the size of the entire image by rearranging the image data. ing. This reduction in resolution is performed for each of the small regions according to the required image quality. On the other hand, the JPEG encoder 14 performs compression processing on the image data according to the JPEG standard to generate compressed data with a further reduced data amount.

<Receiving unit>
FIG. 3 is a block diagram showing a configuration example of the receiving side unit Ur of FIG. The receiving side unit Ur includes an RGB conversion unit 22, an image expansion unit 21, and a data reception unit 20. The compressed data sent from the transmitting unit Ut to the communication network 100 is received by the data receiving unit 20. The received compressed data is decompressed by the image decompressing unit 21, and image data in the YUV format is restored. The image data is converted into the RGB format by the RGB conversion unit 22 and output to the image output device 102.

  The image expansion unit 21 includes a JPEG decoder 23 and an image enlargement converter 24. The compressed data from the data receiving unit 20 is decompressed by the JPEG standard in the JPEG decoder 23, and the image data before JPEG compression is restored. The expanded image data is increased in resolution by the image enlargement converter 24 to increase the image size, and the image data before the resolution reduction is restored.

  The RGB conversion unit 22 converts the YUV format image data output from the image enlargement converter 24 into the RGB format and outputs the converted data to the image output device 102. Note that the RGB converter 22 is omitted when YUV format image data is input to the image output apparatus 102 or when JPEG compression is performed with the RGB format image data as it is.

<Image compression unit>
FIG. 4 is a block diagram showing a configuration example of the image compression unit 11 in FIG. 2, and shows detailed configuration examples of the image reduction converter 13 and the JPEG encoder 14. The image reduction converter 13 includes a downsampling area dividing unit (DS area dividing unit) 130, a non-reducing area specifying unit 131, a downsampling unit 132, and a rearrangement unit 133.

[DS area division unit]
The DS area dividing unit 130 divides the YUV-converted image data into a plurality of downsampling areas (hereinafter referred to as DS areas). Each DS area can be an area having an arbitrary number of pixels and an arbitrary shape. However, in consideration of compression efficiency, a block used in the JPEG encoder 14 is a basic unit, and an aggregate of the blocks is used. It is desirable to use the DS region.

  Since the block size of JPEG is 8 × 8 pixels, 16 × 16 pixels, 32 × 32 pixels, 24 × 16 pixels, 8 × 32 pixels, etc. are conceivable as DS region sizes that are integer multiples thereof.

  In addition, each DS region may have a different size or a different shape, but it is desirable to have the same size and the same shape in consideration of the ease of rearrangement processing described later. A rectangular shape (particularly a square shape) is more desirable. Hereinafter, in the present embodiment, an example in which the entire image area is divided into DS areas of the same size composed of 16 × 16 pixels will be described.

[Non-reduced area specification part]
The non-reduced area designating unit 131 performs an operation of designating a DS area where no downsampling is performed as a non-reduced area. This non-reduced area is designated based on an operation input by an operator, for example, and can be changed for each input image. Specifically, the image quality is maintained in the resolution reduction processing of the image data, and an adjacent region including one or more DS regions is designated as an unreduced region as an important region.

  Here, a rectangular area is designated as an important area (non-reduced area). Note that an important area in an image may be determined based on an output signal of a sensor (not shown) and automatically designated as a non-reduced area. It is also possible to determine the important area based on the image data. For example, based on a comparison result between frames of moving images, it is possible to determine a region with motion as an important region. It is also possible to determine a flat image area with relatively little change as an unimportant area.

[Downsampling section]
The down-sampling unit 132 performs down-sampling of each pixel data constituting the DS area for each DS area excluding a part of designated areas (important areas), and converts the data into reduced data in which the number of pixels is reduced. . The size of the reduced data is preferably set to an integral multiple of the block size in consideration of the compression rate in the JPEG encoder, and here it is assumed to match the block size (8 × 8 pixels).

  The downsampling process is performed only for the DS area other than the non-reduced area specified by the non-reduced area specifying unit 131, and the input data is output as it is for the DS area in the non-reduced area, and the DS area outside the non-reduced area For, reduced data is output.

  Such a downsampling process is realized by a thinning process for extracting pixels in the DS region, a filter process using a low-pass filter, or the like. At this time, the downsampling process may be realized only by the thinning process, but by using the filter process and the thinning process in combination, a highly accurate downsampling process with less distortion can be realized.

  For example, in processing using a Gaussian filter as a low-pass filter, a one-dimensional Gaussian filter is applied in the horizontal direction of the image data, and as a result, the same filter is applied in the vertical direction to the obtained data. By performing the thinning process on the image data filtered by the Gaussian filter in this way, downsampling can be performed with high accuracy.

[Relocation section]
The rearrangement unit 133 rearranges the output data of the downsampling unit 132 to generate reduced image data in which the size of the entire image is reduced, and outputs the reduced image data to the JPEG encoder 14. Specifically, the input data including the non-reduced area and each reduced data is arranged in order from the top in an image area having a predetermined horizontal width (horizontal resolution). At that time, the non-reduced region is arranged by slicing the same vertical width (8 pixels) as the reduced data. Reduced image data with a reduced image size is generated by such rearrangement of image data.

  Here, it is assumed that reduced image data including, as header information, a flag for determining whether or not downsampling has been performed for each block is generated. The receiving unit Ur can restore the non-reduced area and the arrangement of each reduced data based on the header information included in the reduced image data. When image data having a different horizontal width is rearranged in the transmission side unit Ut, the horizontal width of the image data before the rearrangement may be included in the header information. In this way, the arrangement of the non-reduced area and the reduced data can be correctly restored even for image data having different horizontal widths.

  Further, the JPEG encoder 14 cannot process image data having a shape other than a rectangle. For this reason, the rearrangement unit 133 adds a block composed of fill data (hereinafter, such a block is referred to as a fill block) to the end of the image data, and shapes the reduced image data into a rectangular shape. This fill data is predetermined arbitrary pixel data. From the viewpoint of improving the data compression rate by reducing the number of fill blocks inserted into the reduced image data, it is desirable that the horizontal width of the reduced image data be shorter than that of the original image data. In this way, the fill block inserted at the end of the image data is reduced and the data amount of the reduced image data is reduced, so that the data compression rate can be improved.

  Further, since the image data in the YUV format is composed of one luminance data Y and two color difference data U and V, the DS area dividing unit 130, the downsampling unit 132, and the rearrangement unit 133 have these three data. Each of the above-described processes is performed.

<JPEG encoder>
The JPEG encoder 14 includes a block dividing unit 140, a DCT processing unit 141, a quantization processing unit 142, and an encoding unit 143 (see FIG. 4). The reduced image data output from the image reduction converter 13 is divided into a plurality of block areas (JPEG blocks) each having 8 × 8 pixels by the block dividing unit 140. A DCT (Discrete Cosine Transform) processing unit 141 obtains a DCT coefficient by performing a discrete cosine transform for each divided block region. Each DCT coefficient obtained in this way is quantized by the quantization processing unit 142 using the quantization table.

  This quantization table is made up of data for defining the quantization step width, and is predetermined for each horizontal and vertical frequency component. Usually, different quantization tables are used for quantization processing of luminance data and color difference data constituting image data.

  When quantizing the DCT coefficient, the quantization processing unit 142 reads data corresponding to the DCT coefficient from the quantization table, and further quantizes a value obtained by multiplying the read data by a quantization factor (quantization coefficient) Q. This is used as the step size. The quantization factor Q is an arbitrary value for adjusting the compression rate and the image quality, and is determined in advance. If the quantization factor Q is increased, the quantization step width is increased and the data compression rate can be improved, but with this, block distortion occurs and the image quality is lowered.

  The encoding unit 143 performs a run-length transform process in the block area for the AC coefficient that is an AC component among the quantized DCT coefficients, and the DC coefficient that is a DC component is between the block areas. After performing the difference process, an encoding process using an entropy code is performed on these data. The entropy code is a code system having a code length corresponding to the appearance probability, and the Huffman code is widely known. Such an entropy code is read out from the code table and encoded, and compressed data is generated.

<Image expansion part>
FIG. 5 is a block diagram showing an example of the configuration of the image expansion unit 21 in FIG. 3, in which detailed configuration examples of the JPEG decoder 23 and the image enlargement converter 24 are shown.

[JPEG decoder]
The JPEG decoder 23 includes a decoding unit 230, an inverse quantization processing unit 231, and an inverse DCT processing unit 232. The JPEG decoder 23 performs processing opposite to that of the JPEG encoder 14 to decompress the compressed data, and converts the image data before JPEG compression. Restoring. Note that the quantization table and the code table need to use the same data table as that of the JPEG encoder 14, and can be added to the compressed data as necessary and transmitted from the transmission side unit Ut to the reception side unit Ur.

  The decoding unit 230 decodes the Huffman code in the compressed data using the code table, and further decodes the differentially processed DC coefficient and the run-length converted AC coefficient. The decoded data is inversely quantized using a quantization table in the inverse quantization processing unit 231 to restore the DCT coefficients for each block area. This DCT coefficient is subjected to inverse DCT processing in the inverse DCT processing unit 232, and image data before JPEG compression processing, that is, image data reduced by the image reduction converter 13 is restored.

[Image Enlargement Converter]
The image enlargement converter 24 includes a DS region dividing unit 240, an arrangement restoration unit 241, and an interpolation processing unit 242, and performs reverse processing to the case of the image reduction converter 13 to restore image data before the reduction process. . First, the DS area dividing unit 240 divides the image area into a plurality of DS areas in order to arrange the image data output from the JPEG decoder 23. The DS area has the same size (here, 16 × 16 pixels) as the DS area used by the DS area dividing unit 130 of the transmission side unit Ut.

  The arrangement restoration unit 241 arranges the non-reduced area and each reduced data in each DS area based on the flag included as header information in the image data, and restores the arrangement of the image data before the reduction process.

  The interpolation processing unit 242 restores the original size before downsampling (that is, the size of the DS region) by interpolating between the reduced data output from the arrangement restoring unit 241 to increase the number of pixels. For this interpolation process, an LP enlargement method, linear interpolation, quadratic interpolation, cubic interpolation, or the like can be used. The LP enlargement method is a method of restoring an image by estimating high-frequency components lost during downsampling by the downsampling unit. On the other hand, no interpolation processing is performed for the non-reduced region. In this way, the image data before the size reduction by the image reduction converter 13 is restored and output to the RGB conversion unit.

  Since the compressed data of the image having the YUV format is composed of compressed data for one luminance data Y and two color difference data U and V, the DS area dividing unit 240, the arrangement restoring unit 241, and the interpolation processing unit. 242 performs the above-described processing on these three pieces of compressed data.

  6A and 6B are diagrams showing an example of DS area division and downsampling in the image compression unit 11. (A) in the figure shows a case where the entire image area is divided into a DS area A1 of 16 × 16 pixels. BL in the figure is an 8 × 8 pixel JPEG block.

  The DS area dividing unit 130 matches the reference position (upper left of the image area in the drawing) with the block division in the JPEG encoder 14 so that the DS area A1 is an aggregate of JPEG blocks BL. The down-sampling unit 132 down-samples the DS area A1 other than the non-reduced area so that the number of pixels in the DS area is halved in both the vertical and horizontal directions, and converts it into reduced data of 8 × 8 pixels. Similarly, (b) in the drawing shows a case where the entire image area is divided into a DS area A2 of 32 × 32 pixels. The DS region A2 divided by the DS region dividing unit 130 is converted by the downsampling unit 132 into 8 × 8 pixel reduced data obtained by downsampling each of the vertical and horizontal directions to ¼.

  FIG. 7 is a diagram illustrating an example of the operation of the image reduction converter 13. The processing by the DS region dividing unit 130, the downsampling unit 132, and the rearrangement unit 133 is started from the upper left of the image data B1, and sequentially shifts from left to right in the horizontal direction (x-axis direction). . If the processing is completed up to the right end of the image data B1, the processing target is shifted in the vertical direction (y-axis direction), and the same processing is repeated for the region located immediately below the processed region.

  The DS area dividing unit 130 divides the entire image area into 12 × 7 DS areas composed of 16 × 16 pixels with respect to the image data input from the image input device 101, and downsamples the image data of each DS area. Sequentially output to the unit 132.

  The downsampling unit 132 determines, for each DS region sequentially output from the DS region dividing unit 130, whether or not the DS region is an important region based on an instruction from the non-reduced region specifying unit 131. As a result, when it is determined that the region is an important region, pixel data of 16 × 16 pixels is output without performing downsampling. On the other hand, if it is determined that the region is not an important region, downsampling is performed to reduce the number of vertical and horizontal pixels to ½, and reduced data composed of 8 × 8 pixels is output. When the image data B2 is output, the reduced data is output for each DS region, and the important region is sliced into data having the same vertical width (8 pixels) as the reduced data (hereinafter referred to as non-reduced data). Is output.

  In the case where the important area has a rectangular shape formed by arranging four DS areas in the horizontal direction and two DS areas in the vertical direction, the pixel data constituting the important area is divided into four non-reduced data b1 to b4. Is output. At that time, the leading non-reduced data b1 is sequentially output following the output of the reduced data located on the left side on the horizontal line to which the non-reduced data belongs, and when the output of the non-reduced data b1 is completed, Output of the reduced data located on the right side on the line is started. When all the output of the reduced data on the horizontal line is completed, the next non-reduced data b2 is output. When the output of the non-reduced data is completed, the left side on the next horizontal line to which the non-reduced data b3 belongs is displayed. The output of the reduced data positioned is started.

  FIG. 8 is a diagram showing an example of the operation of the image reduction converter 13, and shows the reduced image data B3 after the important area and the reduced data are rearranged. The rearrangement unit 133 arranges the image data from the downsampling unit 132 in order from the upper left of the image area.

  At that time, the reduced data is arranged for each block region from left to right, and the non-reduced data b1 to b4 are arranged in order for each non-reduced data. When all the data in the image data B2 is rearranged, fill data is inserted at the end of the image data, and the reduced image data B3 is completed. In this example, reduced image data composed of 16 × 7 JPEG blocks is generated, and four fill blocks are added.

  According to the present embodiment, it is possible to improve the data compression rate while using a general-purpose image compressor. In particular, since the reduced image data is generated by including information related to downsampling in the header information, the processing load is reduced and image compression / expansion processing is performed as compared with the case where an identification fill block is arranged in the DS area. The speed can be increased.

  In the present embodiment, an example in which compressed data is transmitted between a transmission side unit and a reception side unit via a communication network has been described, but the present invention is not limited to this. For example, the present invention can also be applied to an image compression / decompression apparatus that inputs / outputs compressed data to / from a data storage device such as an HDD.

Embodiment 2. FIG.
In the first embodiment, the example in which the resolution of the image data is reduced as the preprocessing of the image compression processing by the JPEG encoder has been described. On the other hand, in the present embodiment, a case will be described in which the resolution of image data is reduced as preprocessing of image compression processing by an MPEG encoder.

<Image compression unit>
FIG. 9 is a block diagram showing a configuration example of the image compression unit 30 according to the second embodiment of the present invention, and shows a detailed configuration example of the image reduction converter 31 and the MPEG encoder 32. The image reduction converter 31 includes a DS area dividing unit 130, a non-reduction area specifying unit 131, a downsampling unit 132, and a rearrangement unit 33.

  The DS area dividing unit 130 uses the blocks used for the encoding process in the MPEG encoder 32 as a basic unit when dividing the YUV-converted image data into a plurality of DS areas, and uses the aggregate as the DS area. In particular, from the viewpoint of appropriately performing motion prediction based on inter-frame differences at the time of encoding, each DS region is composed of an integer multiple of the motion compensation region used for motion prediction in the MPEG encoder 32.

  Specifically, since the area size of the motion compensation area in MPEG is 16 × 16 pixels, the DS area size that is an integral multiple of 16 × 16 pixels, 32 × 32 pixels, 32 × 16 pixels, etc. Can be considered. Here, it is assumed that each DS region has the same size and shape (16 × 16 pixels) as the motion compensation region.

  The downsampling unit 132 generates reduced data that is an integral multiple of the block size (8 × 8 pixels) used in the encoding process in the MPEG encoder 32 when performing downsampling for each DS region except the important region. . Here, it is assumed that reduced data matching the block size is generated.

  When the rearrangement unit 33 rearranges the output data of the downsampling unit 132, the rearrangement unit 33 arranges the important region (non-reduction region) at the top of the reduced image data, that is, the upper left of the image region, and sets each reduction data to the remaining region. Are arranged in order. At that time, the horizontal width of the image area is determined in accordance with the horizontal width of the important area. That is, here, reduced image data having the same width as the horizontal width of the non-reduced region is generated.

  In addition, the rearrangement unit 33 generates reduced image data by including the size and position information of the important region in the header information. As the position information of the important area, for example, the horizontal and vertical positions of the first block of the important area in the image data before rearrangement are used. In the receiving side unit Ur, even when the size or position of the important area is changed in the transmitting side unit Ut, the non-reduced area and each reduced data are arranged based on the header information included in the reduced image data. Can be restored properly. Other configurations are the same as those of the image compression unit 11 (Embodiment 1) of FIG.

<MPEG encoder>
The MPEG encoder 32 includes a motion compensation prediction unit 34, a DCT and quantization processing unit 35, an encoding processing unit 36, a decoding processing unit 37, an inverse DCT and inverse quantization processing unit 38, a motion compensation prediction unit 39, and a frame memory 40. Consists of. The reduced image data output from the image reduction converter 31 is divided into a plurality of motion compensation regions composed of 16 × 16 pixels in the motion compensation prediction unit 34.

  In the motion compensation prediction unit 34, motion position information for each motion compensation region is generated based on the inter-frame difference between the current frame image and the previous and subsequent frame images stored in the frame memory 40. The DCT and quantization processing unit 35 performs discrete cosine transform and quantization processing on the difference value between frames for each block region (MPEG block) constituting the motion compensation region.

  In the encoding processing unit 36, entropy encoding processing is performed on the quantized DCT coefficients, and compressed data including motion position information in header information is generated. In the decoding processing unit 37 and the inverse DCT / inverse quantization processing unit 38, a process opposite to the process described above is performed, and the difference value before DCT conversion is restored from the compressed data. Based on the motion position information from the motion compensation prediction unit 34, the motion compensation prediction unit 39 uses the previous and next frame images stored in the frame memory 40 and the difference value from the inverse DCT and inverse quantization processing unit 38 to determine the current compensation value. The frame image is restored.

<Image expansion part>
FIG. 10 is a block diagram showing a configuration example of the image expansion unit 50 according to the second embodiment of the present invention, and shows detailed configuration examples of the MPEG decoder 51 and the image enlargement converter 52.

[MPEG decoder]
The MPEG decoder 51 includes a decoding processing unit 53, an inverse DCT and inverse quantization processing unit 54, and a motion compensation prediction unit 55, and includes a decoding processing unit 37, an inverse DCT and inverse quantization processing unit 38 in the MPEG encoder 32, and The same processing as that of the motion compensation prediction unit 39 is performed to decompress the compressed data and restore the image data before MPEG compression.

[Image Enlargement Converter]
The image enlargement converter 52 includes a DS region dividing unit 57, an arrangement restoring unit 58, and an interpolation processing unit 242. The arrangement restoration unit 58 arranges the important area and each reduced data based on the size and position information of the important area (non-reduced area) included as header information in the image data, and arranges the image data before the reduction process. Is restoring.

  FIG. 11 is a diagram showing an example of the data structure of compressed data generated by the MPEG compression process, and shows the structure of a sequence constituting one video program. In MPEG, the entire encoded signal constituting one video program is called a sequence. One sequence begins with a sequence header and ends with a sequence end. This sequence header includes information about the entire sequence such as the image size, encoding speed, and communication speed.

  One sequence includes one or more GOPs (Group Of Pictures). One GOP includes a GOP header, an I picture that is subjected to intra-frame coding, a P picture that is subjected to forward motion prediction using a past frame image, and a previous and a future frame image using a past and future frame image. It is composed of B pictures for which bidirectional motion prediction is performed later. This GOP header includes a time stamp for synchronizing with audio or the like at the time of image restoration.

  One picture is a frame image constituting a moving image signal, and includes a picture header and one or more slices. This picture header includes identification information for identifying an I picture, a P picture, and a B picture, and arrangement information of each picture.

  One slice is composed of slice information and one or more macroblocks. This slice information includes coding information used in the slice, for example, a code table and a quantization table.

  One macro block is composed of macro block information, a total of six blocks including four MPEG blocks in the luminance data Y and one MPEG block in each of the color difference data U and V. The macro block information includes coding information used in the macro block.

  One block consists of 8 × 8 encoded coefficient data by DCT encoding. The motion compensation prediction is performed on the luminance data Y by using the motion compensation area composed of the four MPEG blocks constituting the macro block as a processing unit. The color difference data U and V are processed based on motion compensation prediction for the luminance data Y.

  12A and 12B are diagrams illustrating an example of the image reduction operation in the image compression unit 30 in FIG. In FIG. 12A, image data B1 is divided into 12 × 7 DS regions by the DS region dividing unit 130, and an important region composed of 4 × 2 DS regions is designated by the non-reduced region designating unit 131. It is shown.

  FIG. 12B shows reduced image data C1 composed of 8 × 14 MPEG blocks generated from the image data after downsampling by the rearrangement unit 33. Of the image data from the downsampling unit 132, the rearrangement unit 33 first arranges the important region at the head of the image region.

  At that time, the horizontal width of the image area is adjusted to the same width as the important area. Next, each reduced data is sequentially arranged from left to right behind the important area. When all the reduced data is rearranged, fill data is inserted at the end of the image data, and the reduced image data C1 is completed. In this example, reduced image data C1 composed of 8 × 14 MPEG blocks is generated, and four fill blocks are added.

  According to the present embodiment, downsampling is performed on a part of the image area, the important area and each reduced data are rearranged, and the image data is spatially compressed as preprocessing of the MPEG compression process. it can. Further, since the rearrangement is performed without dividing the important area when the reduced image data is generated, the motion prediction for the important area can be appropriately performed. In particular, in the reduced image data, since the important region is arranged so that the side in contact with the reduced data is only one side in the horizontal direction, the difference value from the region different from the important region is encoded by motion prediction. Can be effectively suppressed.

  Also, by generating the reduced image data in accordance with the horizontal width of the important area, the number of fill blocks inserted at the end of the image data is reduced, so that the data compression rate can be further improved.

  Further, since the arrangement of the important area and each reduced data is restored based on the header information, even if the size or position of the important area is changed, the spatially compressed image data is processed by the MPEG decompression process. It can be properly restored by post-processing.

  In the present embodiment, an example in which reduced image data is generated in accordance with the horizontal width of an important region has been described, but the present invention is not limited to this. For example, reduced image data different from the horizontal width of the important area may be generated.

  FIG. 13 is a diagram showing another example of the image reduction operation in the image compression unit 30 of FIG. 9, and the reduced image data C2 composed of 17 × 7 MPEG blocks generated from the image data after downsampling is shown. It is shown. In this example, an important area is arranged at the head position of the reduced image data C2 formed by arranging 17 MPEG blocks in the horizontal direction, and reduced data is arranged in the remaining area.

  In this example, there are 11 fill blocks added to the end of the image data, which is larger than when reduced image data is generated by matching the image area to the horizontal width of the important area. Therefore, when this is done, the data compression rate is slightly reduced.

  In this embodiment, an example in which only one important region is designated has been described. However, the present invention is not limited to this, and reduced image data is designated by designating two or more important regions. It may be generated.

  14 and 15 are diagrams showing another example of the image reduction operation in the image compression unit 30 of FIG. FIG. 14 shows image data B11 in which an important area consisting of 4 × 2 DS areas and an important area consisting of 2 × 2 DS areas are designated. FIG. 15 shows reduced image data C11 composed of 8 × 14 MPEG blocks generated from the image data after downsampling.

  In the rearrangement of the image data, each important area is sequentially arranged at the head position of the image data. In this example, the horizontal width of the image area is defined in accordance with the horizontal width of one important area composed of 4 × 2 DS areas, and the other important area is arranged behind the important area in a left-justified manner. . When restoring the reduced image data C11, the size and position information of each important area included as header information is read, and the positions of the first blocks D1 and D2 in the image are determined. The arrangement of each important area and each reduced data can be correctly restored.

  The image compression unit and the image expansion unit in each of the above embodiments can be provided as a computer program that can be executed on a computer such as a personal computer. The JPEG encoder, JPEG decoder, MPEG encoder, and MPEG decoder can be configured by general-purpose hardware capable of high-speed processing such as a PC add-on board, and the image reduction converter and the image enlargement converter can be configured by a computer program. These computer programs are provided by being stored in, for example, an optical storage medium such as a CD-ROM, a magnetic storage medium such as a flexible disk, or a semiconductor storage medium such as an IC memory. It can also be provided via a telecommunication line such as the Internet or a LAN (Local Area Network).

1 is a block diagram illustrating a configuration example of an image compression / decompression system according to Embodiment 1 of the present invention, and illustrates an example of an image transmission system. FIG. It is the block diagram which showed one structural example of the transmission side unit Ut of FIG. It is the block diagram which showed one structural example of the receiving side unit Ur of FIG. FIG. 3 is a block diagram illustrating a configuration example of the image compression unit 11 in FIG. 2, in which detailed configuration examples of the image reduction converter 13 and the JPEG encoder 14 are illustrated. FIG. 4 is a block diagram illustrating a configuration example of the image expansion unit 21 in FIG. 3, in which detailed configuration examples of the JPEG decoder 23 and the image enlargement converter 24 are illustrated. It is the figure which showed an example of DS area division and downsampling in the image compression part. FIG. 10 is a diagram illustrating an example of the operation of the image reduction converter 13. It is a figure showing an example of operation of image reduction converter 13, and reduction image data B3 after rearrangement of an important field and reduction data is shown. It is the block diagram which showed the example of 1 structure of the image compression part 30 by Embodiment 2 of this invention. It is the block diagram which showed the example of 1 structure of the image expansion | extension part 50 by Embodiment 2 of this invention. It is the figure which showed an example of the data structure of the compression data produced | generated by MPEG compression process, and the structure of the sequence which comprises one video program is shown. It is the figure which showed an example of the image reduction operation | movement in the image compression part 30 of FIG. It is the figure which showed another example of the image reduction operation | movement in the image compression part 30 of FIG. It is the figure which showed another example of the image reduction operation | movement in the image compression part 30 of FIG. It is the figure which showed another example of the image reduction operation | movement in the image compression part 30 of FIG.

Explanation of symbols

11, 30 Image compression unit 13, 31 Image reduction converter 14 JPEG encoder 21, 50 Image expansion unit 23 JPEG decoder 24, 52 Image enlargement converter 31 Image reduction converter 32 MPEG encoder 33 Relocation unit 34 Motion compensation prediction unit 35 DCT and quantum Decoding processing unit 36 Encoding processing unit 37 Decoding processing unit 38 Inverse DCT and inverse quantization processing unit 39, 55 Motion compensation prediction unit 40 Frame memory 51 MPEG decoder 53 Decoding processing unit 54 Inverse DCT and inverse quantization processing unit 130 DS region dividing unit 131 Non-reduced region specifying unit 132 Downsampling unit 133 Relocation unit 140 Block dividing unit 141 DCT processing unit 142 Quantization processing unit 143 Encoding unit 230 Decoding unit 231 Inverse quantization processing unit 232 Inverse DCT processing unit 240 Arrangement of DS area dividing unit 241 Based on section 242 interpolation processing unit A1, A2 DS area

Claims (8)

A down-sampling area dividing step for dividing the image data into two or more down-sampling areas having an integral multiple of the block size;
A non-reduced area designating step for designating an adjacent area consisting of the downsampling area as a non-reduced area;
A downsampling step for reducing the number of pixels by performing downsampling for each downsampling region excluding the non-reduced region, and generating reduced data consisting of an integral multiple of the block size;
A rearrangement step of arranging the non-reduced area and the respective reduced data, and generating reduced image data including the size and position information of the non-reduced area as header information;
The reduced image data is divided into block areas having the block size, motion compensation prediction is performed for each motion compensation area including a plurality of adjacent block areas, and compressed image data is generated by performing an encoding process for each block area. Data compression step,
A data decompression step for restoring image data based on the decoding process for each block area and the motion compensation prediction for each motion compensation area for compressed image data;
An arrangement restoration step for restoring the non-reduced area and the arrangement of each reduced data included in the restored image data based on the header information;
An image compression / expansion method comprising: an interpolation step of interpolating pixel data for each reduced data included in the restored image data and restoring the image data including the downsampling area. Down-sampling area dividing means for dividing image data into two or more down-sampling areas consisting of an integral multiple of the block size;
Non-reduced area designating means for designating an adjacent area consisting of the downsampling area as a non-reduced area;
Downsampling means for reducing the number of pixels by performing downsampling for each downsampling area excluding the non-reduced area, and generating reduced data consisting of an integral multiple of the block size;
Rearrangement means for arranging the non-reduced area and the respective reduced data, and generating reduced image data including the size and position information of the non-reduced area as header information;
The reduced image data is divided into block areas having the block size, motion compensation prediction is performed for each motion compensation area including a plurality of adjacent block areas, and compressed image data is generated by performing an encoding process for each block area. An image compression apparatus comprising:   3. The image compression apparatus according to claim 2, wherein the down-sampling area dividing unit divides the image data into down-sampling areas that are integer multiples of the motion compensation area.   3. The image compression apparatus according to claim 2, wherein the non-reduced area designating unit designates a rectangular adjacent area consisting of an integral multiple of the motion compensation area as a non-reduced area.   5. The image compression apparatus according to claim 4, wherein the rearrangement unit arranges the non-reduced area at the head of the reduced image data.   6. The image compression apparatus according to claim 5, wherein the rearrangement unit generates reduced image data having the same width as the horizontal width of the non-reduced region.   3. The image compression apparatus according to claim 2, wherein the rearrangement unit adds a block area made up of fill data to the image data, shapes the image data into a rectangular shape, and generates reduced image data. Based on motion compensation prediction for each motion compensation area consisting of a plurality of adjacent block areas, rearranged the reduced data consisting of an integral multiple of the block size obtained by downsampling part of the image data together with the non-reduced area An image expansion device that expands compressed image data that has undergone encoding processing,
Data decompression means for performing decoding processing for each block area having the block size on the compressed image data, performing motion compensation prediction for each motion compensation area, and restoring the image data;
An arrangement restoration means for restoring the arrangement of the non-reduced area and each reduced data included in the restored image data based on the size and position information of the non-reduced area included as header information in the image data;
An image expansion apparatus comprising: interpolation means for interpolating pixel data for each reduced data included in the restored image data and restoring the image data to a plurality of downsampling areas having an integral multiple of the block size .

Images