JPH11146400A - Image processing unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing unit that modifies an image based on information denoting the degree of magnification/reduction of each part of the image in the case of decoding coded image data. SOLUTION: Based on a modification instruction of an image from a user, a modified information generating means generates information denoting a degree of magnification/reduction with respect to each block shown in figure (a) and an IDCT means uses the information denoting the degree of magnification/reduction to decode MPEG image data for each block shown in the figure (a) while magnifying/reducing each block so as to be compatible with each corresponding block shown in figure (b). Furthermore, in order to allow the processing unit to be compatible with MPEG image data consisting of difference data and motion vectors, the IDCT means decodes all MPEG image data of each block independently of the information denoting the degree of magnification/reduction stated as above and the decoded image data are stored in a frame buffer.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for deforming an image displayed on a display of a personal computer or the like.

[0002]

2. Description of the Related Art Conventionally, digital image data which is an original image of a computer graphic displayed on a display of a personal computer or the like includes coded image data of an orthogonal transform system such as MPEG and DV.

[0003] MPEG, DV as shown in FIG.
Is decoded, and the decoded image 15
As shown in FIG. 7 (b), there is a mapping device for mapping a display 16 such as a personal computer, for example, on a wall 17 of a room, which is created by computer graphics.

Hereinafter, a conventional MPEG image mapping apparatus for mapping an image from MPEG image data and a DV image mapping apparatus for mapping an image from DV image data will be described with reference to the drawings. explain.

First, a conventional MPEG image mapping apparatus will be described.

FIG. 8 shows a block diagram of a conventional MPEG image mapping apparatus.

[0007] The VLD means 1 is a variable length coded MP.
The EG image data is input and subjected to variable length decoding. The inverse quantization means 2 inversely quantizes the variable length decoded MPEG image data, and the IDCT means 3 outputs the inversely quantized MP image data.
The EG image data is decoded. When decoding, IDCT
The means 3 decodes all MPEG image data for each block of the MPEG image having a plurality of 8 × 8 DCT coefficients.

[0008] The MPEG image data input by the VLD means 1 is an intra-frame coded image (hereinafter referred to as an I picture).
In the case of the data, the IDCT means 3 outputs the decoded image to the format conversion means 12 through the addition means 7 and also to the frame buffer 8. That Fr
The ame Buffer 8 stores the decoded image from the IDCT means 3.

On the other hand, the MPEG image data input by the VLD means 1 is used to encode an inter-frame forward prediction encoded image (hereinafter referred to as P
In the case where the frame buffer 8 is data of an inter-frame bidirectional predictive coded image, the Frame Buffer 8 already receives an I-picture once, and the I-picture or the I-picture includes a motion vector and a difference image. This means that the added complete decoded picture of the P picture has already been stored. At that time, the IDCT means 3 is the inverse quantization means 2
Is output to the motion compensating means 9 via the frame buffer 8, and the motion compensating means 9 adds only the motion vector to the complete image stored in the frame buffer 8, so that only the motion vector is compensated. A motion compensated image is generated. Then, the adding means 7 adds the motion compensated image from the motion compensating means 9 and the decoded difference image from the IDCT means 3, outputs the added image to the format converting means 12, and outputs the added image to the frame buffer 8. I do.
The Frame Buffer 8 stores the complete decoded image from the adding means 7.

Next, the format conversion means 12 outputs the ID
The image in YUV format from the CT unit 3 or the adding unit 7 is converted into an image in RGB format and output to the image transforming unit 18.

The image transforming means 18 receives the RGB format image from the format converting means 12 and outputs the information on the degree of enlargement / reduction of each part of the image from the CG original data producing means 4 to the transformation information producing information. Based on the information of the degree of enlargement / reduction input through the means 5, the image from the format conversion means 12 is transformed by interpolating or thinning out pixels for each part of the image,
Output to the mapping means 13.

The mapping means 13 maps the image from the image deforming means 18 to a predetermined position on a display of a personal computer or the like based on the mapping position information of the image from the CG original data creating means 4.

Next, a conventional DV image mapping apparatus will be described.

FIG. 9 is a block diagram showing a conventional DV image mapping apparatus.

The VLD means 1 comprises a variable-length coded DV.
Inputting image data and performing variable length decoding, inverse quantization means 2
Dequantizes the variable-length-decoded DV image data, and the IDCT means 3 decodes the inverse-quantized DV image data. During the decryption, the IDCT means 3
All DV image data is decoded for each block of the DV image having a plurality of 8 × 8 DCT coefficients. And
The IDCT means 3 converts the partial image decoded for each block into
Output to the deshuffling means 14.

The partial image output by the IDCT means 3 and decoded for each block is, for example, the uppermost block to the lowermost block from the leftmost column to the lowermost block of the entire image composed of the partial images. Are not output in order from the uppermost block to the lowermost block in the second column from. That is, the order of the partial images input by the deshuffling means 14 is shuffled. Therefore, the deshuffling unit 14 deshuffles the partial images from the IDCT unit 3 and arranges them in order to generate a complete DV image.

Next, the format conversion means 12 converts the image in the YUV format from the deshuffling means 14 into an image in the RGB format and outputs it to the image transformation means 18. The subsequent operation of the DV image is the same as the operation of the above-described MPEG image mapping apparatus.

[0018]

As described above, in the conventional MPEG image mapping device and DV image mapping device, the image input by the format conversion means 12 is a full-size image, In the deformation means 18, pixel interpolation or thinning is performed for each part of the image based on information on the degree of enlargement / reduction of each part of the image.

However, the format conversion means 12
Is more convenient if it is an image that has been transformed during the decoding process based on information on the degree of scaling for each part of the image.

According to the present invention, in consideration of the conventional problem that the above-described image transformation is not performed when decoding the encoded image data, the encoded image data is decoded. It is an object of the present invention to provide an image processing apparatus for deforming an image based on information on the degree of enlargement / reduction of each part of the image.

[0021]

According to a first aspect of the present invention, there is provided image deformation information input means for inputting information on the degree of enlargement / reduction of each part of an image, and coded image data of the image. At the same time, information on the degree of scaling for each part of the image is input from the image deformation information input means, and the image data is decoded for each part of the image based on the information on the degree of scaling. A decoding unit for decoding all of the image data irrespective of the information of the degree of enlargement / reduction; an image in which all the image data from the decoding unit is decoded; and an image deformation information input unit. Reference image generation that generates a scaled reference image for each part of the image using information on the degree of scaling for each part of the image from the image and the motion vector of the image data An image processing apparatus comprising: a stage; an image decoded for each part of the image from the decoding unit; and an adding unit configured to add a reference image from the reference image generating unit. is there.

According to a third aspect of the present invention, there is provided an image transformation information inputting means for inputting information of a degree of enlargement / reduction for each part of an image, and an apparatus for inputting encoded image data of the image, and Decoding means for inputting information on the degree of scaling for each part of the image from the input means and decoding the image data for each part of the image based on the information on the degree of scaling. An image processing apparatus characterized in that:

[0023]

Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1) First, the configuration of an image processing apparatus according to Embodiment 1 of the present invention will be described.

FIG. 1 shows a block diagram of an image processing apparatus according to the first embodiment of the present invention. The image processing apparatus according to the first embodiment of the present invention includes a VLD unit 1, an inverse quantization unit 2, an ID
CT means 3, CG original data creating means 4, deformation information creating means 5, IDCT control means 6, first adding means 7
, Frame Buffer 8, motion compensating means 9, and correcting means 1
0, second adding means 11, format converting means 12
And mapping means 13.

The VLD means 1 outputs a variable-length coded MP
A means for inputting EG image data and performing variable length decoding.

The inverse quantization means 2 is a means for inputting the variable length decoded MPEG image data from the VLD means 1 and performing inverse quantization.

The IDCT means 3 is a means for receiving the inversely quantized MPEG image data from the inverse quantization means 2 and decoding the MPEG image data for each block described later. The block will be described later in detail.

The CG original data creating means 4 includes the VLD means 1
Converts the MPEG image from the input MPEG image data into
When the information is displayed on a display of a personal computer or the like, an instruction is received from the user as to which position on the display is to be mapped and in what shape and size, and the information is output to the deformation information creating means 5. . The display displays an MPEG image from the image processing device.

The deformation information creating means 5 is a means for creating information on the degree of enlargement / reduction of each of a plurality of blocks, which will be described later, based on an instruction to deform the MPEG image from the CG original data creating means 4.

The IDCT control means 6 executes a decoding method when the IDCT means 3 decodes the MPEG image data for each block based on the information of the degree of enlargement / reduction for each of the plurality of blocks from the deformation information creating means 5. It is a means to control.

The first adding means 7 converts the MPEG image data inputted by the VLD means 1 into an inter-frame forward prediction coded image (hereinafter referred to as a P picture) or an inter-frame bidirectional prediction coded image (hereinafter referred to as a P picture). B data), the decoded difference image from the IDCT unit 3 is added to the motion compensation image corresponding to the difference image from the motion compensation unit 9 and only the motion vector is compensated. It is a means to do.

The Frame Buffer 8 is a means for temporarily storing the I picture decoded by the IDCT means 3 and the completely decoded picture of the P picture or B picture from the first adding means 7.

When the MPEG image data inputted by the VLD means 1 is P-picture or B-picture data, the motion compensating means 9 is stored in the frame buffer 8 using only the motion vectors of the data. This is a means for generating a motion-compensated image in which only a motion vector is compensated from a completely decoded image of an I picture and / or a P picture.

If the MPEG image data input by the VLD means 1 is P-picture data, the frame buffer
The er8 has already stored at least an I picture from the IDCT means 3. Similarly, when the MPEG image data input by the VLD means 1 is B-picture data, the Frame Buffer 8 has already stored the fully decoded images of the I-picture and the P-picture. This is because the data of the P picture or the B picture is only the data of the difference data and the data of the motion vector, and therefore, in order to create a complete image, an image to be a reference for them is necessary.

When the MPEG image data inputted by the VLD means 1 is P picture or B picture data, the correction means 10 performs the motion compensation from the motion compensation means 9 based on the deformation information from the deformation information creating means 5. This is a means for interpolating or thinning out the pixels of the image and transforming the motion compensated image to generate a reference image.

When the MPEG image data inputted by the VLD means 1 is P picture or B picture data, the second adding means 11 decodes the data from the IDCT means 3 based on the transformation information from the transformation information creating means 5. And a reference image from the correction unit 10 corresponding to the obtained difference image. If the MPEG image data input by the VLD means 1 is I picture data, the second adding means 11 passes the I picture decoded based on the transformation information from the IDCT means 3.

The format conversion means 12 is a YUV format image in which the I-picture from the IDCT means 3 or the P-picture or B-picture from the second addition means 11 which has passed through the second addition means 11 is completely decoded. Therefore, it is means for converting an image in that format into an image in RGB format.

The mapping means 13 is a means for displaying an image from the format conversion means 12 on a display of a personal computer or the like based on an instruction of a mapping position from the CG original data creating means 4.

Next, the operation of the image processing apparatus according to the first embodiment of the present invention will be described.

The operation of the image processing apparatus is based on VLD means 1
Are different depending on whether the input MPEG image data is I-picture data or P-picture or B-picture data.

First, the operation of the image processing apparatus when the MPEG image data input to the VLD means 1 is I-picture data will be described.

First, the CG original data creation means 4 receives the MPE input by the VLD means 1 as shown in FIG.
When the MPEG image from the G image data is transformed and displayed on a display of a personal computer or the like as shown in FIG. 2B, an instruction on how to transform is received from the user, and the information is received. Output to the deformation information creating means 5. FIG. 2A shows a case where the MPEG image data input by the VLD means 1 is data of an intra-frame coded image (hereinafter, referred to as an I picture).
It is assumed that all DCT coefficients of the data are decoded images. On the other hand, FIG.
It is assumed that the image of FIG.

The deformation information creating means 5 generates the image shown in FIG. 2B from the image shown in FIG. 2A based on the instruction to transform the MPEG image from the CG original data creating means 4. 3A corresponding to FIG. 2A is generated for each of the plurality of blocks in FIG. FIG. 3A is a schematic diagram in which the image of FIG. 2A is divided into 25 blocks, and the blocks are partial images constituting FIG. 3A. Similarly, FIG.
FIG. 2B is a schematic diagram in which the image of FIG. 2B is divided into 25 blocks. The numbers given to the blocks in FIGS. 3A and 3B are for distinguishing the respective blocks, and the blocks having the same numbers in FIGS. Shall correspond. For example, FIG.
The seventh block in FIG. 3A corresponds to the seventh block in FIG. When the image of FIG. 2B is generated from the image of FIG. 2A, the image of FIG. 2A is deformed by enlarging or reducing each block of FIG. 3A. Assume that an image of 2 (b) is generated. That is, each block in FIG. 3B is obtained by enlarging or reducing the corresponding block in FIG. 3A, or equivalently. Here, the deformation information creating means 5 is configured as shown in FIG.
The size of each block in FIG. 3B is the same as the size of the corresponding block in FIG. 3/4 in both the vertical and horizontal directions of the corresponding block in FIG.
As for the 16th to 25th blocks, the size of each of the 16th to 25th blocks is reduced so that both the vertical and horizontal sizes of the corresponding blocks in FIG. Create degree information. Note that FIG.
When each block in (a) is simply reduced, for example, the seventh block and the eighteenth block are reduced only in size as shown in FIG. 4, and the seventh block and the eighteenth block in FIG. It does not match the shape of 18 blocks. However, here, it is assumed that each of the blocks in FIG. 3A is simply enlarged or reduced to approximate each of the blocks to the corresponding block in FIG. 3B.

Thereafter, the IDCT control means 6 decodes the image data when the IDCT means 3 decodes the MPEG image data for each block based on the information of the degree of enlargement / reduction for each of the plurality of blocks from the deformation information creation means 5. Control the way.

On the other hand, the VLD means 1 inputs the variable length encoded MPEG image data and performs variable length decoding, and the inverse quantization means 2 inversely quantizes the variable length decoded MPEG image data. .

The IDCT means 3 converts the inversely quantized MPEG image data from the inverse quantizing means 2 into its M code based on the control of the decoding method from the IDCT control means 6.
Decoding is performed for each block of the PEG image. The block is
That is, each block in FIG. 3A is a partial image in which all 8 × 8 DCT coefficients are decoded. Therefore, the IDCT means 3 is the first
As for the blocks Nos. To 5, as shown in FIG.
All of the 8 × 8 DCT coefficients are decoded, and for the sixth to fifteenth blocks, as shown in FIG.
Decode the low-frequency 6 × 6 DCT coefficients of the T coefficients,
As for the 16th to 25th blocks, as shown in FIG. 5C, the 4 × 4 DCT coefficients on the lower side of the 8 × 8 DCT coefficients are decoded. Thereafter, the IDCT unit 3 outputs the decoded image, that is, the image of FIG. 2B to the format conversion unit 12 via the second addition unit 11.

The IDCT means 3 applies the inversely quantized MPEG image data from the inverse quantization means 2 to the MCT, regardless of the control of the decoding method from the IDCT control means 6.
8 × 8 DCT for any block of PEG image
All the coefficients are decoded, and the decoded image, that is, FIG.
The image of (a) is output to the Frame Buffer 8. And
The Frame Buffer 8 temporarily stores the image of FIG.
This storage is necessary when the MPEG image data input to the VLD means 1 is P-picture or B-picture data.

Next, the format conversion means 12
The format of the image from the IDCT unit 3 that has passed through the second adding unit 11 is converted from YUV to RGB.

Finally, the mapping means 13 displays the image from the format conversion means 12 on a display of a personal computer or the like based on the indication of the mapping position from the CG original data creation means 4.

Next, the operation of the image processing apparatus when the MPEG image data input to the VLD means 1 is P picture or B picture data will be described.

The MPE input by the VLD means 1
If the G image data is P picture data, Frame
Buffer 8 has already stored at least an I picture. If the MPEG image data input by the VLD means 1 is B picture data, the frame buffer
No. 8 has already stored the fully decoded images of the I picture and the P picture. This is because the data of the P picture or the B picture is only the data of the difference data and the data of the motion vector, and therefore, in order to create a complete image, the images to be referred to are necessary.

Now, as described above, the CG original data creating means 4 determines how to deform the MPEG image from the MPEG image data input by the VLD means 1 on a display of a personal computer or the like. Is received from the user, and the information is output to the deformation information creating means 5.

Then, based on the instruction to deform the MPEG image from the CG original data generating means 4, the deformation information creating means 5 enlarges each of the plurality of divided blocks of the MPEG image as shown in FIG. Create information about the degree of reduction. Here, the deformation information creating means 5 is configured as shown in FIG.
The size of each block in FIG. 3B is the same as the size of the corresponding block in FIG. 3A for the first to fifth blocks. 3/4 in both the vertical and horizontal directions of the corresponding block in FIG.
As for the 11th to 20th blocks, the 21st to 25th blocks are reduced so that both the vertical and horizontal sizes of the corresponding blocks in FIG. For (2), information on the degree of enlargement / reduction for each block is created so that the corresponding block in FIG.

Thereafter, the IDCT control means 6 decodes the image data when the IDCT means 3 decodes the MPEG image data for each block based on the information of the degree of enlargement / reduction for each of the plurality of blocks from the deformation information creation means 5. Control the way.

On the other hand, the VLD means 1 receives the variable length encoded MPEG image data and performs variable length decoding, and the inverse quantization means 2 inversely quantizes the variable length decoded MPEG image data. .

The IDCT means 3 converts only the differential data of the inversely quantized MPEG image data from the inverse quantizing means 2 based on the control of the decoding method from the IDCT control means 6 to a block of the MPEG image. Decode each time.
That is, the IDCT means 3 performs, for example, the first to fifth blocks of the MPEG image as shown in FIG.
As shown in FIG. 5A, all 8 × 8 DCT coefficients are decoded, and for the sixth to tenth blocks, as shown in FIG. 5B, low-frequency 6 × 6 DCT coefficients are decoded. And the first
As for the blocks 1 to 20, as shown in FIG. 5C, the low-frequency side 4 × 4 DCT coefficients are decoded and the 21st to 2nd blocks are decoded.
As for the block No. 5, as shown in FIG. 5D, the low-frequency side 2 × 2 DCT coefficients are decoded. After that, IDC
The T means 3 outputs the difference image obtained by decoding only the difference data to the second adding means 11 based on the control of the decoding method from the IDCT control means 6.

The IDCT means 3 stores only the difference data of the inversely quantized MPEG image data from the inverse quantization means 2 irrespective of the control of the decoding method from the IDCT control means 6 for any block. , And 8 × 8 DCT coefficients, and outputs the decoded difference image to the first adding means 7. The size of the difference image is the same as the size of the image in FIG.

Next, the motion compensating means 9 inputs the motion vector input by the VLD means 1 and a complete image to be referred to by the motion vector, for example, an I-picture from the Frame Buffer 8, and obtains the motion in which only the motion vector is compensated Generate a compensation image. The motion-compensated image uses I-picture data,
In each block, all the 8 × 8 DCT coefficients have the same size as the decoded image. That is, FIG.
The size is the same as the image of FIG. Then, the motion compensating means 9 outputs the motion compensated image to the correcting means 10 and also to the first adding means 7.

The first adder 7 applies to any block from the IDCT 3 the difference image obtained by decoding all the 8 × 8 DCT coefficients and the motion compensation from the motion compensator 9 corresponding to the difference image. Add the image and Frame Buffer
8, and the Frame Buffer 8 temporarily stores the image obtained by adding the difference image from the first adding means 7 and the motion compensation image as a complete P-picture or B-picture.

On the other hand, the correction means 10 receives the motion-compensated image from the motion-compensation means 9 and the blocks in FIG. 3A of the motion-compensated image from the deformation information creating means 5 are shown in FIG. In step b), information on the degree of enlargement / reduction for each of the 25 blocks is input. Then, based on the information on the degree of enlargement / reduction, interpolation or thinning out of the pixels of the motion compensation image is performed, and the motion compensation image is transformed to generate a reference image. That is, the IDCT means 3 is
The correction means 10 adjusts each of the motion-compensated images corresponding to the difference image so that the size of each block of the difference image obtained by decoding only the difference data based on the control of the decoding method from Each block is enlarged or reduced by interpolating or thinning out the pixels of the block. For example, ID
The CT means 3 calculates the low frequency side 6 ×
When decoding the 6 DCT coefficients, the correcting unit 10 determines that the 6 × 6 DCT coefficients of the seventh block of the motion compensation image are equivalent to the size of the decoded partial image.
Pixel thinning is performed. After that, the correcting unit 10 outputs the reference image to the second adding unit 11.

Next, the second adding means 11 performs the decoding based on the control of the decoding method of the IDCT control means 6 from the IDCT means 3 and the difference image obtained by decoding only the difference data, and the difference image corresponding to the difference image. The reference image from the correction unit 10 is added, and the added image is output to the format conversion unit 12.

The format conversion means 12 converts the format of the image from the second addition means 11 from YUV to RGB.

Finally, the mapping means 13 displays the image from the format conversion means 12 on a display of a personal computer or the like based on the indication of the mapping position from the CG original data creation means 4.

(Embodiment 2) First, the configuration of an image processing apparatus according to Embodiment 2 of the present invention will be described.

FIG. 6 is a block diagram showing an image processing apparatus according to the second embodiment of the present invention. The image processing device according to the second embodiment of the present invention includes the same components as those included in the image processing device according to the first embodiment. That is, the image processing apparatus according to the second embodiment includes a VLD unit 1, an inverse quantization unit 2, an IDCT unit 3, a CG original data creation unit 4, a deformation information creation unit 5, an IDCT control unit 6,
Deshuffling means 14 and format conversion means 12
And mapping means 13.

The VLD means 1 outputs a variable-length coded DV
This is a means for inputting image data and performing variable length decoding.

The inverse quantization means 2 is a means for inputting the variable length decoded DV image data from the VLD means 1 and performing inverse quantization.

The IDCT means 3 receives the inversely quantized DV image data from the inverse quantizing means 2 and outputs the data from the first embodiment.
This is means for decoding DV image data for each block as shown in FIG.

The CG original data creating means 4 is composed of the VLD means 1
When displaying a DV image from DV image data input by a user on a display such as a personal computer, an instruction as to which position, what shape and size to map on the display is received from a user, and the information is received. Is output to the deformation information creating means 5. Note that the display is a D
A V-image is displayed.

The deformation information creating means 5 determines the degree of enlargement / reduction of each of a plurality of blocks as shown in FIG. 3 used in the first embodiment, based on the instruction to transform the DV image from the CG original data creating means 4. This is a means for creating information.

The IDCT control means 6 performs a decoding method when the IDCT means 3 decodes DV image data for each block based on the information on the degree of enlargement / reduction for each of the plurality of blocks from the deformation information creating means 5. It is a means to control.

The deshuffling means 14 is a means for correctly arranging the partial images decoded for each block from the IDCT means 3 and completing the image of the frame for each frame.

The format conversion means 12 is a means for converting the format of the image from the deshuffling means 14 from the YUV format to the RGB format.

The mapping means 13 is a means for displaying an image from the format conversion means 12 on a display of a personal computer or the like based on an instruction of a mapping position from the CG original data creating means 4.

Next, the operation of the image processing apparatus according to the first embodiment of the present invention will be described.

First, the CG original data creating means 4 converts the DV image from the DV image data input by the VLD means 1 into a display such as a personal computer as shown in FIG. At the time of deformation and display as shown in (1), an instruction on how to deform is received from the user, and the information is received by the deformation information creation means 5.
Output to Note that, in the second embodiment, FIG.
It is assumed that all the DCT coefficients of the DV image data are decoded, and FIG. 2B is an image obtained by transforming the image of FIG. 2A.

Then, the deformation information creating means 5 generates the image of FIG. 2B from the image of FIG. 2A based on the instruction to transform the DV image from the CG original data creating means 4. 3A corresponding to FIG. 2A is generated for each of the plurality of blocks in FIG. Also in the second embodiment, FIG. 3A is a schematic diagram in which the image of FIG. 2A is divided into 25 blocks, and the blocks are the parts constituting FIG. 3A. It is assumed to be an image. Similarly, FIG.
It is assumed that the image shown in FIG. 2B is a schematic diagram obtained by dividing the image into 25 blocks. The numbers given to the blocks in FIGS. 3A and 3B are for distinguishing the respective blocks, and the blocks having the same numbers in FIGS. Shall correspond. FIG. 2 (a)
When the image of FIG. 2B is generated from the image of FIG.
By performing scaling for each block in FIG.
It is assumed that the image in FIG. 2A is transformed to generate the image in FIG. That is, each block in FIG.
It is assumed that the corresponding block in (a) is scaled or equivalent. Here, the deformation information creating means 5 sets the size of each block in FIG. 3B to be the same as the size of the corresponding block in FIG. 6th to 15th
The 16th block is reduced to 3/4 in both the vertical and horizontal directions of the corresponding block in FIG.
3 to 25, the vertical and horizontal sizes of the corresponding blocks in FIG.
Create information on the extent of scaling for each block. If each block in FIG. 3A is simply reduced, for example, only the size of the seventh block and the eighteenth block is reduced as shown in FIG.
The shape does not match the shape of the seventh block or the eighteenth block in (b). However, here, it is assumed that each of the blocks in FIG. 3A is simply enlarged or reduced to approximate each of the blocks to the corresponding block in FIG. 3B.

Thereafter, the IDCT control means 6 performs the decoding when the IDCT means 3 decodes the DV image data for each block based on the information of the degree of enlargement / reduction for each of the plurality of blocks from the deformation information creating means 5. Control the way.

On the other hand, the VLD means 1 receives the variable length coded DV image data and performs variable length decoding, and the inverse quantization means 2 inversely quantizes the variable length decoded DV image data. .

Then, based on the control of the decoding method from the IDCT control means 6, the IDCT means 3 converts the inversely quantized DV image data from the inverse quantization means 2 into its DV image, that is, FIG. a) For each block in the schematic diagram of FIG.
The image data is decoded and output to the deshuffling means 14 in order from the decoded data. Note that, in the second embodiment, the order in which the IDCT means 3 decodes each block in FIG. 3A is as follows: first → third → fifth → seventh →.
It is assumed that the order is 3 → 25 → 2 → 4 → 6 → 8 →... → 20 → 22 → 24. Each block in FIG. 3A is an 8 × 8 DC.
It is assumed that the T coefficients are all decoded partial images. The decoding method is as described below. That is, as shown in FIG. 5A, the IDCT means 3 converts the 8 × 8 DC into the first to fifth blocks in FIG.
As shown in FIG. 5B, all the T coefficients are decoded, and as shown in FIG. 5B, the lower 6 × 6 DCT coefficients of the 8 × 8 DCT coefficients are decoded. 16-2
For the block 5, as shown in FIG.
The 4 × 4 DCT coefficients on the low frequency side of the × 8 DCT coefficients are decoded.

Thereafter, the deshuffling means 14 outputs the ID
FIG. 3 (a) decoded for each block from the CT means 3.
Are correctly arranged at the positions of the corresponding blocks of the number in FIG. 3B to complete the image. For example, the partial image of the thirteenth block in FIG.
That is, it is arranged at the position of the thirteenth block in FIG. 3B, and the partial image of the twenty-first block in FIG. 3A is arranged at the position of the twenty-first block in FIG. Then, the deshuffling unit 14 outputs the completed image to the format conversion unit 12.

Next, the format conversion means 12 converts the format of the image from the deshuffling means 14 into YUV
To RGB.

Finally, the mapping means 13 displays the image from the format conversion means 12 on a display of a personal computer or the like based on the mapping position instruction from the CG original data creating means 4.

In the second embodiment, the IDCT means 3
However, the order of decoding each block in FIG. 3A is as follows: first → third → fifth → seventh →... → 21st → 23rd → 25th → 2nd →
Although the order of the fourth, sixth, eighth,..., Twentieth, twenty-second, and twenty-fourth blocks has been described, the order is not limited. In short, the IDCT of the second embodiment
Means 3 does not decode each block in the order of the numbers given to the blocks in FIG.

In the present invention, CG original data creation means 4, deformation information creation means 5, and IDCT control means 6 are used as image deformation information input means. As decoding means, VLD means 1, inverse quantization means 2, and IDCT means 3 were used. First adding means 7 as a reference image generating means
, Frame Buffer 8, motion compensating means 9, and correcting means 1
0 was used. The second adding means 11 is used as the adding means.
Using the deshuffling means 14 as a rearranging means. However, the image deformation information input means of the present invention is not limited to the above-described CG original data creation means 4, deformation information creation means 5, and IDCT control means 6, but in short, inputs information on the degree of enlargement / reduction for each part of the image. It just needs to be done. Further, the decoding means of the present invention is not limited to the VLD means 1, the inverse quantization means 2 and the IDCT means 3, but in short, it inputs coded image data and outputs each part of the image from the image deformation information input means. , The image data is decoded for each part of the image based on the information of the degree of enlargement / reduction, and the image data is decoded independently of the information of the degree of enlargement / reduction. All that is required is to decrypt everything. Or, the decoding means of the present invention, while inputting the encoded image data, from the image deformation information input means input information of the degree of scaling for each part of the image,
Based on the information of the scale, the image data is
All that is required is to decode each part of the image.
Further, the reference image generating means of the present invention comprises a first adding means 7, a Frame Buffer 8, a motion compensating means 9, and a correcting means 1.
Not limited to 0, in short, the image obtained by decoding all the image data from the decoding means, the information on the degree of enlargement / reduction of each part of the image from the image deformation information input means, and the motion vector of the image data It is only necessary to generate a reference image that is enlarged and reduced for each part of the image.

FIGS. 3 (a) and 3 (b) are schematic diagrams in which the image of FIG. 2 (a) or (b) is divided into 25 blocks. (B) The number of blocks when each is divided into blocks is not limited to 25. That is, the first to fifth blocks in FIG. 3A may be regarded as one block. In short, Figure 3
FIG. 3A only needs to divide the image into blocks each having the same degree of enlargement / reduction when the image is deformed as shown in FIG. 3B. FIG.
3B only needs to be divided into blocks each having the same degree of enlargement / reduction of the image so as to correspond to the block division in FIG. 3A.

When the image shown in FIG. 2A is transformed into the image shown in FIG. 2B, each block shown in FIG. 3A is reduced in size to the same size or reduced. Although the example of the case where decoding is performed has been described, each block in FIG. 3A may be enlarged according to the degree of image deformation. The expansion is performed, for example, by adding 0-value data to the ninth or higher order high-frequency component of the 8 × 8 DCT coefficient to obtain a 10 × 10 DCT coefficient.
This is done by generating CT coefficients and decoding them. When the image is reduced, the entire image may be reduced so that some blocks in FIG. 3A do not exist in FIG. 3B depending on the degree of the reduction.
That is, the reduction ratio of the block may be 100%. In this case, when an image is created, a difference between a block that no longer exists and a neighboring block is required, so that only the DC component of the 8 × 8 DCT coefficient is decoded.

In the above-described embodiment, the IDCT
Although the format from the image from the means 3, the second adding means 11, and the deshuffling means 14 is converted by the format converting means 12, it is not always necessary to convert. In short, when an image from the image processing apparatus of the present invention is displayed on a display such as a personal computer, the format of the image only needs to match the display. Therefore, when the format of the image is not converted by the format conversion means 12, the mapping means 13 comprises the IDCT means 3, the second addition means 11
Alternatively, the image from the deshuffling means 14 is displayed on the display.

Further, in the above-described embodiment, a case has been described in which only one image is displayed on the display of a personal computer or the like. As described above, when the image is divided into blocks and the image data is decoded, it is only necessary to perform scaling on each block and deform the image.

[0091]

As is apparent from the above description, according to the present invention, when the encoded image data is decoded, the image is transformed based on the information of the degree of enlargement / reduction for each part of the image. It is possible to provide an image processing apparatus for performing the processing.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention;

FIG. 2 is a view for explaining image deformation when the image processing apparatus according to the embodiment of the present invention deforms an image;

FIG. 3 is a schematic diagram obtained by dividing the image of FIG. 2 into 25 blocks.

FIG. 4 is a diagram simply reduced from the block in FIG. 3;

FIG. 5 is a diagram for explaining decoding when image processing apparatus IDCT means according to the embodiment of the present invention decodes image data;

FIG. 6 is a block diagram of an image processing apparatus according to a second embodiment of the present invention.

FIG. 7 is a diagram for explaining mapping when a decoded image is transformed and mapped on a wall of a room, which is created by computer graphics on a display such as a personal computer.

FIG. 8 is a block diagram of a conventional MPEG image mapping apparatus.

FIG. 9 is a block diagram of a conventional DV image mapping apparatus.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 VLD means 2 Inverse quantization means 3 IDCT means 4 CG original data creation means 5 Deformation information creation means 6 IDCT control means 7 First addition means, addition means 8 Frame Buffer 9 Motion compensation means 10 Correction means 11 Second addition means 12 Format conversion means 13 Mapping means 14 Deshuffling means 15 Image 16 Display 17 Wall 18 Image deformation means

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Yoshimori Nakase 1006 Kazuma Kadoma, Kadoma City, Osaka Prefecture Inside Matsushita Electric Industrial Co., Ltd. (72) Satoru Inagaki 1006 Kazama Kadoma Kadoma City, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. 72) Inventor Yoshihisa Nishikori 1006 Kazuma Kadoma, Osaka Pref.Matsushita Electric Industrial Co., Ltd.

Claims (6)

[Claims] 1. An image transformation information input means for inputting information on a degree of enlargement / reduction for each part of an image, and coded image data of the image are inputted. Information on the degree of scaling for each part is input, and the image data is decoded for each part of the image based on the information on the degree of scaling, and the information on the degree of scaling is Decoding means for decoding all of the image data independently, an image from which all the image data is decoded from the decoding means, and a degree of scaling for each part of the image from the image deformation information input means. Reference image generating means for generating a reference image enlarged and reduced for each part of the image using the information of the image data and the motion vector of the image data, and the image from the decoding means. An image decoded for each portion, an image processing apparatus characterized by comprising an adding means for adding the reference image from the reference image generating unit. 2. The image processing apparatus according to claim 1, wherein when the image data is data of an intra-frame encoded image, the adding unit does not use a reference image from the reference image generating unit. . 3. An image transformation information inputting means for inputting information of a degree of enlargement / reduction for each part of an image, and coded image data of the image are inputted. Decoding means for inputting information about the degree of enlargement / reduction for each part and decoding the image data for each part of the image based on the information about the degree of enlargement / reduction. Image processing device. 4. The image processing apparatus according to claim 3, further comprising a reordering unit for reordering the partial images decoded for each part of the image from the decoding unit in a predetermined order. apparatus. 5. The decoding means, the adding means, or
5. The image processing apparatus according to claim 1, further comprising a format conversion unit configured to convert a format of the image from the rearrangement unit. 6. The apparatus according to claim 1, further comprising mapping means for mapping an image from said decoding means, said adding means, said rearranging means, or said format converting means to a predetermined position on a predetermined screen. Item 6. The image processing apparatus according to any one of Items 1 to 5.