JP3500112B2 - Image encoding device and image decoding device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding device and an image decoding device, and more particularly to an image encoding device and an image decoding device for encoding / decoding digital image data with high efficiency.

[0002]

2. Description of the Related Art Conventionally, in image coding, a method has been proposed in which the image quality of a specific area is better than that of other areas.

For example, in the system disclosed in Japanese Patent Laid-Open No. 5-130603, among a plurality of people in a TV conference,
The minimum quantization width is used in the speaker area, and the quantization width is controlled in other areas according to the bit rate.

Similarly, the document “Image Quality Improvement Technology in Moving Image Coding” (Sharp Technical Report, No. 6, December 1994).
In the method described in P. 25 to 30), the face area in the face area is detected by detecting the face area in the image and reducing the quantization width specified in macroblock units in the detected area. The image quality of is better than that of other areas.

Alternatively, in the method disclosed in Japanese Patent Laid-Open No. 5-75867, all DCT (discrete cosine transform) coefficients are used for coding in a designated area, and DCT coefficients are used in other areas. The image quality of an important region in the image is improved by encoding using only a part of.

Both methods are based on the H.264 standard. The feature is that the image quality of the selected area is made better than the image quality of other areas without changing the standard encoding method represented by H.261.

Further, conventionally, in encoding of a TV telephone or the like, it has been considered to store an image of a background portion in a memory and apply it to predictive encoding.

For example, in the system disclosed in Japanese Patent Application Laid-Open No. 64-32788, a background image memory is successively stored each time a portion of a background image hidden behind a person appears as the person moves. Will be added to. This gives a more complete background image.

[0009]

[Problems to be Solved by the Invention] As described above,
In order to improve the image quality of the selected area better than other areas without changing the standard encoding method, in the method of controlling the quantization width or limiting the DCT coefficient, other encoding parameters are used. The image quality cannot be controlled due to the difference.

Specifically, the number of pixels of a frame (spatial resolution), the number of dropped frames of a frame (temporal resolution), and the like are the same in the selected area and other areas. Therefore, in the above-described image coding method, the image quality of the selected region cannot be made better than other regions by adjusting the coding parameters other than the quantization width and the number of DCT coefficients.

Further, in the method of storing the image of the background portion in the memory, it is assumed that the background image is stationary. However, in reality, for example, when a different person moves behind the speaker, or when using a TV phone in a car, when a scene outside the window is included in the background, a moving image with a motion in the background is encoded. In some cases. In this case, the background image does not have to be completely stationary but needs to have some movement.

The present invention has been made in order to solve the above problems, and is image coding for coding / decoding a selected region in a moving image with better image quality than others without increasing the amount of data. An object is to provide an apparatus and an image decoding apparatus.

[0013]

An image coding apparatus according to a first invention of the present application is a plurality of rectangular images having a rectangular shape.
An image encoding device for encoding a rectangular image
Parameter adjuster for adjusting parameters related to image quality
And a parameter encoding unit for encoding parameters,
Fixed-length encoding unit for encoding the number of horizontal and vertical pixels of a shape, and an image
Image information that divides the image into blocks and encodes the pixel values of the image.
The block division position is a rectangular image.
The feature is that each is set independently. In addition, the parameter
Data is at least the quantization width, the number of dropped frames, and the spatial resolution.
There is also one.

An image decoding apparatus according to the second invention of the present application is
Recovers the encoded data of multiple rectangular images with a rectangular shape.
The image decoding device for
Parameter for decoding the encoded data of the related parameter
Decoding unit and decodes the coded data of the number of vertical and horizontal pixels
Fixed-length decoding unit and the pixel value of the image divided into blocks
Image information decoding unit for decoding the encoded data of
The lock division position is set independently for each rectangular image.
It is characterized by The parameters are the quantization width,
At least one of the number of dropped frames and the spatial resolution.

[0015]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First, a first embodiment of the image coding apparatus of the present invention will be described.

FIG. 1 is a block diagram showing the configuration of this embodiment. The encoding apparatus according to the present embodiment includes an area selection unit 101 that selects a specific area in an image and an area position / shape encoding unit 102 that encodes the position and shape of the selected area.
By adjusting various parameters used for controlling image quality and data amount in moving image encoding, the image quality of the region selected by the region selection unit 101 is encoded better than the image quality of other regions. So as to control the encoding parameter adjusting unit 104.

A parameter coding unit 105 for coding the various parameters, a moving image coding unit 106 for coding the input moving image data using the various parameters, and a region position / shape code. The encoding unit 102, the parameter encoding unit 105, and the encoded data integration unit 103 that transmits or accumulates the data encoded by the moving image encoding unit 106 in combination.

The method in the area selection unit 101 is, for example, in the case of selecting a face area in an image of a videophone or the like, the document "Real-time Face Image Tracking Method" (Preliminary Report of the Institute of Image Electronics Engineers, 93-04-04, page 13-). (P. 16, 1994)
Can be used.

In the moving picture coding unit 106, motion compensation prediction, orthogonal transformation, quantization, variable length coding and the like are combined in the same manner as in the conventional case, and high efficiency coding is realized.

In the coding apparatus of this embodiment, the area selection unit 1
A specific region is selected from the original image input by 01, and the position and shape of the selected region are encoded by the region position / shape encoding unit 102. Various parameters used when the moving image is encoded are adjusted by the encoding parameter adjusting unit 104 so that the image quality of the selected region is better than the image quality of other regions, and the parameter encoding unit 105 Various parameters are encoded.

The moving picture input by the moving picture coding unit 106 is coded using various parameters, and the coded data integration unit 103 codes the area position / shape coding unit 10.
2. Parameter encoding unit 105 and moving image encoding unit 10
The data encoded in 6 is combined and transmitted or stored.

As described above, the coding is performed so that the image quality of the selected area in the image is better than that of other areas.

Next, a first embodiment of an image decoding device corresponding to the image encoding device of the present invention will be described.

FIG. 2 is a block diagram showing the arrangement of the decoding apparatus according to this embodiment. The decoding device according to the present exemplary embodiment is a coded data separation unit 2 that separates and outputs input coded data.
01 and a region position / shape decoding unit 202 for decoding the position and shape of the region selected from the separated encoded data.
And a parameter decoding unit 203 that decodes a coding parameter from the separated coded data, and a moving image decoding unit 204 that decodes a moving image from the separated coded data using the coding parameter.

In the decoding apparatus of the present embodiment, whether the coded data is separated by the coded data separation means 201 and whether the area position / shape decoding unit 202 has selected the coding parameter or another area. The positions and shapes of the regions are distinguished and decoded, the parameters encoded by the parameter decoding unit 203 are decoded, and the moving image decoding unit 204
Thus, the moving image is decoded using the encoding parameter.

As described above, the decoding is performed so that the image quality of the selected area in the image is better than that of other areas.

Next, regarding the representation of the position and shape of the area handled by the image coding apparatus and the image decoding apparatus of the present invention,
Description will be given with reference to the drawings.

FIG. 3 is a first example of a region in an image handled by the device of the present invention. The entire image is divided into blocks for coding. The size of each block is, for example, vertical 8 pixels × horizontal 8 pixels. At this time, the selected area is represented by a rectangle such as a shaded area. The position and shape of the area are, for example, the coordinates (4, 1) of the upper left block of the area,
It is represented by the vertical and horizontal sizes (3, 4) of the area, and these pieces of information are encoded. Fixed-length coding or variable-length coding is used for coding the position and shape information of the area.

FIG. 4 is a second example of a region in an image handled by the device of the present invention. The entire image is divided into blocks for encoding, and the selected area is represented by a set of blocks such as a shaded area. The position and shape of the area are
For example, the coordinates (4, 1) of the block that is the starting point and the sequences 3, 4, 5, of the 8-direction quantization codes of the blocks around the area
It is represented by 6,0,0. As shown in FIG. 5, the 8-direction quantized code is a numerical value indicating the direction to the next point, and is generally used when representing a digital figure.

FIG. 6 is a third example of a region in an image handled by the device of the present invention. The selected area is represented by a rectangle such as a shaded area, and the position and shape of the area are represented by, for example, the coordinates of the upper left pixel of the area (102, 49) and the number of vertical and horizontal blocks of the area (3, 4). Then, these pieces of information are encoded. The difference from the first example is that the position of the area is selected in pixel units.

FIG. 7 is a fourth example of a region in an image handled by the device of the present invention. The selected area is represented by a set of blocks such as a shaded area. The position and shape of the area are
For example, the coordinates (10
2, 49) and the sequence of directional quantization codes 3, 4, 5, 6, 0, 0 of the blocks around the area. The difference from the second example is that the position of the area is selected in pixel units.

FIG. 8 is a fifth example of a region in an image handled by the device of the present invention. The selected area is represented by a rectangle such as a shaded area, and the position and shape of the area are represented by, for example, the coordinates of the upper left pixel of the area (102, 49) and the number of vertical and horizontal pixels of the area (31, 50). Then, these pieces of information are encoded. The difference from the first example is that the position and size of the region are selected in pixel units. However, the position of the area can be expressed in block units and the size of the area can be expressed in pixel units.

FIG. 9 is a sixth example of a region in an image handled by the device of the present invention. The selected area is represented by an arbitrary shape such as a shaded area. The position and shape of the region are, for example, the coordinates (102, 49) of the upper left pixel of the block at the start point.
Is represented by a sequence of 8-direction quantization codes of pixels around the area. The difference from the second example is that the shape of the region is free, and the position and shape of the region are selected in pixel units.

Next, a method of setting a block which is a unit of coding processing in the moving picture coding unit 106 of the first embodiment of the image coding apparatus of the present invention will be described.

FIG. 10 is a diagram showing a first example of block setting in the image coding apparatus of the present invention. The selected area 11 indicated by hatching in the drawing is divided into blocks each having the same size. The unselected area is divided into blocks of equal size at the center of the image, but may be divided into blocks of irregular size at the edge of the image as shown by the shaded portion 12.

These irregularly sized blocks are coded by using motion compensation prediction, orthogonal transform, quantization, variable length coding, etc., like other blocks. Alternatively, if some distortion is allowed because it is not in the central portion of the image, the coding may be performed using only motion compensation prediction. That is, only the motion vector is encoded, and the prediction error may not be encoded.

FIG. 11 is a diagram showing a second example of block setting in the image coding apparatus of the present invention. The entire image is divided into rectangular blocks of the same size, but the block including the boundary of the selected area is the shaded area 2.
1 and a portion indicated by a shaded portion 22. The shaded area 21 is an arbitrary shape block located at the boundary of the selected area, and the shaded area 22 is an arbitrary shape block located at the boundary of the unselected area. At the time of encoding, these are encoded separately.

A block of such an arbitrary shape is coded by using motion compensation prediction, orthogonal transform, quantization, variable length coding, etc., like other blocks. Examples of the orthogonal transform include an arbitrary shape DCT and a document “Variable block of color image” as described in the document “A Study on Basis Selection Method in Arbitrary Shape DCT” (1993 IEICE Spring Conference D-251). Improvement of image quality of shape KL transform coding "(IEICE Spring Conference D-1 in 1992)
It is also possible to use an arbitrary shape KLT or the like as described in 34).

Alternatively, when some distortion is allowed at the edge of the area, the coding may be performed using only motion compensation prediction. That is, only the motion vector is encoded, and the prediction error may not be encoded.

FIG. 12 is a diagram showing a third example of block setting in the image coding apparatus of the present invention. The difference from the second example is that the block position of the selected area is determined independently of the block position of the non-selected area. Therefore, in a general case, it is possible to set more rectangular blocks of the same size in the selected area.

In FIG. 12, the shaded area 31 is an arbitrarily shaped block located at the boundary of the selected area, and the shaded area 32 is an arbitrarily shaped block located at the boundary of the unselected area. At the time of encoding, these are encoded separately. The handling of the arbitrarily shaped block is the same as in the second example.

As a block setting method, there are various methods other than those described here. For example, as shown in FIG. 13, the blocks around the image can be set independently of other parts. This example is similar to the first example, but the method of setting the blocks at the upper and lower ends of the image is different, and the number of blocks in the shaded portion 42 is reduced.

Next, the quantization width control in the coding parameter adjusting unit 104 of the first embodiment of the image coding apparatus of the present invention will be described.

Reference “MPEG2 Quantization and Coding Control”
(Technical report of Television Society Vol. 16, No. 61.
The method described on pages 43 to 48) adjusts the quantization width for each block to control the average number of bits in a given section (usually a dozen frames) to be constant. Document "Image quality improvement technology in video coding" (Sharp technique, No. 6, December 1994, pages 25 to 30)
In the method described in (Page), the quantization width once determined for data amount control is set small for blocks in the selected area and large for other blocks. There is. By doing so, the image quality of the selected area can be improved.

Similarly, in the image coding apparatus of the present invention, the quantization width in the selected area is adjusted to be smaller than that in other areas. The difference from the conventional method is that the encoding device of the present invention encodes the coordinates of the area. Therefore, for example, if the offset of the quantization width is set in advance in each area, the amount of information of the quantization width is determined by the conventional method. It is possible to reduce the number and it is convenient for high efficiency coding.

FIG. 14A shows an example of the quantum width of each block in the conventional example. For high efficiency coding, it is desirable to predictively code these values. That is, when the quantization width is encoded in the raster scan order, the difference between the quantization width of the current block and the quantization width of the block on the left is encoded.

In FIG. 14 (b), the shaded area is the selected area, and the quantization width is reduced by 3 for the blocks in the selected area (that is, the offset is -3), and for the other blocks. Is set to be larger by 3 (that is, the offset is +3).

In the conventional method, since the quantization width of FIG. 14B is encoded as it is, the difference in the size of the quantization width becomes large at the boundary between the selected area and the other area. It is not possible to improve the efficiency by predictive coding in which is the coding target.

However, in comparison with this, in the method of the image coding apparatus of the present invention, since the coordinates of the area are coded,
As the quantization width information, the information in FIG. 14A, that is, the information having a small difference in quantization width between adjacent blocks can be encoded, resulting in high efficiency. In this method, a method for obtaining the quantization width actually used from the encoded quantization width is decided on the encoding side and the decoding side, and as the quantization width actually used on the encoding side and the decoding side, FIG. It will be used after being changed as in b).

In the above example, a method has been described in which the quantization width is given an offset in each area, but the quantization width may be multiplied by a predetermined coefficient in each area. further,
The offset and the coefficient may be set in advance, or the value may be adaptively controlled and incorporated into the encoded data.

Next, the frame dropping control in the coding parameter adjusting unit 104 of the first embodiment of the image coding apparatus of the present invention will be described.

International standard encoding system for moving images used in video telephones and video conferences. In H.261, there are dropped frames, that is, frames that are not encoded. The frame number is encoded and incorporated into the encoded data to indicate the temporal position of the encoded frame. In the encoding device of the present invention, in addition to such frame dropping, there is provided a case where only the region other than the selected region is frame dropped.

As described above, in order to represent the case where the areas other than the selected area are dropped, that is, not encoded,
1-bit information is used for each frame and incorporated into encoded data. For example, as shown in FIG. 15, 1-bit data is incorporated into encoded data. When this data is 1, only the selected area is encoded, and when it is 0, the entire screen is encoded.

Alternatively, as shown in FIG. 16, 1-bit data is incorporated in the encoded data, and this data is 1
When, only the selected area is coded, and when 0, only the area other than the selected area is coded. 16 and 1
In FIG. 7, the hatched portion represents the encoded portion.

In either case, in the corresponding decoding device,
With the 1-bit data and the data representing the coordinates of the area, it is possible to know which part of the image data is present in the encoded data.

As a method of determining the above-mentioned 1-bit data by the encoding device, a method of determining a cycle in advance,
A method of monitoring the data rate of the coded data and controlling so that the value 1 is frequently generated when the rate is equal to or higher than a predetermined rate and the value 0 is frequently generated when the rate is lower than the predetermined rate is used. In the former method, it is not necessary to add the above 1-bit data to the encoded data.

In any case, the image quality of the selected area can be improved by making the time resolution of the selected area higher than that of the other areas. Further, by providing the time resolution in the area other than the selected area, it is possible to deal with the case where the background of the videophone changes, for example.

Next, the spatial resolution control in the coding parameter adjusting unit 104 of the first embodiment of the image coding apparatus of the present invention will be described.

In the coding apparatus of the present invention, the pixels in the selected area are coded without thinning, and the other areas are coded after the number of pixels is reduced by thinning to reduce the spatial resolution. The thinning rate may be determined in advance, or the thinning rate may be adaptively controlled and the thinning information may be encoded.

In the decoding device, when the encoded data is the data of the selected area, it is displayed as it is, and when it is the data of the other area, it is displayed after being converted to the original resolution by pixel interpolation. To do. By controlling the spatial resolution in this way, the image quality of the selected area can be improved.

The method for adjusting the quantization width, the number of dropped frames, and the spatial resolution as the encoding parameters has been described above, but these parameters can be used in combination. An encoding apparatus and a decoding apparatus that use these parameters in combination will be described below.

FIG. 17 is a block diagram showing a second embodiment of the image coding apparatus of the present invention.

The coding apparatus according to the present embodiment has a frame drop determining unit 111 that determines which part of an image is dropped and a frame number coding unit 112 that codes the frame number of a frame to be coded. And a region selection unit 114 for selecting a specific region in the image, and a region position / shape encoding unit 115 for encoding the position and shape of the selected region.

Further, the quantization width control unit 1 for controlling the average number of bits in a given section to be constant while adjusting the quantization width to be smaller in the selected area than in other areas.
16, a quantization width encoding unit 117 that encodes the obtained quantization width, a thinning unit 118 that controls the thinning rate of pixels for each region, and a moving image code that encodes input moving image data. The encoding unit 119 and the encoded data integration unit 113 that integrates and incorporates encoded data are provided.

For example, as shown in FIG. 15, the frame omission determining unit 111 encodes the entire screen without frame omission, performs frame omissions on frames 1, 2 and 3, and selects a frame 4 on the selected area. Only the code is encoded and the other areas are dropped, and the dropping method is determined.

In the quantization width encoding unit 117, as already described, it may be determined in advance as a method for obtaining the quantization width actually used from the quantization width obtained for controlling the data amount. Alternatively, the information may be adaptively changed and the information may be encoded. In any case, it is possible to avoid an increase in the code amount due to a large change in the quantization width at the boundary of the selected area.

The thinning section 118 maintains a high spatial resolution in the selected area and lowers the spatial resolution in other areas. The thinning rate also includes a mode in which the original image is not thinned and is delayed as it is. High spatial resolution data is encoded for the selected region. In the other areas, the data that has been thinned out to have a low spatial resolution is encoded.

Although not shown, generally, in the coding using motion compensation prediction, a "local decoder" is set in the coding device.
A decoding unit and a frame memory called as are provided, and the same decoded image as that obtained by the decoding apparatus is obtained and stored in the frame memory.

The selected area in the image is encoded as described above.

According to the encoding device configured as described above, the quantization width and DC for the selected area in the image are
Not only the number of T coefficients, but also the number of dropped frames, the resolution, and the like, the image quality of the selected region can be encoded better than others.

FIG. 18 is a block diagram showing a second embodiment of the image decoding apparatus of the present invention.

The decoding apparatus of this embodiment has a coded data separation unit 211 for separating the input coded data, a frame number decoding unit 212 for decoding the frame number of the coded frame, and a quantization width. Quantization width decoding unit 2 for decoding
13, a region position / shape decoding unit 214 that decodes the position and shape of a region from encoded data, a moving image decoding unit 215 that decodes moving image data, and an interpolation that interpolates data in regions with different spatial resolutions. It includes a unit 216 and a frame memory 217 that stores the decoded image.

The encoded data separation unit 211 separates the input encoded data and is required for the frame number decoding unit 212, the quantization width decoding unit 213, the area position / shape decoding unit 214, and the moving image decoding unit 215, respectively. Coded data is supplied.

In the quantization width decoding unit 213, when the quantization width obtained for controlling the data amount is decoded,
The quantization width of the area selected by the predetermined method is reduced, and the quantization width is changed in the other areas to be used for decoding the image data.

The interpolating unit 216 interpolates data in areas having different spatial resolutions so that the number of pixels becomes the same in all areas. The decoded image data is displayed on a display or the like, and is also stored in the frame memory 217 in the figure and used as a reference image for motion compensation prediction.

As described above, the coded data of the selected area in the image is decoded.

According to the decoding device configured as described above, the position and shape of the area selected at the time of encoding are decoded, and not only the quantization width and the number of DCT coefficients but also the number of dropped frames,
Due to the difference in resolution, the image quality of the selected area can be decoded better than the image quality of other areas.

[0079]

According to the image coding apparatus and the image decoding apparatus of the present invention, coding / decoding can be performed such that the time resolution of the selected area becomes higher than the time resolution of the other areas. .

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of an encoding device of the present invention.

FIG. 2 is a block diagram showing a first embodiment of the decoding device of the present invention.

FIG. 3 is a first example of a region in an image handled by the encoding device of the present invention.

FIG. 4 is a second example of a region in an image handled by the encoding device of the present invention.

FIG. 5 is a diagram showing an eight-direction quantization code.

FIG. 6 is a third example of a region in an image handled by the encoding device of the present invention.

FIG. 7 is a fourth example of regions in an image handled by the encoding device of the present invention.

FIG. 8 is a fifth example of regions in an image handled by the encoding device of the present invention.

FIG. 9 is a sixth example of areas in an image handled by the encoding device of the present invention.

FIG. 10 is a first example of block setting in an image handled by the encoding device of the present invention.

FIG. 11 is a second example of block setting in an image handled by the encoding device of the present invention.

FIG. 12 is a third example of block setting in an image handled by the encoding device of the present invention.

FIG. 13 is a fourth example of block setting in an image handled by the encoding device of the present invention.

FIG. 14 is a diagram showing an example of quantization width for each block.

FIG. 15 is a diagram showing a first example of frame dropping in the encoding device of the present invention.

FIG. 16 is a diagram showing a second example of frame dropping in the encoding device of the present invention.

FIG. 17 is a block diagram showing a second embodiment of the encoding device of the present invention.

FIG. 18 is a block diagram showing a second embodiment of the decoding device of the present invention.

[Explanation of symbols]

101, 114 area selection unit 102,115 area position / shape coding unit 103,113 coded data integration unit 104 Coding Parameter Adjusting Unit 105 Parameter Encoding Unit 106,119 Moving image coding unit 111 Frame drop decision section 112 frame number encoder 116 Quantization width control unit 117 Quantization width coding unit 118 Thinning section 201 and 211 encoded data separation unit 202, 214 area position / shape decoding unit 203 Parameter Decoding Unit 204,215 Video decoding unit 212 frame number decoding unit 213 Quantization width decoding unit 216 Interpolator 217 frame memory

Continuation of the front page (56) Reference JP-A-4-354489 (JP, A) JP-A-4-219089 (JP, A) JP-A-6-245082 (JP, A) JP-A-8-140096 (JP , A) R.S. A. F. Belfor and M.D. P. A. Hesp, Spatially Adaptive Subsam pling of Image Sequences, IEEE Transactions on Image Processing, September 1994, Vol. 3, No. 5, p. 492-500 (58) Fields surveyed (Int.Cl. ⁷ , DB name) H04N ^7/ 24-7/68 JISST file (JOIS)

Claims (4)

(57) [Claims] 1. A plurality of rectangular images each having a rectangular shape are encoded.
An image encoding device for encoding, which adjusts parameters relating to image quality of each of the rectangular images.
Parameter adjusting unit , a parameter encoding unit that encodes the parameters, and a fixed-length encoding unit that encodes the number of vertical and horizontal pixels of the rectangle.
And divide the image into blocks and sign the pixel values of the image
And an image information coding unit for converting the block, and the division position of the block is set independently for each rectangular image.
An image encoding device characterized by the following. 2. The parameters are quantization width and frame drop
Characterized by at least one of number and spatial resolution
The image coding apparatus according to claim 1, wherein 3. A plurality of rectangular image marks having a rectangular shape.
An image decoding device for decoding encoded data , wherein encoding of parameters relating to image quality of each of the rectangular images
A parameter decoding unit that decodes data, and a fixed unit that decodes coded data of the vertical and horizontal pixels of the rectangle
Long decoding unit and encoded data of pixel values of the image divided into blocks
And an image information decoding unit for decoding the block information, and the division position of the block is set independently for each rectangular image.
An image decoding device characterized in that 4. The parameters are quantization width and frame drop
Characterized by at least one of number and spatial resolution
The image decoding device according to claim 3.