JPH0879757A - Moving image encoding device - Google Patents

Abstract

PURPOSE: To provide a moving image encoding device for which compression for an area is set corresponding to the time when the area appears in moving images and to reduce a data amount further while preventing the degradation of the picture quality to the naked eye of moving image data. CONSTITUTION: An area division part 10 divides image data into the plural area corresponding to luminance and chrominance, etc., a buffer part 15 stores the divided image data for 30 frames (for one second) and an encoding part 20 recognized the continuous appearance time of the respective area by referring to the buffer part 15. Then, the area whose continuous appearance time is long is encoded with low compression (high accuracy) and the area whose continuous appearance time is short is encoded with high compression (low accuracy). Thus, this moving image encoding device capable of reducing the data amount of codes without degrading the visual picture quality is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture coding device. In particular, the present invention relates to a moving picture coding apparatus capable of realizing a high compression rate by adapting to each scene.

[0002]

2. Description of the Related Art In recent years, with the progress of image processing technology, not only still images but also moving images have been extensively processed. For example, various kinds of image processing are performed in image communication such as videophone and videoconference, or processing of video signals. Since the data amount of such a moving image is generally extremely large, various moving image coding methods have been proposed for compressing the data amount of the moving image. Encoding for compression of a moving image includes, as one of general methods, for data compression using, for example, the correlation of image signals in space-time. Also,
It is also proposed to use some of human visual characteristics to compress data.

The former method of utilizing the correlation of data, that is, the method of utilizing statistical properties, utilizes the fact that the moving image data has a very strong correlation between frames or between pixels. For example, as this type of encoding method, a predictive encoding method can be cited. The predictive coding system predicts a pixel value of interest from a previous frame or surrounding pixels and codes a prediction error. It is expected that this prediction error will be a scattered value centered on zero. As another encoding method of this kind, a conversion encoding method can be cited. This transform coding method is a coding method that utilizes the fact that when moving image data is converted into the spatial frequency domain, the energy concentrates in the low frequency domain. That is,
This is a method in which an important component in the low frequency region and an unimportant component in the high frequency region are separated to provide a difference in encoding accuracy.

An example of the latter method of utilizing the human visual characteristics is a method of utilizing the fact that the human visual sense is insensitive to high frequency noise. this is,
This is a method of roughly (with low accuracy) encoding the high-frequency component of moving image data. Also, since human vision is generally sensitive to luminance but insensitive to chromaticity,
A method of coarsely (with low accuracy) encoding the color difference signals is also conceivable.

Other usable human visual characteristics include a visual characteristic that is sensitive to noise in a flat portion but insensitive to noise in an edge portion, and a characteristic that sensitivity to an oblique component is low. Conventionally known.

[0006] In particular, as a human visual characteristic peculiar to a moving image, there is a perceptual characteristic of motion. That is, human vision is insensitive to objects moving at a certain speed or higher. Further, as the speed of the object increases, the visual sensitivity to high frequency components deteriorates. Further, the moving body visual acuity deteriorates as the illuminance decreases. For example,
When the eyes are fixed, the change in viewing angle per second is 15 deg.
It is generally known that the shape of the object is perceived for movements up to / sec, but it appears as a mere band of light for movements over 50 deg / sec. It is known that when the eyes are moved, it is possible to follow the movement up to about 25 to 30 deg / sec. Such visual characteristics could be used, for example, in sports broadcasting and in the coding of images of car racing.

There is also a visual characteristic due to human psychological factors. For example, it is known that human vision is insensitive to distortion of a portion (background) other than the attention portion.

A moving picture coding apparatus utilizing the above human visual characteristics is described in, for example, Japanese Patent Laid-Open No. 3-16490. The moving picture coding device described here detects a motion vector between frames of a moving picture signal and determines whether or not the block is moving by this motion vector, and determines that the block is moving. In this case, the coding device sets a coarse quantization characteristic in the case of doing so and sets a dense quantization characteristic in the case of determining that the block is not moving. That is, the accuracy of encoding is changed depending on whether or not each block is moving to compress data.

[0009]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
According to the moving picture coding apparatus described in Japanese Patent Laid-Open No. 90, since the coding can be performed in consideration of the motion of the picture, the data can be efficiently compressed without deteriorating the visual quality of the reproduced moving picture. Is.

However, in the above-described conventional moving picture coding apparatus, it is determined whether or not the moving picture is moving, in blocks, that is, in each block. Therefore, even in a block which is determined to be moving, a still portion may exist. In such a case, the image quality of the still portion may be visually deteriorated. On the contrary, even in a block determined to be still, there may be a moving part. In such a case, the moving part may be encoded with high accuracy and the data amount may increase. Further, since the actual boundary of the moving / static region and the boundary of the block do not coincide with each other, discontinuity of the quantization density occurs at the boundary portion of the moving / static region, resulting in distortion.

In particular, a problem in the above-mentioned conventional example is that a stationary object is accurately coded even if it appears in a moving image for only a short time.

Generally, an object that appears in a moving image for an extremely short time is difficult to be recognized accurately by human eyes.
Therefore, even if the object (region) that appears in the image for an extremely short time is encoded with a high compression rate, the deterioration of the image quality is hardly perceived with the naked eye. In the above-mentioned conventional example, as long as the area that appears only for a short time is controlled, the data is coded with high accuracy, so that the data amount of the code cannot be significantly reduced.

The present invention has been made in view of the above problems, and an object thereof is to provide a moving picture coding apparatus in which a compression rate for the area is set according to the time during which the area appears in the moving picture. It is an object of the present invention to provide a moving picture coding device capable of further reducing the amount of data while preventing deterioration of the image quality of moving picture data with the naked eye.

[0014]

In order to solve the above-mentioned problems, a first aspect of the present invention is to divide a moving image into a plurality of regions, and to divide the divided regions into continuous appearance times. An encoding device for encoding each area with accuracy according to the length, and a moving image encoding device. Here, the continuous appearance time means the time during which a certain divided area continuously appears in the displayed image.

A second aspect of the present invention, in order to solve the above problems, in the moving picture coding apparatus according to the first aspect of the present invention, the area dividing means divides the moving picture into a plurality of irregular areas. It is a moving picture coding device characterized by the above.

In order to solve the above-mentioned problems, a third aspect of the present invention is the moving image encoding apparatus according to the second aspect of the present invention, wherein the area dividing means is the direction / size of the movement of each pixel in the image. The moving picture coding device is characterized by dividing a plurality of regions according to the above.

In order to solve the above-mentioned problems, a fourth aspect of the present invention is the moving image encoding apparatus according to the second aspect of the present invention, wherein the area dividing means is responsive to the magnitude of the luminance of each pixel in the image. The moving picture coding apparatus is characterized in that a plurality of areas are divided into two areas.

A fifth aspect of the present invention, in order to solve the above-mentioned problems, in the moving image encoding apparatus according to the second aspect of the present invention, the area dividing means is provided with a plurality of pixels in accordance with the color of each pixel in the image. A moving picture coding apparatus characterized by dividing an area.

[0019]

The encoding means in the first aspect of the present invention encodes each area with the accuracy corresponding to the continuous appearance time, that is, the compression rate. Therefore, by roughly coding (at a high compression rate) an area that appears for only a short time, it is possible to reduce the data amount of the code with almost no deterioration in image quality with the naked eye. .

The area dividing means in the second aspect of the present invention divides the image into irregular areas, and the encoding means encodes the accuracy, that is, the compression rate, according to the continuous appearance time of the area. Therefore, the discontinuity of the quantization density along the boundary of the block does not occur as compared with the case of dividing into rectangular block areas and encoding.

The area dividing means in the third aspect of the present invention divides the area according to the direction and size of the movement of each part of the image. That is, a portion that moves in the same (similar) direction is grasped as one area, and the encoding means performs a predetermined encoding on the thus grasped area.

The area dividing means in the fourth aspect of the present invention divides the area according to the brightness of each pixel of the image. That is,
A plurality of pixels having the same (similar) brightness are grasped as one region, and the encoding means performs a predetermined encoding on the region thus grasped.

The area dividing means in the fifth aspect of the present invention divides the area according to the color (chromaticity, saturation, color difference, etc.) of each pixel of the image. That is, an equivalent (similar) color (chromaticity,
A plurality of pixels having saturation, color difference, etc.) are grasped as one region, and the encoding means performs a predetermined encoding on the region thus grasped. Here, the color means values such as chromaticity, saturation, and color difference.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a moving picture coding apparatus which is a preferred embodiment of the present invention. FIG.
As shown in, the moving image data is first divided into a plurality of predetermined regions by the region dividing unit 10. The division of this area is performed by a motion vector (for example, optical flow), brightness, color, and the like. Here, the color specifically refers to values such as chromaticity, saturation, and color difference. What is characteristic of the present embodiment is that this area division is performed in an indeterminate area, and is not simply division into rectangular blocks. As described above, in the conventional encoding device that performs division into simply rectangular blocks, block-shaped distortion is likely to occur, but in the present embodiment,
Since the area is divided into the irregular areas by the above-mentioned predetermined characteristic amount (luminance, color, etc.), block-like distortion as in the prior art does not occur.

When area division is performed by the brightness of pixels,
Region division is performed by clustering into pixel groups having similar luminance values, but as a concrete method, for example, a K-means algorithm or the like is suitable. K
The averaging algorithm is a method of area division by clustering, and is a method of classifying a set of pixel groups having certain R-type feature values X1 to XR into K clusters.

There is another simple method for area division, for example, an area expansion method. In this method, if there are adjacent areas having similar attribute values, they are integrated to form a larger area.

After the area is divided into the irregular areas in this way, a moving image of a predetermined frame is stored in the buffer unit 15, and then the encoding is performed for each area. This encoding is performed by the encoding unit 20 in FIG. This encoding unit 20
Is performed by, for example, DCT encoding.

The encoding unit 20 performs encoding with a different compression rate, that is, accuracy for each area. That is, the visually important region is compressed (encoded) with high accuracy, and the non-important region is encoded with low accuracy, that is, a high compression rate. In this way, since the encoding accuracy is changed according to the importance of each area, efficient encoding can be performed and the amount of encoded data can be reduced. As a result, the amount of data to be transmitted can be reduced in a videophone or the like, and a moving image that can sufficiently follow the movement of an object can be provided.

A feature of this embodiment is that the setting of the coding accuracy for each area is determined by the continuous appearance time of each area.

The continuous appearance time is the time from when the area appears in the moving image to when the area disappears. An explanatory diagram of this continuous appearance time is shown in FIG. FIG. 2 shows a time chart of a moving image. According to the sequence of moving images shown in FIG. 2A, the car appears for 5 frames. On the other hand, according to the sequence of moving images shown in FIG. 2B, the car appears over three frames. The frame rate of each moving image is 30 frames /
In terms of seconds, the continuous appearance time of the car in the sequence shown in FIG. 2A is 5 × (1/30) = 1/6 seconds. Then, the continuous appearance time of the vehicle in the sequence shown in FIG. 2B is 3 × (1/30) = 1 /
It will be 10 seconds.

According to the characteristics of the human naked eye, an object (area) that appears in an image for a shorter period of time has less influence on vision. Therefore, the encoding unit 20 in the present embodiment encodes each region with a higher compression rate (lower accuracy) in a region in which the continuous appearance time is shorter. In the calculation of the continuous appearance time in this embodiment, the number of frames accumulated in the buffer unit 15 is the maximum value. In this embodiment, 30 frames (corresponding to 1 second) of images are stored in the buffer unit 15, and the encoding unit 20 stores the images in the buffer unit 1.
The continuous appearance time of each region is checked by checking the images of 30 frames stored in 5.
That is, in this embodiment, the continuous appearance time is a value from 0 second to a maximum of 1 second.

As described above, in the present embodiment, since a predetermined number of frames of images are temporarily stored in the buffer unit 15, real-time encoding in a strict sense cannot be performed. Therefore, for example, although there is no problem in reproducing a video tape, there is a slight inconvenience in an application field such as a videophone that requires real time. However, by taking measures such as reducing the number of frames stored in the buffer unit, it is possible to make the time lag due to the presence of the buffer unit 15 almost unnoticeable. I think it can be applied.

In order to adjust the accuracy of image coding,
For example, it is preferable to adjust the sampling density or the quantization level at the time of quantization, but various conventionally known parameters can be adjusted.

In the above embodiment, the compression rate (encoding accuracy) of the area is adjusted by the value of the continuous appearance time, but the continuous appearance time is less than a certain value (for example, 1/10 second or less). It is also preferable not to encode the area. That is, it is possible to consider a region having a very short continuous appearance time as noise, and by excluding such a region having a short continuous appearance time from the encoding target, it is possible to further reduce the amount of code data. is there.

Further, in the above-mentioned embodiment, the setting of the compression rate of the encoding is performed only on the basis of the continuous appearance time of the area, but it is also preferable to combine it with other criteria. For example, even if the continuous appearance time of the area is long, if the area with large motion is encoded at a high compression rate, extremely efficient encoding according to the characteristics of the actual moving image can be achieved. You can do it.

As described above, according to the present embodiment, the moving image is divided into a plurality of irregular areas, and each area is encoded with a different compression rate (accuracy) according to its continuous appearance time. Therefore, it is possible to obtain a moving picture coding apparatus capable of reducing the code data amount without causing a significant deterioration in the visual quality of human eyes.

Further, if the region where the continuous appearance time is equal to or less than a certain value is excluded from the encoding target, there is an effect that the data amount of the code can be further reduced.

[0039]

As described above, according to the first aspect of the present invention, since the encoding is performed with accuracy according to the length of the continuous appearance time, the moving image code which enables more efficient encoding. There is an effect that a chemical conversion device can be obtained.

As a result, it is possible to obtain an image processing apparatus such as a videophone and a video reproducing apparatus in which the data amount of the code is reduced and the image quality is improved without degrading the image quality with the naked eye.

According to the second aspect of the present invention, the coding can be performed without causing the distortion due to the block shape, and
The present invention has the effect of providing a moving picture coding apparatus that enables efficient coding in consideration of image brightness.

As a result, it is possible to obtain a videophone or the like in which the code data amount is reduced and the image quality is improved without degrading the image quality with the naked eye.

According to the third aspect of the present invention, since the area division is performed so that the portion having the same movement is defined as one area, it is possible to obtain the area division that matches the contour of the moving object in the image. By changing the encoding accuracy for each, there is an effect that a moving image encoding apparatus capable of more efficient encoding can be obtained.

According to the fourth aspect of the present invention, since the area division is performed so that the portion having the similar brightness is regarded as one area, it is possible to obtain the area division that matches the contour of the object in the image.
By changing the encoding accuracy for each area, it is possible to obtain a moving image encoding apparatus capable of more efficient encoding.

According to the fifth aspect of the present invention, since the area division is performed so that the portions having the same color are regarded as one area, it is possible to obtain the area division that matches the contour of the object in the image, By changing the encoding accuracy to 1, it is possible to obtain a moving image encoding device capable of more efficient encoding.

[Brief description of drawings]

FIG. 1 is a conceptual diagram of functional blocks of a moving picture coding apparatus that is a preferred embodiment of the present invention.

FIG. 2 is an explanatory diagram of a continuous appearance time of a region.

[Explanation of symbols]

 10 area division unit 15 buffer unit 20 encoding unit

[Procedure amendment]

[Submission date] September 22, 1994

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0010

[Correction method] Change

[Correction content]

However, in the above-described conventional moving picture coding apparatus, it is determined whether or not the moving picture is moving, in blocks, that is, in each block. Therefore, even in a block which is determined to be moving, a still portion may exist. In such a case, the image quality of the still portion may be visually deteriorated. On the contrary, even in a block determined to be still, there may be a moving part. In such a case, the moving part may be encoded with high accuracy and the data amount may increase. Further, in order to the actual boundary and the block boundary of the moving / static image region does not match, the discontinuity of the coded accuracy is caused distortion at the boundary of the moving / static image region.

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0012

[Correction method] Change

[Correction content]

Generally, an object that appears in a moving image for an extremely short time is difficult to be recognized accurately by human eyes.
Therefore, even if the object (region) that appears in the image for an extremely short time is encoded with a high compression rate, the deterioration of the image quality is hardly perceived with the naked eye. In the above-mentioned conventional example, as long as the region that appears for only a short time is stationary , the data is encoded with high accuracy, so that the data amount of the code cannot be significantly reduced.

[Procedure 3]

[Document name to be amended] Statement

[Correction target item name] 0020

[Correction method] Change

[Correction content]

The area dividing means in the second aspect of the present invention divides the image into irregular areas, and the encoding means encodes the accuracy, that is, the compression rate, according to the continuous appearance time of the area. Therefore, never discontinuity of coding accuracy along the boundary of the block as compared with the case of coding by dividing a rectangular block area may occur.

Claims (5)

[Claims] 1. A region dividing means for dividing a moving image into a plurality of regions, and an encoding device for encoding each region with accuracy according to the length of continuous appearance time of each divided region. A moving picture coding apparatus comprising: 2. The moving picture coding apparatus according to claim 1, wherein the area dividing unit divides the moving picture into a plurality of irregular areas. 3. The moving picture coding apparatus according to claim 2, wherein the area dividing unit divides a plurality of areas in accordance with the direction and size of the movement of each pixel in the image. Video encoding device. 4. The moving picture coding apparatus according to claim 2, wherein the area dividing unit divides a plurality of areas according to the magnitude of the brightness of each pixel in the image. Image coding device. 5. The moving picture coding apparatus according to claim 2, wherein the area dividing unit divides a plurality of areas according to a color (chromaticity, saturation, color difference, etc.) of each pixel in the image. A moving picture coding device characterized by: