KR100188534B1 - Method for reducing shape information by using motion estimation and predictive contour - Google Patents

Abstract

The present invention relates to a motion estimation method for reducing the shape information transmission amount by transmitting a motion compensation prediction of a contour in a continuous image and transmitting a prediction error for each prediction error region, and a shape information reduction method using a prediction contour. The shape information is reduced by using the proposed method, which approximates the overall contour of the motion region, and then transfer-compensates and predicts the vertices used for the approximation and transmits the error. In the present invention, there is a drawback that the predictive contour which is compensated for by using the previous frame contour and the motion information and the transmission contour which is the contour of the current frame have similar shapes. By approximating the contours, video phones, video conferencing, Reducing the shape information of the motion object in the system in which the shape information of the moving object required as audio coding and computer vision and it is possible to obtain an improvement in image quality by ensuring that the transmitted data is reduced drastically.

Description

Shape Information Reduction Method Using Motion Estimation and Predictive Contours

1 is an exemplary diagram of video segmentation of a head and shoulder image.

2 is a state diagram of a previous frame and a current frame of a head and shoulder image.

3 is a diagram illustrating a comparison state between an image obtained by moving compensation of a previous frame and a current frame.

4 is a configuration block diagram of a motion compensation contour prediction coding scheme according to the present invention.

Figure 5 (a-c) is a polygon / splin approximation process for the present invention.

6 (a to c) is an approximation process diagram using the prediction contour for the present invention.

7 is a flowchart illustrating the motion compensation contour prediction coding method according to the present invention.

* Explanation of symbols for main parts of the drawings

11: subtraction means 12: critical performance means

13: contour prediction means 14: contour reconstruction means

15: addition means 16: mobile compensation prediction means

20 Transmission Contour 30 Polygon

40: splin

The present invention relates to a motion estimation method for reducing the shape information transmission amount by transfer compensation prediction of a contour line in a continuous image and transmitting a prediction error for each prediction error region, and a shape information reduction method using a predictive contour line.

Conventionally, the shape information is reduced by using the method proposed in the paper of HOETTER. In this method, the overall contour of the motion region is approximated, and then the motion compensation prediction of the vertices used for the approximation is performed, and the error is estimated. Send it.

As described above, the entire shape information is approximated and the motion compensation is predicted, and then the error is transmitted. Therefore, the image quality cannot be improved.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problem, and by performing motion compensation prediction for each prediction error region, transmission information can be selected and the outline approximation encoding process is simplified for the purpose of drastically reducing the shape information of the moving object. It is characterized by.

That is, in video encoding by object, the input image is divided into a changed region having a different signal from a previous image, and an unchanged region (background) with no signal change by the movement of an object. . The background area does not need any further analysis or transmission of information, and the receiving end reproduces using the signal of the previous image as it is. In the extracted motion region, motion information of an object is estimated using an object model and a motion model.

The motion information and shape information of the motion area are transmitted to the receiver. The receiving end reproduces an image with motion compensated prediction using this information.

Since the above method estimates motion information for each moving object in comparison with the conventional block-by-block coding method, the prediction performance is excellent and the block effect can be reduced, thereby improving subjective picture quality.

Here, subjective picture quality, for example, when high quality and low picture quality appear on one screen, the boundary of blocks is shown as a mosaic. In this block unit encoding, the boundary of blocks is called blacking phenomenon and this blacking phenomenon is invisible. When the subjective picture quality is good.

In addition, since shape information is transmitted to two adjacent pixels with the motion of an object interposed therebetween, a spot phenomenon and an edge business appearing in a block-by-block method do not appear.

On the other hand, there are regions where movement compensation prediction errors occur in the movement region. These regions are complex regions where the assumed object model does not fit or the movement model does not fit. do.

Since these areas are sensitive to the user's vision, color information is encoded and transmitted so that the receiver can faithfully reproduce the original signal. In addition, a newly uncovered background appears due to the movement of the object, and since the information is not included in the previous image, it is necessary to transmit information about it.

FIG. 1 shows an example of dividing a head and shoulder image into a background, a moving object, a moving compensable object, an uncovered object, and the like.

As described above, according to the present invention, the shape information of the moving object can be significantly reduced as a method of encoding shape information of the moving area.

Here, the shape information of the motion region may be represented by a binary image representing an area / non-region or a contour of a boundary thereof. Redundancy is large between shape information of a motion region by the same object in successive images.

The technique of moving compensation predicting a contour using such a property and transmitting a prediction error to reduce a shape information transmission amount is called a predective contour coding technique.

The contour prediction coding scheme for the present invention performs a thresholding operation and contour approximation to select a transmission prediction error in order to transmit the prediction error.

This method has the advantage that it is easy to control the large data rate and control the coding parameters without degrading the subjective picture quality compared to the conventional contour prediction coding.

The outline prediction coding scheme for the present invention will be described with reference to the accompanying drawings.

In object-specific video coding, shape information along with motion information should be transmitted at the highest priority for this video prediction.

The transmission of the shape information gives different motion information to pixels adjacent to each other around the boundary of the object, thereby avoiding a deterioration in the subjective quality due to spots, boundary distortions, etc. that appear in block-by-block coding.

Therefore, in low-rate coding, object-to-object coding becomes a clue that the shape information transmission has better subjective quality than block-by-block coding.

As a method for representing the boundary of an area, many techniques have been studied in computer graphics, character recognition, object synthesis, and the like.

Examples include chain difference coding, s, curves, polygon approximation, spline approximation, Fourier Descriptor, and the like.

However, since these methods do not consider transmission, they are difficult to use due to the high data rate when the contour of the motion region must be transmitted in every frame.

There are many similarities in shape and position between the shape information of the motion region generated by the same object on successive frames. Therefore, the present shape information can be predicted from the past shape information.

Furthermore, the motion information of the moving object can be estimated, and the shape information can be used to predict the motion compensation using this information. If the motion region extraction and the motion information estimation are ideally accurate, the transmission of the shape information is unnecessary.

However, as the data rate decreases, the shape information occupies a larger portion. Therefore, a significant reduction in shape information is required to obtain a coding gain as compared with the block-by-block coding method in which shape information transmission is not required.

To this end, the present invention uses a thresholding operation and contour approximation to select the transmission prediction error.

In this case, isolated prediction error regions are generated when the motion compensation prediction of the current shape information is performed.

Here, FIG. 2 shows shape information of a previous frame (N-1th frame) and a current frame (Nth frame), and FIG. 3 shows shape information of a previous frame (N-1th frame) according to motion information. When moving compensation predicts, 9 isolated prediction error areas are shown.

For each of the nine prediction error areas, a portion corresponding to shape information of the current frame is encoded and transmitted. Transmission of the prediction error itself may include information that does not affect human vision.

Important information affecting subjective picture quality should be transmitted, and transmission rate coding can be performed only when transmission of other information is suppressed. Therefore, information that does not affect subjective picture quality cannot be removed and transmitted by performing a critical operation. Performance uses the characteristics of the region where the error occurs.

In order to transmit the contour for transmission of the prediction error region determined to be transmitted, shape information is reduced by using an approximation method rather than an accurate transmission.

As the contour approximation method, a polygon / splin approximation method and an approximation method using predictive contours are used.

As described above, the configuration of the motion compensation contour prediction encoding method for shape information reduction is configured as shown in FIG.

That is, motion compensation prediction means 16 for motion compensation prediction of shape information of the previous frame and shape information of the current frame, and subtraction means 11 for obtaining a difference between the output of the motion compensation prediction means 16 and the motion region. By calculating the isolated prediction error areas through the subtraction means 11, critical information affecting the human vision is transmitted by performing a critical operation on the isolated prediction error areas. Threshold performing means 12 for suppressing, contour prediction means 13 for reducing shape information by using an approximation method rather than an accurate transmission for transmitting the contour for transmission of the prediction error region determined to be transmitted, and performing the threshold Contour reconstruction means 14 for reconstructing the contour encoded by means 12 and contour prediction means 13 into an isolated prediction error region and Adder means 15 for forming the shape information of the current frame by adding the isolated prediction error region by the contour reconstruction means 14 and the output of the motion compensation prediction means 16 to input to the motion compensation prediction means 16. Is done.

The contour approximation method used to transmit the contour for the transmission of the error region is as follows.

First, the error region in which transmission is determined performs contour coding for transmission.

The contour of the error region includes a contour (prediction contour) of the motion compensation prediction shape information. Only the contour (transmission contour: transmission contour) from which the prediction contour is removed is encoded and transmitted.

In the head and shoulder image, there are many error regions with narrow bands, and the amount of transmission data is considerable when using chain difference coding without error in the transmission contour coding of the region. Become.

Therefore, the contour coding method using the approximation expression is required, but the local position error of the contour occurs, but since the human visual characteristics are sensitive to the geometric shape of the whole object, this small position error does not lead to degrading image quality. Do not.

The polygon / splin approximation process used for such contour approximation is shown in FIG.

The transmission contour 20 of the error region in which the transmission is determined is first approximated by the polygon 30 as shown in FIG. 5 (a). At this time, the number of vertices of the polygon 30 depends on the degree of approximation between the actual transmission contour 20 and the approximate polygon 30.

This approximation is represented using the maximum difference between the actual transmission contour 20 and the approximate polygon 30, where a rough approximation is obtained when the maximum difference is large, but the number of vertices is reduced, and a fine approximation when the maximum difference is small. The number of vertices increases.

Then, as shown in (b) of FIG. 5, the splins 40 passing through the obtained approximate vertices are obtained, and the distances to the transmission contours 20 are examined at each pixel on the splins 40, and the distance is determined by the threshold value ( dMAX) or more, the approximate section including the pixel is approximated by the polygon 20 instead of the spline 40 as shown in FIG.

In addition, an index indicating whether the positions of the vertices and the intervals between the vertices are a polygon approximation or a splin approximation is transmitted to the receiver to reproduce the polygon / splin. Compared to polygon approximation, this contour approximation, combined with polygons and splins, shows a visually natural contour shape with fewer vertices.

The conventional contour prediction coding method is a method of first approximating an entire contour to a polygon / sprinkle, and then performing a motion compensation prediction of the contour with an approximate vertex. In contrast, polygon / sprinkle approximation for the present invention divides the entire contour into several transmission contours and approximates it.

In addition, the removal of the small error region approximates the reduction of the transmission amount, and the simplicity of execution is achieved by approximating several small transmission contours, respectively. In addition, the shape information transmission amount can be easily controlled by using a threshold value dMAX that can be given differently for each transmission outline and a threshold value tsm used for a small error area.

An approximation method using predictive contour, which is another method for contour approximation, is used as shown in FIG.

That is, although the motion region extraction is relatively good, the prediction contour and the transmission contour have a similar form in the error region generated due to the problem of motion information estimation.

In this case, it is possible to approximate the transmission contour by using the predictive contour, and this method eliminates the need for spline approximation, thereby simplifying performance and reducing the amount of information.

FIG. 6 shows an approximation process using the predictive contour. In order to approximate the transmission contour of the error regions (a, b) generated as a problem of motion information estimation rather than the problem of motion region extraction as shown in FIG. First, as shown in (b) of FIG. 6, an error magnitude with respect to the prediction contour is obtained, approximated as shown in (c) of FIG. 6, and then an approximation error is drawn on the frame with respect to the prediction contour.

This technique eliminates the need for a spline approximation to produce a visually natural shape since the reproduction contour is generated in dependence on the prediction contour.

Polygon approximation can be considered as a technique for approximating the error magnitude. In the contour approximation, since the error magnitude has a low curvature, the vertex can be approximated with fewer vertices on the frame.

However, polygon approximation coding is efficient when the frame has a high curvature like the transmission contour on the frame. In the approximation method using the prediction contour, the polygon approximation is approximated using the peaks and the position where the maximum error occurs and the both ends of the transmission contour are approximated vertices. .

Polygon / Splin approximation of high curvature transmission contours on the frame requires several vertices, but the contour approximation technique requires only three vertices to reduce the number of transmission vertices.

At the receiving end, as shown in (C) of FIG. 6, the error is approximated by two straight lines connecting both end points and the maximum error position, and then the approximate error is drawn on the frame.

Considering all the error areas occurring within a frame, the extraction of the area is good, but it can be divided into the case where the error area is generated and the error area is not well extracted.

In case of error domain but good extraction of domain, the similarity between transmission contour and prediction contour is high.

Therefore, in this case, the prediction contour can be used when approximating the transmission contour.

If the error region and the region are not well extracted, the similarity between the transmission contour and the prediction contour is not large, and in this case, it is difficult to use the prediction contour when approximating the transmission contour.

Reflecting these characteristics, the method of applying an approximation method to each of the above two regions includes a prediction contour that is compensated by movement compensation using the shape information contour of the previous frame and the motion information of the image continuous to the contour, and the shape of the current frame. A method of reducing shape information in video encoding for each object by using a transmission outline that is an information outline, the method comprising: detecting a number of pixels corresponding to a length of the prediction outline and a transmission outline, and each of the detected outlines A second process of determining the completeness of the extraction of the entire object region by using the number of pixels, and encoding a transmission contour of the corresponding region by using the polygon / splin approximation method in the case of the completeness of the object extraction. Approximation method using predictive contours in the third process and in the region of excellent object extraction A fourth process is performed to encode a transmission contour of a corresponding region by using an equation.

The fourth process may further include extracting a maximum error value between the prediction contour and the transmission contour of the corresponding region, and comparing the extracted maximum error value with a threshold value used in the polygon / splin approximation method. A process of determining whether the maximum error value is larger than a threshold value and approximating the point of the transmission contour where the maximum error occurred and both end points of the transmission contour as approximate vertices and encoding and transmitting only three vertices. And when the maximum error value is less than the threshold, the corresponding area is not transmitted.

The above process will be described in more detail with reference to FIG. 7 as follows.

First, the prediction contour and the transmission contour are found in the transmission error area, and their length, that is, the number of pixels, is obtained (step S1). It is judged whether or not the object extraction is a region using the obtained number of pixels of the two outlines (step S2). At this time, the evaluation function D as a criterion is as follows.

In the region where object extraction is not good, the difference between the two pixels is large, and thus the evaluation function D becomes large. Given the threshold value Dth, if the evaluation function D of the error region in which the transmission is determined is larger than the threshold value Dth, it is determined that there is a problem in object extraction and transmitted using a polygon / sprinkle approximation method that does not consider the prediction contour. The outline is encoded (step S3).

If the evaluation function D of the error region is smaller than the threshold Dth, an encoding method using a prediction contour is used. The process is as follows.

First, the maximum error value MAX ERROR is extracted (step S4), and it is determined whether the extracted maximum error value is larger than the threshold value dMAX used in the polygon / sprinkler approximation (step S5).

If the maximum error value (MAX ERROR) is smaller than the threshold value (dMAX), the area is not transmitted. This is not a problem since the approximation error is smaller even without transmission, and the long band error region belongs to this case.

If the maximum error value MAX ERROR is greater than the threshold value dMAX, the position where the maximum error occurs and both end points of the transmission contour are approximated as the peaks, and only three vertices are transmitted (step S6).

In addition, the approximation method using the contour has the advantage of reducing the transmission amount and the spline approximation by approximating only three vertices for each transmission contour compared to the conventional method.

As described above, according to the present invention, in a system requiring shape information of a moving object, such as a video telephone, video conferencing, video encoding, and computer vision, the shape data of the moving object is reduced to significantly reduce transmission data. You will get an improvement.

Claims (3)

In the method of reducing shape information in video encoding for each object by using the motion contour of the previous frame shape information and the motion contour of the image consecutive to the circumference line, and the transmission contour which is the shape information contour of the current frame. The method may further include a first process of detecting the number of pixels corresponding to the lengths of the prediction contour and the transmission contour, and a second determination of the completeness of the extraction of the entire object region using the number of pixels of the detected contours. A third process of encoding a transmission contour of the corresponding region using a polygon / sprinkle approximation method in the case of regions in which the object extraction is less than perfect, and an approximation using a predictive contour in the case of the excellent object extraction. Characterized in that a fourth process of encoding a transmission contour of a corresponding region is performed using the method. Reduction of shape information using motion estimation method and predictive contour. The method of claim 1, wherein the fourth process comprises: extracting a maximum error value between the prediction contour and the transmission contour of the corresponding region, and using the extracted maximum error value in the polygon / sprinkle approximation method. Determining whether the value is larger than the maximum value, and when the maximum error value is larger than the threshold value, approximates the point of the transmission contour where the maximum error occurs and both endpoints of the transmission contour as approximate vertices and encodes only three vertices. And a process of transmitting a region and a process of not transmitting a corresponding region when the maximum error value is smaller than a threshold as a result of the comparison. According to claim 1, The completeness of the object extraction in the third process is determined by the evaluation function (D) obtained by the equation given below, when the completion of the object extraction is less than the difference between the number of two pixels The shape information reduction method using the motion estimation method and the predictive contour, characterized in that the function (D) is also large, and the difference between the two pixels is small when the object extraction is good.