KR100928889B1 - Image Segmentation and Synthesis Method Using Image Polyhedral Structure - Google Patents

Abstract

The present invention relates to segmentation, contour extraction, and image synthesis for a specific region in an image based on a clustering method for maintaining an image polyhedron structure. By providing a more automated technique and user interface for each image frame, most of the manual process provides an efficient and reasonable area segmentation, contour extraction, and image synthesis method.

Digital Video Special Effect, Video Rotoscoping, Image Segmentation, Geodesic Distance, Data Manifolds

Description

Image segmentation and synthesis using image polyhedral structure {METHOD FOR VIDEO SEGMENTATION AND COMPOSITION BASED ON SCENE MANIFOLDS}

The present invention relates to a rotoscoping operation, one of digital video special effects (DVFX) mainly used in image post-processing, and in particular, segmentation of a specific region within an image frame. And an image segmentation and synthesis method using an image polyhedron structure suitable for more rational implementation of composition.

Rotoscoping refers to a series of tasks that extract the contour information of a specific object within a given image frame and use it to perform composition such as color change or texture change or deletion. Means changing the image and geometric characteristics of a specific object in a video.

Recently, with the development of computer graphics and computer vision technology, interest in digital image special effects is increasing. However, the post-processing work on this specific area is almost entirely using computer graphics software in film or video image production. The situation is done by hand. For example, compositing a specific object in a given image requires effort that requires independent post-processing of the object present on every frame of the image.

More specifically, existing commercial software for rotoscoping includes "Commotion" and "Adobe After Effects", which use contour points drawn by a user for a specific image image on a given image frame. By tracking by using optical flow to extract the object region in the remaining image frame.

However, this method is sensitive to the initialization and the error rate is gradually increased for the frame which is far from the initialization frame on the time axis, which still requires a lot of manual work of the user.

Automatically extracting the contours of moving objects is one of the more difficult techniques in computer vision, and in traditional image compositing for image frames, these tasks are independent on each 2D (2D x and y axis) image. There is a limit that only consists of. In other words, it overlooks consistent information that exists in the region of interest on the time axis.

However, recalling that the image is a series of 2D images accumulated on the time axis, it can be expected that it is more reasonable to apply region segmentation using information on the time axis.

Accordingly, the present invention is to provide a cluster analysis method using a manifold for each image existing in the image frame in order to more consistent region segmentation and contour extraction on the time axis.

In addition, the present invention aims to provide an interface environment that can be operated by the user on the x-axis and time-axis, away from the methodology limited to the image work of the existing 2D (2D x-axis and y-axis) image.

According to a preferred embodiment of the present invention, a process of extracting contour information of a specific image region of an input image frame, and extracting the contour information of the specific image region based on the x-axis and time axis according to the extracted contour information of the specific image region An image segmentation and synthesis method may be provided that includes segmenting an image frame and synthesizing the image frame.

According to the present invention, a rotoscoping process by a user within a given image frame, cluster analysis using an image manifold on a time axis centered on the region, and synthesis are performed on existing x and y axes. By providing a work processing environment on the x-axis and time-axis away from the image processing work that has been made, it is possible to obtain a consistent result from a more diversified perspective.

A video segmentation and image synthesis method using an image polyhedron structure according to the present invention includes a region portion defined by a user in a given image frame and a cluster analysis portion using an image manifold on a time axis around the region, And it is characterized by consisting of synthetic parts.

The present invention is a clustering using a geometric manifold structure of individual images in an image for segmentation, contour extraction, and synthesis of moving objects in the image, which is one of the most important techniques in digital image special effects. By applying the analysis method, more consistent segmentation and synthesis results can be obtained for all frames.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

Prior to the description of the embodiment, the area portion defined by the user provides various methods for expressing a curve such as Bezier or Spline, and provides a method for starting and ending frame images to which an image processing operation is to be applied. Applies for both key frames.

In the cluster analysis part using the image manifold existing on the time axis, a cluster analysis method of mean shift is applied. Image segmentation is performed by applying a center transformation analysis method, which is one of non-parametric methods. In addition, in order to maintain the manifolds structure for each image existing on the time axis, the geodesic distance measurement method instead of the euclidean distance measurement method when applying the center transformation is used. It is characterized by applying.

In the image synthesizing portion, color and texture modification may be performed through various interfaces on the divided images obtained from cluster analysis using a specific image manifold existing on the time axis.

1 is a system block diagram illustrating an image segmentation and synthesis method using an image polyhedron structure according to the present invention, which includes an image input unit 100, an image segmentation and contour extracting unit 102, and an image synthesis unit 104. do.

As shown, the image input unit 100 provides an image frame for the image input by the user to the image segmentation and contour extracting unit 102, and outlines a specific region according to a rotoscoping process by the user. The definition is provided to the image segmentation and contour extracting unit 102.

At this time, rotoscoping refers to a series of operations that extract the contour information of a specific object in a given image frame and perform composition such as color change or texture change or deletion of the object. To change the image and geometric characteristics of a particular object.

The image segmentation and contour extracting unit 102 performs image segmentation and contour extraction according to the input image frame and the contour of a specific region defined by the user. The video segmentation at this time is the video segmentation which holds the video manifold existing on the time axis according to the present embodiment.

The image synthesizing unit 104 performs an image synthesizing process for the divided image region provided by the image dividing and contour extracting unit 102.

Hereinafter, the image segmentation and synthesis method using the image polyhedron structure according to the preferred embodiment of the present invention will be described in detail with reference to the flowchart of FIG. 2.

First, when an arbitrary image frame is input through the image input unit 100 (S200), an outline of a specific region of the image frame is defined according to the result of the rotoscoping process by the user (S202).

Afterwards, the image segmentation and contour extracting unit 102 performs image segmentation and contour extraction. In performing a specific region division on the image frame, the image segmentation and contour extracting unit 102 performs operations performed on the existing x-axis and y-axis 2D images. More consistent region division is performed using the time axis (t-axis) (S204).

That is, in order to maintain the image manifold structure existing on the time axis, a method based on geodesic distance is used instead of the method based on general Euclidean distance.

In addition, the mean shift, which is one of the above distance measurement method and non-parametric method, for an image existing at the center of the x-axis and the time-axis instead of the region division of the 2D image on the x-axis and the y-axis ) To perform image segmentation more consistently.

Then, an image synthesis process is performed on the divided image region (S206). In this case, the image synthesis process is a process of performing texture and color conversion on the region based on the x-axis, y-axis, and time-axis (t-axis) information on the region obtained from the geodesic distance-based center transformation technique. .

FIG. 3 is a flowchart illustrating steps S202 and S204 of FIG. 2 in more detail, and illustrates an automation method for image segmentation and an interface for user rotoscoping.

As shown in FIG. 3, after defining the contours of the ROIs on the two key frames through the rotoscoping process by the user (S300), the contours defined in the step S300 are defined by the x-axis and the time-axis (t). Axis) is specified centering (S302).

Thereafter, in step S304, image segmentation using an image manifold is performed on the time axis (t-axis). That is, the image is segmented using the contour information on the time axis (t axis). In this case, the present embodiment is characterized by applying the image segmentation method based on the geodesic distance described above.

In operation S306, an image segmentation result of an image frame between two key frames is obtained.

At this time, in the present embodiment, the efficiency can be increased by complementarily using the two relationships with each other. That is, by using a mapping relationship between a point on an image existing on a time axis t and a point present on a 2D image of an x-axis, a user may be provided with an environment in which the user can work at various angles.

4A and 4B show the image on the x-axis and the y-axis and the image on the x-axis and the time-axis (t-axis) for the given image sequence, in the part described above.

Unlike the image on the x-axis and the y-axis as shown in FIG. 4A, the image centered on the time axis (t-axis) (FIG. 4B) has a long road shape with respect to the corresponding area. It can be expected that the center transformation methodology based on geodesic distance would be more reasonable for such images. The reason is that the geodesic distance is based on the shortest distance calculation and is more suitable for the image segmentation existing on the time axis (t-axis) as illustrated in FIG. 4.

Meanwhile, FIGS. 5A and 5B illustrate an example of a center transformation methodology based on the geodesic distance.

사이의 지오데식 거리(∥x-

∥_g)를 의미 한다. 도 5a에서 공간 (c)는 데이터들이 존재하고 있는 임의의 매니폴드 공간을 의미한다.In FIG. 5A, the dotted line (a) means the Euclidean distance (∥xp∥ _e ) between x and p, and the thick solid line (b) is the x and p according to the present embodiment.

Geodesic distance between (∥ x-

_G ) means. Space (c) in FIG. 5A means any manifold space in which data exists.

First, as shown in FIG. 5A, when the distribution of the corresponding data is examined, a distance closer to that of the data within the same cluster is measured in the different clusters under the conventional Euclidean distance (∥xp∥ _e ). It is easy to see that there is a problem.

∥_g) 측정을 적용한 것을 특징으로 한다.In order to solve this problem, in the present embodiment, the geodesic distance (∥x−

_G ) characterized by applying the measurement.

As described above, the embodiments of the present invention have been described in detail, but the present invention is not limited to these embodiments, and various modifications may be made by those skilled in the art within the spirit and scope of the present invention described in the claims below.

1 is a system block diagram for performing an image segmentation and synthesis method using an image polyhedron structure according to the present invention;

2 is an overall flowchart of an image segmentation and synthesis method using an image polyhedron structure according to an embodiment of the present invention;

3 is a flowchart illustrating a user-defined area input and image segmentation process of FIG. 2 in detail;

4 is an exemplary diagram of an image manifold structure shown in image frames and a time axis according to the present invention;

5 is an exemplary diagram for data clustering using a data manifold structure in accordance with the present invention.

<Explanation of symbols for the main parts of the drawings>

100: image input unit 102: image segmentation and contour extraction unit

104: image synthesis unit

Claims (6)

Extracting contour information of a specific image region of an input image frame; Dividing the image frame around an x-axis and a time-axis according to the extracted contour information of the specific image region; Synthesizing the image frame to be divided into images; Image segmentation and compositing method. The method of claim 1, The extraction process, Specifying contour information of the specific image area about the x-axis and time-axis Image segmentation and compositing method. The method of claim 1, The image segmentation process, Acquiring an image segmentation result for the image frame by segmenting the image frame using an image manifold on the time axis Image segmentation and compositing method. The method of claim 3, wherein The acquisition process, And a mutual mapping relationship between an image centered on the x-axis and a time axis and an image of the x-axis and the y-axis. The method of claim 1, The division process, Image segmentation and synthesis method implemented by centroid transformation based on geodesic distance. The method of claim 5, wherein The synthesis process, And the texture and color transformation are performed on the divided image region obtained from the center transformation technique based on the x-axis, time-axis, and y-axis information.

Images