KR20010000250A - Method and apparatus for segmenting an image by using symmetric property - Google Patents

Abstract

PURPOSE: An apparatus and a method for object division of MPEG-4 using a symmetrical characteristic is provided to separate a specific object from a background picture by using a symmetrical characteristic of the object. CONSTITUTION: An apparatus and a method for object division of MPEG-4 using a symmetrical characteristic comprise a pre-processing portion(22), an assignment and object division portion(24), and a post-processing portion(26). The pre-processing portion(22) generates a simplified picture by filtering an input picture, obtains information of current input picture frame, and finds motion information between a current picture frame and a previous picture frame. The assignment and object division portion(24) performs line scanning for objects of a corresponding picture by using the simplified picture information and the current picture and finds information object information separated from a background. The post-processing portion(26) selects only a moving area as a final object and generates an object picture of a divided object.

Description

METHOD AND APPARATUS FOR SEGMENTING AN IMAGE BY USING SYMMETRIC PROPERTY}

The present invention relates to an object segmentation apparatus of MPEG-4 (Moving Picture Experts Group) and a method thereof. In particular, an image of a next generation multimedia portable terminal is divided by segmenting only a specific object excluding a background image by using a symmetric characteristic of an object before image compression. It is a technology that can greatly reduce the throughput.

In general, MEPG 4 is a method of encoding standard and audio-visual materials on a per-object basis for multimedia applications. The MEPG 4 achieves very high compression efficiency and can transmit at very low bit rates. Accordingly, since the transmission bandwidth required for transmitting a video can be reduced to about tens of Kbps and has a strong advantage in transmission error, multimedia communication is possible in an error environment such as a mobile communication network and an internet network as well as a telephone line.

On the other hand, the object segmentation technology of the image is to separate the desired object in the image based on the homogeneity or heterogeneity between the objects. This object segmentation technique is mainly used to segment only a specific object, that is, a person, from the background before image compression in order to reduce the amount of compressed information of the portable terminal applying MPEG-4. Various examples of such an object partitioning method are as follows.

As a first conventional object segmentation method, there is an image segmentation technique having a hierarchical structure of a top-down method. References to this technology are described in P. Salembier and J. Serra "Hierarchical morphological segmentation for image sequence coding" IEEE Trans. on IP, Vol 3, no. 5, 1994, pp. 639-651.

This image segmentation method is simplified using an open_close by reconstruction filter to make it easier to separate the image from the input image. In addition, the pixel information in the image is classified based on homogeneity with elements such as basic color and brightness within the portion of the image information to be objectified. Then, the boundary information of the desired object is obtained using a watershed algorithm. Based on the data obtained in the previous step, each segmented area is modeled by the polynomial approximation method, and then the boundary information and the segmented image are output based on this.

However, this method is difficult to apply to a portable terminal using a central processing unit with limited performance because the same process is repeated several times to improve the characteristics of the open-close filter. In other words, the object segmentation method using the open-close filter separates the object in the form of merging pixels having similar information in the image information, so the target image is more form than distinguishing the background and target person necessary for the video terminal. The disadvantage is that it is separated into objects.

As a second object segmentation method, the method of using brightness information and motion information together is described by J. G Choi, S.W Lee, S. D. Kim "Satio-temporal video segmentation using a joint similarity measure," IEEE Trans. on Circuits and Systems for Video Technology. Vol. 7, no. 2, pp 279-286, 1997.

This second method obtains a segmented image in the following order. First, the original image information is simplified by using a morphological filter. The pixels in the current image are classified with respect to brightness, and the pixels in the current image are classified into groups having similar values in the direction of motion through motion comparison analysis with the previous image frame. The image is then segmented by merging similarly shaped pixels with the analyzed motion and brightness information.

However, the second object segmentation method is difficult to apply to a mobile terminal requiring a fast transmission speed because it needs to calculate not only motion estimation information that requires a large amount of computation but also brightness information.

The third object segmentation method is a method of detecting a change region between the current frame and the previous frame to separate the changed part into objects so as to facilitate videophone image encoding.

In this third method, the brightness difference is obtained for two pieces of image information having a temporal difference in order to detect a change area, and a part having a difference in brightness is classified as a part having no motion. Thus, the part where motion does not exist is classified as the background and the part where motion does exist. The new background in this classification process is continuously reconstructed into mosaic to improve the coding efficiency.

However, in this third case, since the motion is estimated using only the brightness difference, when applied to a portable terminal that looks at the camera, only a part of the motion is objectified, rather than an object partition that separates the background from the person. In other words, when a person speaks without moving, an error occurs that processes all the image parts except the mouth.

The fourth object segmentation method is to utilize continuous three-frame image information.

First, the motion of the entire image is estimated using an affine motion model to obtain a change between the previous image and the current image, and the two images are matched through motion compensation between frames to be compared with the current frame. In addition, the average value of pixel difference values between the previous image and the current image is obtained to confirm whether there is a sudden change between the two images. Then, the brightness change of the previous image and the current image is calculated, and the portion with the large change in brightness is set as the movement, and based on this, the clue for extracting the moving portion is generated. Then, based on the statistics on the brightness difference, the system divides the motion into the nonexistent and nonexistent parts, obtains a rough boundary of the object from the difference between the frames, and obtains a finer boundary with the current frame information. Output

However, this object segmentation method also uses an open-close filter for motion estimation, which limits the application of portable terminals.

Therefore, the conventional object partitioning method has a disadvantage in that a computation apparatus having a high data processing capability is required because the processing speed and the calculation time are increased because the object and the object are usually divided using an open close filter. Also, in order to estimate the movement with the brightness of the light for the sake of accuracy, if the user moves from a dark place to a bright place or if the brightness of the light changes due to the ambient light, all the parts with the change of light are estimated as objects and eventually compressed. There was a problem that the data amount is increased.

An object of the present invention is to focus on the symmetry of the human body form in order to solve the problems of the prior art as described above pre-processing unit to divide the person (object) and the background image from the source image input through the camera of the mobile phone , MPEG-4 using a symmetrical feature to improve accuracy while greatly reducing the processing time and time required to divide the image information (object) to be transmitted from the mobile terminal using MPEG-4 by providing a background, an object dividing unit and a post processing unit. We want to provide a 4 object splitter.

Another object of the present invention is to pre-process the process of simplifying the image, to divide the background and the object by using symmetry, to compare the motion information and the divided object information to extract the image of the object with a motion and through a monoscopic filter Since the background and the object are divided through the post-processing process that generates the final object image after processing the boundary line, the processing speed and calculation time of the object segmentation can be shortened than before, and the symmetry characteristic can be greatly improved. An object segmentation method of MPEG-4 is provided.

In order to achieve the above object, the present invention provides an MPEG-4 object splitting apparatus for selectively dividing an object of an image to generate a simplified image by filtering an input image, and to obtain information of a currently input image frame. Pre-processing unit for finding motion information between the current image frame and the previous image frame stored, and through the pre-processing unit line scan the objects of the image with the simplified image information and the information of the current image frame to find the symmetrical part in the frame The background and object divider that creates object information to be separated from the background by finding the area with movement, and only the area with movement is selected as the final object according to the object information of the background and object divider, and the object is divided by boundary processing. And a post processor for generating an object image of the.

In order to achieve the above object, the present invention provides an MPEG-4 object segmentation method for selectively segmenting any object of an image. The present invention filters the input image to generate a simplified image and obtains information on the currently input image frame. Finding motion information between the current image frame and the previously stored image frame, and scanning the objects of the image with the simplified image information and the information of the current image frame to find the symmetrical part, and Searching for a region, generating object information to be separated from the background, and selecting only a region having movement according to the object information as a final object, and generating an object image of a target to be segmented by processing the boundary of the object.

1 is an overall configuration diagram of an object segmentation apparatus of MPEG-4 using symmetrical characteristics according to the present invention;

2 is a circuit diagram of a preprocessor of an object partitioning apparatus according to the present invention;

3 is a circuit diagram illustrating a background and an object division unit in the object division apparatus according to the present invention;

4 is a circuit diagram of a post-processing unit in an object splitter according to the present invention;

5A to 5C are flowcharts illustrating a method of segmenting an object of MPEG-4 using a symmetrical characteristic according to the present invention in a preprocessing step, a background and an object partitioning step, and a post-processing step, respectively.

<Description of the code | symbol about the principal part of drawing>

10: input image 20: object splitting device

22: preprocessor 24: background and object division

26: post-processing unit 30: the divided object image

220: HPF 222: video summary

224: frame buffer 226: motion estimation analyzer

242: line scan and edge recording unit

244: symmetry analysis unit 246: motion analysis unit

248: Merging the same motion area

260: movement area determiner

262: Morphological Filter

264: segmented object image generating unit

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

An embodiment of the present invention is to divide a background and an object using MPEG-4 object-based compression scheme to generate information of a binary alpha plane mask, which is an image of a divided object.

1 is an overall configuration diagram of an object segmentation apparatus of MPEG-4 using a symmetric characteristic according to the present invention. Referring to this, the object segmentation apparatus of the present invention is a preprocessor 22 that simplifies an image, and separates an object from a background. It is composed of a background and object dividing unit 24, a post-processing unit 26 for processing the boundary line processing and the divided image.

2 is a circuit diagram of a preprocessor of an object partitioning apparatus according to the present invention.

Referring to FIG. 2, in the present invention, the preprocessor 22 controls the contrast (Y) and the color (Cr, Cb) of the image 10 (CIF / QCIF YUV 4: 2: 0 format) input from the camera of the portable terminal. A high frequency filter 220 for filtering the signal, an image simplified portion 222 for generating a simplified image by detecting a boundary line, a frame buffer 224 for obtaining information (Y frame) of an input image frame, and a current image. The motion estimation analyzer 226 finds motion information between the frame and the stored previous image frame. Here, CIF is a 352 × 288 pixel size image and QCIFF is a 176 × 144 pixel size image.

The high frequency filter 220 obtains a boundary portion from luminance information (Y image) having brightness image information (Cr, Cb). An example of the high frequency filter in the present invention uses the following sobel operator.

When the color image is calculated by applying the above operator, the brightness information of the boundary part is large.

Then, the image simplified unit 222 acquires the binary plane information having the boundary information which sets the portion where the ambient value and the color information value rapidly change to 1 for all pixel values and the non-zero portion for all pixel values. .

The motion estimation analyzer 226 obtains motion information of the current image information through motion estimation between the current image frame and the previous image for the macroblock. This part adopts the same method as the motion estimator used for video compression in portable video terminal using MPEG-4.

3 is a circuit diagram illustrating a background and an object division unit in the object division apparatus according to the present invention. Referring to FIG. 3, the background and object division unit 24 of the present invention may further include an image division unit 240a and a motion detection unit ( 240b).

Here, the image dividing unit 240a uses the preprocessing unit 22 to perform line scanning and edge recording for line scanning the objects of the corresponding image and detecting edges with the simplified image information and the information of the current image frame (Y frame). The unit 242 includes a symmetry analyzer 244 that finds a symmetrical part and finds a moving area in the frame to generate object information (that is, a binary alpha plane) to be separated from the background.

The background and object dividing unit 24 of the present invention divides the background and the object by focusing on the fact that a person who is an object has a symmetrical form.

First, the line scan and edge recording unit 242 of the background and object divider 24 scans the simplified image information obtained by the preprocessor 22 in a low direction and records the current pixel position when the boundary point appears. do. Then, when the scanning of one line is finished, the symmetry analyzer 244 obtains two points having the highest similarity in the line by using the color image information at the position where the boundary point appears and the color image information at the corresponding position, and then calculates the '1' between the two pixel positions. 'Value, otherwise the value is set to' 0 'value. In this case, for comparison of similarity between two pixels, it is determined that the difference between the color image information values of the two pixels is the smallest, and it is determined that the point is symmetrical only when the distance between the two smallest points is more than a predetermined threshold value.

As such, the image divider 240a needs an iterative process required when separating an object by a conventional open-close filter method by separating an object from a background by using an object (person) having a symmetrical shape. There is no such thing as partitioning objects in a relatively simple way.

The motion detector 240b merges the motion analyzer 246 analyzing the motion information detected by the motion estimation analyzer 226 of the preprocessor 22 with the same motion region between frames to generate the same motion region. Is composed of the same motion region merging unit 248.

The motion detector 240b has the same motion and motion through the motion analyzer 246 in order to remove a symmetrical portion of the background image that may appear when the object is divided, in addition to the symmetry information of the image splitter 240a. A divided image of a part with and without a motion is obtained for the macro block. The same motion region merger 248 merges the same motion region between frames to generate the same motion region.

4 is a circuit diagram of a post-processing unit in an object partitioning apparatus according to the present invention. Referring to this, the post-processing unit 26 receives binary alpha of an object by receiving the same motion region as the background and object information of the object-dividing unit 24. The motion region determiner 260 which determines an area having the same motion in the plane, the morphological filter 262 which processes the boundary of the object, and selects only the area having the motion as the final object to display the object image to be divided ( And a divided object image generation unit 264 for generating a binary alpha plane mask).

The post-processing unit 26 includes the same motion region of the motion detector 240b among the divided object information obtained through symmetry in the background and the image divider 240a of the object divider 24 through the motion region determiner 260. In comparison, the part with no movement is classified as the background image. The symmetrical part of the background image is removed.

Next, the post-processing unit 26 processes the boundary of the divided object using the morphological filter 262 which is frequently used in the existing image processing, and then the final object image (binary alpha) through the object image generating unit 264. Plane mask).

Therefore, the object segmentation apparatus of MPEG-4 using the symmetry characteristic of the present invention having the above-described configuration receives the image information input from the camera of the portable terminal, and simplifies the input image through the preprocessor 22, and the object and the background. After detecting the symmetrical image through the divider 24 to separate the background and the object, the post-processing unit 26 generates only the moving part in the object information as the final object to process the boundary line and MPEG-4 source. Generates mask information for objects that can be used by the codec.

5A to 5C are flowcharts illustrating the object segmentation method of MPEG-4 using the symmetrical characteristics according to the present invention, respectively, in the preprocessing step, the background and the object partitioning step, and the post-processing step. With reference to the object segmentation method of the present invention will be described in detail.

Referring to FIG. 5A, the preprocessing step of the present invention is a high frequency filter for contrast (Y) and color (Cr, Cb) of the image 10 (CIF / QCIF YUV 4: 2: 0 format) input from the camera of the portable terminal. The filter is filtered through 220 and the boundary line is detected by the image simpler 222 to generate a simplified image. At the same time, while obtaining the information (Y frame) of the image frame input through the frame buffer 224, the motion estimation analyzer 226 finds the motion information between the current image frame and the stored previous image frame. . (S12 to S14)

Referring to FIG. 5B, in the background and object segmentation step according to the present invention, the line scan and edge recording unit 242 has a simplified image information and information (Y frame) of the current image frame through the preprocessor 22. Line scans the objects in the image and detects edges. That is, the line scans the objects of the image with the simplified image information and the current image frame information. When the boundary point appears during line scanning, record the boundary position of the current pixel. (S20 to S24)

When the line scanning ends (S26), the symmetry analysis of the object is performed through the symmetry analyzer 244. The symmetry analyzer 244 connects all edge points of the line-scanned object to find a part having symmetry and repeats the connection routine of the symmetry points. When the boundary position is the end of the frame, the symmetry investigation of the object is terminated, and the object area (that is, binary alpha plane) to be separated from the background is found by finding the moving area in the frame. (S26-S32)

In addition, the motion analyzer 246 of the motion detector 240b removes macro blocks having the same motion and motion through motion analysis to remove portions having symmetry among the background images that may appear when the object is divided, in addition to the motion symmetry information. Analyze (S34)

Next, the same motion region merging unit 248 of the motion detector 240b acquires a segmented image of a part with and without a part of the object. (S36)

Therefore, the background and object division scheme of the present invention can easily separate the background and the object by using the symmetrical feature of the person who is the object, considering that the use of the portable terminal is mainly a video conference.

Referring to FIG. 5C, in the post-processing step of the present invention, the motion detector 240b is used among the divided object information obtained through the symmetry of the image divider 240a of the background and object divider 24 through the motion region determiner 260. Compared to the same motion area of), the motionless part is separated into the background image, and the motion part is separated into the object area. That is, even in the case of symmetric information, only the moving region is separated into the final object (that is, the person) according to the object information without the movement. (S40 to S44)

At this time, for reference, the portion having the symmetry is removed from the background image to remove the background image having the symmetry and movement.

Next, after processing the boundary of the object using the morphological filter 262, the mask information (binary alpha plane mask) which is the final object image that can be used in the MPEG-4 source codec through the object image generating unit 264 ) (S46 to S48)

As described above, the present invention separates an object (person) and a background image by using symmetry in object segmentation of a next-generation multimedia (eg, IMT-2000) MPEG-4 portable terminal. The operation process is simplified while demanding.

Accordingly, when transmitting the object image separated from the background in the portable terminal, it is possible to deliver accurate object information, and the time and process required to separate the object information is shorter than in the related art, thereby providing a high quality video communication service.

On the other hand, the present invention is not limited to the above-described embodiment, various modifications are possible by those skilled in the art within the spirit and scope of the present invention described in the claims to be described later.

Claims (4)

In the MPEG-4 object splitter for selectively dividing any object of the image, A pre-processing unit generating a simplified image by filtering the input image, obtaining information on a currently input image frame, and searching for motion information between the current image frame and a stored previous image frame; A background for generating object information to be separated from the background by finding a symmetrical part and finding a moving region in the frame by scanning the objects of the corresponding image with the simplified image information and the information of the current image frame through the preprocessor; Object splitting unit; And MPEG-based using a symmetric feature characterized in that it comprises a post-processing unit for selecting only the moving region according to the background and the object information of the object dividing unit as the final object and generating the object image of the object to be segmented by processing the boundary of the object. 4, object divider. The apparatus of claim 1, wherein the background and object dividing unit has a simplified image information and current image frame information through the preprocessing unit, and a symmetry information extracting unit which finds a symmetrical portion by line scanning the objects of the corresponding image; An object segmentation apparatus of MPEG-4 using a symmetry feature, comprising a motion region extracting unit for generating a region having the same movement and having the same movement as object information to be divided. In the MPEG-4 object segmentation method for selectively segmenting any object of the image, Generating a simplified image by filtering the input image, obtaining information of a currently input image frame, and finding motion information between the current image frame and a stored previous image frame; Generating the object information to be separated from the background by finding a symmetrical part and finding a moving area in the frame by scanning the objects of the corresponding image with the simplified image information and the information of the current image frame; And Selecting only a region having movement according to the object information as a final object and generating an object image of a target to be segmented by processing the boundary of the object. The method of claim 1, wherein the generating of the object information comprises: scanning the objects of the corresponding image with the simplified image information and the current image frame information to find a symmetric part, and having the same movement with the motion information. An object segmentation method of MPEG-4 using a symmetric characteristic, characterized in that a region of a region is generated as object information of a segmentation target.