CN103914822A - Interactive video foreground object extraction method based on super pixel segmentation - Google Patents
Interactive video foreground object extraction method based on super pixel segmentation Download PDFInfo
- Publication number
- CN103914822A CN103914822A CN201210592951.3A CN201210592951A CN103914822A CN 103914822 A CN103914822 A CN 103914822A CN 201210592951 A CN201210592951 A CN 201210592951A CN 103914822 A CN103914822 A CN 103914822A
- Authority
- CN
- China
- Prior art keywords
- segmentation
- super pixel
- foreground object
- video
- interactive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses an interactive video foreground object extraction method based on super pixel segmentation. The method includes an S1 step of sampling each frame of a video sequence, and then pre-segmenting based on a super pixel; an S2 step of calculating based on video object interactive extraction of the super pixel; and an S3 step of sampling the original video resolution from the segmented results by interactive calculation, and refining the edge of the segmented results. The invention can help users to quickly and efficiently extract an object in a video interactively.
Description
Technical field
The present invention relates to video editing technical field, particularly a kind of interactive video foreground object extracting method based on super pixel segmentation.
Background technology
The efficient editor of image and video is research topic important in computer graphics.Compared with image, the data volume of video is larger and structure content is complicated, thereby video editing has more challenge than picture editting.Video editing is important technological means during postproduction of movies making, advertisement are synthesized.Existing famous video editing instrument has professional Adobe Premiere, Ulead MediaStudio and free VirtualDub at present.
Cutting apart of video foreground object is an important and basic problem in video editing, obtained in the last few years research widely.Because the complexity of background in video and the motion of prospect have arbitrariness, therefore do not have a kind of method can automatically be partitioned into the foreground object in all types of videos.Interactively video foreground Object Segmentation, i.e. the given a small amount of interactive information of user, algorithm is automatically partitioned into foreground object by these information, and feeds back timely to user, and user is optimized according to feedback information again.In interactively video foreground Object Segmentation method, complicacy, algorithm response speed and the segmentation effect of user interactions is the criterion of evaluating an interactive approach.
At present existing comparatively famous interactive video dividing method has " Discontinuity-Aware Video Object Cutout " that the people such as " Video Snapcut " and Zhong Fan that the people such as Bai Xue proposed in 2009 proposed in 2012 etc." Video Snapcut " method has adopted based on manifold local classifiers, and first user carries out Interactive Segmentation to key frame, and the method can automatically propagate into subsequent frame by segmentation result in the borderline feature of prospect according to key frame.But the method generally can only have reasonable effect within the shorter time interval, when the prospect of scene more similar with background color, while there is change in topology and foreground object generation rapid movement in the border of foreground object, method based on propagating is difficult to obtain good result, therefore needs more user interactions to revise the result of every frame.
The current domestic Patents in this field has: a kind of exchange method for object video rapid extraction (application number 201110219610.7).
Summary of the invention
(1) technical matters that will solve
The technical problem to be solved in the present invention is: how to extract exactly the foreground object in video sequence by user's interactive information rapidly.
(2) technical scheme
For solving the problems of the technologies described above, the invention provides a kind of interactive video foreground object extracting method based on super pixel segmentation, comprise the following steps:
S1: each frame to video sequence carries out down-sampling, then carries out the pre-segmentation based on super pixel;
S2: the result of pre-segmentation is carried out to the interactive calculating of extracting of object video based on super pixel;
S3: interactive mode is extracted to the segmentation result calculating and be upsampled to former video resolution, and the edge of segmentation result is carried out to refinement.
Further, the described pre-segmentation based on super pixel comprises the following steps: use mean shift algorithm to carry out pre-segmentation to each frame, the pixel that color is close is polymerized to a class, forms a super pixel.
Further, the interactive calculating of extracting of the described object video based on super pixel comprises the steps: to build a three-dimensional graph model according to the stroke of the sign foreground object of user's input, then uses figure segmentation method to calculate foreground object region.
Further, after the stroke length of the sign foreground object that described user inputs exceedes a setting threshold, just the graph model of described three-dimensional is upgraded, and use figure segmentation method to recalculate foreground object region.
Further, adopt stingy drawing method to carry out refinement to the edge of described segmentation result.
(3) beneficial effect
The present invention carries out pre-segmentation to each two field picture, make the pixel that color is close be polymerized to a class, form a super pixel, simultaneously in order to accelerate computing velocity, make user can see in real time that segmentation result, the present invention have adopted two strategies to accelerate: on the one hand by first video sequence being carried out to down-sampling, then computed segmentation result under low resolution, and then be upsampled under high resolving power segmentation result is carried out to edge thinning, thereby greatly reduce calculated amount.
Brief description of the drawings
Fig. 1 is a kind of interactive video foreground object extracting method process flow diagram based on super pixel segmentation of the embodiment of the present invention;
Fig. 2 carries out the result after pre-segmentation based on super pixel in step S1;
Fig. 3 is the prospect result calculating according to the interactive information of user's input in step S2, and wherein left figure is with representing foreground object in wire frame, and right figure represents foreground object with white portion;
Fig. 4 is that step S3 carries out the segmentation result after refinement to segmenting edge, and wherein left figure is with representing foreground object in wire frame, and right figure represents foreground object with white portion.
Embodiment
Below in conjunction with drawings and Examples, the specific embodiment of the present invention is described in further detail.Following examples are used for illustrating the present invention, but are not used for limiting the scope of the invention.
As shown in Figure 1, be the processing flow chart of an embodiment of the interactive video foreground object extracting method based on super pixel segmentation, comprising:
Step S1, carries out down-sampling to each frame of video sequence.Down-sampling carries out level and smooth rear interlacing every row ground capture element to each frame, image length and wide 1/2nd of the source images that is after down-sampling by gaussian kernel each time.Carry out (n=1 or 2, determines according to the resolution of video, and the higher down-sampling number of times that carries out of resolution is more) after n down-sampling, each frame is carried out to pre-segmentation with average drifting (MeanShift) algorithm, Fig. 2 is out four two field pictures of pre-segmentation.
Step S2, carries out the interactive calculating of extracting of object video based on super pixel to the image of pre-segmentation.User carries out prospect mark in foreground object region entering stroke, builds the three-dimensional plot model of a video data according to the stroke of the sign foreground object of user's input, and then use figure (graphcut) method of cutting is calculated foreground object region.Method of the present invention can by solve fructufy time offer user, if user is dissatisfied to segmentation result, can be by the correct and wrong foreground area of simple stroke mark.So that improve counting yield, after stroke length exceedes certain threshold value, three-dimensional plot model is upgraded, and use figure segmentation method to recalculate the foreground area of rebuild three-dimensional plot model, segmentation result is returned to user.Fig. 3 is the 1st frame of a video sequence and the interactive mode of the 20th frame is extracted the result of calculating.The figure on the left side uses wire frame representation foreground object, white region representation foreground object for the figure on the right.
Step S3, the segmentation result that interactive computing is gone out is upsampled to former video resolution, and the edge of segmentation result is carried out to refinement.Adopt the method for scratching more accurately figure (matting) to carry out refinement for the edge of segmentation result, make final segmentation result more accurate.Fig. 4 has chosen the segmentation result of two frames identical with Fig. 3.Can find out from the white portion on the right, the segmentation result carrying out after edge thinning is obviously better than the result in Fig. 3, and the details at the edges such as hair has all been cut apart out well.
Above embodiment is only for illustrating the present invention; and be not limitation of the present invention; the those of ordinary skill in relevant technologies field; without departing from the spirit and scope of the present invention; can also make a variety of changes and modification; therefore all technical schemes that are equal to also belong to category of the present invention, and scope of patent protection of the present invention should be defined by the claims.
Claims (5)
1. the interactive video foreground object extracting method based on super pixel segmentation, is characterized in that, comprises the following steps:
S1: each frame to video sequence carries out down-sampling, then carries out the pre-segmentation based on super pixel;
S2: the result of pre-segmentation is carried out to the interactive calculating of extracting of object video based on super pixel;
S3: interactive mode is extracted to the segmentation result calculating and be upsampled to former video resolution, and the edge of segmentation result is carried out to refinement.
2. the interactive video foreground object extracting method based on super pixel segmentation as claimed in claim 1, it is characterized in that, the described pre-segmentation based on super pixel comprises the following steps: use mean shift algorithm to carry out pre-segmentation to each frame, the pixel that color is close is polymerized to a class, forms a super pixel.
3. the interactive video foreground object extracting method based on super pixel segmentation as claimed in claim 1, it is characterized in that, the interactive calculating of extracting of the described object video based on super pixel comprises the steps: to build a three-dimensional graph model according to the stroke of the sign foreground object of user's input, then uses figure segmentation method to calculate foreground object region.
4. the interactive video foreground object extracting method based on super pixel segmentation as claimed in claim 3, it is characterized in that, after the stroke length of the sign foreground object that described user inputs exceedes a setting threshold, just the graph model of described three-dimensional is upgraded, and use figure segmentation method to recalculate foreground object region.
5. the interactive video foreground object extracting method based on super pixel segmentation as claimed in claim 1, is characterized in that, adopts stingy drawing method to carry out refinement to the edge of described segmentation result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210592951.3A CN103914822A (en) | 2012-12-31 | 2012-12-31 | Interactive video foreground object extraction method based on super pixel segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210592951.3A CN103914822A (en) | 2012-12-31 | 2012-12-31 | Interactive video foreground object extraction method based on super pixel segmentation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103914822A true CN103914822A (en) | 2014-07-09 |
Family
ID=51040479
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210592951.3A Pending CN103914822A (en) | 2012-12-31 | 2012-12-31 | Interactive video foreground object extraction method based on super pixel segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103914822A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657976A (en) * | 2013-11-19 | 2015-05-27 | 汤姆逊许可公司 | Method and apparatus for generating superpixels |
| CN107481261A (en) * | 2017-07-31 | 2017-12-15 | 中国科学院长春光学精密机械与物理研究所 | A kind of color video based on the tracking of depth prospect scratches drawing method |
| CN108765428A (en) * | 2017-10-25 | 2018-11-06 | 江苏大学 | A kind of target object extracting method based on click interaction |
| CN108965739A (en) * | 2018-06-22 | 2018-12-07 | 北京华捷艾米科技有限公司 | video keying method and machine readable storage medium |
| CN111199547A (en) * | 2018-11-20 | 2020-05-26 | Tcl集团股份有限公司 | Image segmentation method and device and terminal equipment |
| CN114820652A (en) * | 2022-04-07 | 2022-07-29 | 北京医准智能科技有限公司 | Method, device and medium for segmenting local quality abnormal region of mammary X-ray image |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102289847A (en) * | 2011-08-02 | 2011-12-21 | 浙江大学 | Interaction method for quickly extracting video object |
| CN102707864A (en) * | 2011-03-28 | 2012-10-03 | 日电（中国）有限公司 | Object segmentation method and system based on mixed marks |
-
2012
- 2012-12-31 CN CN201210592951.3A patent/CN103914822A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102707864A (en) * | 2011-03-28 | 2012-10-03 | 日电（中国）有限公司 | Object segmentation method and system based on mixed marks |
| CN102289847A (en) * | 2011-08-02 | 2011-12-21 | 浙江大学 | Interaction method for quickly extracting video object |
Non-Patent Citations (5)
| Title |
|---|
| DINGDING LIU ET AL: "Fast Interactive Image Segmentation by Discriminative Clustering", 《MCMC’10》 * |
| 刘嘉 等: "一种基于图割的交互式图像分割方法", 《电子与信息学报》 * |
| 眭博聪 等: "基于三维体的多分辨率视频分割算法", 《计算机工程》 * |
| 蒋建国 等: "层次式图切分快速分割算法", 《图学学报》 * |
| 韩守东 等: "基于高斯超像素的快速Graph Cuts图像分割方法", 《自动化学报》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657976A (en) * | 2013-11-19 | 2015-05-27 | 汤姆逊许可公司 | Method and apparatus for generating superpixels |
| CN107481261A (en) * | 2017-07-31 | 2017-12-15 | 中国科学院长春光学精密机械与物理研究所 | A kind of color video based on the tracking of depth prospect scratches drawing method |
| CN107481261B (en) * | 2017-07-31 | 2020-06-16 | 中国科学院长春光学精密机械与物理研究所 | Color video matting method based on depth foreground tracking |
| CN108765428A (en) * | 2017-10-25 | 2018-11-06 | 江苏大学 | A kind of target object extracting method based on click interaction |
| CN108965739A (en) * | 2018-06-22 | 2018-12-07 | 北京华捷艾米科技有限公司 | video keying method and machine readable storage medium |
| CN111199547A (en) * | 2018-11-20 | 2020-05-26 | Tcl集团股份有限公司 | Image segmentation method and device and terminal equipment |
| CN111199547B (en) * | 2018-11-20 | 2024-01-23 | Tcl科技集团股份有限公司 | Image segmentation method and device and terminal equipment |
| CN114820652A (en) * | 2022-04-07 | 2022-07-29 | 北京医准智能科技有限公司 | Method, device and medium for segmenting local quality abnormal region of mammary X-ray image |
| CN114820652B (en) * | 2022-04-07 | 2023-05-23 | 北京医准智能科技有限公司 | Method, device and medium for segmenting partial quality abnormal region of mammary gland X-ray image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11017586B2 (en) | 3D motion effect from a 2D image | |
| CN103914822A (en) | Interactive video foreground object extraction method based on super pixel segmentation | |
| Wang et al. | Video tooning | |
| CN102388391B (en) | Video matting based on foreground-background constraint propagation | |
| CN110827312B (en) | Learning method based on cooperative visual attention neural network | |
| CN108388882A (en) | Based on the gesture identification method that the overall situation-part is multi-modal RGB-D | |
| CN110276354A (en) | A kind of training of high-resolution Streetscape picture semantic segmentation and real time method for segmenting | |
| CN108509917A (en) | Video scene dividing method and device based on shot cluster correlation analysis | |
| Dong et al. | Summarization-based image resizing by intelligent object carving | |
| Kiess et al. | A survey on content-aware image and video retargeting | |
| Hu et al. | Hybrid shift map for video retargeting | |
| CN100487732C (en) | Method for generating cartoon portrait based on photo of human face | |
| CN111414938B (en) | Target detection method for bubbles in plate heat exchanger | |
| WO2013144418A1 (en) | Image segmentation | |
| Wang et al. | Semantic segmentation of high-resolution images | |
| CN104091336A (en) | Stereoscopic image synchronous segmentation method based on dense disparity map | |
| CN107909602A (en) | A kind of moving boundaries method of estimation based on deep learning | |
| CN115761574A (en) | Weak surveillance video target segmentation method and device based on frame labeling | |
| CN111354067A (en) | Multi-model same-screen rendering method based on Unity3D engine | |
| RU2710659C1 (en) | Simultaneous uncontrolled segmentation of objects and drawing | |
| CN101276476A (en) | Process for the separating prospect background of 2D cartoon animation | |
| CN114549703B (en) | Quick-action image generation method, system, device and storage medium | |
| Shi et al. | Context saliency based image summarization | |
| CN110853040B (en) | Image collaborative segmentation method based on super-resolution reconstruction | |
| CN113139539B (en) | Method and device for detecting characters of arbitrary-shaped scene with asymptotic regression boundary |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140709 |