CN108765428A - A kind of target object extracting method based on click interaction - Google Patents
A kind of target object extracting method based on click interaction Download PDFInfo
- Publication number
- CN108765428A CN108765428A CN201810478354.5A CN201810478354A CN108765428A CN 108765428 A CN108765428 A CN 108765428A CN 201810478354 A CN201810478354 A CN 201810478354A CN 108765428 A CN108765428 A CN 108765428A
- Authority
- CN
- China
- Prior art keywords
- image
- target object
- user
- click
- interaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000003993 interaction Effects 0.000 title claims abstract description 28
- 230000011218 segmentation Effects 0.000 claims abstract description 22
- 230000002452 interceptive effect Effects 0.000 claims abstract description 21
- 238000003709 image segmentation Methods 0.000 claims abstract description 14
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims 3
- 238000010276 construction Methods 0.000 claims 1
- 238000005192 partition Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000012958 reprocessing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了一种基于点击交互的目标对象提取方法,简称为点击图像切分(Click‑cut)。为了简化交互式分割中的交互行为,使得交互式图像分割更简单自动,本发明公开了一种新的交互式图像分割方法。为了选择图像中目标区域,用户只需在所需区域输入一点,实现了用户一点交互就能分割图像前景部分的Click‑cut方法,本方法将图像分割成包含用户交互的前景区域和其余的背景区域。本方法大致可分成二部分,先自动将图像划分出个若干个超区域,再根据用户交互点使用区域选择策略确定用户所需区域。本发明具有操作简单,性能优异,提取的目标对象轮廓清楚,能满足未来对各种新增功能的需求等优点,可应用于各种场合。
The invention discloses a method for extracting a target object based on click interaction, which is referred to as click-cut for short. In order to simplify the interactive behavior in the interactive segmentation and make the interactive image segmentation simpler and automatic, the invention discloses a new interactive image segmentation method. In order to select the target area in the image, the user only needs to input a point in the desired area, and realizes the Click-cut method that can segment the foreground part of the image with a little user interaction. This method divides the image into the foreground area containing user interaction and the rest of the background area. This method can be roughly divided into two parts. First, the image is automatically divided into several super-regions, and then the region required by the user is determined according to the user interaction point using the region selection strategy. The invention has the advantages of simple operation, excellent performance, clear outline of the extracted target object, and can meet the demand for various new functions in the future, and can be applied to various occasions.
Description
技术领域technical field
本发明涉及一种图像中通过用户交互进行感兴趣区域(目标对象)提取的方法,更具体的是涉及一种用户通过输入一个点击行为进行图像中前景区域抽取的图像分割识别方法,属于图像处理技术领域。The present invention relates to a method for extracting an area of interest (target object) in an image through user interaction, and more specifically relates to an image segmentation and recognition method for extracting a foreground area in an image by a user inputting a click action, which belongs to image processing technology field.
背景技术Background technique
图像分割是将图像分成各具特性的区域并提取出感兴趣目标的技术和过程,它是由图像处理到图像分析的关键步骤。图像分割技术的研究已有几十年的历史,但至今人们并不能找到通用的方法能够适合于所有类型的图像。自动分割技术是最普遍的方式,自动分割就是开发一种方法自动让图像分割出各种轮廓,不需要用户进行干预。近年来,也提出了不少自动分割算法,然而自动分割技术得到的分割结果不够正确,用户不能根据自己的意愿抽取出目标区域,由于性能限制无法满足用户需求。Image segmentation is the technology and process of dividing the image into regions with different characteristics and extracting the target of interest. It is a key step from image processing to image analysis. The research on image segmentation technology has a history of several decades, but so far people cannot find a general method that is suitable for all types of images. Automatic segmentation technology is the most common way. Automatic segmentation is to develop a method to automatically segment images into various contours without user intervention. In recent years, many automatic segmentation algorithms have been proposed. However, the segmentation results obtained by automatic segmentation technology are not accurate enough, and users cannot extract the target area according to their own wishes. Due to performance limitations, they cannot meet user needs.
为了解决此问题,出现了有用户干涉的交互式图像分割方法,这种方式避免了机器不能很好的自动分割,交互式图像分割整合了用户交互动作到了图像分割之中,以一种有监督的分割方法,能有效地提取图像中有意义的前景区域。典型的交互手段包括用一把画刷在前景和背景处各画几笔以及在前景的周围画一个方框等。但这些交互式方法都需要输入不同的前景和背景画笔,需要用户很好地估计前景和背景区域的分布。In order to solve this problem, an interactive image segmentation method with user intervention has emerged. This method avoids the automatic segmentation of the machine. Interactive image segmentation integrates user interaction actions into image segmentation, in a supervised The segmentation method can effectively extract meaningful foreground regions in the image. Typical interaction methods include using a brush to draw a few strokes on the foreground and background, and drawing a box around the foreground. But these interactive methods all need to input different foreground and background brushes, and require users to estimate the distribution of foreground and background regions well.
由于现有的交互式分割方法都需要复杂的画笔交互信息,如专利号“201310548279.2”一种交互式图像分割方法等,或需要输入矩形框框选,如“201310587120.1”一种交互式图像分割方法等等。所有这些技术使用非常不方便,如何能进一步简化用户交互信息,如用户只需要输入一点就能得到想要提取的目标区域,目前还没有此类方法问世。Because the existing interactive segmentation methods require complex brush interaction information, such as an interactive image segmentation method in patent number "201310548279.2", or need to input a rectangular frame selection, such as an interactive image segmentation method in "201310587120.1", etc. Wait. All these technologies are very inconvenient to use. How to further simplify the user interaction information, such as the user only needs to input a little to get the target area to be extracted. There is no such method available yet.
本发明提出了一种新的交互式分割方法,为了选择图像中目标区域,用户只需在所需区域输入一点,实现了一个点击交互就能分割图像,本发明将其定义为Click-cut方法。The present invention proposes a new interactive segmentation method. In order to select the target area in the image, the user only needs to input a point in the desired area, and realizes a click interaction to segment the image. The present invention defines it as the Click-cut method .
发明内容Contents of the invention
为了解决现有交互式分割方法中交互信息较复杂的难题,本发明的主要目的在于提供一种非常简单的交互式图像分割方法,该方法仅需要用户在所需选择区域输入一点,就能将图像分割成包含用户交互信息的前景区域和其余的背景区域。In order to solve the problem of complex interactive information in the existing interactive segmentation methods, the main purpose of the present invention is to provide a very simple interactive image segmentation method, which only requires the user to input a point in the desired selection area, and the The image is segmented into a foreground region containing user interaction information and the remaining background region.
为达到上述目的,本发明的一种基于用户交互点的交互式图像分割方法的技术方案是:假设给定一副要处理的图像,首先对图像进行自动分割,然后根据用户交互进行前景抽取,实现了一点进行图像交互分割(Click-cut)。In order to achieve the above-mentioned purpose, the technical scheme of an interactive image segmentation method based on user interaction points in the present invention is as follows: assuming that an image to be processed is given, the image is first automatically segmented, and then the foreground is extracted according to user interaction. Implemented a little image interactive segmentation (Click-cut).
本发明技术实现包括2步,先进行图像初始自动划分:首先根据一副图像,可先对其进行预处理,如进行超像素分割,如何结合图像的颜色特征和纹理特征,分别抽取每个超像素点的颜色特征和纹理特征等,构建新的超像素点相似度矩阵,然后采用一种自动的社团划分方法,将图像划分成若干个社团区域,完成图像的自动分割后,到此完成了发明方法的第一步。接下来需要采用发明的区域选择策略:用户只需要在想选定的区域输入一点,区域选择策略将用户交互点所在附近区域进行合并,输出用户所需区域,将图像分割成前景和背景部分,最终达到了仅用一点这个交互行为就得到前景或者目标对象的目的。The technical implementation of the present invention includes 2 steps, first, the initial automatic division of the image: first, according to a pair of images, it can be preprocessed first, such as performing superpixel segmentation, how to combine the color features and texture features of the image to extract each superpixel respectively The color features and texture features of pixels, etc., construct a new superpixel similarity matrix, and then use an automatic community division method to divide the image into several community areas. After the automatic segmentation of the image is completed, this is completed The first step in the method of invention. Next, the inventive area selection strategy needs to be adopted: the user only needs to input a point in the area to be selected, and the area selection strategy will merge the areas near the user interaction point, output the area required by the user, and divide the image into foreground and background parts. Finally, the goal of obtaining the foreground or the target object is achieved with only a little interaction.
本发明的有益效果是:The beneficial effects of the present invention are:
本发明将用户交互式分割中的交互行为降低到最简单的行为,即用户只需要输入一点,就能很好地抽取图像的目标区域,且能得到目标对象很好地轮廓信息。基于本发明的方法能够开发Click-cut图像交互处理软件,或者在现有图像处理软件中集成Click-cut的图像交互式处理功能。The invention reduces the interaction behavior in the user interactive segmentation to the simplest behavior, that is, the user only needs to input a little, and the target area of the image can be extracted well, and the outline information of the target object can be obtained well. Based on the method of the invention, Click-cut image interactive processing software can be developed, or the image interactive processing function of Click-cut can be integrated in existing image processing software.
附图说明Description of drawings
图1是本发明的总框架图。Fig. 1 is a general frame diagram of the present invention.
图2是本发明的图像自动分割部分功能图。Fig. 2 is a functional diagram of the automatic image segmentation part of the present invention.
具体实施方式Detailed ways
以下结合附图和具体实施例对本发明进一步进行详细描述,但并不用于对本发明的限定。The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, but it is not intended to limit the present invention.
如图1和图2,本发明的具体实现:在图1中,给定原始图像(1)作为交互式分割算法的输入,该图像可为直接摄像设备采集到静态图片或从视频文件中提取出的一帧帧图像,然后进行该图像的自动分割,图像格式也不限,可为jpg或bmp等格式(2)。可将一副图像按照像素的自相似性划分成个数很少的区域,一般一副图像最终划分成十几个区域(3)。用户可在原始图像想提取的目标区域输入一点,或在经过自动分割成若干个区域的图像上输入一点(4)。然后根据区域选择策略,已得到用户一点位置所在的区域如B1,寻找B1的所有邻居区域T,如果某个邻居区域Tj与所有相邻区域的相似度最小的区域为B1,则说明Tj与B1最相似,则Tj区域也被添加前景区域中B中。这一策略为递归执行,直到集合B中的区域都被搜索过一次。最后得到B中的区域被列为前景区域,即为需提取的目标对象,剩余的区域被列为背景区域(6)。As shown in Fig. 1 and Fig. 2, concrete realization of the present invention: in Fig. 1, given original image (1) is as the input of interactive segmentation algorithm, and this image can be that direct camera equipment collects static picture or extracts from video file The frame-by-frame image is obtained, and then the image is automatically segmented, and the image format is not limited, it can be in jpg or bmp format (2). An image can be divided into a small number of regions according to the self-similarity of pixels. Generally, an image is finally divided into more than a dozen regions (3). The user can input a point on the target area to be extracted from the original image, or input a point on the image that has been automatically segmented into several areas (4). Then according to the area selection strategy, the area where the user’s point position is located, such as B1, is obtained, and all neighboring areas T of B1 are searched. If the area with the smallest similarity between a certain neighboring area Tj and all adjacent areas is B1, it means that Tj and B1 Most similarly, the Tj region is also added to B in the foreground region. This strategy is performed recursively until all regions in set B have been searched once. Finally, the region in B is classified as the foreground region, which is the target object to be extracted, and the remaining regions are classified as the background region (6).
在图2所示实施例中,原始图像预分割,这一步可采用一些现成的技术,如超像素分割(1)。一张图像中包含的像素点是非常多的,直接以像素点作为图节点进行网络划分需要较长的处理时间。因此,使用初始分割方法对图像进行过分割,如超像素分割方法。经过初始分割后,得到了一张图的很多超像素小区域,每个区域可以被当做网络的一个节点,为了使用图切的方法进行超像素区域的自动分割,需要构建每个区域的连接关系,需要使用某些特性来描述每个区域,比如颜色特征、边、纹理特征,几何特征,其中颜色特征和纹理特征能有效性表示目标的特性(2)。本发明综合利用了这两种特征,经过组合,形成了图像每个超像素区域的相关连接关系,即得到一个统一的相似度矩阵(3)。将包含数千个超像素进行再处理,也就是将包含几千个节点的网络进行划分,将图像划分成只包含几十个或者几个区域。由于不知道需要划分的个数,因此需要借助一种图的自动划分方法,如社团划分方法(4)。当进行一次社团划分后,将一副图像划分成比超像素区域数量更少的区域,由于只进行了一次社团划分,可能依然包含至少几十个超区域,重复执行网络划分过程(5),当区域个数不再变化或已达到理想的划分区域时,终止自动分割方法(6)。In the embodiment shown in Fig. 2, the original image is pre-segmented, and some ready-made techniques can be used in this step, such as superpixel segmentation (1). An image contains a lot of pixels, and it takes a long processing time to directly use pixels as graph nodes to divide the network. Therefore, the image is over-segmented using an initial segmentation method, such as a superpixel segmentation method. After the initial segmentation, many small superpixel regions of a graph are obtained. Each region can be regarded as a node of the network. In order to use the graph cut method to automatically segment the superpixel region, it is necessary to build the connection relationship of each region , some features need to be used to describe each region, such as color features, edges, texture features, and geometric features, where color features and texture features can effectively represent the characteristics of the target (2). The present invention comprehensively utilizes these two features, and forms the relevant connection relationship of each superpixel region of the image through combination, that is, obtains a unified similarity matrix (3). Reprocessing contains thousands of superpixels, that is, dividing the network containing thousands of nodes, and dividing the image into only dozens or a few regions. Since we don't know the number of graphs to be divided, we need to use an automatic graph division method, such as the community division method (4). After a community division, an image is divided into regions with fewer superpixel regions. Since only one community division is performed, it may still contain at least dozens of superregions, and the network division process (5) is repeated. When the number of regions no longer changes or the ideal division region has been reached, the automatic segmentation method (6) is terminated.
上文所列出的一系列的详细说明仅仅是针对本发明的可行性实施方式的具体说明,它们并非用以限制本发明的保护范围,凡未脱离本发明技艺精神所作的等效实施方式或变更均应包含在本发明的保护范围之内。The series of detailed descriptions listed above are only specific descriptions for feasible implementations of the present invention, and they are not intended to limit the protection scope of the present invention. Any equivalent implementation or implementation that does not depart from the technical spirit of the present invention All changes should be included within the protection scope of the present invention.
Claims (8)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2017110682674 | 2017-10-25 | ||
CN201711068267 | 2017-10-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
CN108765428A true CN108765428A (en) | 2018-11-06 |
Family
ID=64007256
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810478354.5A Pending CN108765428A (en) | 2017-10-25 | 2018-05-18 | A kind of target object extracting method based on click interaction |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN108765428A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109783732A (en) * | 2019-01-10 | 2019-05-21 | 珠海金山网络游戏科技有限公司 | A kind of micro- end construction method of game based on click density |
CN115293996A (en) * | 2022-10-08 | 2022-11-04 | 杭州群核信息技术有限公司 | Image toning method, device and storage medium |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101329763A (en) * | 2007-06-22 | 2008-12-24 | 西门子公司 | Method for segmenting structures in image data records and image processing unit |
CN103116630A (en) * | 2013-02-03 | 2013-05-22 | 苏州市职业大学 | Color image search method based on scale invariant feature transform (sift) seed region growing |
CN103914822A (en) * | 2012-12-31 | 2014-07-09 | 清华大学 | Interactive video foreground object extraction method based on super pixel segmentation |
CN105701810A (en) * | 2016-01-12 | 2016-06-22 | 湖南中航天目测控技术有限公司 | Unmanned aerial vehicle aerial image electronic delineating method based on click type image segmentation |
CN105809673A (en) * | 2016-03-03 | 2016-07-27 | 上海大学 | SURF (Speeded-Up Robust Features) algorithm and maximal similarity region merging based video foreground segmentation method |
CN106127791A (en) * | 2016-07-06 | 2016-11-16 | 武汉大学 | A kind of contour of building line drawing method of aviation remote sensing image |
-
2018
- 2018-05-18 CN CN201810478354.5A patent/CN108765428A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101329763A (en) * | 2007-06-22 | 2008-12-24 | 西门子公司 | Method for segmenting structures in image data records and image processing unit |
CN103914822A (en) * | 2012-12-31 | 2014-07-09 | 清华大学 | Interactive video foreground object extraction method based on super pixel segmentation |
CN103116630A (en) * | 2013-02-03 | 2013-05-22 | 苏州市职业大学 | Color image search method based on scale invariant feature transform (sift) seed region growing |
CN105701810A (en) * | 2016-01-12 | 2016-06-22 | 湖南中航天目测控技术有限公司 | Unmanned aerial vehicle aerial image electronic delineating method based on click type image segmentation |
CN105809673A (en) * | 2016-03-03 | 2016-07-27 | 上海大学 | SURF (Speeded-Up Robust Features) algorithm and maximal similarity region merging based video foreground segmentation method |
CN106127791A (en) * | 2016-07-06 | 2016-11-16 | 武汉大学 | A kind of contour of building line drawing method of aviation remote sensing image |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109783732A (en) * | 2019-01-10 | 2019-05-21 | 珠海金山网络游戏科技有限公司 | A kind of micro- end construction method of game based on click density |
CN109783732B (en) * | 2019-01-10 | 2023-07-04 | 珠海金山数字网络科技有限公司 | Game micro-terminal construction method based on click density |
CN115293996A (en) * | 2022-10-08 | 2022-11-04 | 杭州群核信息技术有限公司 | Image toning method, device and storage medium |
WO2024074060A1 (en) * | 2022-10-08 | 2024-04-11 | 杭州群核信息技术有限公司 | Image toning method and apparatus and storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Liew et al. | Regional interactive image segmentation networks | |
Bao et al. | Tree filtering: Efficient structure-preserving smoothing with a minimum spanning tree | |
CN111738318B (en) | A Large Image Classification Method Based on Graph Neural Network | |
CN103559719B (en) | A kind of interactive image segmentation method | |
CN103262119B (en) | For the method and system that image is split | |
US8213726B2 (en) | Image labeling using multi-scale processing | |
CN108229504B (en) | Image analysis method and device | |
CN111145209B (en) | Medical image segmentation method, device, equipment and storage medium | |
US20110216976A1 (en) | Updating Image Segmentation Following User Input | |
Chen et al. | An efficient and robust algorithm for 3D mesh segmentation | |
Peng et al. | Iterated graph cuts for image segmentation | |
JP6194692B2 (en) | Trimap generation device, Trimap generation method, and program | |
CN109409376A (en) | For the image partition method, terminal and storage medium of solid waste object | |
Ullah et al. | A foreground extraction approach using convolutional neural network with graph cut | |
Scheuermann et al. | Slimcuts: Graphcuts for high resolution images using graph reduction | |
EP3053137B1 (en) | Method and apparatus for generating superpixel clusters | |
JP6389742B2 (en) | Image segmentation method, apparatus, and program | |
CN108765428A (en) | A kind of target object extracting method based on click interaction | |
Liu et al. | Unsupervised image segmentation based on analysis of binary partition tree for salient object extraction | |
EP3018626B1 (en) | Apparatus and method for image segmentation | |
Spina et al. | Fomtrace: Interactive video segmentation by image graphs and fuzzy object models | |
JP6343998B2 (en) | Image processing apparatus, image processing method, and program | |
Hanbury | How do superpixels affect image segmentation? | |
Dimiccoli et al. | Hierarchical region-based representation for segmentation and filtering with depth in single images | |
Happ et al. | Towards distributed region growing image segmentation based on MapReduce |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181106 |
|
RJ01 | Rejection of invention patent application after publication |