CN111311604A - Method and apparatus for segmenting an image - Google Patents
Method and apparatus for segmenting an image Download PDFInfo
- Publication number
- CN111311604A CN111311604A CN201811519372.XA CN201811519372A CN111311604A CN 111311604 A CN111311604 A CN 111311604A CN 201811519372 A CN201811519372 A CN 201811519372A CN 111311604 A CN111311604 A CN 111311604A
- Authority
- CN
- China
- Prior art keywords
- image
- pixel
- segmented
- super
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
Abstract
The embodiment of the application discloses a method and a device for segmenting an image. One embodiment of the method comprises: performing superpixel segmentation on an image to be segmented to generate at least two superpixel blocks; setting a first identification area in a foreground area of an image to be segmented, and setting a second identification area in a background area of the image to be segmented; extracting image characteristics of at least two super-pixel blocks to obtain first image characteristics of the super-pixel blocks in the first identification region and second image characteristics of the super-pixel blocks in the second identification region; determining the similarity between the pixels in the image to be segmented and the regions between the first identification region and the second identification respectively based on the first image characteristic and the second image characteristic; based on the determined similarity, the image to be segmented is segmented into foreground regions and background regions. This embodiment improves the accuracy of image segmentation.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a method and a device for segmenting an image.
Background
Image segmentation is a technique and process that divides an image into several specific regions with unique properties and proposes an object of interest. It is a key step from image processing to image analysis, and the image cannot be correctly analyzed without correct segmentation.
In the prior art, an image segmentation technology such as an image automatic segmentation method or an image segmentation method requiring manual interaction is generally adopted to segment an image. The existing image segmentation technology generally performs foreground and background segmentation on an image according to the difference of pixel values such as brightness, color and the like of pixels in the image, and the obtained segmentation result often has a lot of noises, for example, many holes appear after segmentation, and the pixels of the background and the foreground pixels are difficult to distinguish due to large similarity, so that the image segmentation result is inaccurate.
Disclosure of Invention
The embodiment of the application provides a method and a device for segmenting an image.
In a first aspect, an embodiment of the present application provides a method for segmenting an image, where the method includes: performing superpixel segmentation on an image to be segmented to generate at least two superpixel blocks; setting a first identification area in a foreground area of an image to be segmented, and setting a second identification area in a background area of the image to be segmented; extracting image characteristics of at least two super-pixel blocks to obtain first image characteristics of the super-pixel blocks in the first identification region and second image characteristics of the super-pixel blocks in the second identification region; determining the similarity between pixels in the image to be segmented and the first identification region and the second identification region respectively based on the first image feature and the second image feature; based on the determined similarity, the image to be segmented is segmented into foreground regions and background regions.
In some embodiments, segmenting the image to be segmented into foreground and background regions based on the determined similarity comprises: constructing an energy function based on graph cut theory; and determining the minimum energy value of the energy function based on the determined similarity, and determining the pixel in the image to be segmented as a foreground pixel or a background pixel.
In some embodiments, performing superpixel segmentation on an image to be segmented to obtain at least two superpixel blocks comprises: and performing superpixel segmentation on the image to be segmented by adopting a simple linear iterative clustering algorithm to generate at least two superpixel blocks.
In some embodiments, the energy function includes a data term and a smoothing term; and determining a minimum energy value of the energy function based on the determined similarity, determining a pixel in the image to be segmented as a foreground pixel or a background pixel, comprising: determining a data item based on the similarity between the image characteristics of the super-pixel block where the pixels of the image to be segmented are located and the first image characteristics and the second image characteristics respectively; determining a smoothing term based on image characteristics of pixels of an image to be segmented; a minimum energy value of an energy function is determined based on the determined data item and the smoothing item to determine a pixel in the image to be segmented as a foreground pixel or a background pixel.
In some embodiments, the image features of the superpixel blocks comprise at least one of the following features of the superpixel blocks: mean of pixel values, variance of pixel values, histogram of pixel values, boundary information, texture information.
In a second aspect, an embodiment of the present application provides an apparatus for segmenting an image, the apparatus including: the super pixel segmentation unit is configured to perform super pixel segmentation on an image to be segmented to generate at least two super pixel blocks; an identification region setting unit configured to set a first identification region in a foreground region of an image to be segmented and a second identification region in a background region of the image to be segmented; the image feature extraction unit is configured to extract image features of at least two super-pixel blocks to obtain first image features of the super-pixel blocks in the first identification region and second image features of the super-pixel blocks in the second identification region; the determining unit is configured to determine the similarity between pixels in the image to be segmented and the first identification region and the second identification region respectively based on the first image feature and the second image feature; a foreground region dividing unit configured to divide the image to be divided into a foreground region and a background region based on the determined similarity.
In some embodiments, the segmentation unit comprises: an energy function construction module configured to construct an energy function based on graph cut theory; a pixel determination module configured to determine a minimum energy value of the energy function based on the determined similarity, determine a pixel in the image to be segmented as a foreground pixel or a background pixel.
In some embodiments, the super-pixel segmentation unit is further configured to: and performing superpixel segmentation on the image to be segmented by adopting a simple linear iterative clustering algorithm to generate at least two superpixel blocks.
In some embodiments, the energy function includes a data term and a smoothing term; and the pixel determination module is further configured to: determining a data item based on the similarity between the image characteristics of the super-pixel block where the pixels of the image to be segmented are located and the first image characteristics and the second image characteristics respectively; determining a smoothing term based on image characteristics of pixels of an image to be segmented; a minimum energy value of an energy function is determined based on the determined data item and the smoothing item to determine a pixel in the image to be segmented as a foreground pixel or a background pixel.
In some embodiments, the image features of the superpixel blocks comprise at least one of the following features of the superpixel blocks: mean of pixel values, variance of pixel values, histogram of pixel values, boundary information, texture information.
The method and the device for segmenting the image provided by the embodiment of the application firstly carry out superpixel segmentation on the image to be segmented to generate at least two superpixel blocks, then a first identification region and a second identification region are respectively arranged in a foreground region and a background region of the image to be segmented, then image feature extraction is carried out on the at least two superpixel blocks to obtain a first image feature of the superpixel block in the first identification region and a second image feature of the superpixel block in the second identification region, then the similarity between pixels in the image to be segmented and the first identification region and the second identification region is determined based on the first image feature and the second image feature, and finally the image to be segmented is segmented into the foreground region and the background region based on the determined similarity. Therefore, semi-automatic segmentation of manual interaction on the image by using the super pixel blocks is realized, and the accuracy of image segmentation is improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which one embodiment of the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a method for segmenting an image according to the present application;
FIG. 3 is a flow diagram of one implementation of segmenting an image to be segmented into foreground and background regions based on determined similarities, according to the present embodiment;
FIG. 4 is a schematic illustration of an application scenario of a method for segmenting an image according to the present application;
FIG. 5 is a schematic block diagram of one embodiment of an apparatus for segmenting images according to the present application;
FIG. 6 is a schematic block diagram of a computer system suitable for use in implementing an electronic device according to embodiments of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows an exemplary system architecture 100 to which embodiments of the present method for segmenting images or apparatus for segmenting images may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications, such as image viewing software, image processing software, web browsers, search-type applications, instant messaging tools, mailbox clients, social platform software, and the like.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a display screen and supporting image processing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 players (Moving Picture Experts Group Audio Layer IV, mpeg compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The server 105 may be a server that provides various services, such as a background server that performs segmentation processing on images transmitted on the terminal devices 101, 102, 103. The background server may perform processing such as identifier setting and feature extraction on the received image, and feed back a processing result (for example, a segmented image to be segmented) to the terminal device.
It should be noted that the method for segmenting the image provided by the embodiment of the present application is generally performed by the server 105, and accordingly, the method for segmenting the image is generally provided in the server 105.
The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein. It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
It should be further noted that the terminal devices 101, 102, and 103 may also be installed with an image processing application, and the terminal devices 101, 102, and 103 may also perform foreground and background segmentation processing on the image to be segmented based on the image processing application, in this case, the method for segmenting the image may also be executed by the terminal devices 101, 102, and 103, and accordingly, the apparatus for segmenting the image may also be installed in the terminal devices 101, 102, and 103. At this point, the exemplary system architecture 100 may not have the server 105 and the network 104.
With continued reference to FIG. 2, a flow 200 of one embodiment of a method for segmenting an image according to the present application is shown. The method for segmenting an image comprises the following steps:
In this embodiment, an execution subject (for example, a server shown in fig. 1) of the method for segmenting an image may perform a superpixel segmentation process on the image to be segmented, so as to segment the image to be segmented into at least two superpixel blocks. Here, a super-pixel block generally refers to an irregular pixel block having a certain visual meaning, which is composed of adjacent pixels having similar characteristics of texture, color, brightness, and the like.
In general, the execution subject may perform superpixel segmentation on the image to be segmented by using a superpixel segmentation technique. The super-pixel segmentation technique is to group pixels by using the similarity of features between pixels, and to express image features by replacing a large number of pixels with a small number of super-pixel blocks. Therefore, the superpixel segmentation technique can greatly reduce the complexity of image processing. Further, superpixel block segmentation can avoid the occurrence of pixel holes and noise, as compared with pixel-level segmentation of an image.
In some optional implementation manners of this embodiment, among various superpixel segmentation algorithms, an SLIC algorithm (simple iterative clustering algorithm) has the advantages of low memory occupation, high speed, few parameters, high accuracy of extracted boundary information, and the like, so that the execution main body may perform superpixel segmentation on an image to be segmented by using a simple linear iterative clustering algorithm, so as to segment the image to be segmented to generate at least two superpixel blocks. It is understood that the executing subject may also perform superpixel segmentation on the image to be segmented by using other methods, which are not limited herein.
In this embodiment, the image to be segmented may include a foreground region and a background region, and the executing entity (e.g., the server shown in fig. 1) may set the first identification region in the foreground region of the image to be segmented and the second identification region in the background region of the image to be segmented in response to a user operation. Here, the execution body may set a first flag region in the foreground region using a first flag, and set a second flag in the background region using a second flag, wherein the first flag for setting the first flag region and the second flag for setting the second flag region are different flags. As an example, the executing body may set red and white lines in a foreground region and a background region of the image to be segmented, respectively, in response to a user operation, where regions where the red and white lines are located are the first identification region and the second identification region. It is understood that the image to be segmented may include a plurality of independent foreground regions and/or background regions, and thus the image to be segmented may include a plurality of first identification regions and/or second identification regions.
In this embodiment, based on the super pixel blocks of the image to be segmented generated in step 201, the execution subject may perform image feature extraction on each super pixel block, so that the image features of each super pixel block may be determined. The image characteristics of the super-pixel blocks in the first identification area of the image to be segmented are first image characteristics, and the image characteristics of the super-pixel blocks in the second identification area of the image to be segmented are second image characteristics.
In some optional implementations of this embodiment, the execution subject may extract at least one of the following feature values from each pixel block: mean of pixels within a superpixel block, variance of pixel values within a superpixel block, histogram of pixel values within a superpixel block, boundary information (e.g., gradient magnitude, gaussian difference) of pixels within a superpixel block, texture information (e.g., eigenvalues of a hessian matrix) of pixels within a superpixel block.
And 204, determining the similarity between the pixels in the image to be segmented and the first identification region and the second identification region respectively based on the first image characteristic and the second image characteristic.
In this embodiment, based on the first image feature and the second image feature obtained in step 203, the executing entity may calculate the similarity between the image feature of each super pixel in the image to be segmented and the first image feature and the second image feature, so as to further determine the similarity between the pixel in each super pixel block in the image to be segmented and the first identification region and the second identification region, respectively. As an example, the executing entity may calculate a similarity between an image feature of any super pixel in the image to be segmented and the first image feature, and determine that the pixel in the super pixel block is similar to the first identification region corresponding to the first image feature when determining that the similarity is greater than a preset threshold.
In this embodiment, after determining the similarity between the pixels in each pixel block in the image to be segmented and the first identification region and the second identification region, the execution subject may divide the pixels in the super pixel block in the image to be segmented into foreground pixels and background pixels by various means. As an example, the executing subject may aggregate pixels in a super-pixel block in the image to be segmented into foreground pixels and background pixels by a clustering algorithm or the like. Then, the executing body can segment the foreground pixels and the background pixels, so as to segment the image to be segmented into a foreground region where the foreground pixels are located and a background region where the background pixels are located.
In some optional schemes, as shown in fig. 3, the step 205 of "segmenting the image to be segmented into the foreground region and the background region based on the determined similarity" may also be implemented by the following steps:
In this implementation, the execution subject may construct an energy function (which may also be referred to as a loss function) based on graph cut theory (graph cut), and the energy function may associate a graph cut problem with a graph minimal cut problem, thereby realizing segmentation of an image. First, an image to be segmented is represented by a directed graph g ═ ρ, ∈ where ρ is a pixel of the image to be segmented and ∈ is a boundary of the image to be segmented. The image to be segmented may contain a system of neighboring pixels Ν containing all unordered pairs of neighboring pixels { p, q } in ρ of the image to be segmented. Here, the executing body may make foreground or background labeling for each pixel p ∈ ρ through an energy function constructed by minimizing similarity between a pixel in the image to be segmented and the first identification region and the second identification region, respectively, so as to label each pixel in the image to be segmented as a background pixel or a foreground pixel. As an example, the background pixel of the image to be segmented may be labeled as 0 and the foreground pixel may be labeled as 1, i.e. fpE {0,1}, where fpIs the label of the pixel p in the image to be segmented.
Optionally, the energy function may include a data term and a smoothing term. Wherein the data item can be used for evaluating the label f of each pixel p in the image to be segmentedpThe smoothing term may be used to assign a cost to the corresponding pixelMarking the label f of each disordered adjacent pixel pair { p, q } in the image to be segmented in the evaluationpAnd fqLabeled as penalty costs of different values. The "determining the minimum energy value of the energy function based on the determined similarity, and determining the pixel in the image to be segmented as the foreground pixel or the background pixel" of the above step 302 can be implemented by the following steps: determining a data item based on the similarity between the image characteristics of the super-pixel block where the pixels of the image to be segmented are located and the first image characteristics and the second image characteristics respectively; determining a smoothing term based on image characteristics of pixels of an image to be segmented; a minimum energy value of an energy function is determined based on the determined data item and the smoothing item to determine a pixel in the image to be segmented as a foreground pixel or a background pixel.
Specifically, the energy function configured by the execution main body may be:
Here, the smoothing term of the energy function may be determined by image features of pixels in a super-pixel block in the image to be segmented. Specifically, the above E can be calculated using the following formulasmooth(fp,fq) The smoothing term described above is obtained:
In the prior art, for each pixel p in the image to be segmented, the execution subject may calculate the E for each pixel pdata(fp) Thereby obtaining the value of the data item. Wherein, E is as defined abovedata(fp) Can be calculated by the following formula:
Edata(fp)＝-lgPr(p|fp) Wherein, Pr (p | f)p) Denoted f, which may represent a pixel ppThe probability of (c).
Therefore, the existing image segmentation technology needs to calculate the data item of each pixel when the image to be segmented is segmented, and if the image to be segmented is an image with the number of millions of pixels, a large amount of time is wasted when the image to be segmented is segmented.
However, according to the image segmentation method provided by the foregoing implementation manner of this embodiment, the data items in the energy function may be determined by using the similarity between the image feature of the super-pixel block where the pixel is located and the first image feature and the second image feature, so that it is clear that the data items corresponding to the pixels in the image to be segmented do not need to be calculated one by one, thereby reducing the amount of calculation for image segmentation and increasing the speed of image segmentation. In particular, E in the data item in the energy function described abovedata(fp) The following formula can be used for calculation:
Edata(fp)＝1-Pr(p|fp) Wherein, Pr (p | f)p) Denoted f, which may represent a super-pixel block in which the pixel p is locatedpThe probability of (c).
Optionally, the number of pixels included in the super pixel block generated in the image to be segmented may affect the value of the image characteristic of the super pixel block, so as to further affect the accuracy of the segmentation result of the image to be segmented. Therefore, in order to further improve the accuracy of image segmentation, multi-scale superpixel blocks can be used for image segmentation, i.e. the segmented superpixel blocks can be packagedComprising a plurality of numbers of pixels, wherein a plurality of scales can be set
M is the number of pixels included in the preset superpixel block, H ∈ {1, 2,3 … … H }, and H can be set according to an empirical value. When H is 1,2,3 … … H, the multi-scale superpixel block can be set, and at this time,
wherein the content of the first and second substances,
can represent a superpixel block in which the pixel p is located and which has a scale of
Can represent superpixel blocks
Is marked by fspThe probability of (c).
Representing superpixel blocks corresponding to values of 1,2,3 … … H for H
Is marked by fspThe probabilities of (a) are summed. f. ofspMay be a superpixel block
Of (a), where fspMay be used to indicate that each pixel in the super pixel block is a foreground pixel or a background pixel, and as an example, the background super pixel block of the image to be segmented may be marked as 0, and the foreground super pixel block may be marked as 1, that is, fsp∈{0,1}。
Optionally, the probability
The probability value obtained by learning of a support vector machine can be adopted, and the positive and negative samples used in the learning process can be a foreground region and a corresponding first image feature and a background region and a corresponding second image feature in the image to be segmented. Of course, the present implementation manner can also adopt other manners to calculate the above probability
There is no unique limitation here.
With continued reference to fig. 4, fig. 4 is a schematic diagram of an application scenario of the method for segmenting an image according to the present embodiment. 4a-4d, for the image to be segmented, as shown in fig. 4a, the server may perform superpixel segmentation on the image to be segmented to generate a plurality of superpixel blocks, as shown in fig. 4b, where the mesh information in the graph is an edge of the generated superpixel block; then, the server may set a first identification region in the foreground region of the image to be segmented and a second identification region in the background region of the image to be segmented, as shown in fig. 4c, specifically, the server may draw a red line and a white line in the foreground region and the background region of the image to be segmented, respectively, in response to a user operation, where the red line is the first identification region and the white line is the second identification region (where the red line and the white line are shown as gray and white lines in fig. 4 c); then, the server can extract image characteristics of at least two superpixel blocks to obtain first image characteristics of the superpixel blocks in the first identification region and second image characteristics of the superpixel blocks in the second identification region; then, the server determines the similarity between the pixels in the image to be segmented and the first identification area and the second identification area respectively based on the first image characteristic and the second image characteristic; finally, the server may segment the image to be segmented into foreground and background regions based on the determined similarity, as shown in fig. 4 d.
The method for segmenting the image provided by the above embodiment of the present application, firstly, super-pixel segmentation is performed on the image to be segmented to generate at least two super-pixel blocks, then, a first identification region and a second identification region are respectively arranged in a foreground region and a background region of the image to be segmented, then extracting image characteristics of at least two super-pixel blocks to obtain first image characteristics of the super-pixel blocks in the first identification region and second image characteristics of the super-pixel blocks in the second identification region, then determining the similarity between the pixels in the image to be segmented and the first identification region and the second identification region respectively based on the first image characteristic and the second image characteristic, finally segmenting the image to be segmented into a foreground region and a background region based on the determined similarity, therefore, semi-automatic segmentation of manual interaction on the image by using the super pixel blocks is realized, and the accuracy of image segmentation is improved.
With further reference to fig. 5, as an implementation of the methods shown in the above-mentioned figures, the present application provides an embodiment of an apparatus for segmenting an image, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable in various electronic devices.
As shown in fig. 5, the apparatus 500 for segmenting an image of the present embodiment includes: a superpixel segmentation unit 501, an identification region setting unit 502, an image feature extraction unit 503, a determination unit 504, and a foreground region segmentation unit 505. The super-pixel segmentation unit 501 is configured to perform super-pixel segmentation on an image to be segmented to generate at least two super-pixel blocks; the identification region setting unit 502 is configured to set a first identification region in a foreground region of an image to be divided, and a second identification region in a background region of the image to be divided; the image feature extraction unit 503 is configured to perform image feature extraction on at least two super-pixel blocks to obtain a first image feature of the super-pixel block in the first identification region and a second image feature of the super-pixel block in the second identification region; the determining unit 504 is configured to determine similarities between pixels in the image to be segmented and the first identification region and the second identification region, respectively, based on the first image feature and the second image feature; the foreground region dividing unit 505 is configured to divide the image to be divided into a foreground region and a background region based on the determined similarity.
In some optional implementations of the present embodiment, the foreground area dividing unit 505 includes: an energy function construction module configured to construct an energy function based on graph cut theory; a pixel determination module configured to determine a minimum energy value of the energy function based on the determined similarity, determine a pixel in the image to be segmented as a foreground pixel or a background pixel.
In some optional implementations of this embodiment, the super-pixel segmentation unit 501 is further configured to: and performing superpixel segmentation on the image to be segmented by adopting a simple linear iterative clustering algorithm to generate at least two superpixel blocks.
In some optional implementations of the present embodiment, the energy function includes a data term and a smoothing term; and the pixel determination module is further configured to: determining a data item based on the similarity between the image characteristics of the super-pixel block where the pixels of the image to be segmented are located and the first image characteristics and the second image characteristics respectively; determining a smoothing term based on image characteristics of pixels of an image to be segmented; a minimum energy value of an energy function is determined based on the determined data item and the smoothing item to determine a pixel in the image to be segmented as a foreground pixel or a background pixel.
In some optional implementations of this embodiment, the image characteristic of the super-pixel block comprises at least one of the following characteristics of the super-pixel block: mean of pixel values, variance of pixel values, histogram of pixel values, boundary information, texture information.
The units recited in the apparatus 500 correspond to the various steps in the method described with reference to fig. 2. Thus, the operations and features described above for the method are equally applicable to the apparatus 500 and the units included therein, and are not described in detail here.
Referring now to FIG. 6, shown is a block diagram of a computer system 600 suitable for use in implementing the electronic device of an embodiment of the present application. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the method of the present application when executed by a Central Processing Unit (CPU) 601. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present application may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a superpixel segmentation unit, an identification region setting unit, an image feature extraction unit, a determination unit, and a foreground region segmentation unit. The names of these units do not in some cases constitute a limitation on the unit itself, for example, a super-pixel segmentation unit may also be described as a "unit for super-pixel segmentation of an image to be segmented, generating at least two super-pixel blocks".
As another aspect, the present application also provides a computer-readable medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The computer readable medium carries one or more programs which, when executed by the apparatus, cause the apparatus to: performing superpixel segmentation on an image to be segmented to generate at least two superpixel blocks; setting a first identification area in a foreground area of an image to be segmented, and setting a second identification area in a background area of the image to be segmented; extracting image characteristics of at least two super-pixel blocks to obtain first image characteristics of the super-pixel blocks in the first identification region and second image characteristics of the super-pixel blocks in the second identification region; determining the similarity between pixels in the image to be segmented and the first identification region and the second identification region respectively based on the first image feature and the second image feature; based on the determined similarity, the image to be segmented is segmented into foreground regions and background regions.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (12)
1. A method for segmenting an image, comprising:
performing superpixel segmentation on an image to be segmented to generate at least two superpixel blocks;
setting a first identification region in a foreground region of the image to be segmented, and setting a second identification region in a background region of the image to be segmented;
extracting image characteristics of the at least two super-pixel blocks to obtain first image characteristics of the super-pixel blocks in the first identification region and second image characteristics of the super-pixel blocks in the second identification region;
determining the similarity between the pixels in the image to be segmented and the first identification region and the second identification region respectively based on the first image feature and the second image feature;
based on the determined similarity, the image to be segmented is segmented into a foreground region and a background region.
2. The method of claim 1, wherein the segmenting the image to be segmented into foreground and background regions based on the determined similarity comprises:
constructing an energy function based on graph cut theory;
determining a minimum energy value of the energy function based on the determined similarity, determining a pixel in the image to be segmented as a foreground pixel or a background pixel.
3. The method of claim 1, wherein said superpixel segmenting the image to be segmented resulting in at least two superpixel blocks, comprises:
and performing superpixel segmentation on the image to be segmented by adopting a simple linear iterative clustering algorithm to generate at least two superpixel blocks.
4. The method of claim 2, wherein the energy function comprises a data term and a smoothing term; and
the determining a minimum energy value of the energy function based on the determined similarity, determining a pixel in the image to be segmented as a foreground pixel or a background pixel, comprising:
determining the data items based on the similarity between the image characteristics of the super-pixel block where the pixels of the image to be segmented are located and the first image characteristics and the second image characteristics respectively;
determining the smoothing term based on image characteristics of pixels of the image to be segmented;
determining a minimum energy value of the energy function based on the determined data item and a smoothing item to determine a pixel in the image to be segmented as a foreground pixel or a background pixel.
5. The method according to one of claims 1 to 4, wherein the image features of the superpixel blocks comprise at least one of the following features of the superpixel blocks: mean of pixel values, variance of pixel values, histogram of pixel values, boundary information, texture information.
6. An apparatus for segmenting an image, comprising:
the super pixel segmentation unit is configured to perform super pixel segmentation on an image to be segmented to generate at least two super pixel blocks;
an identification region setting unit configured to set a first identification region in a foreground region of the image to be segmented and a second identification region in a background region of the image to be segmented;
the image feature extraction unit is configured to extract image features of the at least two super-pixel blocks to obtain first image features of the super-pixel blocks in the first identification region and second image features of the super-pixel blocks in the second identification region;
a determining unit configured to determine similarities between pixels in the image to be segmented and the first identification region and the second identification region respectively based on the first image feature and the second image feature;
a foreground region dividing unit configured to divide the image to be divided into a foreground region and a background region based on the determined similarity.
7. The apparatus of claim 6, wherein the segmentation unit comprises:
an energy function construction module configured to construct an energy function based on graph cut theory;
a pixel determination module configured to determine a minimum energy value of the energy function based on the determined similarity, determine a pixel in the image to be segmented as a foreground pixel or a background pixel.
8. The apparatus of claim 6, wherein the superpixel splitting unit is further configured to:
and performing superpixel segmentation on the image to be segmented by adopting a simple linear iterative clustering algorithm to generate at least two superpixel blocks.
9. The apparatus of claim 7, wherein the energy function comprises a data term and a smoothing term; and
the pixel determination module is further configured to:
determining the data items based on the similarity between the image characteristics of the super-pixel block where the pixels of the image to be segmented are located and the first image characteristics and the second image characteristics respectively;
determining the smoothing term based on image characteristics of pixels of the image to be segmented;
determining a minimum energy value of the energy function based on the determined data item and a smoothing item to determine a pixel in the image to be segmented as a foreground pixel or a background pixel.
10. The apparatus according to one of claims 6-9, wherein the image features of the superpixel blocks comprise at least one of the following features of the superpixel blocks: mean of pixel values, variance of pixel values, histogram of pixel values, boundary information, texture information.
11. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.
12. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811519372.XA CN111311604A (en) | 2018-12-12 | 2018-12-12 | Method and apparatus for segmenting an image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811519372.XA CN111311604A (en) | 2018-12-12 | 2018-12-12 | Method and apparatus for segmenting an image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111311604A true CN111311604A (en) | 2020-06-19 |
Family
ID=71159806
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811519372.XA Pending CN111311604A (en) | 2018-12-12 | 2018-12-12 | Method and apparatus for segmenting an image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111311604A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115049835A (en) * | 2022-08-16 | 2022-09-13 | 众烁精密模架(南通)有限公司 | Data preprocessing method based on die-casting die defect identification |
-
2018
- 2018-12-12 CN CN201811519372.XA patent/CN111311604A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115049835A (en) * | 2022-08-16 | 2022-09-13 | 众烁精密模架(南通)有限公司 | Data preprocessing method based on die-casting die defect identification |
| CN115049835B (en) * | 2022-08-16 | 2022-11-29 | 众烁精密模架(南通)有限公司 | Data preprocessing method based on die-casting die defect identification |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107578017B (en) | Method and apparatus for generating image | |
| CN107911753B (en) | Method and device for adding digital watermark in video | |
| CN107633218B (en) | Method and apparatus for generating image | |
| CN108229419B (en) | Method and apparatus for clustering images | |
| CN109308681B (en) | Image processing method and device | |
| CN107220652B (en) | Method and device for processing pictures | |
| CN109918513B (en) | Image processing method, device, server and storage medium | |
| CN109255337B (en) | Face key point detection method and device | |
| CN109377508B (en) | Image processing method and device | |
| CN109344762B (en) | Image processing method and device | |
| US11164004B2 (en) | Keyframe scheduling method and apparatus, electronic device, program and medium | |
| CN109255767B (en) | Image processing method and device | |
| CN109711508B (en) | Image processing method and device | |
| CN109118456B (en) | Image processing method and device | |
| CN110288625B (en) | Method and apparatus for processing image | |
| CN108182457B (en) | Method and apparatus for generating information | |
| CN109934142B (en) | Method and apparatus for generating feature vectors of video | |
| CN109697452B (en) | Data object processing method, processing device and processing system | |
| CN111523400B (en) | Video representative frame extraction method and device | |
| CN108595211B (en) | Method and apparatus for outputting data | |
| CN109919220B (en) | Method and apparatus for generating feature vectors of video | |
| CN112766284A (en) | Image recognition method and device, storage medium and electronic equipment | |
| CN113627536A (en) | Model training method, video classification method, device, equipment and storage medium | |
| CN110633717A (en) | Training method and device for target detection model | |
| CN109697722B (en) | Method and device for generating trisection graph |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |