JP2010067252A - Object region extraction device and object region extraction program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object region extraction device and program for extracting an arbitrary object only with a target image, without requiring a large number of learning images. <P>SOLUTION: The object region extraction device includes: an image dividing section 12 for dividing an image including an object into sub-regions smaller than the object; a pixel classifying section 13 for classifying all pixels of the image into pixels as boundaries of the sub-regions divided by the image dividing section 12 and non-boundary pixels; a boundary determining section 15 for determining whether or not a pixel classified as a boundary of the sub-regions by the pixel classifying section 13 is a boundary of the object; and a region integrating section 16 for integrating neighboring two sub-regions when a ratio of pixels determined as the boundary of the object by the boundary determining section 15 in pixels on a boundary of the neighboring two sub-regions is smaller than or equal to a predetermined value, regarding all combinations of neighboring two sub-regions. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object area extraction device and an object area extraction program.

  Techniques for extracting objects (people, animals, artifacts, etc.) from images form the basis of image processing, and many researches and developments have been conducted since ancient times. In recent years, face detection, human detection, and the like have been put into practical use, but these are techniques for extracting a specific object, and other objects cannot be extracted using the same algorithm. In other words, it cannot be used for the purpose of extracting an arbitrary object in the image. In order to extract a specific object, prior knowledge (color, shape, parts such as mouth and eyes, movement, etc.) needs to be taken into the extraction device as known information, and this prior knowledge varies depending on the object. Therefore, in order to extract an arbitrary object, it is necessary to create a classifier / discriminator for all objects existing in the image by a method similar to face detection or the like, which is not realistic.

  In order to achieve the goal of extracting an object from this image, research and development of contour extraction technology and region segmentation technology have been actively conducted. For example, technologies that extract objects from the bottom up without giving prior knowledge (KMeans method, hierarchical clustering method, graph cut method, MeansShift method, WaterShed method, etc.) are generally widespread. On the other hand, there is no technique for extracting an arbitrary object. This is considered to be one of the reasons that prior knowledge about the object is not taken in well.

On the other hand, instead of capturing prior knowledge for each object, a method has been recently proposed that focuses on only the boundary between objects and learns the state of the boundary from a correct image (for example, Non-Patent Document 1, Non-patent document 2). These methods require a large amount of learning images. In addition, a correct image in which the object is correctly divided is required at the same time, and it is difficult to avoid manual region division in order to obtain a correct image.
P.Dollar, Z.Tu, and S.Belongie, “Supervised Learning of Edge and Object Boundaries.”, CVPR, 2007 D.Martin, C.Fowlkes, and J.malik, “Learning to Detect Natural Image Boundaries Using Local Brightness, Colar and Texture Cues.” PAMI, 2004

  An object of the present invention is to provide an object region extraction apparatus and an object region extraction program that do not require a large amount of learning images and accurately extract an arbitrary object only from an image including the object even if it is a complex image. To do.

  In order to achieve the above object, the object region extraction device according to claim 1, image dividing means for dividing an image including an object into sub-regions smaller than the object, and all pixels of the image are divided into the image dividing device. A pixel classifying unit that classifies the pixel divided into a pixel that is a boundary of the sub-region and a pixel that is not a boundary, and a pixel that is classified as a boundary of the sub-region by the pixel classifying unit is a boundary of the object A boundary determination unit that determines whether or not the object is detected by the boundary determination unit among all the combinations of two adjacent sub-regions of the sub-regions. An area integration unit that integrates the two adjacent sub-areas when the ratio of pixels determined to be a boundary is equal to or less than a predetermined value; There.

  The object region extraction device according to claim 2 is the object region extraction device according to claim 1, wherein the boundary determination unit uses a feature of pixels classified as not a boundary of the sub-region by the pixel classification unit, It is determined whether the pixel to be determined is a boundary of the object.

  The object region extraction device according to claim 3 is the object region extraction device according to claim 1 or 2, wherein the determination processing by the boundary determination unit and the sub region integration processing by the region integration unit are repeated a predetermined number of times. .

  The object region extraction device according to claim 4 is the object region extraction device according to claim 1, wherein pixels classified into pixels that are not boundaries of the sub-region by the pixel classification unit are grouped into a plurality of groups based on the characteristics of the pixels. A non-boundary pixel classifying unit that classifies the sub-region, and a repetitive process determining unit that determines whether or not the sub-region integration by the region integration unit is repeatedly performed based on a result of sub-region integration by the region integration unit. The boundary determination means includes features of pixels belonging to each group classified by the non-boundary pixel classification means, and features of pixels classified as boundaries of the sub-regions by the pixel classification means. To determine whether a pixel classified as a boundary of the sub-region is a boundary of the object. If it is determined to perform repeated integration of the sub-region by return processing determination unit performs the pixel classification unit, the non-boundary pixel classification means, by repeating a predetermined processing by said boundary determining means and the region integrating unit.

  The object region extraction device according to claim 5 is the object region extraction device according to claim 4, wherein the iterative process determination unit determines the subregions by the region integration unit based on a parameter indicating a degree of integration of the subregions. It is determined whether or not the integration is repeated by using the parameters in order from the lowest degree of integration.

  7. The object area extraction program according to claim 6, wherein the computer divides an image including the object into sub-areas smaller than the object, and all pixels of the image are divided by the image dividing means. Pixel classification means for classifying a pixel that is a boundary of a sub-region and a pixel that is not a boundary; a boundary that determines whether a pixel classified as a boundary of the sub-region by the pixel classification means is a boundary of the object With respect to all combinations of two sub-regions adjacent to each other in the determination unit and the sub-regions, it is determined that the boundary determination unit out of the pixels on the boundary between the two adjacent sub-regions is the boundary of the object When the pixel ratio is less than or equal to a predetermined value, it functions as a region integration unit that integrates the two adjacent sub-regions. .

  According to the first aspect of the present invention, an effect that an arbitrary object region existing in an image can be extracted is obtained as compared with the case where the present configuration is not provided.

  According to the second aspect of the present invention, it is possible to extract an object region by learning only the target image without requiring a large amount of learning images as compared with the case without this configuration.

  According to the third aspect of the present invention, an effect that the accuracy of extracting the object region is improved as compared with the case where this configuration is not provided.

  According to the invention of claim 4, compared to the case without this configuration, a large amount of learning images is not required, and even if the target image is complex, the object region can be extracted by learning only the target image. The effect is obtained.

  According to the fifth aspect of the present invention, it is possible to obtain an effect that the areas are more accurately integrated and the accuracy of extracting the object area is improved as compared with the case where the present configuration is not provided.

  According to the sixth aspect of the present invention, an effect that an arbitrary object region existing in an image can be extracted is obtained as compared with the case where the present configuration is not provided.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to the following embodiment, It is applicable also to what changed the design within the range described in the claim.

[First Embodiment]
The first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an object area extracting apparatus according to the present embodiment.

  As shown in FIG. 1, the object region extraction device includes an image input unit 11, an image division unit 12, a pixel classification unit 13, an object extraction unit 14, a boundary determination unit 15, and a region integration unit 16. I have.

  The image input unit 11 captures an image as an object extraction target. The image input unit 11 may be configured with a device such as a camera.

  The image division unit 12 divides the image input by the image input unit 11 so that a sub-region (cluster) smaller than the object to be extracted is generated. Any known method may be used as the region dividing method. However, it is desirable that there be a sufficient number of regions to include almost 100% of the object boundary when generating the sub-region.

  The pixel classification unit 13 classifies all the pixels in the image into a boundary pixel and a non-boundary pixel after removing noise from the result of region division by the image division unit 12.

  The object extraction unit 14 includes a boundary determination unit 15 and a region integration unit 16, and controls execution of the boundary determination unit 15 and the region integration unit 16.

  The boundary determination unit 15 uses a classifier composed of feature vectors for the pixels determined to be non-boundary by the pixel classification unit 13, and the boundary of the cluster divided by the image division unit 12 is the boundary of the object to be extracted ( It is determined whether it is a true boundary.

  The region integration unit 16 integrates clusters that are in contact with the boundary determined by the boundary determination unit 15 as not being a true boundary.

  Next, the flow of operation of the object area extraction apparatus according to the present embodiment will be described with reference to the flowchart shown in FIG.

  In step 100, the image input unit 11 inputs a target image from which an object is to be extracted. In the present embodiment, a still image is handled as a target image, but the present invention can also be implemented by extending to a moving image or a 3D image. Further, the target image to be input may be stored in an external storage device or via an image output device such as a digital camera or a scanner.

  In step 102, the image dividing unit 12 performs clustering that divides the target image input by the image input unit 11 into clusters smaller than the object to be extracted. In order to perform this clustering, the number of regions to be divided should be increased as much as possible. However, if the number of regions is increased too much, a problem of excessive division occurs, which affects subsequent processing. For example, in the case of an image that can be separated from the figure (an image that includes an object that forms a figure, such as an animal or a person, in an image that includes the background), the region is divided so that the average size is about 1/10 of that object. Can be done. However, this is not limited to images such as landscape images that are difficult to separate, and the number of areas is set as large as possible, for example, the upper limit is set to 1000, and the area division is performed so that the upper limit is reached. Many algorithms for dividing an image by a specified number of areas have been proposed, and any known algorithm may be used, but the characteristics of pixels included in the generated area are similar to each other, and different areas are used. Pixels belonging to should have an algorithm that has as different characteristics as possible. In consideration of performance, speed, etc., KMeans method and graph partitioning (for example, Normalized Cut (Shi & Malik, PAMI, 2000) and FH method (Felzenswalb & Huttenlocher, IJCV, 2004)) are suitable. In this embodiment, the region division is performed using the Normalized Cut among the above-described methods, but the other methods described above may be used.

The Normalized Cut algorithm is a method of dividing a graph into a predetermined number when a pixel is a node, a connection between pixels is a link, and a similarity between pixels is defined as the strength of the link. A square symmetric matrix having the link strength wij of the pixel pi and the pixel pj in the i-th row and j-th column component is W (its size is the image size), each column component is added to the diagonal component, and the non-diagonal component is set to 0. (1) which is a generalized eigenvalue problem when the diagonal matrix is D
(D−W) y = λDy (1)
Can be reduced to computing the second smallest eigenvalue and its eigenvector. Since the size of the matrix W is large, it takes a lot of time to calculate eigenvalues as it is, but if the sparseness of the matrix is assumed by setting the link strength to 0 when the pixels are separated, the speed of the matrix calculation Can be raised.

  In addition, since this method does not need to obtain an accurate eigenvector, the speed can be further increased by using the Lanczos method or the like. As an actual operation, Normalized Cut is realized by repeating 2cut.

  FIG. 3 is a diagram illustrating an example in which target images are clustered using Normalized Cut. FIG. 3A shows an example of a target image, and FIG. 3B shows the result of clustering by the image dividing unit 12. FIG. 4B shows the result of manually dividing an object in the target image, and the boundary of the region output by the image dividing unit 12 shown in FIG. 3B is shown in FIG. It shows that it includes almost the boundary created manually. Therefore, by integrating the regions created by the image dividing unit 12, it is possible to approach an ideal region division result (that is, a manual division result).

  In step 104, the pixel classification unit 13 performs processing (preprocessing) for removing noise from the result of clustering by the image dividing unit 12. Specifically, when the number of pixels belonging to the created cluster is very small, a process of merging the cluster with an adjacent cluster is performed. For example, by looking at the cluster ID to which the eight pixels near the specific pixel belong, the cluster ID of that pixel is changed to the most cluster ID.

  In step 106, the pixel classification unit 13 classifies the target image into pixels that are cluster boundaries and pixels that are not boundaries. Since the result of clustering is given for each pixel, the boundary should be in the middle of the pixel. In this embodiment, as shown in FIG. If a pixel 22 is assigned to a different cluster, it is determined that the pixel 20 is a boundary. As a result, the boundary is shifted to the upper left by 0.5 pixels.

In step 108, the pixel classification unit 13 calculates a feature vector group A corresponding to each pixel (30 in FIG. 3B) that is not a cluster boundary. For example, as shown in FIG. 6, a feature difference between a pixel and eight neighboring points (three-dimensional (l ^* a ^* b ^* ) in the case of a color dimension) × 8 is acceptable. Or a 24-dimensional feature vector). In addition to the color dimension, an information amount based on edge information such as a Sobel filter may be added. Note that when calculating such a feature vector, it is advantageous in terms of learning stability that the number of neighboring points and the filter size of the edge filter are large. However, when calculating the feature vector of the non-boundary part, the filter fits in the cluster in the case of the small size filter 31 as shown in FIG. End up. As described above, if the filter size is too large, it may be impossible to express the feature quantity regarding the accurate non-boundary part. Therefore, it is possible to improve the classification by excluding such feature vectors from the learning data.

  In step 110, the boundary determination unit 15 determines whether the pixel classified as a boundary by the pixel classification unit 13 is a true boundary or a false boundary between objects. As a learning element for determination, a feature of a pixel determined not to be a boundary in the target image input by the image input unit 11 is used instead of using an image for learning. Specifically, when forming a partial space with a feature space of pixels in a non-boundary region, among pixels included in the boundary region, a pixel included in the partial space is determined as a false boundary region, and pixels that are not Is determined to be a true boundary.

  When handling learning data belonging to a plurality of groups when configuring a classifier, various classifiers such as SVM and Adaboost can be used. However, as in this embodiment, only groups belonging to false boundaries are used. In the case of learning data, these cannot be used as a classifier. As a classifier whose learning data is one class, a method such as SVDD (Support Vector Domain Description) has been proposed, but here, OCSVM (One-Class Support Vector Machine: B.) which is an extension of SVM (Support Vector Machine). Scholkopf, J. Platt, JS. Taylor, AJ. Smola, and R. Williamson, Neural Corporation, 2001). OCSVM is a technique that non-linearly maps input data to a higher-order feature space (for example, Gaussian kernel) and obtains a discriminant function on that feature space. Since only one class of learning data is used, unsupervised learning is performed. It is considered to be a kind of

  FIG. 8 is a flowchart showing a flow of a determination process of whether or not the true boundary is determined by the OCSVM.

  In step 202, the boundary determination unit 15 generates an OCSVM from the feature vector group A, determines various parameters, and generates a discriminant function. The boundary determination unit 15 divides the non-linearly mapped feature vector group A by a hyperplane that has a maximum margin from the origin on the feature space. In OCSVM, a discriminant function for an input vector (a feature vector for a pixel located at a boundary divided by the image dividing unit 12) z can be written as in Expression (2).

  Here, x is a learning feature vector, K (x, y) is a kernel function, and α is a Lagrange multiplier satisfying Equation (3) (ν∈ (0,1)). W is a variable and n is the number of feature vectors.

  In step 204, the boundary determination unit 15 generates a feature vector group B of pixels corresponding to the cluster boundaries.

  In step 206, the boundary determination unit 15 inputs the feature vector B into the above-described discriminant function, and determines whether or not the cluster boundary is a true boundary. This determination is performed as follows.

  If the input z of the discriminant function is a vector in the feature space composed of pixels in the non-boundary region, the output returns almost positive. Therefore, when the discriminant function of Expression (2) returns a positive value using the feature vector of the feature vector group B for the pixel on the boundary in FIG. 3B as an input value, the feature vector is constituted by learning data. It can be considered to be included in the space, that is, a false boundary. When returning a non-positive value, it can be regarded as a true boundary that is far from the partial space.

  In the present embodiment, the image dividing unit 12 divides the target image into larger clusters. When a large number of clusters are created in this way, a certain pixel in the boundary is a true boundary that separates the objects from each other, and the other boundary is a false boundary that occurs because the clustering number is too large. That is, it can be said that a cluster boundary includes a true boundary, and a pixel that is not a cluster boundary is not a true boundary. Therefore, a scheme is established that learns the features of a region that is not a cluster boundary and is not a true boundary if it has a feature close to the group, but a true boundary if it does not have a feature close to the group. Therefore, the above-described determination method is possible, in which a pixel having a feature amount close to a local feature in the region 30 that is not a cluster boundary shown in FIG.

  FIG. 9 is a diagram illustrating an output example of the image division unit 12, the pixel classification unit 13, and the boundary determination unit 15. 9A shows the output result of the image dividing unit 12, FIG. 9B shows the output result after the preprocessing by the pixel classifying unit 13, and FIG. 9C shows the output result of the boundary determining unit 15. A pixel indicated by a solid line in FIG. 9C represents a pixel determined to be a true boundary.

  In this embodiment, after pre-processing (step 104 in FIG. 2), all the pixels determined to be non-boundary can be used as learning data for the classifier. You may restrict | limit only to the pixel which does not contain. Further, in the present embodiment, the feature vector is configured using the difference between the feature values of the neighboring 8 points, but the number of neighborhoods may be increased (for example, 15 points, 24 points, etc.). In that case, since the number of pixels that do not include a boundary in the vicinity decreases, the learning data decreases.

  Returning to the flowchart of FIG. 2, in step 112, the region integration unit 16 integrates clusters based on the determination result of the boundary determination unit 15. FIG. 10 is a diagram illustrating a cluster integration method. As shown in the figure, two adjacent clusters (P1 and P2) are taken out and attention is paid to pixels on the boundary. “Adjacent” means that there is at least one common pixel among the pixels located at the boundary between two clusters.

FIG. 11 is a flowchart showing the flow of cluster integration processing. Here, | Pi | represents the number of pixels existing on the boundary of cluster Pi, | Pi∧Pj | represents the number of pixels existing on the boundary common to cluster Pi and cluster Pj, and the feature vector for the pixels in the common part Among these, the number of pixels determined to be a true boundary by the boundary determination unit 15 is m _ij . Further, the number of clusters divided by the image dividing unit 12 is _nk .

  In step 300, the region integration unit 16 initializes i and j indicating the cluster ID to “1” and “2”, respectively.

  In step 302, the region integration unit 16 determines whether Pi and Pj are adjacent to each other. If adjacent to each other, the process proceeds to step 304. If not, the process proceeds to step 306.

In step 304, the region integration unit 16 calculates S = m _ij / | Pi∧Pj | and stores it in a buffer (not shown).

  In step 306, the region integration unit 16 proceeds to step 308 when i + 1 is smaller than j, determines whether i + 1 is smaller than j, and proceeds to step 310 when i + 1 is greater than or equal to j.

  In step 308, the region integration unit 16 adds “1” to i, and then proceeds to step 302 to repeat the processing after step 302.

In step 310, the region integration unit 16 determines whether j is smaller than _nk . When j is smaller than _nk , the process proceeds to hs step 312, and when j is equal to or larger than _nk , the process proceeds to step 314.

  In step 312, the region integration unit 16 adds “1” to j and sets i to “1”, then proceeds to step 302 and repeats the processing from step 302 onward.

  In step 314, the region integration unit 16 sorts the S stored in the buffer in ascending order, and sets the minimum value to s. When the buffer is empty, s is set to a sufficiently large value.

  In step 316, the region integration unit 16 determines whether or not s set in step 314 is larger than a predetermined threshold value b. If s is larger than b, the process proceeds to step 318. This completes the integration process.

In step 318, m _ij / | Pi∧Pj | is equal to or smaller than the threshold value b, and it is determined that the boundary common to Pi and Pj is false. Therefore, the region integration unit 16 integrates Pi and Pj. Further, the region integration unit 16 reassigns the cluster ID according to the cluster ID lost due to the cluster integration, and updates the buffer data.

In 320, after subtracting “1” from _nk , the process proceeds to step 314, and the processes in and after step 314 are repeated.

  In this way, the region integration unit 16 integrates two clusters when the common boundary is a false boundary for all adjacent cluster pairs.

  As another method, when the common part | Pi∧Pj | is smaller than | Pi | and | Pj |, for example, min (| Pi∧Pj | / | Pi |, | Pi∧Pj | / | Pj |). When the threshold value b2 is less than or equal to a certain threshold value b2, a constraint condition that the integration process is not performed may be added. Further, when the difference between the features of the two adjacent regions (for example, the average difference between the feature amounts of the pixels belonging to the region) is greater than or equal to the threshold value b3, it is possible to add a constraint that the integration process is not performed.

  In step 114, the object extraction unit 14 determines whether or not the processing of the boundary determination unit 15 and the region integration unit 16 has been performed a predetermined number of times. If the processing has been performed a predetermined number of times, the processing is terminated and the processing is performed a predetermined number of times. If not, the process returns to step 106, and the processing from step 106 to step 112 is repeated. Each time this iterative process is performed, the number of pixels corresponding to the non-boundary region increases. Along with this, learning data also increases, and a vector with a larger dimension can be created when creating a feature vector.

  FIG. 12 shows an output example when the processing is completed. The clustering result created in this way is equivalent to classifying the target image for each object.

  Note that the present embodiment is not limited to the above-described embodiment. For example, as a cluster integration method, not only this embodiment but also a method of calculating the similarity based on the image features of adjacent clusters and combining the clusters based on the value can be used. . Specifically, the evaluation formula is (q / p) / D (R1, R2), etc., and if this value is smaller than the threshold value θ ′, the process of integration is repeated. D (R1, R2) is a function representing the similarity between the adjacent clusters R1 and R2, and the larger the value, the higher the similarity. As an example, Formula (4) can be considered.

Here, d _R is the feature vector of the cluster R, the average feature values of the pixels belonging to the example cluster R. σ is a constant term. When the average feature is the same, the similarity is 1, and the similarity approaches 0 as the feature moves away.

  FIG. 13 is a block diagram showing a variation of the present embodiment. In this way, the classifier may be configured by combining a plurality of feature amounts (for example, colors and textures) instead of only one feature amount. In that case, the region integration unit 16 integrates the results (for example, sums and products) of the classifiers based on a plurality of feature dimensions, and performs processing to determine the true boundary.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the site | part same as 1st Embodiment, and a different part from 1st Embodiment is mainly demonstrated.

  FIG. 14 is a block diagram showing a configuration of an object area extracting apparatus according to the second embodiment. Unlike the block diagram of the first embodiment in FIG. 1, it further includes a non-boundary pixel classification unit 18 and an iterative process determination unit 19.

  The non-boundary pixel classifying unit 18 classifies the feature vectors for the pixels determined as non-boundary by the pixel classifying unit 13 into a plurality of classifiers. Further, the boundary determination unit 15 uses the plurality of classifiers classified by the non-boundary pixel classifying unit 18 as the boundary (true boundary) of the object from which the boundary part of the cluster divided by the image dividing unit 12 is the extraction target. It is determined whether or not there is.

  The repetitive processing determination unit 19 determines whether or not the cluster integration processing by the region integration unit 16 is further repeated.

  Next, the flow of operation of the object area extraction apparatus according to the present embodiment will be described with reference to the flowchart shown in FIG. Step 400, step 402, and step 404 are different from the flowchart of the first embodiment in FIG. The process from the image input in step 100 to the generation of the feature vector of the non-boundary pixel in step 108 is performed in the same manner as in the first embodiment, and the process proceeds to step 400.

  In step 400, the non-boundary pixel classifying unit 18 classifies the feature vector group corresponding to each pixel that is not a cluster boundary created by the pixel classifying unit 13 into a plurality of feature vector groups. Any known method may be used as the classification method, but the above-described OCSVM is used in the present embodiment. When OCSVM is used, as shown in FIG. 16A, one type of feature vector group 40 (feature space indicating a non-boundary) is used as teacher data, and the entire feature space belongs to a group close to these teacher data (that is, It is classified into two groups depending on whether it belongs to a group far from the teacher data (ie, the true boundary). In OCSVM, these groups far from the teacher data are points near the origin of the feature space. Therefore, the side closer to the origin than the classification standard 41 is a true boundary, and the side farther from the origin than the classification standard 41 is a false boundary. The feature space is usually mapped to a multidimensional space by the kernel method or the like.

  Thus, when there is only one type of feature group corresponding to a false boundary, this space becomes a very complicated space if the complexity of the image increases. For example, in the case of an image like a bird flying in the sky as shown in FIG. 17A (that is, an image with a clear figure and ground), the feature vector that enters such a feature space is a pixel corresponding to the region showing the sky. There are at most two types of features for the region showing the bird and the feature space formed is relatively simple.

  However, in the case where an image that is difficult to separate as shown in FIG. 17B is a target, there are a plurality of types of feature vectors that enter such a feature space. It becomes very complicated. In order to avoid this, in the present embodiment, as shown in FIG. 16B, feature vectors corresponding to non-boundaries are divided into a plurality of sets (feature vector groups 42, 43, 44), and each subspace is divided. A classifier is created independently as a feature space.

Several methods are conceivable as such a subspace classification method. In this embodiment, the KMeans method is used. That is, let the feature vector be f _k = (f ^k ₁ , f ^k ₂ , f ^k ₃ ,..., F ^k _d ) (k is the ID of the pixel determined to be a non-boundary part, and d is the number of feature dimensions). , The number is n, and the number of subspaces is κ. Let the feature vector of the subspace Gi be gi (gi is the centroid vector of the feature vector belonging to the subregion). At this time, the subregion Gj to which the feature vector f _k should belong can be written as in Expression (5).

  The number of sub-spaces is set in advance (for example, fixed at about 10), or the maximum κ such that the value of a predetermined evaluation function (evaluates the number of divisions κ) is equal to or less than a reference value. There are methods such as setting. As an example of an evaluation function for dynamically changing κ, we can increase κ and consider the sum of the distance difference between the feature vector and the feature vector of the subspace to which it belongs each time. As this happens, this value decreases). Further, the space may be divided into subspaces before or after the kernel function is applied, but the front is natural.

  In step 402, the boundary determination unit 15 determines whether the pixel classified as a boundary by the pixel classification unit 13 is a true boundary or a false boundary between objects. In OCSVM, the discriminant function for the input vector z is expressed by the above-described equations (2) and (3). The discriminant function outputs “1” when the input vector z is a feature vector for a pixel belonging to a false boundary region, and “0” when it is not, that is, a feature vector for a true boundary region. return. In the present embodiment, the boundary determination unit 15 includes such a classifier as many as the number of subspaces (that is, κ) generated by the non-boundary pixel classification unit 18.

  The flow of boundary true / false determination processing using κ classifiers will be described with reference to the flowchart shown in FIG.

  In step 500, the boundary determination unit 15 calculates a parameter required for the OCSVM for each classifier based on the learning data belonging to the sub-region Gi. Necessary parameters are ν and K (x, y) in the above equations (2) and (3) when a Gaussian kernel is used.

In step 502, the boundary determination unit 15 configures a discriminant function fi (Z _k ) for each sub-region Gi, and outputs an output vector a _k = (a ^k ₁ , a ^k ₂ ,..., A ^k _k ) in units of pixels. obtain. Here, k is an ID of an input vector, and a ^k _i is an output value of fi (Z _k ) and takes a value of “0” or “1”.

In step 504, the boundary determination unit 15 determines whether or not all elements of the output vector a _k of the discriminant function are “0”. If all the elements are “0”, the process proceeds to step 506, and otherwise, step 508 is performed. Proceed to

In step 506, all the elements of the output vector a _k are “0”, and the boundary determination unit 15 determines that the pixel with the ID k is a false boundary.

In step 508, there is a case where at least one element of the output vector a _{k has} a value other than “0”, and the boundary determination unit 15 determines that the pixel whose ID is k is a true boundary.

  The boundary determination unit 15 performs the above processing in parallel for each sub-region Gi (i = 1 to κ).

In the present embodiment, if even one of the elements of the output vector a _k is other than “0”, the pixel is determined to be a true boundary, but “0” of the elements of the output vector a _k. It may be determined that the boundary is true when anything other than “a” is greater than or equal to a predetermined ratio, and may be determined as a false boundary when less than the predetermined ratio.

  Returning to the flowchart of FIG. 15, the region integration unit 16 performs cluster integration based on the determination result of the boundary determination unit 15, and then proceeds to step 404.

In step 404, the repetitive processing determination unit 19 determines whether or not the cluster integration processing by the region integration unit 16 is to be repeated further. Here, it is assumed that the total number of clusters after being integrated by the region integration unit 16 is n _k (k = 1, 2,...). The subscript k means the number of times of integration processing. Further, as a method of determining whether or not to perform cluster integration, when the number of clusters does not change before and after integration, or the rate of change is equal to or less than a threshold value (that is, n _k / n _(k−1) is a threshold value _). As described above, a method is conceivable in which the parameters of the classifier are changed so that the integration capability becomes stronger, further integration is performed, and the integration process is terminated when the parameter value reaches a certain value.

When OCSVM is used, the relationship between the parameters (σ, ν) and the integration capability is as follows. If the number of cycles of OCSVM and the number of clusters after integration when the value of σ is fixed are sufficiently large (about 10,000), the calculation result converges, and the smaller the value of ν, the stronger the integration ability. When σ is changed, if the number of circulations is sufficiently large (about 10000), the smaller σ is, the stronger the integration ability is. In summary, the classifier C (σ, ν) composed of OCSVM is composed of two parameters, and the integration ability is stronger when σ is smaller and ν is smaller. When performing image region integration using multiple classifiers, the classifiers are used in order from the classifier with the lowest integration capability in order to reduce the effects of integration errors (boundary decision errors) that occur during integration. It is desirable. This is because it is considered that a classifier having a low integration capability has few integration errors due to lack of learning data. When the strength of the integration capability is expressed by an inequality sign, C (σ ₁ , ν ₁ ) <C (σ ₂ , ν ₂ ) (where σ ₁ > σ ₂ , ν ₁ > ν ₂ ).

In the present embodiment, σ is fixed, and parameters of ν and the above-described threshold value b are changed. Specifically, an example in which ν is changed to four types (ν = 0.1, 0.2, 0.4, 0.8) and threshold value b is changed to two types (b = 0.1, 0.2) is shown. Of course, the larger the threshold value b, the stronger the integration capability. In the classifier C (σ, ν, b), C (σ, ν ₁ , b ₁ ) <C (σ, ν ₂ , b ₂ ) (however, , Ν ₁ > ν ₂ , b ₁ <b ₂ ). On the other hand, when ν ₁ > ν ₂ and b ₁ > b ₂ , the strength of the integration ability of the two classifiers C (σ, ν ₁ , b ₁ ) and C (σ, ν ₂ , b ₂ ) is obvious. It depends on the target image and the number of clusters before integration. Therefore, in this embodiment, for convenience, C (σ, 0.8,0.1), C (σ, 0.8,0.2), C (σ, 0.4,0.1),..., C (σ, 0.1,0.2), Basically, a method is adopted in which ν is preferentially changed in order from a classifier having a weak integration ability.

FIG. 19 is a flowchart showing a flow of determination as to whether or not to perform further cluster integration by the iterative process determination unit. The iterative processing determination unit 19 determines whether or not to perform further integration processing on n _k clusters generated using the classifier C (σ, ν, b) in the k-th integration processing by the region integration unit 16. judge.

In step 602, the iterative processing determination unit 19 determines the rate of change (n _k / n _(k−1)) between the number of clusters after the k-th integration processing and the number of clusters after the previous (k−1) integration processing. ) Is calculated and compared with a predetermined value c. If c is equal to or greater than c, the process proceeds to step 604, and if smaller than c, the process proceeds to step 608. Here, c is a threshold value, and is set to, for example, “0.9” in the present embodiment.

  In step 604, the iterative processing determination unit 19 determines whether b is “0.1”. When b is “0.1”, the process proceeds to step 610, and when b is not “0.1”, the process proceeds to step 606.

  In step 606, the iterative processing determination unit 19 determines whether or not ν is “0.1”. When ν is not “0.1”, the process proceeds to step 612, and when ν is “0.1”, the above-described ν and b Since all combinations have been executed, the integration process is terminated.

  In step 608, since the integration process is performed again with the classifier having the same parameter, the process is repeated from the process performed by the pixel classification unit 13.

  In step 610, since the integration processing is further performed by the classifier in which b is set to “0.2”, the processing by the pixel classification unit 13 is repeated.

  In step 612, since the integration process is further performed by the classifier in which ν is divided by 2 and b is set to “0.1”, the process from the pixel classification unit 13 is repeated.

  FIG. 20 is a schematic diagram showing a state in which the parameters of the classifier are changed in this way. As shown in the figure, eight kinds of classifiers are implemented by changing the combination of parameters for one classifier.

  FIG. 21 is a diagram illustrating an example of an implementation result of the present embodiment. 21A shows the result of the initial clustering by the image dividing unit 12, and FIGS. 21B to 21O show the integration results in each process when the region integration is repeatedly performed by the region integration unit 16. FIG. FIG. 22 is a diagram showing fluctuations in parameter values and the number of clusters for each of FIGS. 21 (a) to 21 (o). In (d) to (f) of FIG. 21, the parameters of the classifier do not change, but the number of learning data input to the classifier increases because the number of clusters decreases due to the integration process. Even if the same parameter classification is used, further integration processing can be expected. As shown in these figures, by repeating the integration process, false boundaries are eliminated, the number of clusters is reduced, and actual object boundaries are extracted.

It is a block diagram which shows the structure of the object area | region extraction apparatus which concerns on 1st Embodiment. It is a flowchart which shows the flow of an effect | action of the object area | region extraction apparatus which concerns on 1st Embodiment. It is a figure which shows the target image and its clustering example. It is a figure which shows the example which divided | segmented the object into the target image manually. It is a figure which shows the determination method of the pixel of a cluster boundary. It is a figure which shows the preparation method of the feature vector at the time of using a color. It is a figure which shows the example which the filter for feature vector preparations applies to a boundary region. It is a flowchart which shows the flow of the determination process of whether it is a true boundary. It is a figure which shows the example of an output of an image division part, a pixel classification | category part, and a boundary determination part. It is a figure which shows the integration method of a cluster. It is a flowchart which shows the flow of the integration process of a cluster. It is a figure which shows the example of an output at the time of completion | finish of a process. It is a block diagram of the modification of 1st Embodiment. It is a block diagram which shows the structure of the object area | region extraction apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the flow of an effect | action of the object area | region extraction apparatus which concerns on 2nd Embodiment. It is a figure which shows the feature vector used for learning. It is a figure which compares the complexity of an image. It is a flowchart which shows the flow of the authenticity determination process of the boundary using (kappa) classifiers. It is a flowchart which shows the flow of the determination process of a repetition process determination part. It is a schematic diagram which shows a mode that it implements, varying the parameter of a classifier. It is a figure which shows the implementation result of the object area | region extraction apparatus which concerns on 2nd Embodiment. It is a figure which shows the mode of the fluctuation | variation of the parameter with respect to each figure of FIG. 21, and the number of clusters.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Image input part 12 Image division part 13 Pixel classification part 14 Object extraction part 15 Boundary determination part 16 Area integration part 18 Non-boundary pixel classification part 19 Repetitive process determination part

Claims (6)

Image dividing means for dividing an image including an object into sub-regions smaller than the object;
Pixel classification means for classifying all pixels of the image into pixels that are boundaries of the sub-region divided by the image dividing means and pixels that are not boundaries;
Boundary determination means for determining whether or not a pixel classified as a boundary of the sub-region by the pixel classification means is a boundary of the object;
For all combinations of two adjacent sub-regions of the sub-regions, a ratio of pixels determined to be the boundary of the object by the boundary determination unit among the pixels on the boundary of the two adjacent sub-regions is In the case of a predetermined value or less, area integration means for integrating the two adjacent sub-areas;
An object area extraction apparatus comprising:   2. The boundary determination unit according to claim 1, wherein the boundary determination unit determines whether the pixel to be determined is a boundary of the object by using a feature of a pixel classified as not being a boundary of the sub-region by the pixel classification unit. Object area extraction device.   3. The object region extraction device according to claim 1, wherein the determination processing by the boundary determination unit and the sub-region integration processing by the region integration unit are repeated a predetermined number of times. Non-boundary pixel classification means for classifying pixels classified into pixels that are not boundaries of the sub-region by the pixel classification means into a plurality of groups based on the characteristics of the pixels;
Repetitive process determination means for determining whether to repeatedly perform integration of sub areas by the area integration means based on the result of integration of sub areas by the area integration means,
The boundary determination means compares the characteristics of the pixels belonging to each group classified by the non-boundary pixel classification means with the characteristics of the pixels classified as the boundary of the sub-region by the pixel classification means. To determine whether a pixel classified as a boundary of the sub-region is a boundary of the object,
When it is determined by the repetitive processing determining means to repeat the integration of sub-regions, predetermined processing by the pixel classifying means, the non-boundary pixel classifying means, the boundary determining means, and the region integrating means is repeatedly performed. The object area extraction device according to claim 1.   The iterative process determination means determines whether to repeatedly perform sub-area integration by the area integration means based on a parameter indicating the degree of integration of the sub-areas, and has a low degree of integration of the parameters. 5. The object region extraction device according to claim 4, wherein the object region extraction device is used in order. Computer
Image dividing means for dividing an image including an object into sub-regions smaller than the object;
Pixel classification means for classifying all pixels of the image into pixels that are boundaries of the sub-region divided by the image dividing means and pixels that are not boundaries;
Boundary determination means for determining whether or not a pixel classified by the pixel classification means as a boundary of the sub-region is a boundary of the object, and all combinations of two adjacent sub-regions among the sub-regions When the ratio of the pixels determined to be the boundary of the object by the boundary determination means among the pixels on the boundary between the two adjacent sub areas is equal to or less than a predetermined value, the two adjacent sub areas Area integration means to integrate,
Object area extraction program to function as.