JP2006031114A - Image division processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for dividing an image to a plurality of areas, which performs an area division robust to difference in resolution, change of object, occlusion, sudden environmental change or the like of the image by extracting a proper characteristic quantity by a further objective criterion. <P>SOLUTION: Small areas (windows) are set in the whole image, and a group of partial images obtained by cutting out small areas of the image from each window is formed. An order relation corresponding to the non-similarity among all the cut partial images is defined among them, and each partial image is mapped to a point in an optional distance space based on only the order relation. Clustering is performed for the group of the thus-mapped points in the distance space, and partial images obtained by reversely mapping the points belonging to each cluster are regarded to belong to the same area, and reintegrated. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image division processing system, and more particularly to a system that divides a plurality of partial areas based on image feature amounts.

  As a method for identifying a target object from an image, there is a method of dividing an image into a plurality of small partial regions based on the feature amount. In such a method, it is possible to divide the region by detecting a portion where the feature amount (correlation coefficient thereof) greatly changes and regarding that as a boundary between the partial regions. Here, the feature amount is information that characterizes the structure in each partial region of the image. In order to extract a feature amount, there are a method using average luminance and variance, or a heuristic method for extracting based on information such as edge strength and texture. For example, in Patent Document 1, an amount of average color, representative color, or density is used as a feature amount. However, it is not obvious whether the feature quantity extracted in this way is the optimum standard for segmenting an image, so it is important to find the true optimum feature quantity based on objective evaluation. is there.

  In the process of identifying a target object from an image, an approach for extracting useful features by a statistical method is known. In particular, (1) Principal Component Analysis (PCA) or Multiple Discriminant Analysis (MDA) is generally used to extract feature values obtained by linear transformation from images. It is. For example, Patent Document 2 describes an object recognition method and an object recognition device based on MDA. However, depending on the conditions of the target object and the background (environment), the assumption of linearity may not be satisfied, and a feature quantity useful for region segmentation is not always obtained by linear transformation from an image. As a method for dealing with such a case, there are (2) a method of performing principal component analysis after performing a neural network or nonlinear transformation. However, in these methods, it is necessary to presuppose the shape of the nonlinear transformation in advance, but if the feature amount to be extracted is different from the nonlinear transformation assumed in advance, an appropriate feature amount cannot be obtained. Therefore, as a method that does not assume in advance what kind of nonlinear transformation is performed, (3) a method using a self-organizing map (SOM), a multi-dimensional scaling method (Multi Dimensional Scaling; MDS) ) Etc. can be used. For example, Patent Literature 3 describes an image search device based on SOM, and Patent Literature 4 describes an image similarity scale configuration processing method based on MDS.

Japanese Patent Application Laid-Open No. 11-167634. JP 2000-242784. JP 2000-90239 A. Japanese Patent No. 3155033. Taguchi Yoshihiro et al., 2001, “Expectations and New Perspectives on Non-metric Multidimensional Scaling Methods”, Statistical Mathematics Vol. 49, No. 1, p133-153.

  First of all, the problem that can be generally said about the method of constructing statistical features related to images is that before the statistical analysis, the analyst heuristics such as average luminance and variance, or edge and texture information as primitive features. After extracting the feature quantity introduced in step 1 as the initial feature quantity, statistical processing is performed to further extract the statistical feature quantity. That is, information that the analyst determines as noise before performing statistical processing is removed. However, what information is actually noise cannot be predicted in advance. Further, there is a possibility that noise is included in the information extracted as the feature amount. Rather, it can be said that the conventional statistical feature extraction method performs a statistical analysis only to remove noise contained in the heuristically defined feature quantity. Then, in order to eliminate noise by statistical analysis and extract a statistically meaningful feature value, it is desirable to perform statistical analysis without cutting off information from the image. However, in general, image data is very high-dimensional data, and a huge amount of calculation processing is required to perform discriminant analysis on such high-dimensional data, so information is cut off from the image. It is practically difficult to perform statistical analysis without it. For this reason, in the conventional method, an analyst first extracts heuristic feature amounts, and thus performs statistical analysis on the data whose dimensions have been lowered.

  In the method described in the prior art (1), it is assumed that the feature amount is given as a linear transformation related to image information. However, since such an assumption is not generally valid, the applicable objects are limited. There is a problem that it will be.

  In the method described in the prior art (2), in order to handle the feature amount obtained by the nonlinear transformation, it is necessary to determine in advance what kind of nonlinear transformation is performed. There is a problem that an appropriate feature amount cannot be obtained.

  In the method described in the prior art (3), it is not necessary to presuppose nonlinear transformation in advance, but a metric must be given to the space formed by the feature quantity, and the analysis result is greatly influenced by this metric. However, since it is not self-evident how to provide the metric to the feature space to be obtained, there is a problem that accuracy is deteriorated when the set metric is inappropriate. Furthermore, in the case of SOM, since the feature space is represented by a discrete grid, errors due to discretization may greatly affect.

  In image segmentation processing, noise is generated due to differences in resolution between image measurement devices, changes in target objects, sudden changes in occlusion and the environment (background), and these are difficult to handle statistically. . Since the above-described conventional technology handles data quantitatively, there is a problem that the robustness against such noise generation is low. In addition, it is often difficult to sufficiently correct the influence of such large noise in the preprocessing for the image.

  Furthermore, when applying a conventional method to a three-dimensional image such as an MRI image, in order to extract a feature amount, a two-dimensional cross-sectional image is once taken and analyzed, and then again three-dimensional. However, in such a case, it is difficult to extract global features between cross sections. Also, recent developments in MRI image measurement technology have made it possible to acquire 3D images of tensor values, but tensor value (matrix value) data differs from data such as hue and brightness. The amount of determinants, eigenvalues, and traces is more meaningful than the value of each component itself, and features such as average luminance and edge strength as in the conventional method are effective features. Often not.

  The present invention has been made in view of such problems in the prior art, and is a system that divides an image into a plurality of regions by extracting an appropriate feature amount based on a more objective criterion. It is an object of the present invention to provide a system that can perform region segmentation that is robust against differences in image resolution, changes in target objects, occlusions, and sudden changes in the environment.

  In order to solve the above problem, the present inventor first sets an area (window) having a predetermined shape that is sufficiently smaller than the image size so that the window covers the entire image (window). May be overlapped with each other or may not overlap each other), and a set of partial images obtained by cutting out a small area of the image from each window is created, and between all the cut out partial images Define an order relationship that corresponds to the dissimilarity (or similarity) between them, and map each partial image to a point in an arbitrary metric space based on this order relationship. Clustering is performed on the set of points in the metric space, and the partial images obtained by reverse mapping the points belonging to each cluster are reintegrated as belonging to the same region, so that the image is divided into the target object region and the background region. In It was conceived to be split.

  Compared to the method described in the prior art, the present invention uses only the order relationship between the quantities such as dissimilarity or similarity in the mapping source, especially in comparison with the method described in the prior art (1). Since only the relationship is important, it is highly robust against noise and fluctuations in the brightness of the environment, and it does not depend on how the dissimilarity or similarity scale is selected. The mapping can be constructed with high accuracy without being affected by the definition of the metric, and can be applied even when the mapping is nonlinear. Further, compared to the method described in the prior art (2), the nonlinear mapping is previously performed. Since it is not necessary to determine the shape of, it is possible to construct a non-linear mapping of an arbitrary shape, and compared to the method described in the prior art (3), the space of the mapping destination is Since it can be treated as a continuation space, there is no deterioration in the accuracy of mapping by discretizing the space as in the self-organizing mapping, and it does not depend on the definition of the distance scale, that is, the metric, so self-organizing mapping and metric In order to construct a mapping with high accuracy like the general multidimensional scaling method, it is characterized by being unaffected by the problem that the distance scale must be adjusted very accurately.

  Furthermore, the calculation cost can be greatly reduced because the processing can be performed without being affected by these effects without performing pre-processing on the image such as noise removal, brightness correction, and background fluctuation correction. Also has features.

  Furthermore, even for a three-dimensional image such as an MRI image, in the present invention, it is not necessary to divide the three-dimensional image into two-dimensional cross sections, and it is possible to analyze the three-dimensional image as it is, In addition, with the recent development of MRI image measurement technology, it has become possible to acquire three-dimensional images of tensor values. Even with such tensor value data, objective feature values can be obtained objectively. It also has a feature in that it can be extracted.

  As means for realizing the above-described features, the present invention provides window area setting means for setting a plurality of window areas having a predetermined shape so as to cover the entire area to be divided with respect to the area to be divided. And a set of two different partial images from the set of partial images included in each window region, and the similarity or dissimilarity between the partial images of each set is extracted from each partial image A similarity calculation means for calculating based on a quantity, an order relation determining means for determining an order relation between each set of partial images based on the similarity or dissimilarity, and each partial image set The order relation determined in step (2) is saved, and mapping means for mapping the partial images on the metric space and the points mapped on the metric space form a cluster based on the distance or distribution density. Clustering means and each An image division processing system comprising: an area dividing unit that obtains a partial image corresponding to a point belonging to a raster by inverse mapping of the mapping and integrates the partial images as one area. is there.

  In the image division processing system according to the present invention, the mapping means performs mapping based only on the order relationship determined between the sets of partial images.

  In the image division processing system of the present invention, the mapping means performs natural numbering on each partial image set based on the order relationship determined between the partial image sets, and also sets the distance from the partial image set. A set of two different points is generated from a set of points mapped in the space, and the order relation is determined by the distance between the two points for those sets, and according to this order relation, a number by a natural number is assigned to these sets. A value obtained by adding the squares of the difference between the number assigned to the set of partial images and the number assigned to the set of points where they are mapped to the metric space for all the sets of partial images. The mapping is performed so that is minimized.

  In the image division processing system of the present invention, the mapping means calculates a distance between two points of a set of points where the set of partial images is mapped to the metric space, and an auxiliary distance in the set of mapped points. The auxiliary distance of the distance between the two points is calculated so that the order relation determined by the magnitude relation of the two coincides with the order relation determined between the sets of partial images, and in the set of points to be mapped The mapping is performed such that the sum of the squares of the differences between the distances between the two points and the auxiliary distances is minimized for all the pairs of points to be mapped.

  In the image division processing system of the present invention, the similarity or dissimilarity between each set of partial images is obtained by calculating the positional information of each pixel on each partial image and the hue or gray value (or scalar value) on the pixel. It is defined as a functional of the given function.

  In the image division processing system of the present invention, the similarity or dissimilarity between each set of partial images is any one of a difference image, an optical flow image, and a stereo image generated by image difference or matching. In this case, it is defined as a functional of a function that gives position information of each pixel on each partial image and a difference value, vector value, or distance value on the pixel. Here, the difference is a difference in hue or a difference in gray value, and can assume a negative value. The optical flow corresponds to a velocity field and is represented by a two-dimensional or three-dimensional vector.

  In the image division processing system according to the present invention, the similarity or dissimilarity between each set of partial images is a general function of the histogram function for the positional information of each pixel on each partial image and the hue or gray value on the pixel. It is defined as a function.

  In the image division processing system of the present invention, the degree of similarity or dissimilarity between each set of partial images is calculated by adding the positional information of each pixel on each partial image and the function that gives the hue or gray value on the pixel. It is defined as a functional related to a vector value function obtained by applying a differential operator related to the spatial coordinates of.

  In the image division processing system of the present invention, the degree of similarity or dissimilarity between each set of partial images is calculated by adding the positional information of each pixel on each partial image and the function that gives the hue or gray value on the pixel. It is defined as a functional related to a function obtained by applying an integral operator related to the spatial coordinates of.

  In the image division processing system of the present invention, the degree of similarity or dissimilarity between each set of partial images is calculated by adding the positional information of each pixel on each partial image and the function that gives the hue or gray value on the pixel. A second-order tensor obtained by applying an integral operator related to the spatial coordinates on the image to each component of the second-order tensor value function that takes a vector value function obtained by performing a differential operator on the spatial coordinates of It is defined as a functional related to a value function (structure tensor).

  In the image division processing system of the present invention, the similarity or dissimilarity between each set of partial images is a function of giving positional information of each pixel on each partial image and hue or black and white gray value on the pixel. It is defined as a functional related to an autocorrelation function.

  In the image division processing system of the present invention, the degree of similarity or dissimilarity between each set of partial images is calculated by adding the positional information of each pixel on each partial image and the function that gives the hue or gray value on the pixel. It is defined as a functional related to an autocorrelation function related to a vector value function obtained by applying a differential operator related to the spatial coordinates of

  In the image division processing system of the present invention, the clustering means performs clustering without specifying the total number of clusters in advance.

  In the image segmentation processing system of the present invention, the clustering means is a process for maximizing likelihood or log likelihood using a mixed probability distribution, a process for maximizing a posteriori probability, or an expected value based on a pre-distribution given in advance. Any one of the processes of calculating is performed.

  In the image segmentation processing system according to the present invention, the clustering unit is configured such that each Gaussian component determined by applying a maximum likelihood estimation method, a posteriori probability maximization method, or a Bayesian estimation method for a mixed Gaussian distribution represents one class, The cluster to which the partial image belongs is defined as a Gaussian component that minimizes the Mahalanobis distance for each Gaussian component of the point in the metric space corresponding to the partial image.

  In the image division processing system of the present invention, the clustering means selects a two-point set starting from one cluster including all data as an initial state, and having the maximum distance between two points belonging to the same cluster, and the distance. Is equal to or greater than a predetermined threshold, the cluster is divided into two, the selected two points are the seeds of each cluster, and all points belonging to the same cluster are It is characterized by applying division clustering that calculates the distance between them and executes the process of classifying them so as to belong to the cluster with the smaller distance until the cluster division condition in each cluster is not satisfied.

  In the image division processing system according to the present invention, the clustering means performs the process by specifying the total number of clusters in advance.

  In the image segmentation processing system of the present invention, the clustering means performs clustering that executes a K-nearest neighbor method or a K-average method.

  As described above, according to the image segmentation processing system of the present invention, in statistical analysis, according to the most objective standard, and against the difference, change, occlusion, and large variation of the environment of the target object in the image. In addition, it is possible to extract truly useful features in image segmentation with high robustness, and by using these feature values, segment the target object region and background region with high robustness. Will be able to.

  In addition, because of its robustness, even if there are differences in image resolution, various statistical or non-statistical noises, and large changes in environmental conditions, the area can be processed without any special preprocessing. The range of applicable problems can be widened, and the processing time for such preprocessing can be shortened.

  Further, it is possible to extract a feature amount that characterizes a spatial structure in a three-dimensional or higher image. Furthermore, even if the image is a tensor value (matrix value) (two-dimensional or three-dimensional or more), it is possible to extract a feature amount that characterizes the image.

  The best mode for carrying out the image division processing system of the present invention will be described below in detail with reference to the accompanying drawings. 1 to 5 are diagrams illustrating embodiments of the present invention. In these drawings, the same reference numerals denote the same components, and the basic configuration and operation are the same. To do.

  FIG. 1 is a functional block diagram schematically showing an internal configuration of an image division processing system constructed as an embodiment of the present invention. The image division processing system includes a recording device 100 for storing image data, a display device 101 for displaying an interface screen for a user, a result of division processing of image data, and a keyboard 102 for receiving operation input from the user. A pointing device 103 such as a mouse, a central processing unit 104 that performs arithmetic processing and control processing necessary for image division processing, a program memory 105 that stores a program necessary for image division processing, and image data 110 that is a target of image division processing Is stored.

  FIG. 2 is a functional block diagram schematically showing various processing programs included in the program memory 105 of the image division processing system shown in FIG. In FIG. 2, the program memory 105 includes an image input processing unit B1, an image processing unit B2, a partial image configuration processing unit B3, an initial feature amount calculation processing unit B4, a dissimilarity calculation processing unit B5, and an order relation storage feature mapping configuration. A processing unit B6, a clustering processing unit B7, an image region division processing unit B8, a region calculation processing unit B9, and a region division image output processing unit B10 are included. Hereinafter, processing in these processing units will be described in detail.

  The image input processing unit B1 acquires image data to be divided from the recording apparatus 100 or the like in the memory 106. In the following, a case where a two-dimensional image is divided will be described. However, the image division processing system of the present invention can similarly execute a division process even for a three-dimensional image.

  When the acquired target image is a color image, the image processing unit B2 performs various filtering processes such as conversion to a gray scale image, luminance normalization, histogram smoothing, size change, and noise reduction. However, these image processes are not necessarily performed, and the following process may be performed using the image data acquired by the image input processing unit B1 as it is. In the prior art, if the luminance normalization processing is not performed here, the following processing will be seriously affected. However, the image division processing system of the present invention does not perform the luminance normalization processing here. The system is highly robust in that no problems occur in the following processing. Here, the acquired image data is color image data represented by RGB values, and the image processing unit B2 does not perform the above-described image processing.

  The partial image configuration processing unit B3 configures a plurality of partial images by setting a window so as to cover the entire image data. A window setting method for image data is shown in FIG. The window setting method shown in FIG. 3 is an example in which the region of interest is the entire image and the window is rectangular. Even if the windows are set so as not to cross each other as shown in FIG. Alternatively, it may be set so that there are those that cross each other as shown in FIG. Further, even if the shape of the window is not a rectangle as shown in FIG. 3, the shape may be arbitrary as long as it can cover the entire region of interest. In the following processing, the window setting method as shown in FIG. 3A is adopted, but the sizes of the partial images are not necessarily the same in principle.

  The initial feature quantity calculation processing unit B4 extracts initial feature quantities such as histograms and edge strengths from the configured partial images. However, the given partial image itself may be used as a feature amount in a special case without performing feature extraction. Here, in order to avoid losing any useful information in the initial feature extraction, it is assumed that the following processing is performed using the given partial image as an initial feature amount as it is.

The dissimilarity calculation processing unit B5 calculates the dissimilarity between two partial images belonging to each group for the entire group composed of two partial images selected so as to be different from each other. As the dissimilarity, any mapping from the entire partial image set onto the ordered set can be used. An appropriate dissimilarity may be given according to the target image. In the following description, it is assumed that real numbers are used as the ordered set. Now, each partial image is represented as C _i (i = 1, 2,..., N). N represents the number of partial images cut out (that is, the number of windows set) in the partial image configuration processing unit B3. A pixel belonging to the partial image C _i is represented by k, a pixel belonging to the partial image C _j is represented by l, and RGB values at the pixel k are represented by R _k , G _k , and B _k , respectively. As the dissimilarity, any function from all the pixel values belonging to each of the two partial images to a real number can be used. For example, as a method of giving a dissimilarity that is highly invariant with respect to translation and rotation of an image area, a function d as shown in equation (1) can be used.

  Alternatively, quantities such as color histogram intersection are often used as dissimilarities. Based on the dissimilarity given in this way, the order relationship is naturally defined by the order relationship defined on the ordered set to which the dissimilarity is mapped on the set of the entire partial image set. . As described above, in the dissimilarity calculation processing unit B5, an order relationship is defined between sets of partial images. An order function δ that gives this order relationship is expressed as in equation (2).

  The order function satisfies the relationship as shown in equation (3).

  Thus, a set of partial images in which an order relationship is defined between any two sets of different partial images is subjected to mapping processing in the order relationship preserving feature mapping configuration processing unit B6.

  The order relationship preserving feature mapping configuration processing unit B6 maps a set of partial images to a set of points in a certain metric space on a one-to-one basis. Here, regarding the mapping, given to the set formed by the whole combination of two different points selected from the set consisting of the entire points in the metric space mapped by the mapping, given based on the distance between the two points, The dissimilarity calculation processing unit B5 calculates the order relationship defined on the set consisting of two different sets of points for the set consisting of two sets of different partial images of the mapping source partial image. Any mapping can be used as long as it is chosen to match the ordering relationship being made. Hereinafter, such a map is referred to as a “feature map”. For example, one such map can be constructed using non-metric multidimensional scaling. In particular, in the following processing, only the order relationship defined on the set consisting of the combination of two different partial images selected from the set consisting of the entire partial images is applied to the non-metric multidimensional scaling method. It follows the method of using the feature map to construct (see Non-Patent Document 1).

To construct a feature map, first, the position coordinates of a point on the metric space where the partial image C _i is mapped are randomly selected as the initial state,

で表す。お互いに異なる２つの部分画像の組から成る集合上における順序関係を順序が小さいものから順に1,2,...,Nと番号をつけるものとする。つまり、ある部分画像組(C_i, C_j)の写像元での順番がmであったとき、順序関数δ(C_i, C_j)の値はmで与えられ、（４）式のように表すことができる。

Represented by Assume that the order relation on the set of two different partial images is numbered 1, 2,..., N in ascending order. In other words, when the order of a partial image group (C _i , C _j ) at the mapping source is m, the value of the order function δ (C _i , C _j ) is given by m, as in equation (4) Can be expressed as

同様に、位置ベクトル

Similarly, the position vector

から選ばれるお互いに異なる２つの位置ベクトルの組合せ全体から成る集合上に、位置ベクトル

A position vector on a set of two combinations of two different position vectors selected from

間の距離

Distance between

の大小関係に基づいて順序関係を定義し、順序が小さいものから順に1,2,...,N番号が付けるものとする。この順序関係を与える順序関数を とおく。つまり、位置ベクトルの組

The order relation is defined based on the magnitude relation of, and 1, 2, ..., N numbers are assigned in order from the smallest order. Let us give an order function that gives this order relation. That is, a set of position vectors

の順序がnであったとき、順序関数

When the order of n is n

の値はnで与えられ、（５）式のように表すことができる。

The value of is given by n and can be expressed as in equation (5).

部分画像組(C_i, C_j)の写像元での順位がmで与えられるとき、写像先での位置座標の組

When the rank at the mapping source of the partial image set (C _i , C _j ) is given by m, the set of position coordinates at the mapping destination

の順位をT_m と置く。つまり、（６）式のように表すことができる。

Put the rank and T _m. That is, it can be expressed as in equation (6).

目的の特徴写像を構成するためには、以下の手順（ア）〜（オ）に従う。

In order to construct the target feature map, the following procedures (a) to (e) are followed.

(A) The matrix components defined in the following equation (7) are calculated for all components other than the diagonal components. (All the diagonal components may be set to 0.) Here, m represents the order of the set of partial images (C _i , C _j ), and T _m is a set of position vectors defined by equation (6).

の順位を表す。

Represents the ranking.

それぞれについて以下の（８）式で与えられる差分ベクトル (B) Position vector

The difference vector given by the following equation (8) for each

を計算する。

Calculate

（８）式におけるμは更新率で小さい正の数である。

In the equation (8), μ is a positive number that is a small update rate.

  (C) The position vector is updated according to the following equation (9) with the difference vector obtained in the procedure (a).

を (D) Updated position vector

The

と置きなおし、以下の（１０）式が満たされるように、位置ベクトル

And the position vector so that the following expression (10) is satisfied:

のスケールを変換する。

Convert the scale.

(E) Repeat steps (a) to (d) until the end condition is satisfied. For example, the processing is repeated until all the off-diagonal components of the matrix D _{i, j} given by the equation (7) are all 0 or a predetermined maximum number of repetitions is reached.

  As described above, the feature map is configured in the order relation storage feature map configuration processing unit B6. The state of this process is shown in FIG. FIG. 4 is an example of a feature map that maps to a two-dimensional Euclidean space. Partial images belonging to a certain area have a low degree of dissimilarity with partial images belonging to the same area, and a large degree of dissimilarity with partial images belonging to other basins. In space, it is mapped to a nearby point and is away from points corresponding to partial images belonging to other areas. The set of partial images and the set of points on the metric space mapped by the feature map are further processed by the clustering processing unit B7.

  The clustering processing unit B7 is a position vector that is an image of the entire partial image based on the feature map calculated by the order relation preserving feature map configuration processing unit B6.

の配置に関するクラスタリングを行う。クラスタリング手法としてはクラスタ数を予め与える必要のない混合ガウス分布などの混合確率分布を用いた最尤推定法のようなクラスタリング手法が良いが、位置ベクトル間の距離を用いたdivisive clusteringなどを用いても良い。

Perform clustering for the placement of. As a clustering method, a clustering method such as a maximum likelihood estimation method using a mixed probability distribution such as a mixed Gaussian distribution that does not require the number of clusters to be given in advance is good. Also good.

は、以下の（１１）式で表されるような混合ガウス分布からランダムに選ばれたN個の点であると見なす。 Position vector placement when using a mixed Gaussian distribution

Is assumed to be N points randomly selected from a mixed Gaussian distribution represented by the following equation (11).

（１１）式において、Jは特徴写像の写像先の距離空間の次元を表し、Lはガウス分布の成分数を表し、

In equation (11), J represents the dimension of the metric space to which the feature map is mapped, L represents the number of components of the Gaussian distribution,

は各ガウス分布成分のそれぞれ平均値ベクトル、共分散行列を表し、Θ_l ^-1 は共分散行列Θ_l の逆行列を表し、|Θ_l|は共分散行列Θ_lの行列式を表し、λ_l は各ガウス分布成分の重みを表し、以下の（１２）式を満たす。

Represents the mean vector and covariance matrix of each Gaussian distribution component, Θ _l ⁻¹ represents the inverse of the covariance matrix Θ _l , | Θ _l | represents the determinant of the covariance matrix Θ _l , and λ _l represents the weight of each Gaussian distribution component and satisfies the following equation (12).

各ガウス分布成分が一つのクラスタを表す。このとき、位置ベクトルの配置

Each Gaussian distribution component represents one cluster. At this time, position vector arrangement

に関する対数尤度関数

Log-likelihood function for

は以下の（１３）式で表される。

Is represented by the following equation (13).

この対数尤度関数

This log-likelihood function

をパラメタ

The parameters

について最適化することで、目的のクラスタ数およびクラスタ位置とサイズとが計算される。最適化手法としては、例えば、Greedy-EMアルゴリズム（J. Verbeek, N. Vlassis, and B. Krose, Neural Computation Vol.15, No.2, pages 469-485, 2003.）が使用できる。位置ベクトル がどのクラスタに属するかは、各ガウス分布成分に関するマハラノビス距離を計算し、もっともマハラノビス距離が小さくなるガウス成分として決めることができる。第 ガウス分布成分で表されるクラスタから位置ベクトル

, The target number of clusters and the cluster position and size are calculated. As an optimization method, for example, the Greedy-EM algorithm (J. Verbeek, N. Vlassis, and B. Krose, Neural Computation Vol. 15, No. 2, pages 469-485, 2003.) can be used. Which cluster the position vector belongs to can be determined as a Gaussian component with the smallest Mahalanobis distance by calculating the Mahalanobis distance for each Gaussian distribution component. Position vector from the cluster represented by the Gaussian distribution component

までのマハラノビス距離は以下の（１４）式で計算される。

The Mahalanobis distance is calculated by the following equation (14).

  FIG. 5A shows an example of the clustering result by the mixed Gaussian distribution. Ellipses enclose points that belong to the same cluster.

  As described above, in the clustering processing unit B7, a position vector that is an image of the entire partial image based on the feature map calculated by the order relation preserving feature map configuration processing unit B6.

の配置に関するクラスタリングを行い、クラスタごとにどの位置ベクトルがそのクラスタに属するかを表すリストを構成することができる。

And a list representing which position vector belongs to the cluster for each cluster.

  In the image area division processing unit B8, based on the list configured by the clustering processing unit B7, for each cluster, a window used for cutting out a partial image that is a reverse mapping image by a feature mapping of position vectors belonging to the cluster Calculate the area of. The set sum of the window areas calculated in this way is calculated and set as an area corresponding to the cluster. In this way, the area belonging to each cluster can be calculated. At this time, if there is an overlap in the window area, the following area calculation processing unit B9 calculates the overlap of the window areas and reclassifies the overlap area.

  The area calculation processing unit B9 is a part that performs processing for calculating an overlapping area when there is an overlapping window area. In general, in an area with overlapping windows, first, the divided areas that have been voted the most are voted by weighting the size of the window area to which the overlapping window is determined to belong. A method of determining that the image belongs to the area is taken. In such voting, when multiple divided areas with the same value appear, select one uniformly and randomly from them, or add one special divided area, which is an undecidable area And belong to the divided area. In addition, when using the window area | region which does not overlap, the process by area | region calculation process part B9 can be abbreviate | omitted.

  The area division image output processing unit B10 integrates the areas belonging to each cluster based on the clustering result by the image area division processing part B8 or the reclassification result by the area calculation processing part B9, and displays display data of the area division result of the target image Is generated and output. In this way, the display device 101 of the image division processing system displays the result of dividing the image data as shown in FIG. 5B.

  While the image division processing system of the present invention has been described with reference to specific embodiments, the present invention is not limited to these. A person skilled in the art can make various changes and improvements to the configurations and functions of the invention according to the above-described embodiments or other embodiments without departing from the gist of the present invention.

  The image segmentation processing system of the present invention detects and recognizes an object in an image by segmenting the measured image in a medical system, a monitoring system, an inspection system, an intelligent transportation system (ITS), and the like. It can be used as a system.

It is a functional block diagram which shows roughly the internal structure of the image division processing system constructed | assembled as one Embodiment of this invention. It is a functional block diagram which shows roughly the various processing programs contained in the program memory of the image division | segmentation processing system shown in FIG. It is a figure which shows the method of setting the window with respect to image data by the partial image structure process part of the image division processing system shown in FIG. It is a figure which shows a mode that a feature map is comprised in the order relation preservation | save feature map structure process part of the image division | segmentation processing system shown in FIG. (A) is a figure showing the example of the clustering result by mixed Gaussian distribution. (B) is a diagram showing the result of dividing the image data into regions.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Recording device 101 Display device 102 Keyboard 103 Pointing device 104 Central processing unit 105 Program memory 106 Data memory 110 Image data

Claims (19)

A window area setting means for setting a plurality of window areas having a predetermined shape so as to cover the entire area to be divided with respect to the area to be divided of the image;
A set of two different partial images is generated from the set of partial images included in each window region, and the similarity or dissimilarity between the partial images of each set is used as a feature amount extracted from each partial image. Similarity calculation means for calculating based on;
Order relation determining means for determining an order relation between the sets of partial images based on the similarity or dissimilarity;
Mapping means for mapping each partial image on a metric space in such a manner that the order relationship determined between the partial image sets is stored;
Clustering means for forming a cluster based on the distance or distribution density of the points mapped on the metric space;
An image division processing system comprising: an area dividing unit that obtains partial images corresponding to points belonging to each cluster by inverse mapping of the mapping, and integrates the partial images as one area.   The image division processing system according to claim 1, wherein the mapping unit performs mapping based only on an order relationship determined between the sets of partial images.   The mapping means numbers each partial image set with a natural number based on an order relationship determined between the partial image sets, and from a set of points mapped from the partial image set to the metric space. Generating sets of two points different from each other, determining an order relationship for the sets based on the distance between the two points, and numbering the sets with natural numbers according to the order relationship, So that the sum of the squares of the difference between the number attached to the number and the number assigned to the set of points where they are mapped to the metric space is minimized for all the sets of partial images, The image division processing system according to claim 1, wherein mapping is performed.   The mapping means calculates a distance between two points of a set of points where the set of partial images is mapped to the metric space, and an order relationship determined by a magnitude relationship of auxiliary distances in the set of mapped points. The auxiliary distance of the distance between the two points is calculated so as to coincide with the order relationship determined between the sets of the partial images, and the distance between the two points and the auxiliary distance in the set of mapped points is calculated. 2. The image division processing system according to claim 1, wherein mapping is performed such that a value obtained by adding a square of the difference to a set of all the mapped points is minimized.   The similarity or dissimilarity between the partial images of each set is defined as a functional of a function that gives positional information of each pixel on each partial image and a hue or gray value on the pixel. The image division processing system according to any one of claims 1 to 4.   The degree of similarity or dissimilarity between each set of partial images is the difference between the images, if the image is a difference image generated by matching or matching, an optical flow image, or a stereo image. The image division according to any one of claims 1 to 4, wherein the image division is defined as a functional of a function that gives positional information of each pixel and a difference value, a vector value, or a distance value on the pixel. Processing system.   Similarity or dissimilarity between each set of partial images is defined as a functional of a histogram function with respect to positional information of each pixel on each partial image and hue or gray value on the pixel. The image division processing system according to any one of claims 1 to 4.   The similarity or dissimilarity between each set of partial images is obtained by applying a differential operator related to the spatial coordinates on the image to the function that gives the position information of each pixel on each partial image and the hue or gray value on the pixel. The image division processing system according to claim 5, wherein the image division processing system is defined as a functional related to a vector value function obtained by performing the processing.   The similarity or dissimilarity between each set of partial images is obtained by applying an integral operator related to the spatial coordinates on the image to the function that gives the position information of each pixel on each partial image and the hue or gray value on the pixel. The image division processing system according to claim 5, wherein the image division processing system is defined as a functional related to a function obtained by performing the processing.   The similarity or dissimilarity between each set of partial images causes a function that gives the position information of each pixel on each partial image and the hue or gray value on the pixel to be a differential operator with respect to the spatial coordinates on the image. Is defined as a functional related to the second-order tensor value function obtained by applying an integral operator related to the spatial coordinates on the image to each component. The image division processing system according to claim 5, wherein:   The similarity or dissimilarity between each set of partial images is defined as a functional related to the autocorrelation function of the function that gives the position information of each pixel on each partial image and the hue or black and white gray value on the pixel. The image division processing system according to claim 5, wherein   The similarity or dissimilarity between each set of partial images is obtained by applying a differential operator related to the spatial coordinates on the image to the function that gives the position information of each pixel on each partial image and the hue or gray value on the pixel. 6. The image division processing system according to claim 5, wherein the image division processing system is defined as a functional related to an autocorrelation function related to a vector value function obtained by performing the processing.   The image dividing processing system according to claim 1, wherein the clustering unit performs clustering without specifying the total number of clusters in advance.   The clustering means is one of processing for maximizing likelihood or logarithmic likelihood using a mixed probability distribution, processing for maximizing posterior probability, or processing for calculating an expected value based on a predistribution given in advance. The image division processing system according to claim 13, wherein:   The clustering means assumes that each Gaussian component determined by applying a maximum likelihood estimation method, a posteriori probability maximization method or a Bayesian estimation method for a mixed Gaussian distribution represents one class, and a cluster to which each partial image belongs is represented by the partial image. The image division processing system according to claim 14, wherein the image division processing system is defined as a Gaussian component that minimizes the Mahalanobis distance for each Gaussian component of a point in a metric space corresponding to.   The clustering means starts from one cluster including all data as an initial state, selects a two-point set having the maximum distance between two points belonging to the same cluster, and the distance is equal to or greater than a predetermined threshold. The cluster is divided into two, the two selected points are the seeds of each cluster, and for all points that belonged to the same cluster, the distance between these cluster seeds is calculated, and the distance is 14. The image division processing system according to claim 13, wherein division clustering is executed in which the process of classifying so as to belong to a smaller cluster is performed until the cluster division condition in each cluster is not satisfied.   The image dividing processing system according to claim 1, wherein the clustering unit performs the designation by specifying the total number of clusters in advance.   18. The image segmentation processing system according to claim 17, wherein the clustering means performs clustering that executes a K-nearest neighbor method.   18. The image segmentation processing system according to claim 17, wherein the clustering means performs clustering that executes a K-means method.

Images