JP2004258749A - Method and device for clustering feature vector of image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To classify feature vectors extracted from image data by clustering by specifying the level of optimal classification matching the inputted image data from the image data themselves without the need of inputting any parameter to designate the level of desired classification. <P>SOLUTION: Low resolution images whose resolutions are different stepwise are derived from an original image, and the initial values of the respective components of the representative vectors of respective clusters and the initial value of a distortion threshold being the index of the level of optimal classification in an image whose resolution is the second lowest are calculated from the lowest resolution image whose resolution is the lowest. Then, processing to classify the feature vectors extracted from the image whose resolution is the second lowest and processing to update the representative vectors and the distortion threshold for the optimal classification in the subsequent image are successively repeated by using the initial values. Finally, the classification of the plurality of feature vectors extracted from the original image by clustering is carried out in the level of the optimal classification. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a method and apparatus for classifying a plurality of feature vectors extracted from an image, and in particular, a plurality of feature vectors extracted from an image using a clustering technique in a feature vector space belong to each other. The present invention relates to a method and apparatus for classifying a plurality of clusters.
[0002]
[Prior art]
As a data classification method when features of observation data or image data can be expressed as feature vectors having n parameters representing the features, a plurality of feature vectors are mapped to an n-dimensional feature vector space, and the features A so-called clustering method for classifying these feature vectors into a plurality of clusters based on the distribution in the vector space is conventionally known. A cluster is literally a “lumps” of feature vectors in the feature vector space, and feature vectors that are close to each other are classified into the same cluster using Euclidean distance in the feature vector space as an index. That is, feature vectors belonging to the same cluster represent features that are similar to each other.
[0003]
In general, in the clustering process, a constant value parameter indicating how far the classification is to be performed must be specified prior to the start of the process. For example, the final number of clusters and the maximum allowable distance between the cluster center and feature vectors belonging to the clusters can be specified as such parameters. Even when using a self-converging algorithm that does not fix the number of clusters, etc., some constant value parameter that indicates how far the classification is performed, such as a parameter related to cluster expansion and a parameter related to the distance between clusters. The necessity to specify is not eliminated (for example, see Non-Patent Document 1).
[0004]
[Non-Patent Document 1]
Nagao Makoto, "Image recognition theory", first edition, Corona, February 15, 1983
, P. 120-126
[0005]
[Problems to be solved by the invention]
The degree to which classification should be performed in the clustering process strongly depends on the data contents. In particular, when image data with high diversity is targeted, the appropriate classification level varies greatly depending on the image data. For example, in order to perform region division processing for dividing one image into image regions corresponding to imaging target elements such as “sky”, “sea”, and “building” included in the image, from each pixel of the image When classifying extracted feature vectors by clustering processing, etc., the number of elements to be captured included in each image varies, and it is impossible to set a uniform standard for how much to classify It is. That is, if the image includes only “sky” and “sea” as elements to be photographed, it may be divided into two image regions, so that the degree of classification is coarse, but “building” and “tree” are sufficient. For images including many elements to be photographed such as “soil”, a finer classification is required. In this case, if the standard of the degree of uniform classification according to the former image is adopted, sufficient classification cannot be performed for the latter image, and the uniform standard according to the latter image cannot be achieved. If it is adopted, the former image, in which only “sky” and “sea” are actually photographed, is unnecessarily divided into many image areas.
[0006]
Therefore, in the clustering process for image data, regardless of what image data is input as a classification target, the optimum classification level for the image data is specified from the image data itself, and stable classification performance is realized. A method is strongly desired.
[0007]
In view of such circumstances, the present invention specifies the optimum classification level from the input image data itself and does not require input of a parameter that specifies the preferable classification level, and extracts the feature vector from the image data. It is an object of the present invention to provide a clustering method and apparatus for performing classification.
[0008]
[Means for Solving the Problems]
That is, in the first image feature vector clustering method according to the present invention, a step of deriving a low-resolution image composed of a plurality of pixels having different resolutions in stages from an original image, and the plurality of low-resolution images A step of obtaining an initial value of each component of the representative vector and an initial value of a distortion threshold from the lowest resolution image having the lowest resolution, a step of extracting a plurality of feature vectors from an image having the next lowest resolution, and the above distortion threshold , Using the representative vector as a measure of the optimum classification degree, classifying the plurality of feature vectors, updating the representative vector and the distortion threshold, and a plurality of features extracted from the original image Characterized in that it includes a step of repeating the updating step from the extracting step until the classification according to the optimum classification degree of the vector is obtained. It is the law.
[0009]
In addition, the first image feature vector clustering apparatus according to the present invention includes: a unit for deriving a low-resolution image including a plurality of pixels having different resolutions in stages from an original image; Means for obtaining an initial value of each component of the representative vector and an initial value of a distortion threshold from the lowest resolution image of the lowest resolution; means for extracting a plurality of feature vectors from an image with the next lowest resolution; and the above distortion threshold As an index of the degree of optimal classification, means for classifying the plurality of feature vectors based on the representative vector, means for updating the representative vector and the distortion threshold, and a plurality of features extracted from the original image Means for repeatedly operating the updating means from the extracting means until classification according to the optimum degree of vector classification is required. It is a device for the butterflies.
[0010]
Here, in the present invention, the “low resolution image” is an image composed of a plurality of pixels having a smaller number of pixels than the original image. Based on the original image, Gaussian pyramid, linear interpolation, spline interpolation, wavelet transform, etc. It refers to what is sequentially obtained by the image reduction processing used. Different reduction ratios may be applied to the vertical and horizontal directions of the image.
[0011]
In the present invention, a “feature vector” refers to a vector extracted from an image and including a plurality of parameters (hereinafter referred to as “feature amounts”) indicating the features of the image as components. The feature vector is typically extracted for each pixel constituting the extraction target image. However, the feature vector may be extracted for each block made up of several pixels, or linear interpolation, Gaussian pyramid, etc. may be added to the extraction target image. The image may be extracted from each pixel of the reduced image subjected to the image reduction process. As the feature amount that is a component of the feature vector, for example, a feature amount indicating a color feature, a brightness feature, a texture feature, depth information, an edge feature included in the image, or the like can be used.
[0012]
Further, in the present invention, the “representative vector” refers to a vector representing each cluster defined in the feature vector space, and one representative vector is assigned to each cluster. Here, the “feature vector space” refers to a space having the coordinates of each component of the feature vector. For example, when the feature vector is a vector having the luminance component of each pixel and two color difference components in the YCC color system as components, the three-dimensional YCC color system space is the feature vector space.
[0013]
Further, in the present invention, the “distortion threshold value” is a value serving as a reference for how much classification is performed in each low-resolution image or original image.
[0014]
The second feature vector clustering method according to the present invention includes a step of deriving a low-resolution image including a plurality of pixels having different resolutions in stages from an original image, and a step of extracting the plurality of low-resolution images. Of these, the step of obtaining the initial value of each component of the representative vector and the initial value of the distortion threshold from the lowest resolution image with the lowest resolution, and extracting a plurality of feature vectors from the current image with the next lowest resolution image as the current image A step of mapping the plurality of feature vectors to a feature vector space, and classifying the plurality of feature vectors into clusters represented by each of the representative vectors in the feature vector space. Obtaining a provisional classification, updating a representative vector based on a feature vector included in each of the clusters, and Determining a distortion parameter indicating the provisional classification of the distortion; comparing the distortion parameter with a distortion threshold; and if the distortion parameter is greater than the distortion threshold, adding a new vector to the representative vector; Until the parameter becomes smaller than the distortion threshold, the process of adding the new vector from the process of obtaining the provisional classification is repeated, and the process of obtaining the final classification in the current image and the provisional classification in the current image A step of updating a distortion threshold based on any one of or a final classification, and extracting the plurality of feature vectors until the original image is a current image and a final classification in the original image is obtained. To repeating the step of updating the distortion threshold value.
[0015]
According to the second image feature vector clustering apparatus of the present invention, there is provided a means for deriving a low-resolution image composed of a plurality of pixels having different resolutions from the original image, and a plurality of the low-resolution images. Means for obtaining the initial value of each component of the representative vector and the initial value of the distortion threshold from the lowest resolution image of the lowest resolution, and extracting a plurality of feature vectors from the current image with the next lowest resolution image as the current image Means for mapping the plurality of feature vectors to the feature vector space, and classifying the plurality of feature vectors into clusters represented by each of the representative vectors in the feature vector space, Means for obtaining a provisional classification, means for updating the representative vector based on the feature vector included in each of the clusters, and Means for determining a distortion parameter indicating a provisional classification distortion, and means for comparing the distortion parameter with a distortion threshold, and if the distortion parameter is greater than the distortion threshold, adding a new vector to the representative vector; Until the parameter becomes smaller than the distortion threshold, the means for adding the new vector from the means for obtaining the provisional classification is repeatedly operated to obtain the final classification in the current image, and the provisional in the current image. Based on one of the final classifications or the final classification, the means for updating the distortion threshold and the above feature vectors are extracted until the original image is the current image and the final classification in the original image is required It is an apparatus characterized by comprising means for repeatedly operating the means for updating the distortion threshold value from the means for performing the above.
[0016]
Here, in the present invention, the “distortion parameter” is a parameter that is compared with a distortion threshold value, which is an index that represents the roughness of provisional classification of feature vectors by clusters. The greater the number of clusters and the finer the classification, the smaller the distortion parameter. Note that the second image feature vector clustering method and apparatus according to the present invention adds a new vector to the representative vector when the distortion parameter is larger than the distortion threshold, and the distortion parameter is larger than the distortion threshold. Although the classification process in the current image is repeated until it becomes smaller, a method and an apparatus in which handling when the distortion parameter is equal to the distortion threshold are also included in the scope of the present invention. That is, when the distortion parameter is equal to or greater than the distortion threshold, a new vector is added to the representative vector, and the classification process in the current image is repeated until the distortion parameter becomes smaller than the distortion threshold. If it is larger, a new vector is added to the representative vector, and any one that repeats the classification process in the current image until the distortion parameter becomes equal to or less than the distortion threshold value falls within the scope of the present invention.
[0017]
Here, the second clustering method according to the present invention may further include a step of updating the provisional classification based on the representative vector between the step of updating the representative vector and the step of obtaining the distortion parameter. . Similarly, the second clustering apparatus according to the present invention may further include means for updating the provisional classification based on the representative vector updated by the means for updating the representative vector.
[0018]
Further, the step or means for obtaining the provisional classification classifies each of the plurality of feature vectors into a cluster represented by a representative vector that is closest to the feature vector in the feature vector space among the representative vectors. It may be a thing. In this case, the step or means for updating the representative vector may use a vector representing the centroid of the feature vector included in each cluster as a new representative vector in the feature vector space. Further, the distortion parameter is a value indicating the spread of the cluster having the largest spread in the feature vector space among the clusters, and the step or means for adding the new vector represents the cluster to which the cluster belongs. The same vector as the feature vector farthest from the representative vector may be added to the representative vector as a new vector.
[0019]
Here, the “distance” in the feature vector space typically refers to the Euclidean distance between two vectors in the feature vector space, but any index that appropriately represents the degree of proximity of the two vectors in the feature vector space. Suppose that it is not limited to the Euclidean distance. As the “value indicating the spread” of the cluster in the feature vector space, for example, the maximum value or average value of the distance between each feature vector belonging to the cluster and the representative vector of the cluster can be used.
[0020]
In addition, the step or means for updating the distortion threshold value described above may use any one of the distortion parameters obtained for obtaining the final classification in the current image as a new distortion threshold value.
[0021]
Further, when obtaining the initial value of each component of the representative vector and the initial value of the distortion threshold, one original representative vector is derived from the lowest resolution image consisting of n pixels, and the n pixels The distance in the feature vector space between the n feature vectors extracted from each and the original representative vector is obtained, the feature vector indicating the maximum distance is set as the representative vector candidate, and the highest candidate ranking is given, followed by The distance between each of the feature vectors in the feature vector space between the original representative vector and the representative vector candidate closest to the feature vector is obtained, and the feature vector indicating the maximum distance is added to the representative vector candidate and The process of assigning a higher candidate rank and repeating the process of assigning the next higher candidate rank is repeated several times, and the representative vector is The value of each component of the top few feature vectors among the candidates is set as the initial value of each component of the representative vector, and the value indicating the spread in the feature vector space of the n feature vectors is set as the initial value of the distortion threshold. Also good. In this case, as the top several feature vectors, up to the feature vector having the maximum change amount of the maximum distance when added to the representative vector candidate may be selected.
[0022]
Here, the “original representative vector” is one vector representing the lowest resolution image, and for example, a vector whose component is an average value of feature amounts extracted from each pixel of the lowest resolution image is used. obtain.
[0023]
【The invention's effect】
An image feature vector clustering method and apparatus according to the present invention obtain initial values of respective components of a representative vector and initial values of distortion threshold values based on a minimum resolution image derived from an input original image itself, The value is used to sequentially determine the optimum classification of feature vectors in the next lowest resolution image, and finally to determine the classification of feature vectors extracted from the original image. The feature vector extracted from the original image can be classified according to the optimum classification according to the content of the original image without requiring input of the designated parameter. Therefore, even if any original image is input, stable classification performance can be realized, and this is extremely effective when performing continuous segmentation on a large number of images including various elements to be photographed. is there.
[0024]
The image feature vector clustering method and apparatus according to the present invention sequentially performs classification processing on a low-resolution image, that is, an image with a reduced number of feature vectors, and the classification processing for each low-resolution image includes distortion parameters. Is cut off when the value becomes smaller than the distortion threshold value, and the process proceeds to an image with the next lowest resolution, so that the amount of calculation does not become excessive as compared with the conventional clustering process.
[0025]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.
[0026]
FIG. 1 is a flowchart showing a procedure of clustering processing according to one embodiment of the present invention. The purpose of this clustering process is to classify the feature vectors extracted from each pixel of the original image into a plurality of clusters. The original image is “sky”, “sea”, “building”. It can be used for area division processing that divides into a plurality of image areas corresponding to the elements to be imaged. In the present embodiment, the original image is a digital photographic image having 1024 × 1024 pixels, and the feature vector is a three-dimensional vector having a luminance component and two color difference components expressed in the YCC color system. Suppose there is.
[0027]
When the original image is input, first, in step 10 of FIG. 1, a plurality of low resolution images having different resolutions are derived step by step. In this embodiment, as shown in FIG. 2, starting from the original image, low resolution images with half the vertical and horizontal pixels are sequentially obtained, and finally the ninth low resolution image, that is, Assume that a low-resolution image having 2 × 2 pixels is obtained. This low resolution image having 2 × 2 pixels is the lowest resolution image in this embodiment.
[0028]
Any method of image reduction processing for obtaining each of these low resolution images may be used, but in this embodiment, reduction processing using a Gaussian pyramid is used. The reduction process using the Gaussian pyramid is the step of deriving a blurred image by applying a Gaussian filter to the image to be reduced, and picking up every other pixel in such a blurred image and calculating the number of vertical and horizontal pixels respectively. This is a process of sequentially deriving low-resolution images having different resolutions by sequentially repeating the steps of deriving an image that has been reduced to half. The Gaussian filter used is, for example, as shown in FIG. 3 if the filter size is 5 × 5. Regarding the processing of the image edge, for example, when a 5 × 5 size filter shown in FIG. 3 is used, appropriate pixel values for two rows or two columns on each side of the image to be reduced are used. For convenience, a blurred image having the same size as the original image to be reduced can be derived. As the “appropriate pixel value” of the pixel to be added, a zero value, a value obtained by repeating the pixel value at the image end, or the like can be used.
[0029]
When the derivation of the low resolution image is finished, next, in step 12 of FIG. 1, the initial representative vector, that is, the initial value of each component of the representative vector, is derived based on the lowest resolution image (2 × 2 pixels). The initial representative vector derivation process will be described in detail below with reference to the flowchart of FIG. 4 and the conceptual diagram of FIG.
[0030]
First, in step 40 of FIG. 4, one original representative vector is derived from the lowest resolution image. In the present embodiment, one vector having the average value of each component of the feature vector extracted from the four pixels forming the lowest resolution image as the component is defined as the original representative vector. For example, in the feature vector space shown in FIG. ₁ , OB ₂ , OB ₃ And OB ₄ Is a feature vector extracted from four pixels forming the lowest resolution image, the original representative vector is OA.
[0031]
Next, in step 42 in FIG. 4, the distance between each of the above four feature vectors and the original representative vector is calculated. In the present embodiment, this distance is assumed to be the Euclidean distance in the feature vector space. In the example of FIG. 5A, the calculated distances are d ₁ , D ₂ , D ₃ And d ₄ It is.
[0032]
Subsequently, in step 44, the feature vector indicating the maximum distance among the distances calculated in step 42 is set as the first representative vector candidate in the candidate ranking. In the example of FIG. 5A, the distance d ₁ , D ₂ , D ₃ And d ₄ The maximum distance is d ₁ Therefore, the feature vector indicating the maximum distance is the vector OB ₁ So vector OB ₁ Is the first representative vector candidate in the candidate ranking, that is, the first representative vector candidate.
[0033]
Next, in step 46, the remaining feature vectors or three vectors OB ₂ , OB ₃ And OB ₄ , The original representative vector OA and the representative vector candidate OB ₁ The distance to the one closest to the feature vector is identified. In this example, as shown in FIG. ₃ And OB ₄ , The distance from the original representative vector OA is the representative vector candidate OB ₁ Distance d as shown in the figure. ₁ 'And d ₂ 'Is specified. Vector OB ₂ For representative vector candidate OB ₁ The distance d is smaller than the distance to the original representative vector OA. ₃ 'Is specified.
[0034]
Subsequently, in step 48, the feature vector indicating the maximum distance among the distances calculated in step 46 is set as a representative vector candidate of the next highest candidate rank. In the example of FIG. 5B, the maximum distance d ₁ The feature vector indicating 'is the vector OB ₂ So vector OB ₂ Is the second representative vector candidate in the second candidate ranking.
[0035]
Next, in step 50, it is confirmed whether there is a feature vector that has not yet been assigned a candidate rank. Here, the feature vector OB is still ₂ And OB ₄ Since no candidate rank has been assigned to, the process returns to step 46. In step 46 performed again here, as shown in FIG. ₄ For the original representative vector OA and representative vector candidate OB ₁ And OB ₃ Since the vector having the closest distance is the original representative vector OA, the distance d to the original representative vector OA as shown in the figure. ₁ '' Is specified. Vector OB ₂ For representative vector candidate OB ₁ Distance d ₂ '' Is specified. In the following step 48, the distance d ₁ '' Is d ₂ Is larger than '', so the feature vector OB ₄ Is the third representative vector candidate in the third candidate ranking.
[0036]
In this way, step 46 to step 50 in FIG. 4 are repeated until all four feature vectors are given candidate ranks. ₁ , OB ₃ , OB ₄ , OB ₂ Are added to the representative vector candidates in this order, and the first to fourth candidate rankings are given.
[0037]
Subsequently, in step 52 of FIG. 4, according to the above-described candidate ranking, the top several feature vectors among the representative vector candidates are set as initial representative vectors. In this embodiment, when added to the representative vector candidates, Up to the feature vector having the maximum amount of change in the maximum distance is set as the initial representative vector. That is, as shown in FIG. ₁ Is added to the representative vector candidate as the first representative vector candidate, the maximum distance is d ₁ To d ₁ And then the feature vector OB ₃ Is added to the representative vector candidate, the maximum distance is d ₁ 'To d ₁ '' Has changed. This change in the maximum distance is illustrated in FIG. In this example, the amount of change in the maximum distance is the second highest feature vector OB. ₃ Is added to the representative vector candidates, the feature vectors up to the second candidate rank, that is, the vector OB ₁ And OB ₃ Are the initial representative vector OR ₁ And OR ₂ It is said.
[0038]
Here, in the present embodiment, the initial representative vector is selected on the basis of the change amount of the maximum distance in the feature vector space as described above. However, the change amount of the average distance or the like may be used as a reference. Alternatively, since the number of pixels is considerably reduced in the lowest resolution image (four pixels in this embodiment), all of the feature vectors extracted from each pixel of the lowest resolution image may be used as the initial representative vector. In short, as long as the initial value of each component of the representative vector corresponding to the original image is obtained from the minimum resolution image itself derived from the original image by a predetermined process without requiring input of any parameters from the outside. It belongs to the scope of the present invention.
[0039]
Returning to FIG. 1, subsequently, in step 14, the initial value of the distortion threshold is derived also based on the lowest resolution image. The initial value of the distortion threshold is a value that serves as a reference for how much classification is performed in the next low-resolution image, that is, the eighth low-resolution image having 4 × 4 pixels. The initial value of the distortion threshold is preferably a value indicating the spread in the feature vector space of the feature vectors extracted from n pixels (four in the present embodiment) having the lowest resolution, but is not necessarily limited thereto. As in the case of the initial value of each component of the representative vector described above, it is possible to respond to the original image by performing a predetermined process from the lowest resolution image itself derived from the original image without requiring input of any external parameters. If the initial value of the distortion threshold is determined, it belongs to the scope of the present invention. In the present embodiment, the maximum value of the distance between the four feature vectors extracted from each pixel of the lowest resolution image and the original representative vector, that is, the distance d in FIG. ₁ Is the initial value of the distortion threshold D. Instead, the maximum distance (ie d) in the lower hierarchy of FIG. ₁ ', D ₁ '' Etc.) and distance d ₁ , D ₂ , D ₃ And d ₄ An average value or the like may be used.
[0040]
Next, in step 16 of FIG. 1, an image with the next lowest resolution, that is, an eighth low-resolution image having 4 × 4 pixels is set as the current image, and the classification process for the current image is started. Hereinafter, the classification process in the current image having 4 × 4 pixels will be described in order with reference to FIGS.
[0041]
First, in step 18 in FIG. 1, a feature vector having a luminance component and two color difference components in the YCC color system as components is extracted from each of the 16 pixels forming the current image.
[0042]
Next, in step 20, the 16 feature vectors are mapped into the feature vector space. Let the end points of 16 feature vectors be C ₁ To C ₁₆ In this example, it is assumed that the distribution in the feature vector space is a distribution as shown in FIG.
[0043]
Subsequently, in step 22, the 16 feature vectors are provisionally classified into clusters represented by the representative vectors. The current representative vector is the initial representative vector OR derived from the lowest resolution image. ₁ And OR ₂ Therefore, 16 feature vectors are classified into two clusters represented by these. In this embodiment, it is assumed that each feature vector is classified into a cluster represented by a representative vector having a shorter distance on the basis of the Euclidean distance in the feature vector space. As a result, the provisional classification is as shown in FIG.
[0044]
Next, in step 24, based on the feature vectors included in each of the two clusters, the representative vectors are updated to those that more appropriately represent these clusters. In this embodiment, as shown in FIG. 7B, a representative vector OR is added to a vector indicating the center of gravity of the feature vector belonging to each cluster. ₁ And OR ₂ Update.
[0045]
Subsequently, in step 26, a distortion parameter indicating the current provisional classification distortion is calculated. This distortion parameter is preferably a value indicating the spread of the cluster having the largest spread in the feature vector space among the clusters in the current provisional classification. In this embodiment, the maximum value of the Euclidean distance in the feature vector space between the representative vector and the feature vector included in the cluster represented by the representative vector is used as the distortion parameter. In this example, as shown in FIG. ₁ And the feature vector OC belonging to the cluster ₁ Distance d _max Is the maximum, so this d _max Is a distortion parameter. Note that an average value or the like may be used as the distortion parameter instead of the maximum value.
[0046]
Next, in step 28, it is checked whether or not the distortion parameter obtained in step 26 is equal to or less than the distortion threshold value D. The current distortion threshold D is an initial value of the distortion threshold obtained from the lowest resolution image. In this example, the distortion parameter d at this stage _max Is larger than the distortion threshold value D, and therefore the processing shown in FIG.
[0047]
In step 30, a new vector is added to the representative vector in order to increase the number of clusters by 1 to obtain a finer provisional classification. In this embodiment, as shown in FIG. 8A, the feature vector having the longest distance from the representative vector representing the cluster to which it belongs, that is, the maximum distance d described above. _max Feature vector OC indicating ₁ The same vector as the new representative vector OR ₃ Added as.
[0048]
Thereafter, the processing of steps 22 to 28 is performed again based on the three representative vectors. That is, in step 22, the feature vector OC is divided into three clusters represented by each representative vector on the basis of the Euclidean distance in the feature vector space. ₁ To OC ₁₆ Are classified (see FIG. 8A), and in step 24, the representative vector OR ₁ To OR ₃ Is updated to a vector indicating the center of gravity of each cluster (see FIG. 8B), and in step 26, the maximum distance d ′ between the representative vector and the feature vector included in the cluster represented by the representative vector. _max Is derived as a distortion parameter indicating the distortion of the new provisional classification (see FIG. 8C). In this example, a new distortion parameter d ′ _max Is also larger than the distortion threshold D. Therefore, the process proceeds from step 28 to step 30 again, and the maximum distance d ′ _max Feature vector OC indicating ₂ The same vector as the new representative vector OR ₄ Is added to the representative vector (see FIG. 9A).
[0049]
Subsequently, as shown in (a) to (c) of FIG. 9, the processing of steps 22 to 28 is performed again based on the four representative vectors. Here, the distortion parameter d ″ indicating the provisional classification distortion obtained by the four clusters obtained in the third step 26. _max (See (c) of FIG. 9) is smaller than the distortion threshold D. Then, the process shown in FIG. 1 proceeds from the third step 28 to step 32. The classification at this time is the final classification in the current image.
[0050]
In step 32, it is confirmed whether or not the current image is an original image. In this example, since the current image at the present stage is the 8th low resolution image having 4 × 4 pixels and not the original image, the processing shown in FIG. 1 returns to step 16 through step 34, and the resolution is next. The process proceeds to a classification process for a low image, that is, a seventh low resolution image having 8 × 8 pixels.
[0051]
In step 34, which is performed before proceeding to the next lower resolution image, to what extent will the next image be classified based on any of the above tentative classifications or final classification in the current image A new distortion threshold value serving as a reference for the above is derived, and the distortion threshold value D is updated. As the new distortion threshold value D, for example, the distortion parameter obtained in the classification process in the current image (in this example, the above d _max , D ' _max And d '' _max ) Can be used. In this embodiment, the initially derived distortion parameter, d _max Is adopted as a new distortion threshold value D. Instead, the last derived distortion parameter (d ″ in this example) _max ) May be adopted as the new distortion threshold D.
[0052]
Hereinafter, until the original image is made the current image and the final classification in the original image is obtained, the process of steps 16 to 34 shown in FIG. 1 is repeated, and finally extracted from each pixel of the original image. Classification of the 1024 × 1024 feature vectors thus obtained can be obtained. Such a classification result can be used for an area dividing process for dividing an original image into a plurality of image areas corresponding to imaging target elements such as “sky”, “sea”, and “building”.
[0053]
According to the above embodiment, the initial representative vector and the initial value of the distortion threshold value are obtained based on the 2 × 2 minimum resolution image derived from the input original image itself, and the resolution is determined using these. The feature vectors extracted from each pixel of the original image are finally obtained by sequentially finding the optimum classification of feature vectors in a low image, so there is no need to input parameters from the outside. Classification can be performed according to the optimum degree of classification according to the contents of the original image. Therefore, stable classification performance can be realized no matter what original image is input.
[0054]
In addition, as a modification of the above embodiment, a mode may be adopted that further includes a step of updating the provisional classification based on the representative vector updated in step 24 between steps 24 and 26 in FIG. Good. For example, this includes processing for recalculating the Euclidean distance in the feature vector space between each updated representative vector and each feature vector, and reclassifying each feature vector into a cluster represented by the closer representative vector.
[0055]
In the above embodiment, a three-dimensional vector having a luminance component and two color difference components expressed in the YCC color system is used as a feature vector. However, each component of the feature vector is not limited to these, and The dimension of the feature vector is not limited to 3.
[0056]
In the above embodiment, for convenience of explanation, the original image is a square image of 1024 × 1024 pixels, but it goes without saying that the present invention can be applied to original images having different numbers of vertical and horizontal pixels. Each low-resolution image is not limited to a square image. Further, the number of pixels of the minimum resolution image is not limited to four, and any number of pixels may be used as long as it is a plurality of pixels smaller than the number of pixels of the original image.
[0057]
Furthermore, the above embodiment is intended to classify the feature vectors extracted from each pixel of the original image. However, the feature vector extracted from each pixel of the reduced image by performing image reduction processing on the original image in advance. The form of performing the classification is also within the scope of the present invention. In this case, after classifying the feature vector extracted from each pixel of the reduced image, the image is subjected to image enlargement processing again on the reduced image. Etc. can be used.
[0058]
Note that the clustering process according to the above and other embodiments of the present invention may be executed by a computer program. In addition, the above description has been made with respect to a method for convenience of description, but an apparatus including means for performing each of the above steps is also within the scope of the present invention.
[0059]
Although the embodiments of the present invention have been described in detail above, the above-described embodiments are merely illustrative, and the technical scope of the present invention should be defined only by the claims in this specification. Needless to say.
[Brief description of the drawings]
FIG. 1 is a flowchart showing a procedure of clustering processing according to an embodiment of the present invention.
FIG. 2 is a conceptual diagram showing a method for deriving each low-resolution image in the embodiment of FIG.
FIG. 3 is a diagram showing an example of a Gaussian filter used for deriving each low-resolution image.
4 is a flowchart showing in detail an initial representative vector derivation step in the embodiment of FIG. 1;
5 is a conceptual diagram showing each stage of the initial representative vector derivation process shown in FIG. 4;
6 is a bar graph showing a change in the maximum distance which is a reference for selecting an initial representative vector in the initial representative vector derivation step shown in FIG.
FIG. 7 is a conceptual diagram showing each stage of classification processing in a low-resolution image of 4 × 4 pixels.
FIG. 8 is a conceptual diagram illustrating each stage of classification processing in a low-resolution image of 4 × 4 pixels.
FIG. 9 is a conceptual diagram showing each stage of classification processing in a low-resolution image of 4 × 4 pixels.

Claims (18)

Deriving from the original image a low resolution image composed of a plurality of pixels having different resolutions in stages;
Obtaining an initial value of each component of the representative vector and an initial value of a distortion threshold from the lowest resolution image having the lowest resolution among the plurality of low resolution images;
Extracting a plurality of feature vectors from the next lowest resolution image;
Classifying the plurality of feature vectors based on the representative vector, using the distortion threshold as an index of an optimal classification level;
Updating the representative vector and the distortion threshold;
Clustering of feature vectors of an image, including a step of repeating the updating step from the extracting step until classification according to the degree of the optimum classification of a plurality of feature vectors extracted from the original image is obtained Method. Deriving from the original image a low resolution image composed of a plurality of pixels having different resolutions in stages;
Obtaining an initial value of each component of the representative vector and an initial value of a distortion threshold from the lowest resolution image having the lowest resolution among the plurality of low resolution images;
Extracting a plurality of feature vectors from the current image, with the next lowest resolution image as the current image;
Mapping the plurality of feature vectors to a feature vector space;
Classifying the plurality of feature vectors into clusters represented by each of the representative vectors in the feature vector space to obtain a provisional classification in the current image;
Updating the representative vector based on the feature vectors included in each of the clusters;
Determining a distortion parameter indicative of the provisional classification distortion;
Comparing the distortion parameter with the distortion threshold and, if the distortion parameter is greater than the distortion threshold, adding a new vector to the representative vector;
Repeating the step of adding the new vector from the step of determining the tentative classification until the distortion parameter is less than the distortion threshold, and determining the final classification in the current image;
Updating the distortion threshold based on any of the tentative classifications in the current image or the final classification;
Including the step of repeating the step of updating the distortion threshold from the step of extracting the plurality of feature vectors until the original image is the current image and the final classification in the original image is obtained. Clustering method of feature vectors of images to be performed. 3. The clustering method according to claim 2, further comprising a step of updating the provisional classification based on the representative vector between the step of updating the representative vector and the step of obtaining the distortion parameter. The step of obtaining the provisional classification is a step of classifying each of the plurality of feature vectors into a cluster represented by a representative vector that is closest to the feature vector in the feature vector space among the representative vectors. The clustering method according to claim 2 or 3, wherein: 5. The step of updating the representative vector is a step of setting a vector representing a centroid of the feature vector included in each of the clusters as a new representative vector in the feature vector space. Clustering method. The distortion parameter is a value indicating the spread of the cluster having the largest spread in the feature vector space among the clusters,
The step of adding the new vector is a step of adding the same vector as the feature vector having the longest distance from the representative vector representing the cluster to which the cluster belongs to the representative vector as the new vector. 6. The clustering method according to claim 4, wherein the clustering method is characterized. 3. The step of updating the distortion threshold value is a step of setting any one of the distortion parameters obtained to obtain the final classification in the current image as a new distortion threshold value. 7. The clustering method according to any one of 6 to 6. Obtaining the initial value comprises:
deriving one original representative vector from the lowest resolution image consisting of n pixels;
A distance in the feature vector space between the n feature vectors extracted from each of the n pixels and the original representative vector is obtained, and a feature vector indicating the maximum distance is set as a representative vector candidate and the highest Assigning candidate ranks,
A distance in the feature vector space between each of the remaining feature vectors and the original representative vector and the representative vector candidate closest to the feature vector is obtained, and a feature vector indicating a maximum distance is determined as the representative vector candidate. And adding the next highest candidate ranking,
Repeating the step of assigning the next highest candidate ranking multiple times;
In accordance with the candidate rank, the value of each component of the top several feature vectors of the representative vector candidates is set as the initial value of each component of the representative vector;
The clustering method according to any one of claims 2 to 7, further comprising a step of setting a value indicating a spread of the n feature vectors in the feature vector space as the initial value of the distortion threshold value. 9. The clustering method according to claim 8, wherein, as the top several feature vectors, up to a feature vector having a maximum amount of change in the maximum distance when added to the representative vector candidate is selected. Means for deriving a low resolution image composed of a plurality of pixels having different resolutions in stages from the original image;
Means for determining an initial value of each component of the representative vector and an initial value of a distortion threshold from the lowest resolution image having the lowest resolution among the plurality of low resolution images;
Means for extracting a plurality of feature vectors from the next lowest resolution image;
Means for classifying the plurality of feature vectors based on the representative vector, using the distortion threshold as an index of an optimal classification level;
Means for updating the representative vector and the distortion threshold;
Means for repeatedly operating the updating means from the extracting means until classification according to the optimum classification degree of the plurality of feature vectors extracted from the original image is obtained. Feature vector clustering device. Means for deriving a low resolution image composed of a plurality of pixels having different resolutions in stages from the original image;
Means for determining an initial value of each component of the representative vector and an initial value of a distortion threshold from the lowest resolution image having the lowest resolution among the plurality of low resolution images;
Means for extracting an image having the next lowest resolution as a current image and extracting a plurality of feature vectors from the current image;
Means for mapping the plurality of feature vectors to a feature vector space;
Means for classifying the plurality of feature vectors into clusters represented by each of the representative vectors in the feature vector space to obtain a provisional classification in the current image;
Means for updating the representative vector based on the feature vectors included in each of the clusters;
Means for determining a distortion parameter indicative of the provisional classification distortion;
Means for comparing the distortion parameter with the distortion threshold and, if the distortion parameter is greater than the distortion threshold, adding a new vector to the representative vector;
Means for repeatedly operating means for adding the new vector from means for determining the tentative classification until the distortion parameter is less than the distortion threshold, and for determining a final classification in the current image;
Means for updating the distortion threshold based on any of the provisional classifications in the current image or the final classification;
Means for repeatedly operating the means for extracting the plurality of feature vectors from the means for extracting the plurality of feature vectors until the original image is the current image and the final classification in the original image is obtained; An image feature vector clustering device. 12. The clustering apparatus according to claim 11, further comprising means for updating the provisional classification based on the representative vector updated by the means for updating the representative vector. The means for obtaining the provisional classification is means for classifying each of the plurality of feature vectors into a cluster represented by a representative vector having the closest distance in the feature vector space to the feature vector among the representative vectors. The clustering apparatus according to claim 11 or 12, wherein: 14. The means for updating the representative vector is a means for setting a vector representing the centroid of the feature vector included in each of the clusters as a new representative vector in the feature vector space. Clustering equipment. The distortion parameter is a value indicating the spread of the cluster having the largest spread in the feature vector space among the clusters,
The means for adding the new vector is means for adding, to the representative vector, the same vector as the feature vector having the longest distance from the representative vector representing the cluster to which the new vector belongs. The clustering device according to claim 13 or 14, characterized in that 12. The means for updating the distortion threshold value is a means for setting any one of the distortion parameters obtained to obtain the final classification in the current image as a new distortion threshold value. 15. The clustering device according to any one of 15 to 15. Means for determining the initial value;
means for deriving one original representative vector from the lowest resolution image comprising n pixels;
A distance in the feature vector space between the n feature vectors extracted from each of the n pixels and the original representative vector is obtained, and a feature vector indicating the maximum distance is set as a representative vector candidate and the highest A means to rank candidates,
A distance in the feature vector space between each of the remaining feature vectors and the original representative vector and the representative vector candidate closest to the feature vector is obtained, and a feature vector indicating a maximum distance is determined as the representative vector candidate. Means to add and rank the next highest candidate ranking,
Means for repeatedly operating the means for assigning the next highest candidate ranking;
Means for setting the values of the components of the top several feature vectors of the representative vector candidates according to the candidate rank as the initial values of the components of the representative vector;
17. The clustering apparatus according to claim 11, further comprising a unit that sets a value indicating a spread of the n feature vectors in the feature vector space as the initial value of the distortion threshold. 18. The feature vector having the largest change amount of the maximum distance when added to the representative vector candidate is selected as the top several feature vectors. Clustering device.

Images