WO2012005242A1

WO2012005242A1 - Image processing device and image segmenting method

Info

Publication number: WO2012005242A1
Application number: PCT/JP2011/065356
Authority: WO
Inventors: 小川　雅嗣; 雄馬松田
Original assignee: 日本電気株式会社
Priority date: 2010-07-05
Filing date: 2011-07-05
Publication date: 2012-01-12
Also published as: JPWO2012005242A1

Abstract

The disclosed image processing device has an image segmenting processing unit for segmenting an image into clusters, i.e., sets of pixels. The image segmenting processing unit segments an image by making each pixel in the image the target pixel and setting to the same cluster those pixels that are adjacent to the target pixel and that have a feature value closest to that of the target pixel. Because no threshold value processing is performed, the disclosed image processing device is capable of robustly extracting objects from the image data.

Description

Image processing apparatus and image dividing method

[Description of related applications]
The present invention is based on the priority claim of Japanese patent application: Japanese Patent Application No. 2010-152869 (filed on July 5, 2010), the entire contents of which are incorporated herein by reference. Shall.
The present invention relates to an image processing apparatus and an image dividing method. In particular, the present invention relates to an image processing apparatus and an image dividing method for extracting an object from an image whose content of the object being shown is not known.

In recent years, with the rapid spread of digital video equipment such as digital cameras, there is an increasing expectation for general object recognition that recognizes what objects are included in captured images and videos. Yes. In general object recognition, appropriate classification of image data stored without being classified in the database, retrieval of necessary image data, extraction of a desired scene from a moving image, or only a desired scene It can be applied to various applications such as re-editing.

The various object recognition technologies such as face recognition and fingerprint recognition have been developed so far, but all of them have been limited to special applications. Development of technology that recognizes general objects has been pointed out that recognition technology specialized for one object has been pointed out, such as the recognition rate going down immediately when trying to use it for another purpose. Is expected. As a technique for recognizing a general object, there is known a feature amount called SIFT (Scale Invariant Feature Transform) using a histogram in which local intensity gradients of images are accumulated as shown in Non-Patent Document 1. By using this technique, it is possible to recognize that the same image with geometric transformation and occlusion is the same. However, the present technology is for determining whether or not two images are the same, and does not function unless an object to be recognized is prepared in advance. That is, there is a problem that when an image that is not prepared in advance in the database is displayed in the image, it cannot be recognized.

In addition, recognition using a representation method called CSS (Curvure Scale Space), which is shown in Non-Patent Document 2, which smoothes the image contour step by step and represents the inflection point position of each contour, is known. It has been. By using this technology, it is possible to recognize that the same image with geometric transformation or an image with similar outline is the same or similar. However, the present technology is based on the assumption that the contour of the object has been extracted.

On the other hand, it is very difficult to extract the outline of what seems to be an object when it is not clear what is reflected in the image. A technique related to contour extraction of a typical object will be described below. The most common contour extraction technique is edge extraction. This is a technique for extracting a position where a change in luminance, color, or the like between pixels is large as an edge and estimating that it is an outline of an object. At present, the Canny method is known as one of the excellent edge extraction methods. Although the edge is a quantity that correlates with the contour of the object, it seems a reasonable method, but this method has some drawbacks. The first problem of edge extraction is setting a threshold value of change to be regarded as an edge. Depending on the setting of this threshold, the number of extracted edges is greatly affected. Therefore, when actually performing edge extraction, it is necessary to search for an appropriate threshold value by changing the threshold value. This is a very large computational load and is a problem of edge extraction. In the case of images, even if the subject is the same, it is natural that the amount of light changes depending on the captured image. If the threshold value is fixed, a problem occurs in the performance of edge extraction.

In addition, the second problem of edge extraction is that edges are generated or lost due to subtle adjustments of light, so there are no edges in the middle of the contour, or error edges are not related to the contour of the object. Is to appear. Therefore, it is necessary to consider an algorithm for estimating a broken outline and an algorithm for ignoring error edges. This is also a very big problem.

The third problem of edge extraction is that information about the presence of an object is lost after edge extraction. The image after the edge extraction is an edge-only image, and information regarding which region is the same as the object is lost. It is very difficult to specify an object by relying only on edges. Furthermore, as described above, since it must be performed in a state in which broken contours frequently occur, it is even more difficult. As described above, edge extraction is often used as a simple method, but it can be said that it has a great number of problems.

By the way, when trying to extract what seems to be an object, it is more natural to identify the position where the object is likely to exist and then extract the contour. This is the opposite of edge extraction, but such a concept has existed in the past, and techniques such as segmentation, clustering, or labeling correspond to this. This technique called segmentation uses a certain feature amount to cluster pixels according to a threshold value set for the feature amount. Here, a cluster refers to a set of pixels. If this technology is used, it seems that it is possible to divide the individual objects shown in the image into clusters, but since the generated clusters largely depend on the threshold setting, satisfactory results can be obtained depending on the threshold setting. There is no current situation. However, since the position where the object is likely to be expressed is expressed as a cluster, it has an advantage that the position where the object is likely to be specified is easier than the edge extraction.

Regarding segmentation, there are also problems with color. Basically, there are a lot of techniques for segmentation with black and white shading, but if the image is multicolored, it is very difficult to cluster. This is because, in the case of a black and white image, there is a one-dimensional index called shading, so clustering is possible if the density is histogrammed and the threshold is set to a value that allows the peaks of the histogram to be divided into two and three. In the case of multiple colors, it is unclear how to create a histogram. Therefore, it is more difficult to provide effective segmentation for multicolor images than for black and white.

Patent Document 1 discloses a method for performing image template matching. In this method, after applying a DOG (Difference Of Gaussian) filter to the image, two thresholds th1 and th2 are set, and when the DOG output is smaller than th2, it is set to -1, and when th2 to th1 is set to 0. The pattern matching is performed after the process of ternarizing to +1 when the value is larger than th1. However, in the case of such a processing method, the processing result changes depending on the settings of the thresholds th1 and th2.

Patent Document 2 discloses a processing method for giving a rough instruction to a subject area to be extracted from an image and extracting the subject. In this method, an initial nucleus is set from the designated area, and a representative color of the initial nucleus is extracted. Then, while updating the region growth threshold, the contour line of the subject is extracted and the internal region image is cut out. However, even in this method, the processing result varies depending on the setting of the reference for the representative color and the threshold value for area growth.

Patent Document 3 discloses a method for performing region division based on a plurality of feature amounts obtained from image data. In this method, for example, luminance color information and texture information are obtained as a plurality of feature amounts, and region integration processing is performed for each combination of region division results. In this method, a process of integration of a minute area is performed. Here, a certain threshold value is set, and when the threshold value is smaller than the threshold value, it is determined as a minute area and integrated into another area. Processing is in progress. However, also in this method, the processing result changes depending on the threshold value used to determine the minute region.

Japanese Patent Laid-Open No. 06-076062 JP 2001-043376 A JP 2004-258752 A

The entire disclosures of the above patent documents and non-patent documents are incorporated herein by reference. The following analysis is given by the present invention.

As described above, in each of

Patent Documents

1, 2, and 3, there is a problem that the processing result changes depending on the setting of the threshold value or the reference value. By the way, in general, when the subject content and shooting conditions are limited, there is a possibility that the performance of subject extraction and contour extraction can be ensured by optimizing the threshold setting. However, it is very difficult to specify an object or extract a contour from an image in which what is shown or an image with various shooting conditions. This is because there are many cases where satisfactory processing results cannot be obtained if images with various subject conditions are not limited and images with various shooting conditions are processed with the same threshold setting. In particular, when the brightness of an image taken by adjusting light is changed, even with the same subject, it is difficult to obtain the same extraction result stably with the conventional method.

That is, according to the prior art, when subject extraction and contour extraction are processed for a captured image in which the subject content is not limited and the shooting conditions vary, the processing result changes depending on the threshold setting. There is a problem that a robust processing result cannot be obtained.

An object of the present invention is to make it possible to robustly extract an object from image data by providing an image dividing method that does not require a threshold setting.

According to a first aspect of the present invention, there is provided an image division processing unit that divides an image into clusters that are sets of pixels, and the image division processing unit uses each pixel included in the image as a pixel of interest, and An image processing apparatus is provided that divides an image by making the same cluster as a pixel having a feature amount closest to the pixel of interest among pixels adjacent to the pixel of interest.

According to a second aspect of the present invention, there is provided an image dividing method for dividing an image into clusters that are a set of pixels, wherein one of the pixels included in the image is a pixel of interest and is adjacent to the pixel of interest. Among adjacent pixels, a neighboring pixel selection step that selects a pixel having a feature quantity closest to the target pixel, and a clustering between adjacent pixels in which the target pixel is the same cluster as the pixel selected in the adjacent pixel selection step And an adjacent pixel selection step and the same clustering step between adjacent pixels are repeated using each pixel included in the image as the pixel of interest.

According to the first aspect of the present invention, it is possible to provide an image processing apparatus capable of robustly extracting an object from image data. The reason is that the image is divided by making the pixel of interest the same cluster as the pixel having the closest feature quantity among the adjacent pixels, and the threshold processing is not performed, so that the image division processing can be performed robustly. Because it became.

According to the second aspect of the present invention, it is possible to provide an image dividing method capable of robustly extracting an object from image data. The reason is that the pixel with the closest feature quantity is selected from the pixels adjacent to the target pixel, and the image is divided so that the cluster of the target pixel is the same cluster as the selected pixel, and threshold processing is performed. This is because the image division processing can be performed robustly.

1 is a block diagram of an entire image processing apparatus according to a first embodiment of the present invention. It is a figure explaining the principle of the image division process of this invention. It is a flowchart which shows the image division method of this invention. It is a figure explaining the image processing result which implemented this invention. It is an example of the image which implemented this invention in Example 1. FIG. It is a flowchart for demonstrating Example 1 of this invention. It is a figure explaining the image processing result of Example 1 of this invention. It is a figure explaining the image processing result of Example 1 of this invention. It is a figure explaining the image processing result of Example 1 of this invention. It is an example of the image implemented in Example 1 of this invention. It is a figure explaining the image processing result of Example 1 of this invention. It is a block diagram of the whole image processing apparatus of Example 2 of this invention. It is an example of the image which implemented this invention in Example 2. FIG. It is a flowchart for demonstrating Example 2 of this invention. It is a figure explaining the image processing result of Example 2 of this invention.

The outline of the embodiment of the present invention will be described. Note that the reference numerals of the drawings attached to this summary are merely examples for assisting understanding, and are not intended to be limited to the illustrated modes.

In the present invention, the following modes are possible.
[Form 1]
As in the first viewpoint, the image division processing unit divides an image into clusters that are sets of pixels, and the image division processing unit uses each pixel included in the image as a target pixel and is adjacent to the target pixel. Among the pixels to be processed, an image processing apparatus is provided that divides an image by using the same cluster as a pixel having a feature amount closest to the pixel of interest. In the first embodiment, since the pixel of interest is configured to be divided into images by making the same cluster as the pixel having the closest feature quantity among the adjacent pixels, threshold processing is not performed, so that the object from the image data It is possible to provide an image processing apparatus that can extract the image robustly.
[Form 2]
The feature amount is preferably color information.
[Form 3]
When the RGB color system components of the image are R, G, and B, the color information is a vector in which the three components are 2R-GB, 2G-RB, and 2B-RG. Is preferably represented by:
[Form 4]
When the components in the RGB color system of the image are R, G, and B, and α is a certain coefficient, the feature amount includes two components: 2R-GB, 2G-RB, 2B- It is preferably represented by a vector consisting of RG and α (R + G + B).
[Form 5]
It is preferable to further include a contour extracting unit that extracts a boundary between clusters formed by the image division processing unit.
[Form 6]
An image division method for dividing an image into clusters, which are a set of pixels, as in the second viewpoint, wherein one pixel among the pixels included in the image is a pixel of interest, and is adjacent to the pixel of interest. An adjacent pixel selecting step for selecting a pixel whose feature amount is closest to the target pixel; and an adjacent pixel same clustering step in which the target pixel is the same cluster as the pixel selected in the adjacent pixel selection step. In addition, there is provided an image dividing method in which the adjacent pixel selection step and the same clustering step between adjacent pixels are repeated using each pixel included in the image as the pixel of interest.
[Form 7]
The feature amount is preferably color information.
[Form 8]
When the RGB color system components of the image are R, G, and B, the color information is a vector in which the three components are 2R-GB, 2G-RB, and 2B-RG. Is preferably represented by:
[Form 9]
When the components in the RGB color system of the image are R, G, and B, and α is a certain coefficient, the feature amount includes two components: 2R-GB, 2G-RB, 2B- It is preferably represented by a vector consisting of RG and α (R + G + B).
[Mode 10]
Preferably, the method further includes a step of extracting a boundary of the cluster.

Hereinafter, specific embodiments will be described with reference to the drawings. As shown in FIG. 1, the image processing apparatus 8 according to the first embodiment of the present invention includes an image division processing unit 2 that divides an image into clusters that are sets of pixels. The image is divided by setting each pixel included in the pixel as the target pixel to be the same cluster as the pixel having the feature amount closest to the target pixel among the pixels adjacent to the target pixel.

The image division method according to the second embodiment of the present invention is an image division method for dividing an image into clusters, which are a set of pixels, as shown in FIG. Among the pixels adjacent to the target pixel as the target pixel, an adjacent pixel selection step S13 for selecting a pixel whose feature quantity is closest to the target pixel, and the same cluster as the pixel selected as the target pixel in the adjacent pixel selection step S13 The adjacent pixel same clustering step S14 is repeated, and the adjacent pixel selection step and the adjacent pixel same clustering step are repeated with each pixel included in the image as the pixel of interest.

Here, feature amounts in the first embodiment and the second embodiment of the present invention will be described. Any feature value can be applied as long as it represents the nature of the image, but color information is considered as an optimal example of the feature value. The reason is that the brightness information of the captured image changes depending on the shooting conditions such as the amount of light hitting the subject, shutter speed, exposure, sensor sensitivity, etc., but the color information is not easily affected by the same. This is because the parameter is suitable for robust subject extraction because it tends to maintain the state. However, since the color information alone may not be able to distinguish between subjects having the same color information and different reflectivities, it is desirable to appropriately add brightness information.

Next, as a method suitable for expressing color information, the characteristics and problems of each of the RGB method and HSV method, which are widely used color expression methods, are analyzed. First, the RGB method expresses colors with the output of the three primary colors of light. This representation is very convenient as a color representation because it directly corresponds to the output of the device. It is also close to the structure of a human color vision detector. However, the RGB method has a drawback that it does not know which hue, saturation, or brightness corresponds to the difference between colors.

On the other hand, the HSV method solves that problem. In the HSV method, a color is expressed by three parameters of hue H, saturation S, and brightness V. Conversion from the RGB system to the HSV system is expressed by the following equations (1), (2), and (3).

However, from equation (1), hue H cannot be defined when MAX = MIN. Further, from the equation (2), when MAX = 0, the saturation S cannot be defined. Further, the hue H is not uniformly continuous. As described above, when the HSV method is actually used, there is a problem that singularities and continuity must be taken into consideration and complicated processing must be performed. In view of such a situation, the present invention devised a simple color information expression method compared to HSV. It uses the three-dimensional vector of equation (4) or the four-dimensional vector of equation (5).

In the equations (4) and (5), importance is attached to the convenience and convenience of RGB output that directly corresponds to the output of the device, and the characteristics corresponding to the hue H and the saturation S in the HSV method are as follows: Added linear and continuous expression. In the present specification, the vectors defined by Expression (4) and Expression (5) are referred to as color vectors.

The order of each element in the color vectors of Equation (4) and Equation (5) is arbitrary and can be exchanged. That is, the same processing result can be obtained even if the elements are defined interchangeably. Further, Expression (4) represents only color information, but Expression (5) adds a fourth component representing brightness information to Expression (4). Here, α in Equation (5) is a coefficient. If only color information is used, equation (4) may be used, but if information about brightness information is also considered, equation (5) may be used. Further, the weighting of the color information and the brightness information may be adjusted by the coefficient α. If α is set to a large value, the contribution rate of brightness information increases, and if α is set to a small value, the contribution rate of brightness information decreases.

The embodiment will be described in detail below.

[Configuration of Example 1]
FIG. 1 is a block diagram of the entire image processing apparatus according to the first embodiment of the present invention. First, the image acquisition unit 1 acquires an image. The image may be acquired by the image processing apparatus with a built-in CCD, or an image stored outside may be taken out and supplied to the image acquisition unit 1. The image acquired by the image acquisition unit 1 is input to the image division processing unit 2 and is divided into clusters by the image division processing of the image division processing unit 2. Next, the result of the division into clusters by the image division processing unit 2 is input to the contour extraction unit 3, and the contours of the clusters are extracted. Next, the contour of the cluster is input to the contour recognition unit 4. The contour recognition unit 4 recognizes an object from the contour data obtained by the contour extraction unit 3 by referring to data of the contour data storage unit 5 in which the relationship between many contours and object information is stored in advance. Output the result.

[Operation of Embodiment 1]
Next, the operation of the first embodiment will be described with reference to FIG. FIG. 6 is a flowchart for explaining the first embodiment. First, in step S21, an image is acquired. Next, in step S22, it is determined whether to process between pixels or between clusters. Next, if it is decided to process between pixels in step S22, the image is divided by the same clustering between adjacent pixels in step S23. On the other hand, if it is decided to process between clusters, the adjacent cluster in step S28 is determined. The image is divided by the same clustering. Next, in step S24, the contour extraction processing of the contour extraction unit 3 is performed on the clustered result. Next, in step S25, the contour recognition process of the contour recognition unit 4 is performed. Here, matching processing with a contour database stored in the contour data storage unit 5 in advance is performed, and an object recognition result is output. Next, in step S26, it is determined whether or not the recognition result is satisfactory. If it is determined as No, in step S29, the cluster whose contour has been recognized is removed from the processing target, and the process returns to step S22 and the remaining processing. The processes after step S22 are repeated for the target cluster. On the other hand, if it is determined Yes, the process is terminated. Here, in step S22, generally, the process between pixels is selected in the first iteration process, and the process between clusters is selected in the second and subsequent iteration processes.

Next, FIG. 3 is a flowchart showing details of image division by the same clustering between adjacent pixels performed in step S23 of FIG. First, in step S11, a pixel to be processed is selected. In this specification, a pixel to be processed is referred to as a target pixel.

Next, in step S12 in FIG. 3, the color difference between adjacent pixels is calculated. Hereinafter, step S12 will be described in detail. In the first embodiment of the present invention, the color vector represented by Expression (5) is used as the feature amount used for clustering of the image division processing unit 2. Color vectors are calculated for the target pixel shown in FIG. 2 and the eight pixels adjacent to the target pixel. Here, the color vector of the target pixel is EV0, and the color vectors of the eight pixels adjacent to the target pixel are EV1, EV2, EV3, EV4, EV5, EV6, EV7, and EV8, respectively. Next, the difference between the color vectors of the pixel of interest and the eight adjacent pixels, that is, the color difference ΔEVi (i = 1, 2,..., 8) is calculated by Expression (6). Here, ΔEVi is also a four-dimensional vector. Next, the norm ΔEVi_NORM of the color difference ΔEVi is further calculated. Here, the norm is the magnitude of the vector and the square sum of the squares of the four components.

Next, step S13 in FIG. 3 will be described. Step S13 is an adjacent pixel selection step. In step S13, first, among the eight norms ΔEVi_NORM calculated in step S12, the direction having the smallest value is searched for i_min. Thereby, the pixel of interest is considered to be continuous with the adjacent pixel in the i_min direction. As can be seen from FIG. 2, when i_min = 1, the target pixel is considered to be continuous with the right pixel, and when i_min = 2, the target pixel is continuous with the upper right pixel. If i_min = 3, the pixel of interest is considered to be continuous with the upper pixel, and if i_min = 4, the pixel of interest is considered to be continuous with the upper left pixel. , I_min = 5, the target pixel is considered to be continuous with the left pixel, and i_min = 6, the target pixel is considered to be continuous with the lower left pixel, and i_min = 7 In this case, the target pixel is considered to be continuous with the lower pixel. When i_min = 8, the target pixel is considered to be continuous with the lower right pixel.

Next, step S14 in FIG. 3 will be described. Step S14 is the same clustering step between adjacent pixels. The cluster of the pixel of interest is replaced with the same cluster as the pixel in the direction i_min of the adjacent pixel having the smallest color difference obtained in step S13. If a plurality of pixels have the smallest color difference, all of them may be pixels belonging to the same cluster, or one of them may be selected.

Next, in step S15 in FIG. 3, it is determined whether or not processing has been performed for all pixels. If NO, the process returns to step S11 to repeat the processing. On the other hand, when it determines with YES, a process is complete | finished. As described above, clustering is performed on one image, and the image is divided into regions composed of several clusters.

As described above, the operation of the first embodiment has been described with reference to the flowchart of FIG. 6, and the operation of the image division processing unit in step S23 in FIG. 6 has been described in more detail with reference to the flowchart of FIG.

FIG. 4 shows an example of the result of image division processing by the method shown in the flowchart of FIG. In FIG. 4, the numbers described in each pixel are the cluster numbers, and the same clusters have the same numbers. The area of the object B in FIG. 4 must be the same cluster, but is divided into a cluster 3 and a cluster 4. That is, the same clustering is insufficient between the cluster 3 and the cluster 4. This cause occurs when a pixel of a certain cluster selects a pixel of the same cluster as the other party. In such a case, after removing a cluster that has obtained a good result from the clustering processing performed between pixels from the processing target, clustering between the clusters may be performed on the remaining processing target cluster. . Then, cluster 3 and cluster 4 in FIG. 4 are combined, and clustering can be realized in which each object is neatly divided. Referring to the flowchart shown in FIG. 6, in step S22, the process shown in FIG. 4 is selected by selecting a process between pixels in the first process and selecting a process between clusters in the second process. Has solved.

Here, the image division by the same clustering between adjacent clusters in step S28 in FIG. 6 showing the clustering process between clusters is obtained by replacing the process between pixels in FIG. 3 with the process between clusters. In FIG. 3, in the case of processing between clusters, step S13 in FIG. 3 is an adjacent cluster selection step, and step S14 is an identical clustering step between adjacent clusters. Further, the number of adjacent clusters in step S12 in FIG. 3 is not limited to 8, and is the number of adjacent clusters of interest.

One of the effective points of the present invention is that no threshold is used when dividing into clusters. Since each pixel and each cluster is only combined with the closest pixel difference among adjacent pixels or clusters, the relative relationship does not change even if the light intensity changes. In other words, even if the light intensity changes, it will be clustered in the same way. This is an effective feature not found in the conventional segmentation technique, and it is not necessary to change the threshold many times as in the conventional case. That is, it can be said that it is a very robust method for adjusting light.

Next, the operation of the first embodiment will be further described by showing a specific example of image processing. The image in FIG. 5 is composed of three areas of a round object D, a triangular object E, and a background, and each of the three areas has different color information and brightness information. When image division processing is performed on all the pixels of the image in FIG. 5 by the image division processing unit 2, the images are clustered by color. FIG. 7 shows a clustered state. The numbers described in the pixels are the cluster numbers, and the same cluster has the same number. Although the region of the triangular object E has the same color, clusters having different cluster numbers exist.

At this stage, the data is sent to the next contour extraction unit 3. The contour extraction unit 3 extracts a boundary portion of each cluster clustered by the image division processing unit 2 as a contour. The extracted contour information is sent to the contour recognition unit 4. The contour recognizing unit 4 matches the recognition contour database stored in the contour data storing unit 5 with the contour information extracted by the contour extracting unit 3 to specify an object.

<Through the processing so far, the recognition of the object is once completed, but there may still be a cluster that could not be identified as an object. In particular, as shown in FIG. 7, there is a low possibility that a part that is the same color but has a different cluster is identified as an object. As shown in FIG. 7, a circular outline is extracted for the cluster 2, and a round object is recognized by the outline recognition unit 4. However, in the area divided into the cluster 3 and the cluster 4, no matching with the contour database has been found. Therefore, the contour recognition unit 4 removes the recognized cluster 2 from the processing target, and returns to the clustering process again with the remaining cluster as the processing target. Here, instead of clustering pixels, clustering between clusters is selected, and clusters are joined by clustering between clusters. At this time, the adjacent clusters are combined with the cluster having the smallest color difference. FIG. 8 shows the result of the connection between the clustering. It can be seen that clusters of the same color are combined into one cluster. Subsequent processing is the same as that for pixels, and the contour extracting unit and contour recognizing unit sequentially process to identify the object. Now it can also recognize triangular objects. FIG. 9 shows the result of contour extraction. Finally, it can be seen that round objects and triangular objects are recognized and specified. In this way, according to the present invention, the image is autonomously divided without requiring the setting of a threshold value, and then the object can be specified very simply.

FIG. 10 is an image of the same subject as in FIG. 5 taken in the dark. FIG. 11 shows the result when the image division processing unit 2 processes the image of FIG. It can be seen that an object in the dark can be easily recognized. This is because the present invention does not require threshold setting and can operate robustly regardless of light. From the above, it has been found that the present invention is very simple and can robustly specify an object, extract a contour, and recognize an object.

[Configuration of Example 2]
FIG. 12 is a block diagram of the entire image processing according to the second embodiment of the present invention. The second embodiment further includes a cluster combination unit 6 and a cluster combination data storage unit 7 as compared with the first example. The cluster combination unit 6 is arranged between the image division processing unit 2 and the contour extraction unit 3, and the clustering result by the image division processing unit 2 is based on the data stored in the cluster combination data storage unit 7. It is configured to perform a join process.

[Operation of Embodiment 2]
FIG. 14 is a flowchart for explaining the second embodiment of the present invention. In FIG. 14, step S27 is added to FIG. 6 showing the operation of the first embodiment. In step S27, the clustering process is performed on the result of the clustering process in steps S23 and S28 by using the coupling relationship between the clusters.

As described above, in the second embodiment, the final object identification and contour extraction are performed by combining clusters having different colors using the connection relation and inclusion relation between clusters. The contour can be extracted by taking out the periphery of the combined cluster. The extracted contour is matched with a contour dictionary held by the image processing apparatus to recognize what the object is. The connection and inclusion between the clusters is somewhat a combination problem. However, in the clustering by color according to the present invention, information as an object lump is stored, so the processing is not complicated. This method is simpler than the conventional method of specifying an object from an edge where information as a mass of the object is not stored, or specifying an object while performing segmentation many times by changing a threshold value. As described above, according to the present invention, an object can be specified and the contour of the object can be extracted very simply and effectively.

FIG. 13 shows a humanoid object, which is an image composed of seven areas: face, arm (right), arm (left), T-shirt, pants (right), pants (left), and background. Here, the arm (left) and the arm (right) have the same color information and brightness information, and the pants (left) and pants (right) have the same color information and brightness information. FIG. 13 is an image to be executed in the second embodiment. First, in step S23, clustering is performed. The face is in cluster 2, the arm (right) is in cluster 3, the arm (left) is in cluster 4, the T-shirt is in cluster 5, and the pants (right). Are clustered in cluster 6, pants (left) in cluster 7, and background in cluster 1.

Next, the processing of step S27 is performed by the cluster combining unit 6, and as shown in FIG. 15, the six clusters 2, cluster 3, cluster 4, cluster 5, cluster 6, and cluster 7 are integrated into cluster 2 It is summarized in. Here, in the process of combining clusters in step S27, the clusters are combined using not only the physical quantity difference such as the color difference but also the connection relationship or inclusion relationship between the clusters. For example, if an oblong object is connected to a rectangular object, and a thin rectangular object extends from the oblong object, the torso is connected to the face and the limbs protrude from the torso. Therefore, these regions are determined to be the same and combined. As described above, when the relationship between the clusters satisfying the feature of the object is found, the clusters are combined. The characteristics of various objects are stored in advance in the cluster connection data storage unit 7 and acquired from there. When the clusters are combined in this way, if the respective parts are the same object even if they are formed with different colors and physical quantities, the contour of the object can be extracted. In step S24 of FIG. 14, contour extraction is performed from the combined clusters. From the processing result shown in FIG. 15, it can be seen that the boundary of the combined cluster 2 has been subjected to contour extraction.

Next, in step S25, contour recognition is performed. Here, the contour extraction result is referred to the database of the contour data storage unit, matching processing is performed, and a recognition result that the extracted contour is a human is output. From the above, it has been found that the present invention is also effective for recognizing an object composed of several color segments. Further, in the second embodiment, similarly to the first embodiment, it is possible to recognize the robustness of the light.

In the first and second embodiments, clustering is mainly performed using color information, but a texture or the like may be used as another feature amount. In the embodiment, the recognition is performed using a simple image. However, the present invention is also effective for an object having a more complicated shape or an image including many objects.

Some or all of the above embodiments can be described as in the following supplementary notes, but are not limited thereto.

(Additional remark 1) It has an image division processing part which divides an image into a cluster which is a set of pixels,
The image division processing unit sets each pixel included in the image as a target pixel, and among the pixels adjacent to the target pixel, the feature amount is set to the same cluster as the pixel closest to the target pixel. An image processing apparatus characterized by dividing the image.

(Supplementary note 2) The image processing apparatus according to supplementary note 1, wherein the feature amount is color information.

(Additional remark 3) When the component by the RGB color system of the said image is set to R, G, B,
The image processing apparatus according to appendix 2, wherein the color information is represented by a vector including three components: 2R-GB, 2G-RB, and 2B-RG.

(Supplementary Note 4) When the RGB color system components of the image are R, G, and B, and α is a certain coefficient,
The image according to claim 1, wherein the feature amount is represented by a vector including four components of 2R-GB, 2G-RB, 2B-RG, and α (R + G + B). Processing equipment.

(Supplementary note 5) The image processing apparatus according to any one of supplementary notes 1 to 4, further comprising a contour extraction unit that extracts a boundary between clusters formed by the image division processing unit.

(Additional remark 6) It further has a contour recognition part which performs a contour recognition based on the contour extraction result by the said contour extraction part, and after removing the cluster which could be recognized by the said contour recognition part from the process target, the remainder contained in the said image Each cluster to be processed is a target cluster, and the target cluster is the same cluster as the cluster closest to the cluster among the clusters adjacent to the target cluster, thereby combining the clusters. The image processing apparatus according to appendix 5, which is further performed.

(Supplementary note 7) An image dividing method for dividing an image into clusters which are sets of pixels,
An adjacent pixel selection step of selecting a pixel having a feature quantity closest to the target pixel, among pixels adjacent to the target pixel, using one of the pixels included in the image as the target pixel;
Including the same clustering step between adjacent pixels in which the pixel of interest is the same cluster as the pixel selected in the adjacent pixel selection step, and the adjacent pixel selection step and the same clustering step between the adjacent pixels. An image dividing method, wherein each pixel included in the image is repeated as the pixel of interest.

(Supplementary note 8) The image dividing method according to supplementary note 7, wherein the feature amount is color information.

(Supplementary note 9) When R, G, and B are components in the RGB color system of the image,
9. The image segmentation method according to appendix 8, wherein the color information is represented by a vector including three components: 2R-GB, 2G-RB, and 2B-RG.

(Supplementary Note 10) When the RGB color system components of the image are R, G, B, and α is a certain coefficient,
The image according to claim 7, wherein the feature amount is represented by a vector including four components of 2R-GB, 2G-RB, 2B-RG, and α (R + G + B). Split method.

(Supplementary note 11) The image dividing method according to any one of supplementary notes 7 to 10, further comprising a step of extracting a boundary of the cluster.

(Supplementary note 12) A contour recognition step for recognizing a contour based on a contour extraction result obtained by the step of extracting a boundary of the cluster, a step of removing a cluster recognized in the contour recognition step from a processing target, Each of the remaining clusters removed from the processing target is set as a target cluster, and among the clusters adjacent to the target cluster, the target cluster is a neighboring cluster selection step of selecting a cluster whose feature is closest to the cluster; 12. The image segmentation method according to appendix 11, further comprising: an identical clustering step between adjacent clusters in which the cluster of interest is the same cluster as the cluster selected in the adjacent cluster selection step.

The image segmentation method of the present invention makes it possible to recognize a subject appearing in an image captured by a digital video device, so that the captured image data can be classified for each subject or a subject in the image database can be identified. It can be applied to searching for a scene.

In the frame of the entire disclosure (including claims and drawings) of the present invention, the embodiment or the embodiment can be changed or adjusted based on the basic technical idea. Various combinations and selections of various disclosed elements are possible within the scope of the claims of the present invention. That is, the present invention of course includes various modifications and corrections that could be made by those skilled in the art according to the entire disclosure including the claims and the drawings, and the technical idea.

DESCRIPTION OF SYMBOLS 1 Image acquisition part 2 Image division process part 3 Contour extraction part 4 Contour recognition part 5 Contour data storage part 6 Cluster connection part 7 Cluster connection

data storage part

8, 9 Image processing apparatus

Claims

An image division processing unit that divides an image into clusters that are sets of pixels;
The image division processing unit sets each pixel included in the image as a target pixel, and among the pixels adjacent to the target pixel, the feature amount is set to the same cluster as the pixel closest to the target pixel. An image processing apparatus characterized by dividing the image.
The image processing apparatus according to claim 1, wherein the feature amount is color information.
When the components of the image according to the RGB color system are R, G, B,
The image processing apparatus according to claim 2, wherein the color information is represented by a vector including three components of 2R-GB, 2G-RB, and 2B-RG.
When the RGB color system components of the image are R, G, B, and α is a certain coefficient,
2. The feature quantity according to claim 1, wherein the feature amount is represented by a vector composed of 2R-GB, 2G-RB, 2B-RG, and α (R + G + B). Image processing device.
5. The image processing apparatus according to claim 1, further comprising a contour extracting unit that extracts a boundary between clusters formed by the image division processing unit.
An image dividing method for dividing an image into clusters that are a set of pixels,
An adjacent pixel selection step of selecting a pixel having a feature quantity closest to the target pixel, among pixels adjacent to the target pixel, using one of the pixels included in the image as the target pixel;
The same clustering step between adjacent pixels in which the pixel of interest is the same cluster as the pixel selected in the adjacent pixel selection step;
An image dividing method comprising: repeating the adjacent pixel selecting step and the same clustering step between adjacent pixels as each pixel of interest as the pixel of interest.
The image dividing method according to claim 6, wherein the feature amount is color information.
When the components of the image according to the RGB color system are R, G, B,
8. The image dividing method according to claim 7, wherein the color information is represented by a vector including three components of 2R-GB, 2G-RB, and 2B-RG.
When the RGB color system components of the image are R, G, B, and α is a certain coefficient,
7. The feature amount according to claim 6, wherein the feature amount is represented by a vector composed of 2R-GB, 2G-RB, 2B-RG, and α (R + G + B). Image segmentation method.
10. The image segmentation method according to claim 6, further comprising a step of extracting a boundary of the cluster.