JP2005063309A - Object identification method and device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically identify the kind of an object included in an image. <P>SOLUTION: An object region OR obtained by region-dividing the image P in each the object, and a plurality of block regions BR obtained by dividing the image P into a large number of regions each smaller than the object region OR, each having the set number of pixels are generated. Each type of the plurality of block regions BR is identified by use of a representative vector database DB, and the identified type of the block region BR is totaled by the object regions OR. Thereafter, the type of the object region OR is identified by use of a totaled result. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an object identification method, apparatus, and program for automatically identifying the types of objects constituting an image.

  If image information captured by a digital camera or the like can identify what image is captured in the image information, for example, classification, search, or image processing may be performed for each type of object included in the image. it can.

  For example, when classifying and searching for images, an image search system has been proposed in which similarity is determined using physical feature amounts included in images. That is, there is a technique of extracting and localizing an input image, collating the reference region with a reference image while changing the position and size of the local region, and classifying and searching for the image. Further, in the above method, there is a method for efficiently classifying and searching images by detecting the position of an object by using a color histogram of a local region and comparing the histogram with a color histogram of a reference image (for example, non-patent) Reference 1). However, in any of the above-described methods, since the similarity is identified by the physical feature amount of the image, it may be determined that what is not similar in kind is similar due to the similarity of the physical amount. There is a problem that the accuracy of the search is poor.

When performing image processing, a method of identifying a specific color region and performing different processing is known as an example of high image quality processing (see, for example, Patent Document 1). In this method, noise is removed by identifying an area where a noise component is conspicuous by color. However, since the identification is based only on the color, for example, skin and sand may be confused. If the sand area is mistakenly recognized as a skin area and noise is removed from the sand area, the texture may be lost, resulting in an unnatural image.
JP-B-5-62879 The Institute of Electronics, Information and Communication Engineers, vol. j81-DII, no. 9, pp. 2035-2042, 1998

  As described above, when image classification, retrieval, or image processing is performed based on information obtained directly from an image, appropriate information cannot be provided to the user. As one method for solving this, it is conceivable to classify, search or perform image processing after identifying the type of object. Then, in the image classification / search, the image can be classified / searched according to the identified type. Therefore, the image classification / search can be easily and accurately performed. Even when image processing is performed, image processing can be performed using image processing conditions suitable for the object.

  To identify the type of object included in the image described above, it is necessary to extract the object area included in the image and identify the type for each object area. At this time, for example, the user may extract the object area in the image while looking at the screen and input the type for each object. However, there is a problem that it takes time and effort to assign the object area type by the user.

  Therefore, an object of the present invention is to provide an object identification method, apparatus, and program that can automatically identify the type of an object included in an image.

  An object identification method according to the present invention is an object identification method for identifying the type of an object included in an image. To generate a plurality of block regions, extract a plurality of feature amounts from each of the generated block regions, generate a feature vector having the extracted feature amounts as vector components, Using a representative vector database in which a plurality of representative vectors indicating representative vector values for each type are stored for each type, the representative vector most similar to the feature vector is detected from the representative vector database, and the detected representative vector Is output as the block area type and the identified block The type of click region by summing for each object region, and is characterized in that identifying the type of object.

  The object identification device of the present invention is an object identification device for identifying the type of an object included in an image, an object region generation means for dividing the image into regions for each object to generate a plurality of object regions, and an image with a set number of pixels. A block area generating means for generating a plurality of block areas by dividing into a plurality of areas smaller than the object area, and a feature amount extracting means for extracting a plurality of feature quantities from each block area generated by the block area generating means And feature vector generation means for generating a feature vector having a plurality of feature amounts extracted by the feature amount extraction means as vector components, and a plurality of representative vectors indicating representative vector values in block area types for each type. Using the stored representative vector database, the representative vector database A representative vector search means for searching for a representative vector most similar to a feature vector from a class, a type output means for outputting the type to which the searched representative vector belongs as a type of the block area, and output for each block area And an object identification unit that aggregates the types of the block areas for each object area and identifies the type of the object using the aggregated result.

  The object identification program of the present invention causes a computer to divide an image into a plurality of object areas divided into areas for each object, and divide the image into a plurality of areas smaller than the object area, each having a set number of pixels. Generating and extracting a plurality of feature amounts from each of the generated block regions, generating a feature vector having the extracted plurality of feature amounts as vector components, and a plurality of representatives indicating representative vector values in the types of the block regions Using a representative vector database in which vectors are stored for each type, a representative vector that is most similar to a feature vector is searched from the representative vector database, and the type to which the searched representative vector belongs is identified as the block region type. The types of identified block areas are aggregated for each object area, and the object It is characterized in that to perform the identifying of Type.

  Here, “object” means a subject included in an image such as a person, sky, sea, tree, building, etc., and “object region” means a region occupied by the subject in the image.

  “Identify the type of object” means that the object in the image is identified as a type such as “mountain”, “sea”, “flower”, “sky”, etc. It also includes specifying “unknown” when not sure.

  Furthermore, the “feature amount extraction means” may be any means that extracts a plurality of feature amounts indicating the features of the image. For example, the regularity of change along one direction of the component signal value assigned to each pixel of the image In addition to the correlation feature quantity extraction means for extracting the correlation feature quantity indicating the degree of the image, the edge feature quantity extraction means for extracting the edge feature quantity indicating the edge feature of the image and the color feature quantity indicating the color feature of the image are extracted. It may include color feature amount extraction means.

  The “correlation feature extraction means” extracts, for example, a correlation feature along the vertical direction of the image, a correlation feature along the horizontal direction of the image, or a correlation feature along the diagonal direction of the image. What is necessary is just to extract the correlation feature quantity of at least one direction of the image.

  Further, the “correlation feature amount extraction unit” outputs a correlation value indicating a correlation between two pixel lines from component signal values of a plurality of pixels constituting two pixel lines formed in the same direction in the image. A plurality of correlation values are obtained by inputting the component signal value of the pixel into the cross-correlation function while shifting one of the two pixel lines in the pixel line forming direction one pixel at a time. The largest maximum correlation value is calculated from a plurality of acquired correlation values, the maximum correlation value is calculated for all combinations of pixel lines formed in the same direction of the image, and all the calculated maximum correlation values are calculated. The average value and the standard deviation may be extracted as the correlation feature amount.

  Further, the “block area generating means” generates, for example, a plurality of first block areas obtained by dividing an image into a mesh shape, and a second block area having a phase shifted from the plurality of first block areas divided into a mesh shape. Alternatively, any object that generates a block area having a set number of pixels, such as scanning a cut frame having a set number of pixels in the object area and generating an image surrounded by the cut frame as a block area, may be used. .

  Further, the block area generation means may have a function of generating a plurality of resolution conversion images having different resolutions from the image, and may generate a block area from each of the generated resolution conversion images.

  Further, the representative vector is a distance between the winner representative vector and the feature vector when the winner representative vector is searched using a feature vector whose type is known in advance and the type of the feature vector is different from the type indicated by the winner representative vector. When the feature vector and the type indicated by the winner representative vector are the same, the distance between the winner representative vector and the feature vector is learned to be close. May be.

  The representative vector database may have a structure in which representative vectors are arranged in a two-dimensional space, and a representative vector in a region near the winner representative vector may be learned in the same manner as the winner representative vector.

  Further, the representative vector database may include a plurality of classification representative vectors for classifying the block region categories and a plurality of identification representative vectors for identifying types from the classified categories. Good.

  The representative vector search means searches for a classification winner representative vector that is most similar to the feature vector, and selects a feature vector from among the types of identification representative vectors in the category indicated by the searched classification winner representative vector. The most similar identification winner representative vector may be searched.

  The “type output unit” may output a category indicated by the searched classification representative representative vector and a type indicated by the identification winner representative vector.

  Further, the “vector search means” detects the distance between the feature vector and the winner representative vector, and determines that there is no winner representative vector when the detected distance is smaller than the set threshold value. However, it may be output that the type is unknown when it is determined that there is no winner representative vector.

  In addition, the “set threshold value” may be a fixed value, or the representative vector and the winner representative vector having the largest distance from the winner representative vector among a plurality of representative vectors in the vicinity region of the winner representative vector. It may be a distance.

  According to the object identification method, apparatus, and program of the present invention, by using the block area for identifying the type of the object area, the image structural features are compared with those in the object area as compared with the case of identifying the type for each pixel. Since it can be added to the type determination, the type of the object can be accurately identified.

  In addition, by identifying the type for each block area, the block area type is aggregated for each object area and the object area type is identified to identify the original type for some block areas of the object area Even if there is something that has not been done, the erroneous recognition can be absorbed and the type of object can be accurately and automatically identified.

  If the block area generating means generates a plurality of first block areas obtained by dividing the image into a mesh shape and a second block area having a phase shifted from the plurality of first block areas and the mesh shape. Since the number of block areas used to identify the type of object area can be increased, the accuracy in identifying the type of object area from the identification of the type of block area can be improved.

  In addition, when the block area generation unit scans a cut frame having a set number of pixels in the object area and generates an image surrounded by the cut frame as a block area, the type of the object area is identified. Since the number of block areas to be used can be increased, it is possible to improve accuracy when identifying the type of object area from identifying the type of block area.

  Furthermore, if the block area generation unit has a function of generating a plurality of resolution conversion images having different resolutions from the image, and each block area is generated from the generated plurality of resolution conversion images, the block area generation unit depends on the distance from the subject. Even when the way the object is captured differs depending on the image, the type of the object can be accurately identified.

  In addition, if the feature amount extraction unit includes a correlation feature amount extraction unit that extracts a correlation feature amount indicating the degree of regularity of change along one direction of the component signal value assigned to each pixel of the image. Since the feature quantity can be extracted as an index for distinguishing between an image having a regular pattern often found in artifacts and an image having a random pattern often found in natural objects, the correlation feature quantity is appropriate. Type identification can be performed.

  Further, if the correlation feature quantity extraction means extracts the correlation feature quantity along the vertical direction of the image and the correlation feature quantity along the horizontal direction of the image, the correlation feature quantity extraction means regularly in the vertical direction and the horizontal direction. Can be distinguished from those in which a regular pattern is formed and those in which a regular pattern is formed in either the vertical direction or the horizontal direction.

  Further, the correlation feature amount extraction means calculates a plurality of values calculated by inputting the component signal value of the pixel to a predetermined cross-correlation function while shifting one of the two pixel lines one pixel at a time in the pixel line formation direction. If the correlation feature value is calculated using the largest correlation value among the correlation values, not only the regular pattern formed in the vertical or horizontal direction of the image but also the diagonal direction of the image. Therefore, regular patterns formed in the vertical, horizontal, and diagonal directions of images can be extracted as correlation features. Can do.

  Furthermore, if the edge feature quantity extraction means calculates the average value and standard deviation of the edge components in the vertical and horizontal directions of the image, respectively, for example, “water (wave)” depending on the vertical and horizontal directions. Different edges can be distinguished from those having relatively uniform edges in the vertical and horizontal directions of “plants (flower garden, etc.)” by the edge feature amount.

  Furthermore, when the representative vector searches for the winner representative vector using a feature vector whose type is known in advance, and the type of the feature vector is different from the type indicated by the winner representative vector, the distance between the winner representative vector and the feature vector Is obtained by learning so that the distance between the winner representative vector and the feature vector is close when the type of the feature vector and the type indicated by the winner representative vector are the same. Sometimes it is possible to identify the type using a representative vector with high accuracy.

  In addition, if the representative vector database has a structure in which representative vectors are arranged in a two-dimensional space, and the representative vector in the vicinity area of the winner representative vector is learned in the same manner as the winner representative vector, the same object region Therefore, it is possible to generate a representative vector that can correspond to various variations of feature vectors included in the information, and thus it is possible to improve the accuracy of type identification.

  Further, the representative vector database has a plurality of classification representative vectors for classifying the category of the block region and a plurality of identification representative vectors for identifying the type from the classified categories. The search means searches for the classification winner representative vector that is most similar to the feature vector, and the identification that is most similar to the feature vector from among the types of identification representative vectors in the category indicated by the searched classification winner representative vector. If the winner representative vector is searched, the feature vector used for category classification and type identification can be suppressed to a low dimension, and the type identification accuracy can be improved.

  In addition, if the type output means outputs the category indicated by the searched category representative representative vector and the type indicated by the identification winner representative vector, the category and the category of the block area can be used when identifying the object area. Can be identified, and information related to the category of the object area can be provided to the user.

  Further, the vector search means detects the distance between the feature vector and the winner representative vector, and determines that there is no winner representative vector when the detected distance is smaller than the set threshold value. If it is determined that the type is unknown when it is determined that there is no winner representative vector, the type with a low maximum component having a low reliability of classification is considered to be unknown without identifying the type. Therefore, the reliability of type identification can be improved.

  In addition, when the set threshold is the distance between the representative vector having the largest distance from the winner representative vector among the plurality of representative vectors in the neighborhood area of the winner representative vector and the winner representative vector, It is possible to reflect the difference in the reliability of type identification at each position of the winner representative vector caused by the difference in variety of types of images in the set threshold value.

  Hereinafter, embodiments of an object identification device of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a preferred embodiment of the object identification device of the present invention. The configuration of the object identification device 1 as shown in FIG. 1 is realized by executing an object identification program read into the auxiliary storage device on a computer (for example, a personal computer). The object identification program is stored in an information storage medium such as a CD-ROM or distributed via a network such as the Internet and installed in a computer.

  The object identification apparatus 1 in FIG. 1 identifies the type of each object included in an image, and includes a block area generation unit 10, an object area generation unit 20, a block area identification unit 30, an object identification unit 70, and the like. The block area generation means 10 in FIG. 1 generates a plurality of block areas by dividing an image into a large number of areas having a set number of pixels and smaller than the object area. Specifically, as shown in FIG. 2A, the block area generation unit 10, when the set number of pixels is set to 32 pixels × 32 pixels, has a plurality of block areas composed of 32 × 32 pixels. It is designed to be divided into BR.

  The object area generation means 20 has a function of generating a plurality of object areas by dividing an image into areas for each object. For example, in the case of an image including a plurality of objects as shown in FIG. 2B, the image area P is divided into areas for each object to generate an object area OR. Here, the object means a subject included in an image such as a person, the sky, the sea, a tree, and a building, and the object area means an area occupied by the subject in the image.

  The block area identifying means 30 identifies the type for each generated block area BR. For example, the block area BR is of the type of “mountain”, “sea”, “flower”, “sky”. It is something to identify.

  The object identification unit 70 aggregates the types of the block regions BR identified by the block region identification unit 30 for each object region OR generated by the object region generation unit 20, and identifies the type of object.

  Specifically, the object identification means 70 totals the number of types of each block area BR in the object area OR. At this time, the object identification means 70 does not count the block area BR that extends over the plurality of object areas OR. As described above, it is possible to prevent the reliability of the type identification from being lowered by removing the block region BR including two types from the total for the block region extending over the two object regions.

  Then, the object identification unit 70 identifies the largest type of block area BR among the types of block areas BR counted in a certain object area OR as the type of object. Then, as shown in FIG. 2C, the type is identified in each object area OR.

  In the object identification means 70 of FIG. 1, the type of object is determined by majority vote. However, the ratio of the largest type among the aggregated types (the maximum number of types / all the objects constituting the object). When the number of block areas) is smaller than the type information threshold, the object identification unit 70 may have a function of outputting “unknown” as the type of object. Alternatively, when the difference between the ratio of the largest type and the ratio of the second most common type is small, the object identification unit 70 may output “unknown” as the type of the object. This is because it may be preferable for the user to be determined as “unknown” rather than erroneously identifying the type of object.

  FIG. 3 is a block diagram showing an example of the object area generating means 20, and the object area generating means 20 will be described with reference to FIG. The object area generation means 20 shown below is an example, and may be performed by a method of generating each object area OR by edge detection, for example.

  The object region generation unit 20 extracts a plurality of feature amounts from each pixel constituting the image, classifies the pixels for each similar pixel feature amount, and divides the region for each pixel classification. Area dividing means 101 for generating a plurality of clustering areas, a minimum cluster area extracting means 112 for extracting a minimum clustering area having the smallest number of pixels among the generated clustering areas, and an adjacent adjacent to the extracted minimum clustering area The integrated region determination unit 113 extracts clustering regions and the region integration unit 110 extracts object regions by integrating the generated clustering regions.

  Here, FIGS. 4 and 5 are schematic diagrams showing a process of dividing an image for each object area, and an operation example of the object area generating means 20 will be described with reference to FIG. First, as shown in FIG. 4A, it is assumed that there is an image in which pixels having similar characteristics are arranged. At this time, the feature quantity classifying unit 100 extracts a plurality of feature quantities from each pixel and generates a plurality of feature vectors having each feature quantity as an element. Thereafter, as shown in FIG. 4B, a plurality of feature vectors are classified into similar feature vectors (clustering).

  Thereafter, the result of clustering by the feature amount classifying unit 100 is mapped to an actual image by the region dividing unit 101. Then, as shown in FIG. 5A, a plurality of clustering regions composed of similar pixels are formed and stored in the database 111 as label images with labels.

  Next, an example of region integration will be described. First, the smallest cluster area extraction unit 112 extracts the smallest minimum clustering area from the clustering areas stored in the database. In addition, adjacent clustering regions adjacent to the minimum clustering region extracted by the integrated region determination unit 113 are extracted.

  Here, when the minimum clustering area has a number of pixels equal to or smaller than a predetermined minute pixel threshold (for example, 1/100 of the total number of pixels), in the area integration unit 110, the minimum clustering area has the largest number of boundary pixels (peripheral length). It is integrated with many adjacent clustering regions. Specifically, it is assumed that the clustering area A in FIG. 5A is the minimum clustering area having the number of pixels equal to or smaller than a predetermined minute pixel threshold. Since the clustering area A is adjacent to the clustering areas C and D, the clustering areas C and D are adjacent clustering areas.

  Therefore, the region integration unit 110 calculates the number of adjacent pixels where the minimum clustering region A and the clustering regions C and D are in contact with each other. In FIG. 5A, the number of boundary pixels with the adjacent clustering region D is larger than the number of boundary pixels with the adjacent clustering region C. Therefore, the clustering area A is integrated with the clustering area D as shown in FIG.

  On the other hand, when the minimum clustering area has a number of pixels equal to or smaller than a predetermined small pixel threshold (for example, 1/10 of the total number of pixels), in the area integration unit 110, the adjacent clustering area where the minimum clustering area is close in the feature space Integrated with. Specifically, in FIG. 5B, it is assumed that the clustering region B is a minimum clustering region that is equal to or smaller than a predetermined small pixel threshold value. Then, the adjacent clustering regions of the clustering region B are the clustering regions C and D. Therefore, for example, when the texture information is based on the distance, it is determined which of the clustering regions C and D is close to the texture of the clustering region B. Then, as shown in FIG. 5C, the clustering region B is integrated with the clustering region D which is the closest distance in the feature space.

  In the region integration unit 110, the above-described operation is performed until, for example, the minimum clustering region extracted by the minimum cluster region extraction unit 112 has a number of pixels larger than a predetermined small pixel threshold, and the image is displayed in each object region OR. Each area is divided (see FIG. 2C).

  FIG. 6 is a block diagram showing an example of the feature quantity extraction means 40 and the feature vector generation means 46. The feature quantity extraction means 40 will be described with reference to FIG. The feature quantity extraction means 40 includes an image conversion means 41, an edge image generation means 42, a correlation feature quantity extraction means 43, an edge feature quantity extraction means 44, a color feature quantity extraction means 45, and the like.

  The image conversion means 41 converts the block area expressed by the RGB color system into the YCC color system. At this time, the image conversion means 41 identifies a block area included in one object area OR among a plurality of block areas BR constituting the image. Note that, among the plurality of block areas BR constituting the image, the block area BR that straddles the boundary between the object areas OR is not used for determining the type of the object area OR, so that the feature amount is not extracted. ing.

  The edge image generation means 42 has a function of generating an edge image using the Y component sent from the image conversion means 41 and sending the generated edge image to the correlation feature quantity extraction means 43 and the edge feature quantity extraction means 44. Here, the edge image generation means 42 generates a vertical edge image using the vertical edge detection filter shown in FIG. 7A and uses the horizontal edge detection filter shown in FIG. An image is generated.

  Note that the edge image generation means 42 uses an edge detection filter (prewitt filter) as shown in FIG. 7, for example, an edge detection filter in which a larger weight is given to the upper, lower, left, and right pixels than on the diagonal line. An edge detection method using (Sobel filter) or other known edge detection methods can be used.

  The correlation feature amount extraction unit 43 in FIG. 6 extracts a correlation feature amount indicating the degree of regularity of change along one direction of the component signal value assigned to each pixel of the block region BR. Here, FIG. 8 shows a flowchart showing an example of the calculation method of the correlation feature quantity in the correlation feature quantity extraction unit 43, and the calculation method of the correlation feature quantity will be described with reference to FIG.

In addition, although extraction of the correlation feature-value regarding the change along a horizontal direction is demonstrated in FIG. 8, the correlation feature-value with respect to the change along a vertical direction is also extracted by the same method. Further, F _i (x) shown below indicates the component signal value of the x-th pixel (i = 0 to 31, x = 0 to 31) in the i-th row, and F _j (x) indicates the component signal value in the j-th row. The component signal value of x pixel (i = 0-31, x = 0-31) shall be shown.

First, using the vertical edge image calculated by the edge feature quantity extraction means 44, the change of the component signal values F _i (x) and F _j (x) along each row of the vertical edge image is normalized (step ST1). Specifically, the difference between the component signal value F _i (x) and the average value F _i is divided by the standard deviation δ _i to obtain a normalized component signal value F _i ′ (x).

F _i ′ (x) = (F _i (x) −F _i ) / δ _i
Similarly, the difference between the component signal value F _j (x) of j rows and the average value F _j is divided by the standard deviation δ _i to obtain a normalized component signal value F _i ′ (x).

F _j ′ (x) = (F _j (x) −F _j ) / δ _j
As described above, the correlation feature value is obtained by normalizing the component signal value in order to derive the correlation feature value indicating the cross-correlation of the variation pattern itself by eliminating the difference in the fluctuation range and the average value between the rows. It is. When F _i (x) and F _j (x) are constant values and the standard deviation is 0, F _i ′ (x) = 0 (constant) and F _j ′ (x) = 0 (constant). To do.

Then, with respect to the combination of two different rows (i-th row and j-th row), the cross-correlation function is obtained using the normalized component signal values F _i ′ (x) and F _j ′ (x) regarding these two rows

Is derived (step ST2). Conceptually, the cross-correlation function is obtained by dividing the two component normalized component signal values F _i ′ (x) and F _j ′ (x) by d pixels as shown in FIG. Multiply them by shifting and take the sum. Then, a cross-correlation function G _ij (d) as a function of d as shown in FIG. 9B is obtained.

Next, the maximum correlation value is calculated from the correlation values obtained when d = 0 to 31 is substituted into the calculated cross-correlation function G _ij (d) (step ST3).

  In this operation, the maximum correlation value is calculated for all combinations of two rows (step ST1 to step ST4). Here, in the block region of 32 pixels × 32 pixels, the maximum correlation value of all combinations of the 0th to 31st rows is calculated. Then, an average value and a standard deviation of all the calculated maximum correlation values are calculated, and the average value and the standard deviation are set as correlation feature amounts (step ST5).

  Similarly, the average value and the standard deviation of the maximum correlation values related to changes along the vertical direction are calculated as correlation feature amounts (steps ST1 to ST5).

  The correlation feature amount calculated as described above indicates whether or not there is a regular pattern in the block region BR constituting the object. The larger the average value of the maximum correlation values is and the smaller the standard deviation is, It means that a regular pattern is formed. In general, natural objects included in captured images have few regular patterns, continuous patterns, and periodic patterns, and are often composed of random patterns. On the other hand, artifacts such as buildings and cobblestones are often composed of regular patterns. Therefore, a part of an image of a building or the like in which the block region BR is artificially created by extracting a correlation feature amount indicating whether or not the block region BR constituting the object forms a regular pattern. Or a part of an image of a natural object.

Note that the correlation feature quantity extraction unit 43 may simply extract the average value and the standard deviation of the sum of the normalized component signal values F _i ′ (x) and F _j ′ (x) as the correlation feature quantity. However, as described above, if the average value and the standard deviation of the maximum value of the cross-correlation function are used as the correlation feature amount, for example, the pattern of the block region BR in which a regular pattern or ripple is photographed in an oblique direction Accordingly, it is possible to derive an appropriate correlation feature amount indicating regularity of the block, and it is possible to extract a correlation feature amount that accurately represents the feature amount related to the correlation of the block region. Here, the case where the pixel line is shifted pixel by pixel (d = 0, 1, 2,... 31) is mentioned, but the maximum correlation value is calculated while shifting a plurality of pixels, such as shifting by two pixels. You may make it do.

  The edge feature quantity extraction means 44 extracts the feature quantity of the edge component of the block area BR. Specifically, the edge feature quantity extraction unit 44 calculates an average value and a standard deviation of each component signal value for the vertical edge image and the horizontal edge image generated using the edge detection filter (see FIG. 7). Four edge feature values are output.

  In this way, by using the average value of the edge component as the feature value of the edge component, it is possible to classify “sky” with few edges among natural objects and “water” and “plants” with many edges among natural objects. . Further, by extracting the vertical edge component and the horizontal edge component of the block region BR as edge feature amounts, for example, “water”, an object having different edge component characteristics depending on the direction, “plant”, “ Objects that form relatively uniform edges in the vertical and horizontal directions such as “flower garden” can be classified.

  The color feature amount extraction unit 45 extracts a color feature amount indicating the color feature of the block region BR. Specifically, the color feature amount extraction unit 45 includes a luminance component (Y component) for 32 × 32 pixels and two color difference components (Cr, Cb) constituting the block region BR expressed in the YCC color system. An average value and a standard deviation of each component signal value are calculated, and six color feature amounts are extracted from one block area.

  Note that the color feature quantity extraction unit 45 extracts the color feature quantity after conversion from the RGB color system to the YCC color system. For example, the color feature quantity extraction unit 45 maintains the color for each component (RGB) in the RGB color system. The feature amount may be extracted, or the image conversion unit 41 may convert the RGB color system block area BR into the Lab color system and extract the color feature amount for each component of Lab. Also good. Further, the color feature quantity extraction means 45 extracts the average value and standard deviation of each component signal value as the color feature quantity. For example, other representative values such as the maximum value, the minimum value, and the quantile are used as the color feature. It may be used as a quantity.

  Therefore, the feature quantity extraction unit 40 extracts 14 feature quantities from the block region BR, and the feature vector generation unit 46 uses a 14-dimensional feature vector having the extracted 14 feature quantities as vector components. Is supposed to generate.

  Next, the block area identifying means 30 will be described with reference to FIG. The block area identification means 30 identifies the type of each block area BR using the feature vector. Specifically, the block area identification unit 30 includes a representative vector database DB having representative vectors provided for each type, and a vector search unit 50 for searching for a winner representative vector most similar to a feature vector from the representative vector database DB; And a type output means 60 for outputting the type indicated by the searched winner representative vector.

  Here, as shown in FIG. 10, the representative vector database DB has a structure in which a plurality of representative vectors are arranged in a two-dimensional space, and each type is determined by the position of each representative vector. This representative vector is a 14-dimensional vector, similar to the feature vector. Then, for example, the vector search means 50 searches for a representative vector (winner representative vector) having the closest Eugrid distance to the feature vector, and the kind output means 60 sets the kind to which the searched winner representative vector belongs as the kind of the block area BR. It is designed to output.

  In FIG. 10, the representative vector of the representative vector database DB is shown in a two-dimensional space, but it is sufficient that the representative vector has a plurality of representative vectors. In this case, the representative vector is represented for each type (each class). An appropriate number of vector boxes are prepared, and a winner representative vector is searched from representative vectors in a plurality of boxes.

  The representative vector as shown in FIG. 10 uses a learning vector quantization algorithm (LVQ) (reference documents: Tanibe, Sugawara, Yamaguchi “Neural network and fuzzy signal processing” Corona, 1998). Learned. Specifically, first, a plurality of representative vectors whose initial values are set at random for each type are arranged in a matrix. In this state, when a feature vector whose type is known in advance is input to the vector search means 50, a winner representative vector is searched from the representative vector database DB.

  At this time, when the type of the feature vector and the type of the winner representative vector are the same, the winner representative vector is corrected so that the distance from the feature vector is closer. On the other hand, when the type of the feature vector and the type of the winner representative vector are different, the winner representative vector is corrected so that the distance from the feature vector is longer. The representative vector modified here is only the winner representative vector. By repeating this learning process, the representative vector for each type is learned.

  10 illustrates the case of the LVQ1 algorithm, the LVQ2 algorithm may be used. LVQ2 is an improved version of LVQ1. In LVQ1, the boundary of each type is represented by a line. In LVQ2, this line is represented by an area having a certain line width (window area). When the representative vector search means 50 searches for a winner representative vector, two winner representative vectors, ie, the most similar representative vector and the second most similar representative vector, are extracted. Of the two extracted representative vectors, when one of the types (classes) is the same type as the feature vector and the other type (class) is different from the type of the feature vector, the same type of representative vector is The feature vector is modified so as to approach the feature vector, and the different representative vector is modified to move away from the feature vector. Thereby, when erroneous recognition occurs in the learning process of LVQ1, that is, when the winner representative vector that should not win becomes a winner representative vector with a slight difference, correct learning can be performed. Thus, it is possible to learn representative vectors by enabling classification with higher accuracy.

  Furthermore, an LVQ3 algorithm obtained by improving the LVQ2 algorithm may be used. This LVQ3 solves the problem that the representative vector moves away from the correct type (class) when learning progresses in LVQ2, and this is the case where the representative vector in the window area becomes the winner representative vector. In addition to the learning process in LVQ2, when the two winner representative vector types (class) and the feature vector type (class) are all the same, the two winner representative vectors are brought closer to the feature vector. It is to be corrected. Thereby, even when the learning of the representative vector advances, the classification can be performed with higher accuracy, and the learning of the representative vector can be performed.

  Further, the representative vector search means 50 compares the distance between the winner representative vector and the feature vector, and when the distance is larger than the set threshold value, the type output means 60 outputs that the type of the block region BR is unknown. Is supposed to do. Specifically, FIG. 11 is a flowchart illustrating an example when outputting the types of feature vectors. First, the plurality of learned representative vectors are compared with the feature vector, and the representative vector having the smallest Euclidean distance from the feature vector is specified as the winner representative vector (step ST10).

  Next, the Euclidean distance between the winner representative vector and the feature vector is calculated (step ST11), and the calculated distance is compared with a predetermined threshold (step ST12). As a result, when the Euclidean distance is equal to or smaller than the threshold value, the type to which the winner representative vector belongs is specified as the type of the current block area (step ST13).

  On the other hand, when the Euclidean distance between the winner representative vector and the feature vector is larger than the threshold value, it is determined that the type of the block area BR is unknown, and the fact that it is unknown is output (step ST14). In this way, by using the set threshold value, it is possible to avoid a situation where the type is identified with difficulty even when an image is input in which the two-dimensional space has not been learned in advance. Reliability of the results can be increased. That is, the fact that the type is not unknown but has been identified as a predetermined type means that the type has been identified with a certain degree of certainty, resulting in an increase in the reliability of type identification.

  Here, a fixed threshold value may be set as the set threshold value, but preferably the representative having the largest distance from the winner representative vector among a plurality of representative vectors in the vicinity region of the winner representative vector. The distance between the vector and the winner representative vector is used. This is to reflect the difference in the “defense range” of each representative vector caused by the deviation of the learning contents and the difference in the diversity of each type of image in the threshold value.

  Note that an average value or the like may be used as the threshold value instead of the maximum value described above, and the size of the neighborhood region is not limited to 3 × 3, and for example, the threshold value is calculated from the neighborhood region of 5 × 5. It may be. Alternatively, the standard deviation or the like of the vector length of the representative vector in the vicinity region may be used as a reference.

  FIG. 12 is a flowchart showing a preferred embodiment of the object identification method of the present invention. The object identification method (step ST20 to step ST27) will be described with reference to FIGS.

  First, an object area OR is generated by dividing the image input by the object area generation means 20 into areas for each object (see FIGS. 2 to 5). On the other hand, a plurality of block regions BR having an image input by the block region generating means 10 and having a set number of pixels (for example, 32 × 32 pixels) and smaller than the object region OR are generated (step ST20). Next, a plurality of feature amounts are extracted from the block region BR by the feature amount extraction unit 40 (see FIGS. 6 to 9) (step ST21), and a 14-dimensional feature vector is generated by the feature vector generation unit 46. (Step ST22).

  Thereafter, the block area identification means 30 identifies the type of the block area BR using the generated feature vector (see FIG. 10). Specifically, the representative vector search means 50 searches for the winner representative vector from the representative vector database DB (step ST23). Then, it is determined that the type to which the winner representative vector belongs is the type of the block area BR (see FIG. 11), and is output from the type output means 60 (step ST24). This operation is performed for all the block areas BR (step ST21 to step ST25).

  Thereafter, in the object identification means 70, the types assigned to each object area OR are totaled (step ST26). The most common type is output as the type of the object area OR (step ST27).

  According to the above-described embodiment, by using the block region BR for identifying the type of the object region OR, it is possible to determine the image structural feature by determining the type of the object region OR compared to the case of identifying the type for each pixel. Therefore, the object type can be accurately identified.

  Further, by identifying the type for each block area BR, the types of the block area BR are aggregated for each object area OR, and the type of the object area OR is identified to thereby identify a part of the block area BR of the object area OR. Even if there is something that was not identified as the original type, it is possible to absorb the erroneous recognition and identify the type of the object accurately and automatically.

  In addition, the feature amount extraction unit 40 includes a correlation feature amount extraction unit 43 that extracts a correlation feature amount indicating the degree of regularity of change along one direction of the component signal value assigned to each pixel of the image. By doing so, it is possible to extract a feature value that serves as an index for distinguishing between an image having a regular pattern often found in artifacts and an image having a random pattern often found in natural objects due to the correlation feature value. Appropriate types of identification can be made.

  Further, if the correlation feature quantity extraction means extracts the correlation feature quantity along the vertical direction of the image and the correlation feature quantity along the horizontal direction of the image, the correlation feature quantity extraction means regularly in the vertical direction and the horizontal direction. Can be distinguished from those in which a regular pattern is formed and those in which a regular pattern is formed in either the vertical direction or the horizontal direction.

  Further, the correlation feature amount extraction means calculates a plurality of values calculated by inputting the component signal value of the pixel to a predetermined cross-correlation function while shifting one of the two pixel lines one pixel at a time in the pixel line formation direction. If the correlation feature value is calculated using the largest correlation value among the correlation values, not only the regular pattern formed in the vertical or horizontal direction of the image but also the diagonal direction of the image. Therefore, regular patterns formed in the vertical, horizontal, and diagonal directions of images can be extracted as correlation features. Can do.

  Further, if the edge feature quantity extraction unit 44 calculates the average value and standard deviation of the edge components in the vertical and horizontal directions of the image, respectively, the vertical and horizontal directions such as “water (wave)”, for example. Depending on the edge feature, it is possible to distinguish between those having different edges and those having relatively uniform edges in the vertical and horizontal directions of “plants (flower gardens, etc.)”.

  Further, the block area identification unit 30 is most similar to the feature vector among the representative vector database DB storing a plurality of representative vectors generated for each type of the block area BR and the plurality of representative vectors of the representative vector database DB. If the configuration includes a representative vector search means for searching for the winner representative vector and a type output means for outputting the type indicated by the winner representative vector searched by the representative vector search means 50, the type can be identified with high accuracy. Can do.

  Furthermore, when the representative vector searches for the winner representative vector using a feature vector whose type is known in advance, and the type of the feature vector is different from the type indicated by the winner representative vector, the distance between the winner representative vector and the feature vector Is obtained by learning so that the distance between the winner representative vector and the feature vector is close when the type of the feature vector and the type indicated by the winner representative vector are the same. Sometimes it is possible to identify the type using a representative vector with high accuracy.

  In addition, if the representative vector database DB has a structure in which representative vectors are arranged in a two-dimensional space and the representative vectors in the vicinity area of the winner representative vector are learned in the same manner as the winner representative vector, the same object Since a representative vector that can correspond to various variation feature vectors included in the region OR can be generated, the accuracy of type identification can be improved.

  Further, the vector search means 50 detects the distance between the feature vector and the winner representative vector, and determines that there is no winner representative vector when the detected distance is smaller than the set threshold value. If it is determined that the type is unknown when it is determined that there is no winner representative vector, if the reliability of identification of the type having a low maximum component is low, the type is not identified without identifying the type. Therefore, the reliability of type identification can be improved.

  In addition, when the set threshold is the distance between the representative vector having the largest distance from the winner representative vector among the plurality of representative vectors in the neighborhood area of the winner representative vector and the winner representative vector, It is possible to reflect the difference in the reliability of type identification at each position of the winner representative vector caused by the difference in variety of types of images in the set threshold value.

  The embodiment of the present invention is not limited to the above embodiment. For example, various modifications as shown below are possible. For example, in FIG. 1, the block area BR generated by the block area generation means 10 is sent as it is to the block area identification means 30. For the determination result for each block area BR, for example, morphological processing, closing operation, etc. You may make it send to the block area | region identification means 30 after performing a smoothing process. As a result, the isolated noisy elements included in the block region BR are discarded, and the accuracy of type identification can be improved.

  Further, FIG. 13 is a schematic diagram showing another embodiment of the object identification device of the present invention. In the representative vector learning step shown in FIG. 10 described above, only the winner representative vector is corrected. However, in FIG. 13, not only the winner representative vector but also a representative vector in the vicinity thereof is corrected. That is, in the normal LVQ algorithm, the neighborhood function is not used, and the number of representative vectors is generally reduced. On the other hand, when identifying the type of an image, there are various variations even for the same object, and it is difficult to prepare everything according to these variations as a learning sample. Therefore, it is desirable that the type of an object is learned, but the type can be identified even when a feature vector that is not used as a learning sample for the object is input.

  Therefore, the LVQ update amount is multiplied by the neighborhood function used for the self-organizing map so that the representative vector of the 3 × 3 neighborhood region is updated together with the winner representative vector. Specifically, if the winner representative vector belongs to the correct type (class), the representative vector in the vicinity region of the winner representative vector is corrected so as to approach the feature vector. On the other hand, if the winner representative vector belongs to the wrong type (class), the representative vector in the vicinity region of the winner representative vector is corrected in a direction away from the feature vector. Thereby, not only identification of the block region BR similar to the learned sample but also identification of types corresponding to various variations of the types of learning samples can be performed.

  FIG. 14 is a schematic diagram showing a third embodiment showing a block area identifying means in the object identifying apparatus of the present invention. In FIG. 14, in the representative vector database DB, a two-dimensional space in which representative vectors are arranged has a hierarchical structure. Specifically, the representative vector database DB includes a category map CM for classifying the category of the block region BR and a type map KM for identifying the type of the block region BR.

  The category map CM has a structure in which categories CA1 to CA4 are defined for each region in the two-dimensional space, and a plurality of classification representative vectors are arranged in each category CA1 to CA4. The category CA1 is associated with the type map KM1, the category CA2 is associated with the type map KM2, the category CA3 is associated with the type map KM3, and the category CA4 is associated with the type map KM4. Each type map KM1 to KM4 has a structure in which types KI11 to KI44 are defined for each region in the two-dimensional space, and a plurality of identification representative vectors are arranged in each type KI11 to KI44. ing.

  Specifically, for example, it is assumed that the category CA1 of the category map CM is set to “sky”, the category CA2 is “plant”, the category CA3 is “water”, and the category CA4 is “ground”. Then, the type map KM1 associated with the category CA1 belongs to the category CA1 such that the type KI11 is “blue sky”, the type KI12 is “cloudy sky”, the type KI13 is “evening sky”, and the type KI14 is “night sky”. A type is assigned.

  Here, the representative vector search means 50 classifies the category using the correlation feature quantity and the edge feature quantity in the category map CM, and identifies the type using the color feature quantity in the type map KM. It has become. Therefore, the feature vector generation means 46 generates the first feature vector CB1 having the correlation feature quantity and the edge feature quantity as vector components, and the second feature vector CB2 having the color feature quantity as the feature vector. . The type output means 60 outputs the type to which the winner representative vector belongs and also outputs the classified category.

  In this way, the representative vectors used in the LVQ can be suppressed to a low dimension by hierarchically identifying the types of the block regions BR using different feature amounts, so that the types can be identified efficiently and accurately. It can be carried out. Further, if the type output means 60 outputs the category indicated by the searched classification representative representative vector and the type indicated by the identification winner representative vector, the type of block area is identified when identifying the object area OR. At the same time, it is possible to identify the category in consideration of the category, and to provide the user with information related to the category of the object area OR.

  FIG. 15 is a block diagram showing a fourth embodiment showing a block area generating means in the object identification device of the present invention. The block area generation means in FIG. 15 has a function of generating a plurality of first block areas obtained by dividing an image into a mesh shape and a plurality of first block areas and a second block area having a phase shifted in a mesh form. Is.

  Specifically, the block area generating means described above generates, for example, a block area BR having a set number of pixels of 32 pixels × 32 pixels by mechanically dividing it into a mesh shape (see FIG. 15A). Furthermore, it has a function of generating a mesh-like block region BR shifted by a half block (16 pixels) with respect to the horizontal direction and the horizontal direction as shown in FIG. Then, among the generated block areas BR of FIGS. 15A and 15B, the type identification is performed using the block area BR that does not include the boundary of the object area OR.

  Thus, the accuracy of identification can be improved by increasing the number of block areas BR used for identifying the type of object area OR. In other words, as described above, the block area including the boundary of the object area OR has the object area OR in order to prevent a reduction in identification accuracy due to a mixture of features of a plurality of areas and an edge of the boundary. Not included in the type of aggregation. Therefore, when the object area OR is small, the number of block areas BR to be generated is small, and in the case of an object area OR having a complicated shape, the boundary with other object areas OR is increased. The number of block areas BR to be reduced is reduced. For this reason, the accuracy of the identified type is lowered, and in particular, if the image is a little complicated, it cannot be identified and it is determined that many object regions OR are unknown.

  At this time, if the block region BR is shifted by a half block by generating the block regions BR that are out of phase as shown in FIGS. 15A and 15B, the boundary of the object region OR is not included. The generation of the block region BR can be promoted. For this reason, the number of block regions BR used for type identification can be increased, and more accurate type identification can be performed.

  In FIG. 15B, the block region BR shifted by a half block with respect to the horizontal direction and the vertical direction is generated, but the half block only in the horizontal direction as shown in FIG. 15C. The block region BR shifted by the amount (16 pixels) may be generated, or the block region BR shifted by half a block only in the vertical direction may be generated as shown in FIG. Further, in the block area identification means 30, all of the block areas BR shown in FIGS. 15A to 15D may be used, or any one of the block areas BR may be used in combination. Further, in FIGS. 15A to 15D, the block region generation means 10 is illustrated as being shifted by a half block, but is not limited to shifting by a half block, for example, 1/4 block Processing such as shifting by a minute (8 pixels) may be performed.

  FIG. 16 is a block diagram showing a fifth embodiment showing the block area generation means 10 in the object identification device of the present invention. 16 has a function of generating a plurality of resolution-converted images having different resolutions from an image, and a function of generating a block area having a set number of pixels from the generated resolution-converted images. It is. Specifically, the block area generation unit 10 performs a known resolution conversion technique such as a Gaussian pyramid or a wavelet transform on the entire image to generate a plurality of resolution conversion images. Then, the block area generation unit 10 generates the block area BR by dividing the generated plurality of resolution conversion images into meshes for each set number of pixels.

  Therefore, the block region BR is generated from the entire image as shown in FIG. 2A, and is generated from the entire image having different resolutions as shown in FIGS. 16A and 16B. Then, type identification is performed for each block region BR generated from a plurality of resolution-converted images. At this time, the block area generation unit 10 does not change the number of set pixels (for example, 32 pixels × 32 pixels). This is to prevent the size of the block area BR at the time of learning from being different from the size of the block area at the time of identification when the block area identifying means 30 identifies the type based on the feature amount. It is.

  Thus, by generating the block area BR using resolution-converted images having different resolutions, it is possible to improve the accuracy of identifying the type of the block area BR. That is, in a normal whole image, the way the subject is captured differs between an image obtained by photographing the same subject from near and an image obtained from far away. Even if the subject type cannot be identified when shooting from near, the subject type can be identified when shooting from a distance, or vice versa. Therefore, by using a resolution-converted image, it is possible to prevent a decrease in accuracy due to the difference in the way of capturing and improve the accuracy of identifying the type of the block region BR.

  In FIG. 16, the block area BR is generated by mechanically dividing the entire image into a mesh shape. However, the block area BR may be generated by shifting by half a block as shown in FIG. .

  FIG. 17 is a block diagram showing a sixth embodiment of the object identification device of the present invention. In the object identification device 300 of FIG. 17, parts having the same configuration as the object identification device 1 of FIG. The object identification device 300 in FIG. 17 is different from the object identification device 1 in FIG. 1 in that the block area generation unit 310 scans a cut frame having a set number of pixels in the object area OR as shown in FIG. Thus, it has a function of generating an image surrounded by a cutting frame as a block area.

  FIG. 19 is a flowchart showing an example of the operation of the block area generating unit 310 of FIG. 17, and an example of a method for generating the block area BR will be described with reference to FIGS. First, the object area generation means 20 generates a plurality of object areas from the entire image (step ST30). Thereafter, an area ID is assigned to each generated object area OR (step ST31).

  Then, the object area OR for generating the block area BR is determined from the generated object areas OR (step ST32), and the block area BR is generated (step ST33). This block area generation step (step ST33) is performed for all object areas OR included in the entire image (step ST32 to step ST34). Thereafter, the types of the plurality of generated block areas BR are identified by the block area identifying means 30.

  FIG. 20 is a flowchart showing an example of the block region generation step (step ST33). The step of generating the block region BR will be described with reference to FIG. First, the object area OR for generating the block area BR is determined (step ST33-1), and a cut frame is set at the start point in the object area OR (step ST33-2). Specifically, as shown in FIG. 18, the cut frame is positioned so that the upper left corner of the cut frame is positioned at the upper left corner of the object area OR. Then, it is determined whether or not all the area IDs in the cutting frame match (step ST33-2). If all the area IDs in the cutting frame match, the area surrounded by the cutting frame is blocked. The area BR is generated (step ST33-4).

  Thereafter, the cut frame is shifted in the horizontal direction (right direction) by, for example, 8 pixels (step ST33-5). Here, it is determined whether or not the cutting frame has been scanned to the rightmost end of the object area OR (step ST33-6). If the cutting frame has not been scanned, generation of a block area is subsequently performed (steps ST33-2 to ST33-2). Step ST33-5). On the other hand, when the cutting frame is scanned to the rightmost end of the object area OR, the cutting frame is shifted by, for example, 8 pixels in the vertical direction (downward), and also moved in the horizontal direction to the left end of the object area OR. Positioning is performed (step ST33-7). Thereafter, the block region BR is generated in the horizontal direction (step ST33-2 to step ST33-6). When the cut frame is scanned to the lowest end of the object area OR (step ST33-8), the generation of the block area BR for one object area OR is completed.

  In this way, by generating the block area BR while scanning the cut frame within the object area OR, the number of block areas BR for identifying the type of the object area OR can be increased. The accuracy of identification can be improved.

  Note that the cut frame is illustrated as being shifted by 8 pixels with respect to the horizontal direction and the vertical direction, but may be set to a pixel smaller than the cut frame, such as 2 pixels or 4 pixels. Further, the block area BR to be cut out by the cut frame is determined by changing the area ID. However, the cut frame is mechanically arranged at a pixel pitch smaller than the cut frame, such as 2 pixels, in the vertical direction and the horizontal direction. You may make it scan in a direction. At this time, the type identification may not be performed for the block region BR in which two region IDs are included in the cut frame.

  Although an example in which the area ID of 8 pixels × 8 pixels in the cut frame is rewritten is illustrated, it is only necessary to set a pixel smaller than the cut frame, such as 2 pixels × 2 pixels or 4 pixels × 4 pixels, for example. . Further, the block area BR to be cut out by the cut frame is determined by changing the area ID. However, the cut frame is mechanically arranged at a pixel pitch smaller than the cut frame, such as 2 pixels, in the vertical direction and the horizontal direction. You may make it scan in a direction. At this time, the type identification may not be performed for the block region BR in which two region IDs are included in the cut frame.

  FIG. 21 is a block diagram showing a seventh embodiment of the object identification device of the present invention. The object identification device 500 will be described with reference to FIG. In the object identification device 500 of FIG. 21, parts having the same configuration as the object identification device 1 of FIG.

  In the object identification device 500 of FIG. 21, the object area generation means 20 first generates an object area OR. Then, the feature quantity extraction means 540 extracts the object feature quantity from the generated object area OR. After that, the type of the object area OR is identified by the mapping unit 50 and the type output unit 60 using the extracted object feature amount.

  Further, the feature amount extraction unit 540 extracts an object feature amount using the entire image that has been subjected to the predetermined image conversion process by the image conversion unit 510 and the generated object region OR. Specifically, the image conversion unit 510 has a function of converting the entire image composed of the RGB color system into the YCC color system and generating three images for each component of the YCC color system. Furthermore, the image conversion means 510 generates a vertical edge image and a horizontal edge image generated from the Y component. The feature quantity extraction means 40 extracts object feature quantities from five images, that is, three images for each YCC component, a vertical edge image, and a horizontal edge image.

  Here, the feature amount extraction unit 540 generates a distribution (histogram) of pixel values for each object region OR by combining the region division results of the images. Then, the feature amount extraction unit 40 calculates an average value and a standard deviation from the histogram, and generates an object feature amount. Note that a representative point of the histogram (for example, maximum value, minimum value, median value, quantile, etc.) may be used as the feature quantity. In addition, the learning sample of the self-organizing map SOM is performed using the above-described image conversion on the block region BR and using the feature amount extracted from the above-described histogram.

  Note that the feature quantity extraction unit 540 described above may extract a feature quantity as shown in FIG. 6 from the object region OR and use it as the object feature quantity. Note that the feature quantity extraction means 540 described above may extract feature quantities as shown in FIGS. 6 and 19 from the object region OR and use them as object feature quantities. Further, the above-described image conversion unit 510 uses a plurality of resolution conversion images having different resolutions by performing multi-resolution conversion on the entire image, an image obtained by converting the entire image from the RGB color system to the Lab color system, a morphology filter, and the like. A filtering image or the like obtained by extracting a structure having a specific shape may be generated, and the feature amount extraction unit 540 may extract a feature amount from each image.

  As a result, even when the area shape of the object area OR is complex or small, the type of the object area OR can be reliably identified. That is, when the entire image is divided into block areas BR, the object area OR is divided into a plurality of block areas BR when the object area OR is complicated, and the block area used for type identification when the object area OR is small. The number of BR will decrease.

  On the other hand, the result of identification by the block area identifying means 30 is assigned to all the pixels included in the block area BR, and the type having the largest number of pixels among the types assigned to the pixels constituting the object area OR is assigned to the object area. It may be possible to identify the type of OR. However, it is necessary to identify the type of the block region BR including the boundary of the object region OR. As a result, there is a problem that the accuracy of identifying the type is lowered. Therefore, by extracting the feature amount from the object area OR itself and identifying the type, it is possible to accurately identify the type of the object area OR having a complicated shape or the object area OR having a small shape.

  FIG. 22 is a block diagram showing an eighth embodiment of the object identification device of the present invention. The object identification device 600 will be described with reference to FIG. In the object identification device 600 of FIG. 22, parts having the same configurations as those of the object identification device 1 of FIG. 1 and the object identification device 500 of FIG. The object identification device 600 of FIG. 22 differs from the object identification device 500 of FIG. 21 in that the object identification device 600 further includes a normalization unit 630 that generates a standardized object region obtained by standardizing a circumscribed rectangular image of the object region. Then, the feature quantity extraction unit 40 extracts the object feature quantity from the standardized object area. Therefore, the size of the object area OR when extracting the object feature amount is the same regardless of the object area OR of any image.

  As described above, by extracting the object feature amount after normalizing the object area OR, accurate identification can be performed without depending on the size of the object area included in the entire image and the type identification accuracy. it can. That is, the size of the object included in the entire image varies depending on the situation at the time of shooting. Therefore, it is possible to identify types with high accuracy by extracting object feature amounts and identifying types after standardizing each object region OR. In this way, the object region OR is normalized and then the object feature amount is extracted to provide robustness against the size variation, and the type identification accuracy depends on the size of the object region included in the entire image. And accurate identification can be performed.

The block diagram which shows 1st Embodiment of the object identification device of this invention The figure which shows a mode that a kind is identified for every object contained in an image in the object identification device of this invention. The block diagram which shows an example of the object area | region production | generation means in the object identification apparatus of this invention The figure which shows a mode that an image is divided | segmented into an area | region by the object area | region production | generation means of FIG. The figure which shows a mode that a clustering area | region is integrated by the object area | region production | generation means of FIG. 2, and an object area | region is formed. The block diagram which shows an example of the feature-value extraction means in the object identification device of this invention The figure which shows an example of the edge filter used in the edge image generation means of FIG. FIG. 6 is a flowchart showing an example of the operation of the correlation feature amount extraction unit of FIG. The graph which shows an example of the cross correlation function in the correlation feature-value extraction means of FIG. Schematic showing the structure of the representative vector database in FIG. The flowchart which shows the operation example of the block area identification means of FIG. The flowchart figure which shows preferable embodiment of the object identification method of this invention The block diagram which shows 2nd Embodiment of the object identification device of this invention The block diagram which shows 3rd Embodiment of the object identification device of this invention The block diagram which shows 4th Embodiment of the object identification device of this invention The block diagram which shows 5th Embodiment of the object identification device of this invention The block diagram which shows 6th Embodiment of the object identification device of this invention FIG. 17 is a schematic diagram showing a state of a cutting frame operated in the object identification device of FIG. FIG. 17 is a flowchart showing an operation example of the object identification device of FIG. FIG. 17 is a flowchart showing an operation example of the object identification device of FIG. The block diagram which shows 7th Embodiment of the object identification device of this invention The block diagram which shows 8th Embodiment of the object identification device of this invention.

Explanation of symbols

1, 300, 500, 600 Object identification device 10, 310 Block area generation means 20 Object area generation means 30 Block area identification means 40 Feature quantity extraction means 41 Image conversion means 42 Edge image generation means 43 Correlated feature quantity extraction means 44 Edge features Quantity extraction means 45 Color feature quantity extraction means 46 Feature vector generation means 50 Representative vector search means 60 Type output means 70 Object identification means BR Block area CM Category map DB Representative vector database KM Type map OR Object area P Image

Claims (17)

In an object identification method for identifying the type of object included in an image,
A plurality of object areas obtained by dividing the image for each object, and a plurality of block areas obtained by dividing the image into a plurality of areas smaller than the object area, each having a set number of pixels;
A plurality of feature amounts are extracted from each generated block area,
Generating a feature vector having the extracted feature quantities as vector components;
Using the representative vector database in which a plurality of representative vectors indicating representative vector values in the block area type are stored for each type, the representative vector that is most similar to the feature vector is searched from the representative vector database. And
The type to which the retrieved representative vector belongs is output as the type of the block area,
The object identification method characterized in that the type of the block area that has been output is aggregated for each object area to identify the type of the object. In an object identification device for identifying the type of object included in an image,
Object region generation means for dividing the image into regions for each object to generate a plurality of object regions;
A block area generating unit configured to divide the image into a plurality of areas smaller than the object area, each having a set number of pixels, and generating a plurality of block areas;
Feature quantity extraction means for extracting a plurality of feature quantities from each block area generated by the block area generation means;
Feature vector generation means for generating a feature vector having the plurality of feature quantities extracted by the feature quantity extraction means as vector components;
Using the representative vector database in which a plurality of representative vectors indicating representative vector values in the block area type are stored for each type, the representative vector that is most similar to the feature vector is searched from the representative vector database. Representative vector search means to perform,
Type output means for outputting the type to which the representative vector searched by the representative vector search means belongs as the type of the block area;
An object identification device comprising: object identification means that aggregates the types of block areas output for each block area for each object area, and identifies the types of the objects using the aggregated results . The block area generating means generates a plurality of first block areas obtained by dividing the image into a mesh shape, and a second block area having a phase shifted from the plurality of first block areas in a mesh shape. The object identification device according to claim 2, comprising: The block area generation unit has a function of causing the object area to scan a cutting frame having the set number of pixels and generating the block area from an image surrounded by the cutting frame. The object identification device according to claim 2 or 3. The block area generating means has a function of generating a plurality of resolution conversion images having different resolutions from the image, and has a function of generating the block area from the generated plurality of resolution conversion images, respectively. The object identification device according to any one of claims 2 to 4, wherein the object identification device is characterized. The feature quantity extraction means includes a correlation feature quantity extraction means for extracting a correlation feature quantity indicating the degree of regularity of change along one direction of the component signal value assigned to each pixel of the block region. The object identification device according to any one of claims 2 to 5, wherein the object identification device is characterized in that: The correlation feature quantity extraction unit is configured to extract the correlation feature quantity along a vertical direction of the block area and the correlation feature quantity along a horizontal direction of the block area. 6. The object identification device according to 6. The correlation feature quantity extraction unit outputs a correlation value indicating a correlation between the two pixel lines from component signal values of a plurality of pixels constituting the two pixel lines formed in the same direction in the block region. Having a cross-correlation function of
The plurality of correlation values are acquired by inputting the component signal value of the pixel to the cross-correlation function while shifting one of the two pixel lines pixel by pixel in the formation direction of the pixel line. The largest maximum correlation value is calculated from a plurality of correlation values,
The maximum correlation value is calculated for all combinations of the pixel lines formed in the same direction of the block area, and the average value and standard deviation of all the calculated bond correlation values are extracted as correlation feature amounts. The object identification device according to claim 6, wherein the object identification device is an object identification device. 9. The feature amount extraction unit includes an edge feature amount extraction unit that extracts an edge feature amount indicating a feature of an edge component in a vertical direction and a horizontal direction of the block region. The object identification device according to any one of the above. 10. The feature quantity extracting unit includes a color feature quantity extracting unit that extracts a color feature quantity indicating a feature of a color component of the block region. Object identification device. The representative vector is searched for the winner representative vector using the feature vector whose type is known in advance, and when the type of the feature vector is different from the type indicated by the winner representative vector, the winner representative vector and the Learning so that the distance from the feature vector is long, and when the type of the feature vector and the type indicated by the winner representative vector are the same, the distance between the winner representative vector and the feature vector is reduced The object identification device according to claim 11, wherein the object identification device is obtained by learning. The representative vector database has a structure in which the representative vectors are arranged in a two-dimensional space, and the representative vector in the vicinity region of the winner representative vector is learned in the same manner as the winner representative vector. The object identification device according to claim 11. The representative vector database has a plurality of classification representative vectors for classifying the category of the block region, and a plurality of identification representative vectors for identifying a type from the classified categories;
The representative vector search means searches for a classification winner representative vector that is most similar to the feature vector, and from among the identification representative vectors of the type within the category indicated by the searched classification winner representative vector, 13. The object identification device according to claim 11, wherein the identification winner representative vector most similar to the feature vector is searched. 14. The object identification device according to claim 13, wherein the type output means outputs a category indicated by the searched winner representative vector for classification and a type of the identification winner representative vector. The representative vector search means detects a distance between the feature vector and the winner representative vector, and determines that there is no winner representative vector when the detected distance is smaller than a set threshold value. The object according to any one of claims 11 to 14, wherein the output means outputs that the type is unknown when it is determined that there is no winner representative vector. Identification device. The set threshold is a distance between the representative vector having the largest distance from the winner representative vector and the winner representative vector among a plurality of the representative vectors in the vicinity region of the winner representative vector. The object identification device according to claim 15. On the computer,
A plurality of object areas obtained by dividing an image for each object, and a plurality of block areas obtained by dividing the image into a plurality of areas smaller than the object area, each having a set number of pixels;
A plurality of feature amounts are extracted from each generated block area,
Generating a feature vector having the extracted feature quantities as vector components;
Using the representative vector database in which a plurality of representative vectors indicating representative vector values in the block area type are stored for each type, the representative vector that is most similar to the feature vector is detected from the representative vector database. And
Identifying the type to which the detected representative vector belongs as the type of the block region;
An object identification program for causing the identified block area types to be aggregated for each object area to identify the object type.

Images