JP4285640B2 - Object identification method, apparatus and program - Google Patents

Description

  The present invention relates to an object identification method, apparatus, and program for automatically identifying the types of objects constituting an image.

  If image information captured by a digital camera or the like can identify what image is captured in the image information, for example, classification, search, or image processing may be performed for each type of object included in the image. it can.

  For example, when classifying and searching for images, an image search system has been proposed in which similarity is determined using physical feature amounts included in images. That is, there is a technique of extracting and localizing an input image, collating the reference region with a reference image while changing the position and size of the local region, and classifying and searching for the image. Further, in the above method, there is a method for efficiently classifying and searching images by detecting the position of an object by using a color histogram of a local region and comparing the histogram with a color histogram of a reference image (for example, non-patent) Reference 1). However, in any of the above-described methods, since the similarity is identified by the physical feature amount of the image, it may be determined that what is not similar in kind is similar due to the similarity of the physical amount. There is a problem that the accuracy of the search is poor.

When performing image processing, a method of identifying a specific color region and performing different processing is known as an example of high image quality processing (see, for example, Patent Document 1). In this method, an area where a noise component is conspicuous is identified by color, and noise is removed. However, since the identification is based only on the color, for example, skin and sand may be confused. If the sand area is mistakenly recognized as a skin area and noise is removed from the sand area, the texture may be lost, resulting in an unnatural image.
Japanese Patent Publication No. 5-62879 The Institute of Electronics, Information and Communication Engineers, vol. j81-DII, no. 9, pp. 2035-2042, 1998

  As described above, when image classification, retrieval, or image processing is performed based on information obtained directly from an image, appropriate information cannot be provided to the user. As one method for solving this, it is conceivable to classify, search or perform image processing after identifying the type of object. Then, in the image classification / search, the image can be classified / searched according to the identified type. Therefore, the image classification / search can be easily and accurately performed. Even when image processing is performed, image processing can be performed using image processing conditions suitable for the object.

  To identify the type of object included in the image described above, it is necessary to extract the object area included in the image and identify the type for each object area. At this time, for example, the user may extract the object area in the image while looking at the screen and input the type for each object. However, there is a problem that it takes time and effort to give the object region type by the user.

  Therefore, an object of the present invention is to provide an object identification method, apparatus, and program that can automatically identify the type of an object included in an image.

  The object identification method of the present invention is an object identification method for identifying the type of an object included in an image, the object region comprising: an object region obtained by dividing the image into regions for each object; and the image comprising a set number of pixels. Generating a plurality of block areas divided into a plurality of small areas, identifying a type for each of the plurality of generated block areas, and identifying the type of the identified block area for each object area The method includes a step of counting, and a step of identifying the type of the object region using the totaled result.

  The object identification device of the present invention is an object identification device for identifying the type of an object included in an image, an object region generation means for dividing the image into regions for each object to generate a plurality of object regions, and the image A block area generating unit configured to generate a plurality of block areas by dividing the pixel area into a plurality of areas smaller than the object area, and a block area identifying unit for identifying a type for each of the generated block areas And object identifying means for totalizing the types of the block areas identified for each of the block areas for each object area and identifying the types of the objects using the totaled results.

  The object identification program according to the present invention generates, on a computer, an object area obtained by dividing an image into areas for each object, and a plurality of block areas obtained by dividing the image into a plurality of areas smaller than the object area, each having a set number of pixels. A procedure for identifying the type for each of the plurality of generated block regions, a procedure for counting the types of the identified block regions for each object region, and the object region using the tabulated result And a procedure for identifying the type of the program.

  Here, “object” means a subject included in an image such as a person, sky, sea, tree, building, etc., and “object region” means a region occupied by the subject in the image.

  “Identify the type of object” means that the object in the image is identified as a type such as “mountain”, “sea”, “flower”, “sky”, etc. It also includes specifying “unknown” when not sure.

  In addition, the “block area identifying unit” may be any unit that identifies a type for each block area, and a feature amount extracting unit that extracts a plurality of block feature amounts from the block region, and a plurality of the extracted block feature amounts. A mapping means for mapping an image on a two-dimensional space, a type frequency distribution map in which a type is defined for each coordinate in the two-dimensional space, and a type in which the mapped coordinates in the two-dimensional space are indicated on the type frequency distribution map May be provided as a type of block area.

  The “two-dimensional space” may be a self-organizing map in which a plurality of neurons having a learning function are arranged in a matrix.

  The “type output unit” may be any unit that outputs information regarding the type of the identified block area, and may output one identified type, and the frequency of the type for each coordinate of the self-organizing map. Each type has a type frequency distribution map that defines the value as a type index, and outputs a type vector whose vector component is a plurality of frequency values whose coordinates detected by the mapping means are indicated on each type frequency distribution map It may be.

  The “type output unit” may output a type that is the largest maximum vector component among the vector components of the type vector.

  Further, the “type output means” outputs that the type is unknown when the maximum vector component having the maximum vector component size among the type vectors is smaller than a predetermined maximum component threshold value. May be.

  Further, the “feature amount extraction means” may be any means that can extract a plurality of feature amounts indicating image features, and extracts the color component, brightness component, and image feature component of the block area as block feature amounts. For example, in addition to the correlation feature quantity extraction means for extracting the correlation feature quantity indicating the degree of regularity of change along one direction of the component signal value assigned to each pixel of the image, the edge of the image The image processing apparatus may include an edge feature amount extracting unit that extracts an edge feature amount indicating a feature of the image, and a color feature amount extracting unit that extracts a color feature amount indicating a color feature of the image.

  The “correlation feature extraction means” extracts, for example, a correlation feature along the vertical direction of the image, a correlation feature along the horizontal direction of the image, or a correlation feature along the diagonal direction of the image. What is necessary is just to extract the correlation feature quantity of at least one direction of the image.

  Further, the “correlation feature amount extraction unit” outputs a correlation value indicating a correlation between two pixel lines from component signal values of a plurality of pixels constituting two pixel lines formed in the same direction in the image. A plurality of correlation values are obtained by inputting the component signal value of the pixel into the cross-correlation function while shifting one of the two pixel lines in the pixel line forming direction one pixel at a time. The largest maximum correlation value is calculated from a plurality of acquired correlation values, the maximum correlation value is calculated for all combinations of pixel lines formed in the same direction of the image, and all the calculated maximum correlation values are calculated. The average value and the standard deviation may be extracted as the correlation feature amount.

  Further, the “block region generating means” generates, for example, a plurality of first block regions obtained by dividing the image into a mesh shape and a plurality of first block regions and a second block region having a phase shifted from each other in a mesh shape. Or a block area having a set number of pixels, such as an object area that scans a cut frame having a set number of pixels and generating an image surrounded by the cut frame as the block area. That's fine.

  Furthermore, the “block area generation unit” may have a function of generating a plurality of resolution conversion images having different resolutions from an image, and may generate a block area from each of the generated resolution conversion images.

  The object identification device of the present invention is an object identification device for identifying the type of an object included in an image, an object region generation means for generating a plurality of object regions by dividing the image into regions for each object, and the object region Feature quantity extraction means for extracting a plurality of object feature quantities from the object area generated by the generation means, and object identification for identifying the type of the object area using the object feature quantities extracted by the feature quantity extraction means Means.

  Furthermore, the object identification device may include a normalizing unit that generates a standardized object area obtained by standardizing a circumscribed rectangular image of the object area.

  Note that the “feature amount extraction means” may have a function of extracting a feature amount from the standardized object region.

  According to the object identification method, apparatus, and program of the present invention, by using the block area for identifying the type of the object area, the image structural features are compared with those in the object area as compared with the case of identifying the type for each pixel. Since it can be added to the type determination, the type of the object can be accurately identified.

  In addition, by identifying the type for each block area, the block area type is aggregated for each object area and the object area type is identified to identify the original type for some block areas of the object area Even if there is something that has not been done, the erroneous recognition can be absorbed and the type of object can be accurately and automatically identified.

  The block area identifying means extracts a feature quantity extracting means for extracting a plurality of block feature quantities from the block area, a mapping means for mapping the extracted feature quantities on the two-dimensional space, and a position on the two-dimensional space. A type output unit that has a type frequency distribution map in which a type is defined for each type, and outputs a type of block area from a position in a two-dimensional space where a plurality of feature amounts are mapped using the type frequency distribution map By doing so, the type of the block area can be identified accurately and efficiently.

  Further, if the feature quantity extracting means extracts the color component, brightness component, and image feature component of the block area as the block feature quantity, the type of the block area can be identified more accurately.

  Further, the type output means has for each type a type frequency distribution map in which the type frequency value is determined as a type index for each coordinate of the self-organizing map, and the type output means has coordinates detected by the mapping means. If a type vector having a plurality of frequency values shown on each type frequency distribution map as a vector component is output, a plurality of possible types of block areas may be output instead of outputting one identified type. Since the type of the block area can be identified from among the types, the type identification accuracy can be improved.

  Further, if the type output means identifies the type that is the largest maximum vector component among the types vector components as the type of the block region, the type having a high probability among the plurality of types is selected as the type of the block region. Therefore, the identification accuracy can be improved.

  Furthermore, when the type output means outputs that the type of the image is unknown when the maximum vector component is smaller than the predetermined maximum component threshold value, the identification reliability of the type having the low maximum component is low. Since it can be made unknown without identifying the type, the reliability of the type identification can be improved.

  Further, the block area generation means generates a plurality of first block areas obtained by dividing the image into a mesh shape, and a second block area having a phase shifted from the plurality of first block areas and the mesh shape. Since the number of block areas used to identify the type of object area can be increased, the accuracy in identifying the type of object area from the identification of the type of block area can be improved.

  In addition, when the block area generation unit scans a cut frame having a set number of pixels in the object area and generates an image surrounded by the cut frame as a block area, the type of the object area is identified. Since the number of block areas to be used can be increased, it is possible to improve accuracy when identifying the type of object area from identifying the type of block area.

  Furthermore, if the block area generation unit has a function of generating a plurality of resolution conversion images having different resolutions from the image, and each block area is generated from the generated plurality of resolution conversion images, the block area generation unit depends on the distance from the subject. Even when the way the object is captured differs depending on the image, the type of the object can be accurately identified.

  In addition, if the feature amount extraction unit includes a correlation feature amount extraction unit that extracts a correlation feature amount indicating the degree of regularity of change along one direction of the component signal value assigned to each pixel of the image. Since the feature quantity can be extracted as an index for distinguishing between an image having a regular pattern often found in artifacts and an image having a random pattern often found in natural objects, the correlation feature quantity is appropriate. Type identification can be performed.

  Further, if the correlation feature quantity extraction means extracts the correlation feature quantity along the vertical direction of the image and the correlation feature quantity along the horizontal direction of the image, the correlation feature quantity extraction means regularly in the vertical direction and the horizontal direction. Can be distinguished from those in which a regular pattern is formed and those in which a regular pattern is formed in either the vertical direction or the horizontal direction.

  Further, the correlation feature amount extraction means calculates a plurality of values calculated by inputting the component signal value of the pixel to a predetermined cross-correlation function while shifting one of the two pixel lines one pixel at a time in the pixel line formation direction. If the correlation feature value is calculated using the largest correlation value among the correlation values, not only the regular pattern formed in the vertical or horizontal direction of the image but also the diagonal direction of the image. Therefore, regular patterns formed in the vertical, horizontal, and diagonal directions of images can be extracted as correlation features. Can do.

  Furthermore, if the edge feature quantity extraction means calculates the average value and standard deviation of the edge components in the vertical and horizontal directions of the image, respectively, for example, “water (wave)” depending on the vertical and horizontal directions. Different edges can be distinguished from those having relatively uniform edges in the vertical and horizontal directions of “plants (flower garden, etc.)” by the edge feature amount.

  Further, according to the object identification device of the present invention, by extracting a plurality of object feature amounts from the object region generated by the object region generation means, by using the object feature amount, identifying the type of the object region, Even when the shape of the object area is complicated or the object area is small, the type of the object can be reliably identified.

  The image processing apparatus further includes a normalization unit that generates a standardized object region obtained by normalizing a circumscribed rectangular image of the object region, and the feature amount extraction unit extracts the feature amount from the standardized object region generated by the normalization unit. In this case, the feature amount is extracted from the standardized object region that is standardized to the same size for the object regions having different sizes in the image depending on the distance from the subject. The accuracy of type identification can be improved.

  The object identification device of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of an object identification device of the present invention. The object identification device 1 in FIG. 1 identifies the type of each object included in the entire image P, and includes a block area generation means 10, an object area generation means 20, a block area identification means 30, an object identification means 70, etc. Have

  As shown in FIG. 2A, the block area generating unit 10 in FIG. 1 has a function of generating a block area BR in which the entire image P is divided for each set number of pixels. Then, the block area generation unit 10 sends the generated block area BR to the block area identification unit 30. For example, when the number of set pixels is 32 pixels × 32 pixels, the entire image P is divided into a plurality of block regions BR composed of 32 × 32 pixels.

  As shown in FIG. 2B, the object area generation unit 20 has a function of generating an object area OR by dividing the entire image P into areas for each object. Then, the object area generation unit 20 sends the generated object areas OR to the object identification unit 70.

  The block area identifying means 30 has a function of identifying the type for each generated block area BR. In other words, the block area identifying means 30 identifies that the object in the image is of a type such as “mountain”, “sea”, “flower”, “sky”. The block area identifying unit 30 sends the identified type to the object identifying unit 70.

  The object identification unit 70 has a function of identifying the type of the object area OR by giving type information to each divided object area OR using the type of each sent block area BR. Specifically, the object identification unit 70 totals the types of the block areas BR in the object area OR. Then, the object identifying means 70 identifies the largest type information of the block region BR that is the largest among the types of block regions BR counted in a certain object region OR as the type of object. The object identifying means 70 does not count the block area BR that extends over the plurality of object areas OR. Then, as shown in FIG. 2 (c), each object area OR is in a type, and the object area OR can be identified by the type information.

  In the object identifying means 70 of FIG. 1, the type of object is determined by majority vote. However, the ratio of the largest type information among the aggregated type information (the number of maximum types max / object is configured. If the total number of block areas) is smaller than the type information threshold, the object identification unit 70 may have a function of outputting “unknown” as the type information of the object. Alternatively, when the difference between the ratio of the maximum type information and the ratio of the second largest type information is small, the object identification unit 70 may output “unknown” as the object type. This is because it may be preferable for the user to determine “unknown” rather than erroneously identifying the object type information.

  The object region generation unit 20 extracts a plurality of feature amounts from each pixel constituting the image, classifies the pixels for each similar pixel feature amount, and divides the region for each pixel classification. Area dividing means 101 for generating a plurality of clustering areas, a minimum cluster area extracting means 112 for extracting a minimum clustering area having the smallest number of pixels among the generated clustering areas, and an adjacent adjacent to the extracted minimum clustering area The integrated region determination unit 113 extracts clustering regions and the region integration unit 110 extracts object regions by integrating the generated clustering regions.

  Here, FIGS. 4 and 5 are schematic diagrams showing a process of dividing an image for each object area, and an operation example of the object area generating means 20 will be described with reference to FIG. First, as shown in FIG. 4A, it is assumed that there is an image in which pixels having similar characteristics are arranged. At this time, the feature quantity classifying unit 100 extracts a plurality of feature quantities from each pixel and generates a plurality of feature vectors having each feature quantity as an element. Thereafter, as shown in FIG. 4B, a plurality of feature vectors are classified into similar feature vectors (clustering).

  Thereafter, the result of clustering by the feature amount classifying unit 100 is mapped to an actual image by the region dividing unit 101. Then, as shown in FIG. 5A, a plurality of clustering regions composed of similar pixels are formed and stored in the database 111 as label images with labels.

  Next, an example of region integration will be described. First, the smallest cluster area extraction unit 112 extracts the smallest minimum clustering area from the clustering areas stored in the database. In addition, adjacent clustering regions adjacent to the minimum clustering region extracted by the integrated region determination unit 113 are extracted.

  Here, when the minimum clustering area has a number of pixels equal to or smaller than a predetermined minute pixel threshold (for example, 1/100 of the total number of pixels), in the area integration unit 110, the minimum clustering area has the largest number of boundary pixels (peripheral length). It is integrated with many adjacent clustering regions. Specifically, it is assumed that the clustering area A in FIG. 5A is the minimum clustering area having the number of pixels equal to or smaller than a predetermined minute pixel threshold. Since the clustering area A is adjacent to the clustering areas C and D, the clustering areas C and D are adjacent clustering areas.

  Therefore, the region integration unit 110 calculates the number of adjacent pixels where the minimum clustering region A and the clustering regions C and D are in contact with each other. In FIG. 5A, the number of boundary pixels with the adjacent clustering region D is larger than the number of boundary pixels with the adjacent clustering region C. Therefore, the clustering area A is integrated with the clustering area D as shown in FIG.

  On the other hand, when the minimum clustering area has a number of pixels equal to or smaller than a predetermined small pixel threshold (for example, 1/10 of the total number of pixels), in the area integration unit 110, the adjacent clustering area where the minimum clustering area is close in the feature space Integrated with. Specifically, in FIG. 5B, it is assumed that the clustering region B is a minimum clustering region that is equal to or smaller than a predetermined small pixel threshold value. Then, the adjacent clustering regions of the clustering region B are the clustering regions C and D. Therefore, for example, when the texture information is based on the distance, it is determined which of the clustering regions C and D is close to the texture of the clustering region B. Then, as shown in FIG. 5C, the clustering region B is integrated with the clustering region D which is the closest distance in the feature space.

  In the region integration unit 110, the above-described operation is performed until, for example, the minimum clustering region extracted by the minimum cluster region extraction unit 112 has a number of pixels larger than a predetermined small pixel threshold, and the image is displayed in each object region OR. Each area is divided (see FIG. 2C).

  Next, the block area identifying means 30 will be described with reference to FIG. The block area identification unit 30 includes a feature amount extraction unit 40, a mapping unit 50, a type output unit 60, and the like. The feature amount extraction unit 40 has a function of extracting a plurality of block feature amounts from the block region BR. The mapping means 50 has, for example, a two-dimensional space SOM made up of a self-organizing map, and maps a plurality of block feature values (multidimensional feature values) onto the two-dimensional space SOM. The type output means 60 has a type frequency distribution map KDM that defines types for each position on the two-dimensional space SOM. The type output means 60 outputs the type of the block area BR from the coordinates CI on the two-dimensional space SOM mapped by the mapping means 50 using the type frequency distribution map KDM. Below, each structure of the block area identification means 30 is demonstrated concretely.

  FIG. 6 is a block diagram illustrating an example of the feature quantity extraction unit 40. The feature quantity extraction unit 40 will be described with reference to FIG. The feature quantity extraction means 40 outputs 15 block feature quantities BCQ composed of color components, lightness components, and image feature components, and includes a Lab conversion means 41, a first average value calculation means 42, a first wavelet. A conversion unit 43, a distance image generation unit 46, a second wavelet conversion unit 47, and the like are included.

  The Lab conversion means 41 has a function of converting a block area BR formed of RGB images into a Lab image. The average value calculating means 42 has a function of calculating average values L-ave, a-ave, and b-ave of the L component, a component, and b component of the block region BR subjected to Lab conversion. The calculated average values L-ave, a-ave, and b-ave are the block feature values BCQ from which the color components are extracted.

  The first wavelet transform unit 43 performs wavelet transform on the Lab-converted block region BR to calculate high-frequency components L-LH, L-HL, and L-HH of brightness components. In addition, an average value calculating means 44 and a maximum value calculating means 45 are connected to the first wavelet transform means 43.

  The average value calculating means 44 is the average values L-LH-ave, L-HL-ave, L-HH-ave of the high frequency components L-LH, L-HL, L-HH calculated by the first wavelet transform means 43. Is calculated. The calculated average values L-LH-ave, L-HL-ave, and L-HH-ave are the block feature values BCQ from which the brightness components are extracted.

  The maximum value calculating means 45 calculates a value of 5% from the largest in the frequency distribution of the high frequency components L-LH, L-HL, and L-HH calculated by the first wavelet transform means 43. The maximum values L-LH-max, L-HL-max, and L-HH-max become the block feature value BCQ from which the brightness component is extracted.

  In this way, by using the average value and the maximum value as the block feature value BCQ of the L component, a high frequency component having a constant intensity is distributed on average, and the block region BR and a block having a strong high frequency component in part. The region BR can be distinguished from the region BR, and the type of the block region BR can be accurately identified.

  The distance image generation unit 46 has a function of generating a distance image D from the block region BR subjected to Lab conversion by the Lab conversion unit 41. Here, the distance image D is different from a general distance image, as shown in FIG. 7, from the low-frequency components of the three-variable block region BR subjected to Lab transform and the block region BR generated upon wavelet transform. This is an image of the Euclidean distance from the blurred image. That is, the three-dimensional distance image in the Lab space is an image in which the signal variation in the uniform color space is made into one image, and can be described as representing the variation perceived by a person. By handling fluctuations in the three-dimensional space, image structural features that cannot be obtained from the brightness image can be extracted, so that the types can be identified more accurately.

  In other words, when the type is identified based on the pixel feature value extracted for each pixel, the type cannot be identified by the image structure. For example, the image structure is different such as “sky” and “sea”, but the brightness is different. It is not possible to accurately identify types of similar colors. On the other hand, the type can be identified more accurately by extracting the type using the image structure in which the distance image D is generated for each block region BR.

  The second wavelet transform unit 47 has a function of performing wavelet transform on the generated distance image D and outputting the high frequency components D-LH, D-HL, and D-HH. An average value calculating means 48 and a maximum value calculating means 49 are connected to the second wavelet transform means 47.

  The average value calculating means 48 is the average values D-LH-ave, D-HL-ave, D-HH-ave of the high frequency components D-LH, D-HL, D-HH calculated by the second wavelet transform means 47. Is calculated. The calculated average values D-LH-ave, D-HL-ave, and D-HH-ave are the block feature values BCQ from which the image feature components are extracted.

  The maximum value calculation means 49 calculates a value of 5% from the largest in the frequency distribution of the high frequency components D-LH, D-HL, and D-HH calculated by the first wavelet transform means 43. The maximum values D-LH-max, D-HL-max, and D-HH-max become block feature values BCQ from which image feature components are extracted.

  In this way, by using the average value and the maximum value as the block feature value BCQ of the D (distance) component, the high-frequency component having a constant intensity is distributed on the average, and the block region BR and the high-frequency component strong in part. This makes it possible to distinguish a certain block area BR from a certain block area BR, and to accurately determine the type of the block area BR.

  Next, FIG. 8 is a schematic diagram showing an example of the mapping unit 50 and the type output unit 60. The mapping unit 50 and the type output unit 60 will be described with reference to FIGS. The mapping means 50 and the type output means 60 include a modified counter propagation network using a self-organizing map (reference: Tokutaka, Kishida, Fujimura, “Application of Self-Organizing Map-Two-dimensional Visualization of Multidimensional Information”, Kaibundo, 1999. ) Is used.

  The mapping means 50 has a two-dimensional space SOM composed of a self-organizing map in which a plurality of neurons N are arranged in a matrix, and has a function of mapping a plurality of feature quantities (multidimensional feature quantities) onto the two-dimensional space SOM. Have. Each neuron N has a vector coordinate in the same dimension as the block feature BCQ. In the present embodiment, since the block feature value BCQ is composed of 15 block feature values BCQ, each neuron is composed of a 15-dimensional connection weight vector.

  The mapping unit 50 then approximates the 15 block feature values BCQ extracted from one block region BR among the neurons N on the self-organizing map SOM (for example, the neuron having the closest Euclidean distance or the like). Select Ni (ignition element). As a result, a multidimensional space composed of a plurality of block feature values BCQ is mapped onto the two-dimensional space SOM. Then, the mapping means 50 sends the coordinates CI of the selected neuron Ni to the type output means 60.

  The kind output means 60 has a plurality of kind frequency distribution maps KDM having the same coordinate system as the two-dimensional space SOM, and the kind frequency distribution is calculated from the coordinates CI on the two-dimensional space SOM mapped by the mapping means 50. The map KDM has a function of outputting the type indicated by the part indicated by the coordinates CI. As shown in FIG. 9, in this type frequency distribution map KDM, various types of distributions are formed in the two-dimensional space for each type, and a type frequency distribution map KDM is prepared for each type. . For example, FIG. 9A shows the type frequency distribution map KDM with the type “Empty”, FIG. 9B shows the type frequency distribution map KDM with the type “Building”, and FIG. 9C shows the “Tree” with the type KI. "Type frequency distribution map KDM", and FIG. 9D shows the type frequency distribution map KDM of the type "sea". In FIG. 9, the white range is a frequency value (reliability) of 0.8 to 1.0, the gray range is a frequency value (reliability) of 0.2 to 0.8, and the black range is 0.0 to 0.0. A frequency value (reliability) of 0.2 is shown.

  In addition, although the case where the type frequency distribution map KDM is prepared for each type is illustrated, a plurality of types of distributions may be formed in one type frequency distribution map KDM.

  Here, as the above-described self-organizing map SOM and type frequency distribution map KDM used for identifying types (recognition mode), those learned in advance are used. That is, the two-dimensional space SOM and the type frequency distribution map KDM have a learning function, and each neuron N uses the learning input data including the block feature amount BCQ extracted from the block region BR whose type is known in advance. And the type frequency distribution map KDM is learned.

  Specifically, learning of the self-organizing map SOM will be described first. The neurons of the self-organizing map SOM have random connection weight vectors in the initial state. Then, learning input data whose type is known in advance is input to the mapping means 50. Then, the neuron Ni (firing element) most similar to the learning input data is selected by the mapping means 50. At the same time, for example 3 × 3 neurons surrounding the selected neuron Ni (firing element) are selected. Then, the connection weight vector of the neuron Ni (firing element) and the neuron N in the vicinity thereof is updated in a direction approaching the learning input data, and the neuron N of the self-organizing map SOM is learned.

  This operation is performed using a plurality of learning input data. Further, the learning input data is repeatedly input into the self-organizing map SOM a plurality of times. Here, as the input of a plurality of learning input data is repeated, the range of the neighborhood region of the neuron N in which the connection weight vector is updated becomes narrower, and finally the selected neuron Ni (firing element) is selected. Only the combined load vector is updated.

  Next, learning of the type frequency distribution map KDM will be described. In the type frequency distribution map KDM, initial values of all coordinates are zero. As described above, when the learning input data is mapped to the self-organizing map SOM, the coordinates CI on the self-organizing map SOM are output. Then, a positive integer value (for example, “1”) is added to the portion corresponding to the coordinate CI in the type frequency distribution map KDM corresponding to the type of the input data for learning and the region (for example, 3 × 3) surrounding it.

  Then, as learning input data is input, numerical values are added to a specific area on the type frequency distribution map KDM to increase as learning input data is input. That is, if the same type of block region BR, the block feature amount BCQ is similar. If the block feature values BCQ are similar, they are often mapped to nearby coordinates on the self-organizing map SOM, so that the numerical values of specific coordinates also increase in the type frequency distribution map KDM.

  Finally, when the numerical value at each coordinate of the type frequency distribution map KDM is divided by the total number of input learning data times the number of learnings, the type frequency distribution map KDM in which a probability of 0.0 to 1.0 is input to each coordinate is obtained. Generated. This means that the greater the probability, the greater the probability of that type. In the type frequency distribution map KDM of FIG. 9, the reliability (probability) in the white range is 0.8 to 1.0, the reliability (probability) in the gray range is 0.2 to 0.8, and the black range. Indicates a reliability (probability) of 0.0 to 0.2. In this way, the type frequency distribution map KDM is formed for each type such as “sky”, “building”, “tree”, “sea”, and the like.

  When identifying the type of the actual block region BR (recognition mode), the type output unit 60 extracts the reliability of each part of the coordinate CI from the plurality of type frequency distribution maps KDM. Specifically, when the coordinate CI is sent from the mapping means 50, for example, it corresponds to the coordinate CI on each type frequency distribution map KDM such as “sky”, “building”, “tree”, “sea”, etc. Extract the reliability of the part. And the kind output means 60 produces | generates the kind vector which makes the probability obtained from each kind frequency distribution map KDM a vector component. In this case, a kind vector having the vector components of the reliability of the sky, the reliability of the building, the reliability of the tree, and the reliability of the sea is generated. Thereafter, the type output unit 60 identifies the type having the highest probability as the type information of the block area BR, and sends the type to the object identification unit 70.

  In the type output means 60, when the vector component constituting the above-described type vector is smaller than the predetermined vector component threshold, it is determined that the certainty of identifying the type of the block region BR is low, and “unknown” The types may be sent to the object identification means 70. Alternatively, when the difference between the largest vector component and the second largest vector component is small, similarly, it is determined that the certainty of identifying the type of the block area BR is low, and the type is set to “unknown”, and the object identifying unit 70 You may make it send to. As a result, the block region BR with low reliability regarding the type identification can reduce the influence on the type identification of the object region OR, so that the accuracy of identification of the object region OR can be improved.

  Further, when the plurality of block feature values BCQ sent by the mapping means 50 are mapped onto the self-organizing map SOM, the distance (for example, Euclidean distance) between the most approximate neuron Ni (firing element) and the plurality of block feature values BCQ. Or the like) is larger than a predetermined distance threshold value, the mapping unit 50 may send information indicating that the matching processing is not performed to the type output unit 60. In that case, the type output unit 60 may send a type of “unknown” to the object identification unit 70. Even in this case, since the influence on the type identification of the object area OR can be reduced for the block area BR having low reliability for the type identification, the accuracy of the identification of the object area OR can be improved. it can.

  FIG. 10 is a flowchart showing a preferred embodiment of the object identification method of the present invention. The object identification method will be described with reference to FIGS. First, an object area OR is generated by dividing the image input by the object area generation means 20 into areas for each object. On the other hand, a plurality of block areas BR smaller than the object area OR, which are composed of a set number of pixels (for example, 32 × 32 pixels), are generated from the image input by the block area generating means 10. (Step ST1).

  Next, the feature quantity extraction means 40 extracts 15 feature quantities BCQ from the block region BR (step ST2). Thereafter, the extracted feature value BCQ is mapped to the self-organizing map SOM by the mapping unit 50, and the coordinates CI of the self-organizing map SOM are sent to the type output unit 60 (step ST3). The type output means 60 extracts the type indicated by the coordinates CI in the type frequency distribution map KDM and sends it to the object identification means 70 (step ST4). This operation is performed for all the block areas BR (step ST5).

  Thereafter, in the object identification means 70, the types assigned to each object area OR are totaled (step ST6). The most common type is output as the type of the object area OR (step ST7).

  According to the above embodiment, the type of each block area BR is identified, the type of the block area BR is counted for each object area OR, and the type of the object area OR is identified, thereby accurately identifying the object. The type can be automatically identified. That is, the block area identifying means 30 may identify each block area BR as a type different from the original object type. For example, when the object is “sea”, there is a case where a block area BR in the sea object area OR is determined to be “sky”. At this time, since the type of the object region OR is the largest of the aggregated types, the block region BR partially attached with type information different from the true type information of the object is provided. Even if it exists, it is possible to prevent the object from being given a different type from the real object. Therefore, the object type can be automatically and accurately identified.

  On the other hand, as described above, the color component, the brightness component, and the image feature component of the block region BR are extracted as the block feature value BCQ, and the block feature value BCQ is input to the modified counter propagation network. The type of can be identified. That is, when trying to identify a type by extracting a pixel feature amount for each pixel, the type cannot be accurately identified. This is because pixel information obtained from an image does not include distance information (image information), and only lightness information or color information can be extracted. Therefore, for example, “sea” and “sky” may have the same color, and therefore the “sea” object may be determined to be the “sky” object.

  On the other hand, since the block feature value BCQ is extracted for each block area BR and the type is identified, even objects having similar color information and brightness information such as “sea” and “sky” are identified. It is possible to identify the type accurately.

  FIG. 11 is a block diagram showing a second embodiment of the object identification device of the present invention. The object identification device 200 will be described with reference to FIG. In the object identification device 200 shown in the figure, parts having the same configuration as the object identification device 1 in FIG. 1 are denoted by the same reference numerals and description thereof is omitted.

  The object identification device 200 of FIG. 11 is different from the object identification device 1 of FIG. 1 in that after the object region OR is extracted, the object region OR is divided into block regions BR.

  That is, the object identification device 200 of FIG. 11 includes a block area generation unit 10, an object area generation unit 20, a block area identification unit 30, an object identification unit 70, and the like. Then, after the object area generating means 20 divides the image for each object area OR, the block area generating means 10 divides the object area OR for each block area BR. Then, after identifying the type for each block area BR by the block area identifying means 30, the object identifying means 70 totals the block areas BR in the object area OR to identify the type of the object area OR. Even with this object identification device 200, the same effect as the object identification device 1 of FIG. 1 can be obtained.

  In each of the above embodiments, the mapping means 50 in FIG. 8 has one self-organizing map SOM. However, as shown in FIG. 12, it may have two self-organizing maps. Good. Specifically, the mapping unit 150 includes a first self-organizing map SOM1 and a second self-organizing map SOM2, and a first mapping unit for mapping a plurality of block feature values BCQ to the first self-organizing map SOM1. 151, the first coordinate CI1 in the first self-organizing map SOM1 acquired for each block region BR by the first mapping means 151 is acquired, and the plurality of first coordinates CI1 are converted into the second self-organizing map SOM2. And second mapping means 152 for mapping.

  Here, the first mapping means 151 and the first self-organizing map SOM1 have the same structure as the mapping means 50 and the self-organizing map SOM shown in the figure. On the other hand, the second mapping unit 151 includes a plurality of second output units output from the first mapping unit 151 for a plurality of block regions BR having a specific spatial relationship such as 3 × 3 block regions adjacent to each other. One coordinate CI1 is mapped to the second self-organizing map SOM2. As a result, when identifying the type by the block area BR, it is possible to identify the type using a global feature (structural feature) made up of a plurality of block areas BR. The accuracy of identification can be improved. Furthermore, by using not only the above-described two-stage self-organizing maps SOM1 and SOM2 but also more stages, types can be identified from a more global structure.

  In each of the above embodiments, the block area generation unit 10 may have the following functions. That is, the block area generating means 10 includes a plurality of first block areas BR1 obtained by dividing an image into meshes as shown in FIG. 13A, and a plurality of first blocks as shown in FIG. 13B. The region BR1 and the second block region BR2 in which the phase of partitioning in a mesh shape is shifted are generated. That is, the block area generation means 10 generates the first block area BR1 by mechanically dividing the block area having a set number of pixels of 32 pixels × 32 pixels into a mesh shape (see FIG. 13A), Furthermore, a mesh-like second block region BR2 shifted by a half block (16 pixels) with respect to the horizontal direction and the horizontal direction as shown in FIG. 13B is generated. Then, the type identification is performed using the generated first block region BR1 and second block region BR2. Even in this case, the block areas BR1 and BR2 including the boundary of the object area OR are not used for type identification.

  Thus, the accuracy of identification can be improved by increasing the number of block areas BR used for identifying the type of object area OR. In other words, as described above, the block region BR including the boundary of the object region OR is the object region in order to prevent a reduction in identification accuracy due to the presence of the boundary edges and the mixed features of the plurality of regions. Not included in OR type aggregation. Therefore, when the object area OR is small, the number of block areas BR to be generated is small, and in the case of an object area OR having a complicated shape, the boundary with other object areas OR is increased. The number of block areas BR to be reduced is reduced. For this reason, the accuracy of the identified type is lowered, and in particular, if the image is a little complicated, it cannot be identified and it is determined that many object regions OR are unknown.

  At this time, if block regions BR1 and BR2 having different phases as shown in FIGS. 13A and 13B are generated, the number of block regions BR that do not include the boundary of the object region OR is increased, and more accurate. Different types of identification can be made.

  In FIG. 13B, the second block region BR2 shifted by a half block with respect to the horizontal and vertical directions is generated, but only in the horizontal direction as shown in FIG. 13C. The second block region BR2 shifted by a half block (16 pixels) may be generated, or the second block region BR2 shifted by a half block only in the vertical direction as shown in FIG. It may be. Further, in the block area identification means 30, all of the block areas BR1 and BR2 of FIGS. 13A to 13D may be used, or any one of the block areas BR may be used in combination. Good. Further, in FIG. 13A to FIG. 13D, the block region generation means 10 is illustrated with respect to a case where it is shifted by a half block, but is not limited to a case where it is shifted by a half block. What is necessary is just to shift by a smaller pitch than the set number of pixels, such as shifting by minutes (for 8 pixels).

  Further, in each of the above embodiments, as shown in FIG. 14, the block area generation unit 10 has a function of generating a plurality of resolution conversion images having different resolutions from the image, and setting pixels are generated from the generated plurality of resolution conversion images. It may have a function of generating a block area consisting of numbers. Specifically, the block area generation unit 10 performs a known resolution conversion technique such as a Gaussian pyramid or a wavelet transform on the entire image to generate a plurality of resolution conversion images. Then, the block area generation unit 10 generates the block area BR by dividing the generated plurality of resolution conversion images into meshes for each set number of pixels. Then, type identification is performed for each block region BR generated from a plurality of resolution-converted images.

  At this time, the block area generation unit 10 does not change the number of set pixels (for example, 32 pixels × 32 pixels). This prevents the size of the block area BR when learned from differentiating between the size of the block area BR when learning and the size of the block area BR when identifying when identifying the type based on the feature amount in the block area identifying means 30. This is to prevent a decrease in identification accuracy in the self-organizing map SOM.

  Thus, by generating the block area BR using resolution-converted images having different resolutions, it is possible to improve the accuracy of identifying the type of the block area BR. That is, in a normal whole image, the way the subject is captured differs between an image obtained by photographing the same subject from near and an image obtained from far away. Even if the subject type cannot be identified when shooting from near, the subject type can be identified when shooting from a distance, or vice versa. Therefore, by using a resolution-converted image, it is possible to prevent a decrease in accuracy due to the difference in the way of capturing and improve the accuracy of identifying the type of the block region BR.

  In FIG. 14, the block region BR is generated by mechanically dividing the entire image into a mesh shape, but may be generated by being shifted by a half block as shown in FIG. 13.

  FIG. 15 is a block diagram showing a third embodiment of the object identification device of the present invention. In the object identification device 300 of FIG. 15, parts having the same configuration as the object identification device 1 of FIG. The object identification device 300 in FIG. 15 is different from the object identification device 1 in FIG. 1 in the block region BR generation method in the block region generation means 310.

  Specifically, as shown in FIG. 16, the block area generation unit 10 scans a cut frame having a set number of pixels in the object area OR, and generates an image surrounded by the cut frame as a block area. It has become. FIG. 17 is a flowchart showing an example of the operation of the block area generating unit 310 of FIG. 15, and an example of a method of generating the block area BR will be described with reference to FIGS.

  First, the object area generation means 20 generates a plurality of object areas OR from the entire image (step ST10). Thereafter, an area ID is assigned to each generated object area OR (step ST11). Then, the object area OR for generating the block area BR is determined from the plurality of generated object areas OR (step ST12), and the block area BR is generated (step ST13). This block region generation step (step ST13) is performed for all object regions OR included in the entire image (steps ST12 to ST14). Thereafter, the types of the plurality of generated block areas BR are identified by the block area identifying means 30.

  FIG. 18 is a flowchart showing an example of the block area generation step (step ST13). The generation process of the block area BR will be described with reference to FIG. First, a cutting frame is set at the start point in the object area OR (step ST13-1). Specifically, as shown in FIG. 16, the cutting frame is positioned so that the upper left corner of the cutting frame is positioned at the upper left corner of the object area OR. Then, it is determined whether or not all the area IDs in the cut frame match (step ST13-2). If all the area IDs in the cut frame match, the area surrounded by the cut frame is blocked. The area BR is generated (step ST13-3).

  Thereafter, the cut frame is shifted by, for example, 8 pixels in the horizontal direction (right direction) (step ST13-4). Here, it is determined whether or not the cutting frame has been scanned to the rightmost end of the object area OR (step ST13-5). If not, the block area is generated (step ST13-2 to ST13-2). Step ST13-5). On the other hand, when the cutting frame is scanned to the rightmost end of the object area OR, the cutting frame is shifted by, for example, 8 pixels in the vertical direction (downward), and also moved in the horizontal direction to the left end of the object area OR. Positioning is performed (step ST13-6). Thereafter, the block region BR is generated in the horizontal direction (step ST13-2 to step ST13-6). When the cut frame is scanned to the lowest end of the object area OR (step ST13-7), the generation of the block area BR for one object area OR is completed.

  In this way, by generating the block area BR while scanning the cut frame within the object area OR, the number of block areas BR for identifying the type of the object area OR can be increased. The accuracy of identification can be improved.

  Note that the cut frame is illustrated as being shifted by 8 pixels with respect to the horizontal direction and the vertical direction, but may be set to pixels smaller than the cut frame, such as 2 pixels or 4 pixels. Further, the block area BR to be cut out by the cut frame is determined by changing the area ID. However, the cut frame is mechanically arranged at a pixel pitch smaller than the cut frame, such as 2 pixels, in the vertical direction and the horizontal direction. You may make it scan in a direction. At this time, the type identification may not be performed for the block area BR including two area IDs in the cut frame.

  FIG. 19 is a block diagram showing another embodiment of the feature quantity extraction means in the object identification device of the present invention. The feature quantity extraction unit 140 of FIG. 19 includes an image conversion unit 141, an edge image generation unit 142, a correlation feature quantity extraction unit 143, an edge feature quantity extraction unit 144, a color feature quantity extraction unit 145, and the like.

  The image conversion unit 141 converts the block area expressed by the RGB color system to the YCC color system. At this time, the image conversion unit 141 identifies a block area included in one object area OR among a plurality of block areas BR constituting the image. This is because, among the plurality of block areas BR constituting the image, the block area BR that straddles the boundary between the object areas OR is not used for the determination of the type of the object area OR, and thus the feature amount is not extracted. is there.

  The edge image generation unit 142 has a function of generating an edge image using the Y component generated by the image conversion unit 141. Here, the edge image generation unit 142 generates a vertical edge image using the vertical edge detection filter shown in FIG. 20A and uses the horizontal edge detection filter shown in FIG. An image is generated.

  The edge image generation unit 142 uses an edge detection filter (prewitt filter) as shown in FIG. 20, for example, an edge detection filter in which higher and lower pixels are given higher weights than those on the diagonal line. An edge detection method using (Sobel filter) or other known edge detection methods can be used.

  The correlation feature amount extraction unit 143 in FIG. 19 extracts a correlation feature amount indicating the degree of regularity of change along one direction of the component signal value assigned to each pixel of the block region BR. Here, FIG. 21 is a flowchart showing an example of a calculation method of the correlation feature quantity in the correlation feature quantity extraction unit 143, and the calculation method of the correlation feature quantity will be described with reference to FIG.

In addition, although extraction of the correlation feature-value regarding the change along a horizontal direction is demonstrated in FIG. 21, the correlation feature-value with respect to the change along a vertical direction is also extracted by the same method. Further, F _i (x) shown below indicates the component signal value of the x-th pixel (i = 0 to 31, x = 0 to 31) in the i-th row, and F _j (x) indicates the component signal value in the j-th row. The component signal value of x pixel (i = 0-31, x = 0-31) shall be shown.

First, using the vertical edge image generated by the edge image generation means 142, the change in the component signal values F _i (x) and F _j (x) along each row of the vertical edge image is normalized (step ST21). ). Specifically, the difference between the component signal value F _i (x) and the average value F _i is divided by the standard deviation δ _i to obtain a normalized component signal value F _i ′ (x).

F _i ′ (x) = (F _i (x) −F _i ) / δ _i
Similarly, the difference between the component signal value F _j (x) of j rows and the average value F _j is divided by the standard deviation δ _i to obtain a normalized component signal value F _i ′ (x).

F _j ′ (x) = (F _j (x) −F _j ) / δ _j
As described above, the correlation feature value is obtained by normalizing the component signal value in order to derive the correlation feature value indicating the cross-correlation of the variation pattern itself by eliminating the difference in the fluctuation range and the average value between the rows. It is. When F _i (x) and F _j (x) are constant values and the standard deviation is 0, F _i ′ (x) = 0 (constant) and F _j ′ (x) = 0 (constant). To do.

Then, with respect to the combination of two different rows (i-th row and j-th row), the cross-correlation function is obtained using the normalized component signal values F _i ′ (x) and F _j ′ (x) regarding these two rows.

Is derived (step ST22). Conceptually, the cross-correlation function is obtained by converting the normalized component signal values F _i ′ (x) and F _j ′ (x) of two rows by d pixels as shown in FIG. Multiply by shifting and take the sum. Then, a cross-correlation function G _ij (d) as a function of d as shown in FIG. 22B is obtained.

Next, the maximum correlation value is calculated from the correlation values when d = 0 to 31 is substituted into the calculated cross-correlation function G _ij (d) (step ST23).

  In this operation, the maximum correlation value is calculated for all combinations of two rows (steps ST21 to ST24). Here, in the block region of 32 pixels × 32 pixels, the maximum correlation values of all combinations of the 0th to 31st rows are calculated. Then, an average value and a standard deviation of all the calculated maximum correlation values are calculated, and the average value and the standard deviation are set as correlation feature amounts (step ST25). Similarly, the average value and the standard deviation of the maximum correlation values relating to changes along the vertical direction are calculated as correlation feature amounts (steps ST21 to ST25).

  The correlation feature amount calculated as described above indicates whether or not there is a regular pattern in the block region BR constituting the object. The larger the average value of the maximum correlation values is and the smaller the standard deviation is, It means that a regular pattern is formed. In general, natural objects included in captured images have few regular patterns, continuous patterns, and periodic patterns, and are often composed of random patterns. On the other hand, artifacts such as buildings and cobblestones are often composed of regular patterns. Therefore, a part of an image of a building or the like in which the block region BR is artificially created by extracting a correlation feature amount indicating whether or not the block region BR constituting the object forms a regular pattern. Or a part of an image of a natural object.

Note that the correlation feature quantity extraction means 143 may simply extract the average value and standard deviation of the sum of the normalized component signal values F _i ′ (x) and F _j ′ (x) as the correlation feature quantity. However, as described above, if the average value and the standard deviation of the maximum value of the cross-correlation function are used as the correlation feature amount, for example, the pattern of the block region BR in which a regular pattern or ripple is photographed in an oblique direction. Accordingly, it is possible to derive an appropriate correlation feature amount indicating regularity of the block, and it is possible to extract a correlation feature amount that accurately represents the feature amount related to the correlation of the block region. Here, the case where the pixel line is shifted pixel by pixel (d = 0, 1, 2,... 31) is mentioned, but the maximum correlation value is calculated while shifting a plurality of pixels, such as shifting by two pixels. You may make it do.

  The edge feature quantity extraction means 144 extracts the feature quantity of the edge component of the block area BR. Specifically, the edge feature quantity extraction unit 144 calculates an average value and a standard deviation of each component signal value for the vertical edge image and the horizontal edge image generated using the edge detection filter (see FIG. 20). Four edge feature values are output.

  In this way, by using the average value of the edge component as the feature value of the edge component, it is possible to classify “sky” with few edges among natural objects and “water” and “plants” with many edges among natural objects. . Further, by extracting the vertical edge component and the horizontal edge component of the block region BR as edge feature amounts, for example, “water”, an object having different edge component characteristics depending on the direction, “plant”, “ Objects that form relatively uniform edges in the vertical and horizontal directions such as “flower garden” can be classified.

  The color feature amount extraction unit 145 extracts a color feature amount indicating the color feature of the block region BR. Specifically, the color feature amount extraction unit 145 includes the luminance component (Y component) and the two color difference components (Cr, Cb) for 32 × 32 pixels constituting the block region BR expressed in the YCC color system. An average value and a standard deviation of each component signal value are calculated, and six color feature amounts are extracted from one block area.

  The color feature quantity extraction unit 145 extracts the color feature quantity after conversion from the RGB color system to the YCC color system. The feature amount may be extracted, or the image conversion unit 141 may convert the RGB color system block area BR to the Lab color system and extract the color feature amount for each component of Lab. Also good. The color feature quantity extraction unit 145 extracts the average value and standard deviation of each component signal value as color feature quantities. For example, the color feature quantity extraction unit 145 uses other representative values such as a maximum value, minimum value, and quantile as color features. It may be used as a quantity.

  Then, a 14-dimensional block feature value composed of four correlation feature values, four edge feature values, and six color feature values is input to the mapping unit 50, and the type output unit 60 identifies the type. Become. At this time, the combined load vector of the self-organizing map SOM in the mapping means 50 is composed of 14-dimensional vectors, and the self-organizing map SOM is learned using the 14-dimensional feature vectors. ing.

  Each feature amount may be used with appropriate weighting after adjusting the fluctuation range. Further, the feature quantity extraction unit 140 in FIG. 19 has a function of calculating the feature quantity extracted from the distance image in FIG. 6 and the feature quantity of the high frequency component in addition to the above-described correlation feature quantity, edge feature quantity, and color feature quantity. You may have.

  FIG. 23 is a block diagram showing a fourth embodiment of the object identification device of the present invention. The object identification device 500 will be described with reference to FIG. In the object identification device 500 of FIG. 23, parts having the same configuration as the object identification device 1 of FIG.

  In the object identification device 500, first, the object area generation means 20 generates an object area OR. Then, the feature quantity extraction means 540 extracts the object feature quantity from the generated object area OR. After that, the type of the object area OR is identified by the mapping unit 50 and the type output unit 60 using the extracted object feature amount.

  Further, the feature amount extraction unit 540 extracts an object feature amount using the entire image that has been subjected to the predetermined image conversion process by the image conversion unit 510 and the generated object region OR. Specifically, the image conversion unit 510 has a function of converting the entire image composed of the RGB color system into the YCC color system and generating three images for each component of the YCC color system. Furthermore, the image conversion means 510 generates a vertical edge image and a horizontal edge image generated from the Y component. The feature quantity extraction means 40 extracts object feature quantities from five images, that is, three images for each YCC component, a vertical edge image, and a horizontal edge image.

  Here, the feature quantity extraction means 540 extracts object feature quantities by the following method. That is, the feature amount extraction unit 540 generates a distribution (histogram) of pixel values for each object region OR by combining the region division results for each image described above. The feature quantity extraction unit 540 calculates an average value and a standard deviation from the histogram, and generates an object feature quantity. Note that a representative point of the histogram (for example, maximum value, minimum value, median value, quantile, etc.) may be used as the feature quantity. In addition, the learning sample of the self-organizing map SOM is performed using the above-described image conversion on the block region BR and using the feature amount extracted from the above-described histogram.

  Note that the feature quantity extraction means 540 described above may extract feature quantities as shown in FIGS. 6 and 19 from the object region OR and use them as object feature quantities. Further, the above-described image conversion unit 510 uses a plurality of resolution conversion images having different resolutions by performing multi-resolution conversion on the entire image, an image obtained by converting the entire image from the RGB color system to the Lab color system, a morphology filter, and the like. A filtering image or the like obtained by extracting a structure having a specific shape may be generated, and the feature amount extraction unit 540 may extract a feature amount from each image.

  As a result, even when the area shape of the object area OR is complex or small, the type of the object area OR can be reliably identified. That is, when the entire image is divided into block areas BR, the object area OR is divided into a plurality of block areas BR when the object area OR is complicated, and the block area used for type identification when the object area OR is small. The number of BR will decrease.

  On the other hand, the result of identification by the block area identifying means 30 is assigned to all the pixels included in the block area BR, and the type having the largest number of pixels among the types assigned to the pixels constituting the object area OR is assigned to the object area. It may be possible to identify the type of OR. However, it is necessary to identify the type of the block region BR including the boundary of the object region OR. As a result, there is a problem that the accuracy of identifying the type is lowered. Therefore, by extracting the feature amount from the object area OR itself and identifying the type, it is possible to accurately identify the type of the object area OR having a complicated shape or the object area OR having a small shape.

  FIG. 24 is a block diagram showing a fifth embodiment of the object identification device of the present invention. The object identification device 600 will be described with reference to FIG. In the object identification device 600 of FIG. 24, parts having the same configurations as those of the object identification device 1 of FIG. 1 and the object identification device 500 of FIG. The object identification device 600 of FIG. 24 is different from the object identification device 500 of FIG. 23 in that the object identification device 600 further includes a normalization unit 630 that generates a normalized object region obtained by normalizing a circumscribed rectangular image of the object region OR. Accordingly, the size of the object area OR when extracting the object feature amount is the same regardless of the object area OR of any image.

  As described above, by extracting the object feature amount after normalizing the object area OR, accurate identification can be performed without depending on the size of the object area included in the entire image and the type identification accuracy. it can. That is, the size of the object included in the entire image varies depending on the situation at the time of shooting. Therefore, by extracting the object feature amount after standardizing each object region OR and identifying the type, it is possible to make the type robust with respect to the size variation and to identify the type with high accuracy. .

  Note that the object identification device 1 of FIG. 1 and the object identification device 600 of FIG. 24 may be used in combination. Then, when a part of the object is hidden by the shielding object, the type identification accuracy is lowered from the object area OR, but if the identification summation by the block area BR is used, the object type identification accuracy is lowered. Can be prevented.

  The embodiments of the present invention are not limited to the above embodiments. For example, for the object identification devices 1 and 300 in FIGS. 1 to 22, the types of the block areas BR are identified, the identification results are aggregated to identify the types of the object areas OR, and the object identifications in FIGS. As for the devices 500 and 600, the type is identified from the object region OR itself, but they may be combined. That is, the object identification means identifies the final type of the object area OR using the block area BR type identification by the block area identification means 30 and the type identification means by the type output means. It may be.

  1 sends information such as “sky” and “sea” as types to the object identification unit 70, but the above-described type vector itself is used as a type to the object identification unit 70. You may make it send. In this case, the object identifying means 70 identifies the type that is the largest vector component of the type vectors as the type of the object region OR by simply adding the type vector of each block region BR included in the object region OR. You may make it do. Alternatively, when the maximum vector component is smaller than the maximum threshold value, the object identification unit 70 may set the type of the object area OR to “unknown”.

  In addition, the generation of the object area OR and the generation of the block area BR are exemplified for the case where the resolution of the whole image P to be sent is used as it is, but the input to the object area generation means 20 and the block area generation means 10 You may make it input after reducing resolution before doing. Since the amount of data to be processed can be reduced by reducing the resolution, the processing speed can be improved and the processing efficiency can be improved.

  Furthermore, the resolution for generating the object area OR and the resolution for generating the block area BR do not have to be the same. For example, the block area BR may have a higher resolution than the image of the object area OR. This is because it is necessary to identify the type of each of the block regions BR as described above. However, since the purpose is to roughly divide the block region BR into regions similar to each other, the relatively low-resolution image This is because the purpose can be achieved even if is used.

  In FIG. 1, the block area BR generated by the block area generation means 10 is sent as it is to the block area identification means 30. For example, morphology processing, closing operation, etc. are performed on the determination result for each block area BR. After performing the smoothing process, it may be sent to the block area identifying means 30. As a result, the isolated noisy elements included in the block region BR are discarded, and the accuracy of type identification can be improved.

The block diagram which shows 1st Embodiment of the object identification device of this invention The figure which shows a mode that a kind is identified for every object contained in an image in the object identification device of this invention. The block diagram which shows an example of the object area | region production | generation means in the object identification apparatus of this invention The figure which shows a mode that an image is divided | segmented into an area | region by the object area | region production | generation means of FIG. The figure which shows a mode that a clustering area | region is integrated by the object area | region production | generation means of FIG. 2, and an object area | region is formed. The block diagram which shows an example of the feature-value extraction means in the object identification device of this invention The block diagram which shows the mode of the production | generation of the distance image in the distance image generation means in the object identification device of this invention The block diagram which shows an example of the mapping means and kind output means in the object identification apparatus of this invention The block diagram which shows an example of the kind frequency distribution map in the object identification apparatus of this invention The flowchart which shows preferable embodiment of the object identification method of this invention The block diagram which shows 2nd Embodiment of the object identification device of this invention The block diagram which shows another example of the mapping means in the object identification apparatus of this invention The schematic diagram which shows an example of another production | generation method of the block area production | generation means of FIG. The schematic diagram which shows an example of another production | generation method of the block area production | generation means of FIG. The block diagram which shows 3rd Embodiment of the object identification device of this invention FIG. 15 is a schematic diagram showing an example of a block area generation method in the object identification device of FIG. The flowchart which shows an example of the production | generation method of the block area | region in the object identification apparatus of FIG. The flowchart which shows an example of the production | generation method of the block area | region in the object identification apparatus of FIG. The block diagram which shows another embodiment of the feature-value extraction means in the object identification device of this invention The figure which shows an example of the edge filter used in the image generation means of FIG. The flowchart which shows the operation example of the correlation feature-value extraction means of FIG. FIG. 19 is a graph showing an example of a cross-correlation function in the correlation feature quantity extraction unit of FIG. The block diagram which shows 4th Embodiment of the object identification device of this invention The block diagram which shows 5th Embodiment of the object identification device of this invention

Explanation of symbols

1, 300, 500, 600 Object identification device 10 Block area generation means 20 Object area generation means 30 Block area identification means 30 Block area identification means 30 Type identification means 40, 140 Feature quantity extraction means 41 Conversion means 42 Average value calculation means 43 Wavelet transform means 44 Average value calculation means 45 Maximum value calculation means 46 Distance image generation means 47 Wavelet transform means 48 Average value calculation means 49 Maximum value calculation means 50 Mapping means 60 Type output means 70 Object identification means 100 Feature quantity classification means 101 Region Dividing unit 110 Region integrating unit 111 Database 112 Minimum cluster region extracting unit 113 Integrated region determining unit 130 Block region generating unit 140 Feature amount extracting unit 141 Image converting unit 142 Edge image generating unit 143 Correlation feature amount extraction Means 144 Edge feature amount extraction means 145 Color feature amount extraction means 150 Mapping means 200 Object identification device 201 Block area generation means BR Block area BR1 First block area BR2 Second block area KDM Kind frequency distribution map KI Kind vector OR Object area P Image SOM Self-organizing map (2D space)

Claims (17)

In an object identification method for identifying the type of object included in an image,
Generating an object area obtained by dividing the image for each object, and a plurality of block areas obtained by dividing the image into a plurality of areas smaller than the object area, each having a set number of pixels;
Identify each type for each of the plurality of generated block areas,
Totalize the types of the identified block areas for each object area,
An object identification method comprising: identifying a type of the object region using a totaled result. In an object identification device for identifying the type of object included in an image,
Object region generation means for dividing the image into regions for each object to generate a plurality of object regions;
A block area generating unit configured to divide the image into a plurality of areas smaller than the object area, each having a set number of pixels, and generating a plurality of block areas;
A block area identifying means for identifying a type for each of the plurality of block areas generated by the block area generating means;
Object identification apparatus comprising: object identification means that aggregates the types of the block areas identified for each of the block areas for each object area, and identifies the types of the objects using the aggregated results . The block area identification means;
Feature quantity extraction means for extracting a plurality of block feature quantities from the block region;
Mapping means for mapping a plurality of the block feature values extracted by the feature value extraction means on a two-dimensional space;
A type frequency distribution map in which a type is defined for each coordinate in the two-dimensional space, and the type in which the coordinates in the two-dimensional space mapped by the mapping unit indicate on the type frequency distribution map The object identification device according to claim 2, further comprising: a type output unit that outputs a type.   The object identification apparatus according to claim 3, wherein the two-dimensional space is a self-organizing map in which a plurality of neurons having a learning function are arranged in a matrix.   5. The object identification device according to claim 3, wherein the feature amount extraction unit extracts a color component, a brightness component, and an image feature component of the block area as the block feature amount. .   The type output means has, for each type, the type frequency distribution map in which the type frequency value is defined as the type index for each coordinate of the self-organizing map, and the coordinates detected by the mapping means are 6. The object identification device according to claim 3, wherein a type vector having a plurality of frequency values shown on each type frequency distribution map as a vector component is output.   7. The object identifying apparatus according to claim 6, wherein the type output means outputs a type that is the largest maximum vector component among vector components of the type vector.   8. The type output unit outputs the fact that the type of the block area is unknown when the maximum vector component is smaller than a predetermined maximum component threshold value. Object identification device.   The block area generating means generates a plurality of first block areas obtained by dividing the image into a mesh shape, and a second block area having a phase shifted from the plurality of first block areas in a mesh shape. 9. The object identification device according to claim 2, wherein the object identification device includes:   The block area generation unit has a function of causing the object area to scan a cutting frame having the set number of pixels and generating the block area from an image surrounded by the cutting frame. The object identification device according to any one of claims 2 to 9.   The block area generating means has a function of generating a plurality of resolution conversion images having different resolutions from the image, and has a function of generating the block area from the generated plurality of resolution conversion images, respectively. The object identification device according to any one of claims 2 to 10, wherein the object identification device is characterized in that: A correlation feature amount, wherein the feature amount extraction unit extracts, as a correlation feature amount, whether or not a variation pattern of a vertical or horizontal signal value of the pixel has a correlation between pixel lines in the block region. 12. The object identification device according to claim 3, further comprising an extraction unit.   The correlation feature quantity extraction unit is configured to extract the correlation feature quantity along a vertical direction of the block area and the correlation feature quantity along a horizontal direction of the block area. 13. The object identification device according to 12. The correlation feature quantity extraction unit outputs a correlation value indicating a correlation between the two pixel lines from component signal values of a plurality of pixels constituting the two pixel lines formed in the same direction in the block region. Having a cross-correlation function of
The plurality of correlation values are acquired by inputting the component signal value of the pixel to the cross-correlation function while shifting one of the two pixel lines pixel by pixel in the formation direction of the pixel line. The largest maximum correlation value is calculated from a plurality of correlation values,
The maximum correlation value is calculated for all combinations of the pixel lines formed in the same direction of the block region, and the average value and standard deviation of all the calculated maximum correlation values are extracted as correlation feature amounts. The object identification device according to claim 12, wherein the object identification device is provided.   15. The feature amount extraction unit includes an edge feature amount extraction unit that extracts an edge feature amount indicating a feature of an edge component in a vertical direction and a horizontal direction of the block region. The object identification device according to any one of the above.   16. The feature amount extraction unit according to claim 3, wherein the feature amount extraction unit includes a color feature amount extraction unit that extracts a color feature amount indicating a feature of a color component of the block region. Object identification device. On the computer,
A procedure for generating an object area obtained by dividing an image for each object, and a plurality of block areas obtained by dividing the image into a plurality of areas smaller than the object area, each having a set number of pixels
A procedure for identifying the type for each of the plurality of generated block areas,
A procedure for totalizing the types of the identified block areas for each object area;
An object identification program for executing a procedure for identifying the type of the object area using the totaled result.

Images