JP2010009337A - Annotation device, annotation method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an annotation device for easily attaching meta data to a region included in an image. <P>SOLUTION: This annotation device 1 includes an image selection part 3, a region extraction part 5, a feature point collation part 6, and a region meta data attachment part 7. The image selection part 3 selects at least one image from an image database 2 in which a plurality of images are stored, and the region extraction part 5 extracts at least one image from the selected image, and the feature point collation part 6 collates the feature points included in the extracted region with the feature points included in the plurality of images stored in the image database. Then, the region meta data attachment part 7 attaches the meta data of the image whose feature points are decided to be matched with the feature points of the region as the result as the meta data of the region. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an annotation device that adds metadata to an area included in an image.

In recent years, many images have been made public on a network such as the Internet, and a technique for making it possible to search the images by text has been proposed. In order to search for an image by text, it is necessary to assign a keyword (for example, Tokyo Tower or Mt. Fuji) to the image. Conventionally, as a method of assigning a keyword to an image, a method in which a user manually inputs a keyword when registering an image in a database is known (see, for example, Patent Document 1).
JP-A-11-175556

  By the way, these images include various objects as subjects. For example, an object “Tokyo Tower” is included in one image, and an object “Mt. Fuji” is included in another image. However, no proposal has been made in the past about assigning a keyword to a region included in such an image.

  In general, there are countless images on the network, and these images are uploaded every day. Therefore, a great amount of work is required to find an object from these large images and assign a keyword, and it is not easy for a user to input a keyword by hand. Moreover, the target object to which a keyword is assigned varies depending on the fashion at that time. When a new object is added, the object is searched from a large number of images and a keyword is assigned, and it is not realistic for the user to input the keyword by hand each time.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an annotation device that can easily add metadata to a region included in an image.

  An annotation apparatus according to the present invention includes: an image selection unit that selects at least one image from an image database that stores a plurality of images; an area extraction unit that extracts at least one region from the selected image; Feature point matching means for matching feature points included in the region with feature points included in the plurality of images stored in the image database, and as a result of the matching, it is determined that the region and the feature point match. Region metadata providing means for providing metadata of the image as metadata of the region.

  According to this apparatus, by comparing feature amounts between an area extracted from an image and an image in an image database, an image including a portion similar to the area is found, and metadata attached to the image is determined. It is given as metadata of the area. Thereby, metadata can be automatically assigned to an area extracted from a certain image by using metadata assigned to a large amount of images stored in the image database.

  In the annotation apparatus of the present invention, the region extraction unit may determine the region to be extracted from the image based on a feature point included in the selected image.

  Thereby, since the area | region extracted from the image is determined based on the feature point contained in an image, an appropriate area | region can be extracted from an image.

  In the annotation device of the present invention, the region metadata adding unit includes a plurality of the regions included in the selected image, and the same metadata is added to the plurality of regions. The plurality of areas are combined to generate one area.

  Thereby, when a plurality of regions are extracted from one image and the same metadata is given to the plurality of regions, the plurality of regions are combined to generate one region. That is, if the region extraction is not complete, it can be corrected.

  The annotation method of the present invention selects at least one image from an image database storing a plurality of images, extracts at least one region from the selected image, and includes feature points included in the extracted region. The feature points included in the plurality of images stored in the image database are collated, and the metadata of the image determined to match the region and the feature points as a result of the collation is used as the region metadata. Give.

  According to this method, it is possible to automatically add metadata to an area extracted from a certain image by using metadata provided to a large amount of images stored in the image database.

  The annotation program of the present invention includes a computer that selects at least one image from an image database in which a plurality of images are stored, a process that extracts at least one region from the selected image, and the extracted A process of matching feature points included in a region with feature points included in the plurality of images stored in the image database, and a meta of an image determined to match the region and feature points as a result of the matching And processing for assigning data as metadata of the area.

  This program can also automatically add metadata to a region extracted from a certain image by using metadata assigned to a large number of images stored in the image database.

  According to the present invention, it is possible to easily add metadata to an area included in an image by providing area metadata providing means for assigning image metadata as area metadata.

  Hereinafter, an annotation apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the present embodiment, an example of an annotation device used for metadata assignment for an image search system or the like is illustrated. It can be said that the annotation apparatus according to the present embodiment has a function of performing annotation processing. This function is executed by a program stored in the HDD or memory of the apparatus.

  The configuration of the annotation apparatus according to the embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing the configuration of the annotation apparatus. As shown in FIG. 1, the annotation device 1 includes an image selection unit 3 that selects an image from an image database 2 in which a plurality of images are stored, a feature point extraction unit 4 that extracts a feature point from the selected image, and a selection An area extraction unit 5 is provided for extracting an area from the obtained image.

  The feature point extraction unit 4 calculates SIFT feature amounts of feature points (characteristic points) extracted from the image. The SIFT feature value represents a feature value of a feature point in an image by a 128-dimensional vector (also referred to as a feature value vector). For example, the feature point extraction unit 4 extracts 2000 feature points from one image and calculates a 128-dimensional feature quantity vector for each feature point.

  The region extraction unit 5 extracts a set of feature points extracted from the image by the feature point extraction unit 4 as a region included in the image. This area corresponds to an object (such as Tokyo Tower or Mt. Fuji) included in the image as a subject. In this case, the region is represented by a sequence of feature points included in the image (see FIG. 3).

  In addition, the annotation apparatus 1 includes a feature point matching unit 6 that performs matching between feature points included in the extracted region and feature points included in images in the image database 2. The feature point matching unit 6 calculates feature points that match between the extracted region and the image in the image database 2, and determines whether or not the number of feature points that match between the region and the image is equal to or greater than a predetermined threshold value. judge. If the number of feature points that match the region and the image is equal to or greater than the threshold value, it is determined that the region and the image match. For example, the threshold value for the number of feature points is 50.

  For the calculation of matching feature points, the inner product angle of the 128-dimensional feature vector is calculated for all the feature point combinations of the region and the image, and there is a sufficient difference between the smallest inner product angle and the second smallest inner product angle. In this case, the feature points of the area and the image are regarded as matching. For example, if the second smallest inner product angle is smaller than 50% of the smallest inner product angle, it is considered that there is a sufficient difference and the feature points of the region and the image coincide.

  In addition, the annotation device 1 includes a region metadata adding unit 7 that adds, as a region metadata, metadata of an image that is determined to match the region and the feature point as a result of matching by the feature point matching unit 6. Yes. In addition, the area metadata adding unit 7 has a function of writing the data of the area to which the metadata is added to the area database 8. Note that the metadata includes information such as keywords, comments, and tags.

  Further, the region metadata adding unit 7 includes a plurality of regions in the selected image, and when the same metadata is added to the plurality of regions, the plurality of regions are combined into one region. It has a function to generate. That is, the region metadata adding unit 7 has a function of correcting the region extraction when it is not completely extracted. This function will be described later with reference to the drawings.

  The image database 2 stores data of a plurality of images collected from the Internet, and an image ID and a keyword are assigned to each image. FIG. 2 is a diagram illustrating an example of the image database 2. In the example of FIG. 2, the image database 2 stores an image A to which the keyword “Tokyo Tower” is assigned, an image B to which the keyword “Mt. Fuji” is assigned, and an image C to which the keyword “Tokyo Tower” is assigned. Has been.

  The area database 8 stores data of a plurality of areas extracted from the image, and an area ID and a keyword are assigned to each area. The area data is a point sequence (coordinate sequence) indicating an area included in the image. This point sequence is a set of feature points extracted from the image by the feature point extraction unit 4. That is, it can be said that the area extracted from the image is determined based on the feature points included in the image.

  FIG. 3 is a diagram illustrating an example of the area database 8. The upper part of FIG. 3 shows area data before annotation processing. This area database 8 includes a point sequence area a “(0, 0) (1, 0) (2, 0)...” And “(0, 2) (1, 2) (2, 2 ) ... "is stored. The lower part of FIG. 3 shows area data after the annotation processing. In this example, the keyword “Tokyo Tower” is assigned to the area “a” and the keyword “Mt. Fuji” is assigned to the area “b” by the annotation process.

  The operation of the annotation apparatus 1 configured as described above will be described with reference to FIG. Here, description will be made centering on the operation of giving metadata (annotation processing) to an area, which is a feature of the present invention.

  When metadata is added to an area using the annotation apparatus 1 of the present embodiment, first, one image (for example, image A) is selected from the image database 2 as shown in FIG. Next, one area is selected from the selected image A.

  For example, “Tokyo Tower” is shown as a subject in the image A, and an area “a” corresponding to this “Tokyo Tower” is extracted from the image A. And the feature point contained in this area | region a and the feature point contained in the some image (image A, B, C ...) of the image database 2 are collated. As a result, it is determined that the region a matches the image C. This image C is given the keyword “Tokyo Tower”. Therefore, in this case, “Tokyo Tower” that is the keyword of the image C is given as the keyword of the region a extracted from the image A. Then, the data of the area a to which the keyword “Tokyo Tower” of the area is assigned in this way is stored in the area database 8.

  In addition, “Mt. Fuji” is shown as a subject in the image A, and an area b corresponding to this “Mt. Fuji” is also extracted from the image A. In the same manner as described above, the feature points included in the region b are collated with the feature points included in a plurality of images (images A, B, C...) In the image database 2. As a result, it is determined that the region b matches the image B. The image keyword “Mt. Fuji” is assigned to the image B. Therefore, in this case, “Mt. Fuji” that is the keyword of the image B is assigned as the keyword of the region b extracted from the image A. Then, the data of the area b to which the area keyword “Mt. Fuji” is assigned in this way is stored in the area database 8.

  Next, the region extraction correction function in the annotation process will be described with reference to FIG. As shown in FIG. 5, as a result of extracting the region from the image B selected from the image database 2, the region c “Mt. Fuji” that should be extracted as one region originally is divided into two regions c <b> 1 and c <b> 2. May be. In such a case, when the feature points of the two regions c1 and c2 are collated, the two regions c1 and c2 have the same feature point as the image of “Mt. Fuji”, and the two regions c1 and c2 have “ The keyword “Mt. Fuji” will be given.

  In such a case, the region metadata assigning unit 7 generates a region c obtained by integrating (synthesizing) these two regions c1 and c2, and assigns the keyword “Mt. Fuji” to the region c. To remember. In this way, incomplete region extraction is corrected.

  According to the annotation apparatus 1 of this embodiment as described above, by providing the region metadata adding unit 7 that adds image metadata as region metadata, metadata can be easily added to regions included in the image. can do.

  In other words, in the present embodiment, by comparing the feature amount between the region extracted from the image and the image in the image database 2, an image including a portion similar to the region is found and added to the image. Metadata is assigned as metadata of the area. Thereby, metadata can be automatically given to an area extracted from a certain image using metadata given to a large amount of images stored in the image database 2.

  In the present embodiment, since an area to be extracted from the image is determined based on the feature points included in the image, an appropriate area can be extracted from the image.

  In the present embodiment, when a plurality of areas are extracted from one image and the same metadata is given to the plurality of areas, the plurality of areas are combined to generate one area. . That is, if the region extraction is not complete, it can be corrected.

  The embodiments of the present invention have been described above by way of example, but the scope of the present invention is not limited to these embodiments, and can be changed or modified in accordance with the purpose within the scope of the claims. is there.

  As described above, the annotation apparatus according to the present invention has an effect that metadata can be easily given to a region included in an image, and is useful for use in an image text search system or the like.

It is a block diagram for demonstrating the structure of the annotation apparatus in this Embodiment. It is explanatory drawing of the image database in this Embodiment. It is explanatory drawing of the area | region database in this Embodiment. It is a figure for demonstrating the flow of the annotation process in this Embodiment. It is a figure for demonstrating the correction function of the area | region extraction in annotation processing.

Explanation of symbols

1 Annotation device 2 Image database (image DB)
DESCRIPTION OF SYMBOLS 3 Image selection part 4 Feature point extraction part 5 Area extraction part 6 Feature point collation part 7 Area metadata provision part 8 Area database (area DB)

Claims (5)

Image selection means for selecting at least one image from an image database in which a plurality of images are stored;
Region extracting means for extracting at least one region from the selected image;
Feature point matching means for matching feature points included in the extracted region with feature points included in the plurality of images stored in the image database;
As a result of the collation, region metadata giving means for giving metadata of an image determined to match the region and the feature point as metadata of the region;
An annotation device characterized by comprising: The region extracting means includes
The annotation apparatus according to claim 1, wherein the region to be extracted from the image is determined based on a feature point included in the selected image. The region metadata giving means is
The plurality of areas are included in the selected image, and when the same metadata is given to the plurality of areas, the plurality of areas are combined to generate one area. The annotation apparatus according to claim 1 or 2. Select at least one image from an image database in which multiple images are stored,
Extracting at least one region from the selected image;
The feature points included in the extracted region are compared with the feature points included in the plurality of images stored in the image database,
An annotation method characterized in that, as a result of the collation, metadata of an image determined to match a feature point with the region is added as metadata of the region. On the computer,
Selecting at least one image from an image database storing a plurality of images;
Processing to extract at least one region from the selected image;
A process of matching feature points included in the extracted region with feature points included in the plurality of images stored in the image database;
As a result of the collation, processing for assigning metadata of an image determined to match the region and the feature point as metadata of the region;
An annotation program characterized by running