JP2001357045A - Device and method for managing image, and recording medium for image managing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the manual work when managing an image while applying bibliographical information to the image for facilitating the utilization or management of the image. SOLUTION: A bibliographical information applying part 4 based on schedule reference compares the time information of image data inputted from an image data file 11 with the time information of schedule information, which is external related information, inputted from a schedule file 41 and applies the contents of relevant schedule information as bibliographical information. Besides, a bibliographical information applying part 5 based on similar images collates a feature quantity extracted from the input image data with the feature quantity of a large quantity of sample images previously stored as image feature data 51 for collation, selects any similar image and applies bibliographical information applied to that similar image as bibliographical information of the input image data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is directed to a user of an image, such as a photograph taken by a general digital camera capable of utilizing a time stamp function.
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image management apparatus capable of extracting and managing information such as features related to an image so as to provide an image to be searched, and adding bibliographic information related to the image from external related information.

[0002]

2. Description of the Related Art As a conventional related device, for example, an image management device as shown in FIG. 10 is known. In the image management unit 110, each of a large number of images input by the image input unit 100, the image feature amount extraction unit 111 extracts a feature amount related to color and shape, and the feature amount management unit 1
12 manages the feature amount extracted for each image. Accordingly, if an image serving as a search key is given, it is possible to similarly extract a feature amount from the image, search for a similar image by matching the feature amount, and output the result to the image output unit 120. (Reference Document 1).
[Reference 1] Kazuhiko Kushima, Hiroki Akama, Seiichi Konya, Masashi Yamamuro: "Image Content Retrieval Technology Based on Surface Features such as Color and Shape", Transactions of Information Processing Society of Japan, Vol.40, No.SIG
3 (TOD1).

[0003] In addition to using a feature amount extracted from an image, a device that can refer to bibliographic information of an input image is also known (Reference Document 2). [Reference 2] Kazuhiko Kushima, Michie Sato, Hiroki Akama, Masashi Yamamuro: "Hierarchical Classification Support Function for Viewing Large Number of Images", Transactions of Information Processing Society of Japan, Vol.41, No. SIG1 (TOD5) .

Generally, when searching for an image, the color of the image and the object shown in the image are necessary information, but information not shown in the image is often necessary information about the image. For example, bibliographic information such as a title often includes information necessary for searching for a photograph or the like.

[0005] The above-described apparatus can automatically extract and manage information on the color of an image and an object shown in the image, but the necessary bibliographic information on the image must be manually created.

[0006]

When using or managing photographs and other images, the bibliographic information of the images is necessary and often important together with the information shown in the images. For example, personal names and place names are necessary and important information regarding photographs. Since such information is difficult to obtain directly from an image, if bibliographic information is not added to an image, it may not be possible to provide an image user with an image to be searched in an image search or the like.

When manually assigning bibliographic information to each image, the level of detail varies depending on the convenience of the creator of the bibliographic information or because the standards often differ for each creator. There is. In addition, a photograph taken with great care is often left for a long time, and the memory becomes ambiguous and the work becomes complicated as time passes. For this reason, when a photograph or an image is later searched, it becomes a cause that the search for the photograph or the image becomes difficult, and it is difficult to manually add the bibliographic information.

[0008] For example, suppose that there is the following photograph left for a long time. [Photo 1]: Photo taken when taking a walk in Hakone [Photo 2]: Photo taken when taking a walk in Yugawara If you want to find a photo when you took a walk in "Hakone",
It is not easy to judge which one is "Hakone" and which is "Yugawara" from the color of the photo taken when taking a walk in Hakone and the photo taken when taking a walk in Yugawara Sometimes. In this case, if a place name such as “Hakone” or “Yugawara” is given in advance to each photograph, it is possible to determine whether the photograph is a walk in “Hakone” or a walk in “Yugawara”. Conventionally, there has been a problem that the work of assigning this type of bibliographic information has to rely on manual labor as described above.

SUMMARY OF THE INVENTION It is an object of the present invention to utilize images without inputting bibliographic information for each image of a photograph or the like taken by a general digital camera capable of utilizing a time stamp function. An object of the present invention is to provide a means for assigning and managing bibliographic information relating to an image from external related information as well as features relating to the color and shape of the image so as to provide a desired image to a user.

[0010]

When using and managing an image such as a photograph, objects and colors in the image are important information, but bibliographic information on the image is also important information. Bibliographic information about a place is given to the input image by using a schedule rule by means of giving bibliographic information by referring to a schedule from external related information,
A bibliographic information providing unit using similar images uses the bibliographic information reference rule to provide bibliographic information on images such as photographs.

A program for realizing each of the above-described processing means by a computer can be stored in an appropriate recording medium such as a computer-readable portable medium memory, a semiconductor memory, and a hard disk.

[0012] In addition to extracting and managing information relating to the features of the image, such as a photograph taken by a general digital camera that can use the time stamp function, the bibliographic information relating to the image can be obtained. By assigning and managing, the images can be easily used and distributed. In particular, the task of assigning bibliographic information, which could only be done manually before, can now be performed on a computer, and the number of manual tasks can be reduced, making it possible to easily assign bibliographic information to many images. This makes it easier to use, manage and distribute images.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows a configuration example of an embodiment of the present invention. The image data file 11 is a file in which image data to which bibliographic information is to be added is stored. Image data in the image data file 11 is input to the image management unit 2 by the image input unit 1. The image management unit 2 includes an image feature amount extraction unit 21 and a feature amount management unit 22.

The image feature extraction unit 21 extracts features such as colors and shapes of the input image according to a predetermined feature extraction rule 210 and stores the results in a feature file 211. The feature amount management unit 22 manages the image in association with the feature amount, and enables an image search using the feature amount.

According to the present invention, not only an image can be searched for by a feature amount but also bibliographic information can be searched for automatically (or semi-automatically) adding bibliographic information to the image. It has a bibliographic information providing unit 4 based on a schedule and a bibliographic information providing unit 5 using similar images.

Bibliographic information providing section 4 by referring to schedule
Is the date and time when the input image was generated (photographed) in accordance with the predetermined schedule rule 40 and the schedule file 4
By comparing the date and time in 1 with the schedule information at that time, the place where the image was taken at the corresponding date and time is specified, and the place is added to the image as bibliographic information. The schedule file 41 is generally stored in a personal computer or PDA (Perso
nal Digital Assistants) and other files used by users to manage their schedules. The bibliographic information of the assigned place is stored in the bibliographic information result file 42. Bibliographic information obtained from schedule information is often information on places, but is not necessarily limited to places.

The bibliographic information providing unit 5 based on the similar image compares the feature amount data extracted by the image feature amount extracting unit 21 from the input image with matching image feature data 51 previously extracted from a large number of sample images. Then, several similar sample images are selected, and according to a predetermined bibliographic information reference rule 50, the bibliographic information previously given to the sample images is added as the bibliographic information of the input image.
The bibliographic information resulting from the assignment is stored in the bibliographic information result file 52.
Is stored in The bibliographic information added by the bibliographic information adding unit 5 using similar images is generally information that specifies the content of an image such as a subject category indicating what the subject in the image is.

Bibliographic information providing section 4 by referring to schedule
And automatically adding bibliographic information, which had to be manually added conventionally, based on externally related information such as the schedule file 41 and the image characteristic data for collation 51 by the bibliographic information providing unit 5 using similar images. Will be able to The bibliographic information assigned in this manner can be used for image search and image classification and arrangement.

FIG. 2 shows an example of adding bibliographic information by the device shown in FIG. For each image data file 11 as shown in FIG. 2, the bibliographic information adding unit 4 by referring to the schedule uses the schedule file 41 which is external related information to add, for example, bibliographic information of a place. The similar image-based bibliographic information assigning unit 5 compares bibliographic information such as the category of a subject that has been previously assigned to the similar image by collating features with a sample image prepared as collation image feature data 51. Give.

For example, since the image with the image ID # 3 in the image data file 11 is a photograph taken at 9:00, the image data 6 to which bibliographic information is added by referring to the schedule file 41 As shown, the bibliographic information of “C place name” is provided, and the bibliographic information providing unit 5 using similar images is provided with bibliographic information of “balloon”. Details of these processes will be described later.

In the present invention, it is also possible to implement only one of the addition of bibliographic information by the bibliographic information adding unit 4 by referring to the schedule and the addition of bibliographic information by the bibliographic information adding unit 5 by similar images.

FIG. 3 shows an overall processing flowchart. In step S1, the image data in the image data file 11 is input to the image management unit 2 by the image input unit 1. In step S2, the image feature amount extraction unit 21 of the image management unit 2 extracts the feature amount of the input image.

FIG. 4 shows an example of extraction of image feature values. FIG. 4 shows an example in which a feature amount is extracted from image data having an image ID of # 3 using a predetermined feature extraction rule 210.
The extracted feature amount is statistical information of a color included in the image data, a numerical value indicating a shape feature, and the like, and the result is stored in the feature file 211. Feature extraction rule 21
The processing method at 0 is like a calculation formula for feature extraction. In this feature extraction processing, for example, a technique described in detail in “Image content search technique based on surface features such as colors and shapes” in Reference Document 1 described above can be used. Is a well-known technique, and therefore, further detailed description is omitted here.

Next, in step S3, the bibliographic information providing unit 4 referring to the schedule uses the schedule file 41 to provide, for example, the bibliographic information of the place.
In step S4, the bibliographic information providing unit 5 based on similar images provides bibliographic information on similar images by collating features with sample images.

In step S5, the feature amount management unit 22 manages the input image, the feature amount obtained in step S2, and the bibliographic information obtained in steps S3 and S4 in association with each other. In step S6, the image output unit 3
Then, the image and the bibliographic information attached to the image are output.

The processing by the bibliographic information providing unit 4 with reference to the schedule will be described in more detail with reference to FIGS. FIG. 5 shows a bibliographic information providing unit 4 by referring to a schedule.
It is a processing flowchart of.

First, in step S10, time information of the image data to be processed is obtained. This time information is, for example, time information added to a photograph taken by a digital camera having a time stamp function. If time information is added as additional information separately from the image data itself, it can be used. If the time information is imprinted as a telop, it can be extracted using a simple character recognition technique or the like.

Next, at step S11, time and place and other schedule information are read from the schedule file 41.

In step S12, the schedule information is analyzed in accordance with a predetermined schedule rule 40, and the location at the time of the image data is specified. The schedule rule 40 is, for example, a conversion rule as shown in FIG. In this example, when the time information of the image data is T, the rule is that if T is between "time m" and "time n" of the schedule information, it is "place P", and the image ID is # Focusing on the image data of No. 3, T is "9:00", time m is "8:30", and time n is "1".
0:30 ". Therefore, as place P," C place name "
Is chosen.

In step S13, bibliographic information on a place is added to the image data. As a result, as shown in FIG. 6, bibliographic information of image ID = # 3, time = 9: 00, place name = C place name is stored in the bibliographic information result file 42.

FIG. 7 shows a bibliographic information providing unit 5 using similar images.
8 shows an example of the image feature data for collation 51, and FIG. 7 shows an example of the addition of bibliographic information using the bibliographic information reference rule 50.

The image feature data 51 for comparison stores the feature data of an image previously extracted from a large number of sample images by the same method as the feature extracting method by the image feature extracting unit 21. As shown in the figure, feature data such as color and shape for each image and bibliographic information given in advance by hand or other means are held.

First, in step S20, the feature quantity extracted from the image data to be processed by the image feature quantity extraction unit 21 (for example, data of the feature file 211 shown in FIG. 4).
And the matching image feature data 51 shown in FIG.
Several similar images with similar feature values are selected. FIG.
In the example, three images whose bibliographic information is a1 to a3 are selected as the similar images 7 selected from the matching image feature data 51.

Next, in step S21, the bibliographic information reference rule 50 is used to determine whether or not to use bibliographic information of a similar image. In the bibliographic information reference rule 50 shown in FIG. 9, the bibliographic information is adopted if the bibliographic information of the selected three similar images is all the same, and if not identical, the bibliographic information is not adopted as the bibliographic information. Of course, various other bibliographic information criteria or algorithms can be defined as the bibliographic information reference rule 50. In the example of FIG. 9, since the bibliographic information a1 to a3 are all "balloons" and equal, "balloon" is used as the bibliographic information of the image whose image ID is # 3.
Has been adopted.

In step S22, the bibliographic information of the adopted similar image is added to the image data to be processed. As a result, "balloon" is added to the bibliographic information result file 52 as bibliographic information as shown in FIG.

[0037]

As described above, according to the present invention, bibliographic information such as a place where an input image is taken and a name of an object is automatically or semi-automatically obtained from external related information. Can be granted. This makes it possible to manage bibliographic information on images together with images, thereby facilitating use, management, and distribution of images.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration example of an embodiment of the present invention.

FIG. 2 is a diagram showing an example of adding bibliographic information.

FIG. 3 is an overall processing flowchart.

FIG. 4 is a diagram illustrating an example of extracting an image feature amount.

FIG. 5 is a processing flowchart of a bibliographic information providing unit by referring to a schedule.

FIG. 6 is a diagram showing an example of adding bibliographic information using a schedule rule.

FIG. 7 is a processing flowchart of a bibliographic information providing unit using similar images.

FIG. 8 is a diagram showing an example of collation image feature data.

FIG. 9 is a diagram illustrating an example of adding bibliographic information using a bibliographic information reference rule.

FIG. 10 is a schematic diagram of a conventional image management device.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image input part 2 Image management part 3 Image output part 4 Bibliographic information provision part by schedule reference 5 Bibliographic information provision part by similar image 11 Image data file 21 Image feature quantity extraction part 22 Feature quantity management part 40 Schedule rule 41 Schedule file 42 Bibliographic information result file 50 Bibliographic information reference rule 51 Image feature data for collation 52 Bibliographic information result file 210 Feature extraction rule 211 Feature file

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hiroki Akama 2-3-1 Otemachi, Chiyoda-ku, Tokyo Within Nippon Telegraph and Telephone Corporation (72) Michie Sato 2-chome Otemachi, Chiyoda-ku, Tokyo No. 1 Nippon Telegraph and Telephone Corporation F-term (reference) 5B075 ND06 NK04 NK06 5C052 AA01 AB03 AB04 AC08 CC06 DD02 EE02 EE03

Claims (6)

[Claims] An image management apparatus for managing image data by adding bibliographic information to the image data, comprising: means for inputting image data having time information at the time of shooting or generation; Means for inputting the schedule information at the given date and time separately from the image data, and comparing the time information of the image data with the time information in the schedule information so that the contents of the corresponding schedule information are stored in the bibliographic information of the image data. An image management apparatus comprising: 2. An image management apparatus for managing bibliographic information by adding bibliographic information to image data, comprising: means for inputting image data; means for extracting a feature amount of an image from the input image data; Storage means for storing image feature data for comparison for storing feature amounts extracted from the image data and bibliographic information of each sample image, and input image data obtained by comparing the feature amount extracted from input image data with the image feature data for comparison. Select a sample image similar to
Means for adding bibliographic information added to the sample image as bibliographic information of input image data. 3. An image management method for adding bibliographic information to image data and managing the image data. The method comprises the steps of: inputting image data having time information at the time of shooting or generation; By inputting the schedule information at the given date and time separately from the image data, and comparing the time information in the image data with the time information in the schedule information, the contents of the corresponding schedule information are stored in the bibliographic information of the image data. And a process of assigning as an image management method. 4. An image management method for adding bibliographic information to image data and managing the image data, wherein the step of inputting the image data, the step of extracting a feature amount of the image from the input image data, and the step of extracting from the input image data The extracted image data is compared with the image data of each sample image extracted in advance from a large number of sample images, a sample image similar to the input image data is selected, and the bibliographic information added to the sample image is converted into the input image data. Providing the data as bibliographic information of the data. 5. A recording medium storing a program for adding bibliographic information to image data and managing the image data, the image data having time information when photographed or generated, and a process for inputting image data. The process of inputting the schedule information at the date and time when the image was captured or generated separately from the image data, and comparing the time information of the image data with the time information in the schedule information, the contents of the corresponding schedule information are described above. A recording medium for storing an image management program, wherein a program for causing a computer to execute a process of adding image data as bibliographic information is recorded. 6. A recording medium storing a program for adding bibliographic information to image data and managing the image data, the image data being input, a process of extracting an image feature amount from the input image data, The feature amount extracted from the input image data is compared with the feature amount of each sample image extracted from a large number of sample images in advance, a sample image similar to the input image data is selected, and the bibliography attached to the sample image is selected. A recording medium for an image management program, wherein a program for causing a computer to execute a process of giving information as bibliographic information of input image data is recorded.