JP2006099268A - Image retrieval device, image retrieval support device, image retrieval system, image retrieval method, and image retrieval program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently retrieve desired images while reducing the burden of providing metadata. <P>SOLUTION: Upon downloading image data captured with a camera 7, a terminal device 6 requests a metadata providing server 2 to provide metadata to the image data. Upon receiving the request from the terminal device 6 to provide the metadata to the image data, the metadata providing server 2 provides the metadata to the image data and sends the image data provided with the metadata back to the terminal device 6. Upon receiving the image data with the metadata from the metadata providing server 2, the terminal device 6 changes the metadata provided to the image data into a name that is easy for the user to understand, such as a personal name or nickname, and retrieves the image data according to the metadata changed. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image search device, an image search support device, an image search system, an image search method, and an image search program, and is particularly suitable for application to a method of adding metadata to image data.

With the recent expansion of communication networks and lower prices of devices that handle digital images such as digital cameras, digital images are actively used. Regarding the storage of these digital images, it is common to store them in a personal computer in addition to printing directly as a photograph using a printer.
Further, when the hard disk capacity of the personal computer and the memory capacity of the digital camera are increased, a huge number of digital images are stored in the personal computer. For this reason, it is necessary to organize the digital images so that the user can easily find the desired digital image. Here, when organizing digital images with a personal computer, the digital images taken at the same time can be stored together in one directory, given a name that is easy to understand for each directory, and the directory can be managed in a tree structure. Usually done.

  Further, for example, in Patent Document 1, the system performs the same level of understanding and recognition as an image that is subject to search, and assigns the recognition result to the image as metadata. In searching, a method is disclosed in which a user's desire is expressed as a keyword, and an arbitrary image can be easily found with a small number of steps by comparing metadata and the keyword.

Further, for example, in Non-Patent Document 1, by displaying an object related to an object at the center of a certain object in a network form, a list of relations between objects can be displayed, and a search considering the listability of the objects can be performed. A method of doing so is disclosed.
JP 11-306322 A http: // www. dolog. com / weblog / BlogServet? CMD = GRAPH & userid = 871

  However, in the method of assigning a name to the directory in which digital images are stored and managing them using a tree structure, images are searched for based on the characteristics of individual images instead of the characteristics of the images, or images are searched based on abstract requests. There was a problem that it was difficult to do. For example, if multiple images taken with a digital camera when traveling at a certain time are stored in a directory named “Travel”, “If you forgot the time, the background of when you traveled If you search for images based on features such as “images with mountains in”, “all images with dad”, or “images with flowers,” they are stored in a directory named “Travel”. There is a problem that it is necessary to open and check each image one by one, which is very time-consuming.

  In addition, in the method disclosed in Patent Document 1, when assigning metadata, it starts by selecting and setting a model image to be subjected to image analysis by applying an image recognition algorithm from target images. There was a problem that a lot of time and a large processing capacity were required. In addition, when metadata is input by an individual, there is a problem that although it is accurate, a burden on inputting the metadata becomes very large.

  Furthermore, in the search, when the user inputs the search condition, the system extracts the filtering key and collates the filtering key with the keyword, so that these matched images are output as the search result. Is done. For this reason, the verification operation is multi-staged, and since trial and error such as loosening if the search condition is narrowed down too much is performed, it takes time to search, and a related image is seen due to a slight difference in the search condition. There is also a problem that it is not possible to perform a search considering listability. In addition, when searching for a plurality of images extending over a plurality of directories, it is necessary to browse a plurality of directories, which is troublesome.

In addition, in the method disclosed in Non-Patent Document 1, the display is limited to a two-dimensionally expanded presence / absence and strength of relevance between a plurality of objects. For this reason, the transition of the relationship between multiple objects is displayed in a three-dimensional manner by adding a time axis, and the amount of information held by the object is somewhat (here, the number of images that store metadata described as keywords) There is a problem that cannot be displayed at the same time.
Accordingly, an object of the present invention is to provide an image search device, an image search support device, an image search system, an image search method, and an image search device that can efficiently search for a target image while reducing the burden on adding metadata. To provide an image search program.

  In order to solve the above-described problem, according to the image search device of claim 1, metadata grant request means for requesting addition of metadata to image data, and metadata is given in response to the request. Metadata-added image receiving means for receiving image data, metadata changing means for changing metadata attached to the image data, and image data search for searching the image data based on the changed metadata Means.

  As a result, it is possible to assign metadata at the request destination, eliminating the need to assign metadata at the own terminal, and at the terminal's own side the metadata assigned at the request destination. You can change the name or nickname to a name that is easy for you to understand. For this reason, even when a huge number of digital images are stored in your terminal, it is possible to reduce the burden of adding metadata while protecting personal privacy, By giving it, it becomes possible to reach the target image from various keywords, and it is possible to efficiently perform a search without omission corresponding to the characteristics of each image and abstract search requests.

The image search apparatus according to claim 2 further includes incidental information adding means for adding incidental information including date and time information when the image data is taken to the image data for which a request for adding metadata is made. It is characterized by.
As a result, it is possible to make a request for adding metadata with additional information added. For this reason, it is possible to efficiently extract events from image data, and it is possible to search image data for each event while reducing the burden of adding metadata.

The image search apparatus according to claim 3, further comprising metadata assignment request condition setting means for setting a metadata assignment request condition for image data for which a request for assignment of metadata is performed. The grant request condition includes a disclosure condition of the image data.
As a result, it is possible to give metadata that meets the request of the client, and to disclose image data as necessary.

  The image search apparatus according to claim 4, further comprising: an object display unit that network-displays objects related to each other using the changed metadata as an object, and the object display unit is provided with the same metadata. At least one of an object changing means for changing the size of the object according to the quantity of the image data and a relevance control means for controlling the relevance between the objects in conjunction with the time of shooting. Features.

  This makes it possible to display a list of a plurality of objects in consideration of not only the relationship between objects but also the temporal characteristics and amount of information of the objects. This makes it possible to efficiently find the desired image data from the object display screen, and to reach the image preview screen from the object display screen by a single step (double-click) operation. Time can be shortened.

According to the image search support device of claim 5, metadata grant request accepting means for accepting a request for giving metadata to image data, and metadata grant for giving metadata to the accepted image data And metadata added image providing means for providing the request source with the image data to which the metadata is attached.
As a result, it is possible to support the addition of metadata, and even when a large number of digital images are stored in a personal terminal, it is possible to reduce the burden of giving metadata individually. It becomes possible.

According to another aspect of the image search support apparatus of the present invention, the metadata assigning means assigns the same metadata to the same person, the same event, or the same phenomenon.
Thereby, even when the same person, the same event, or the same phenomenon is reflected in a huge number of digital images, the digital images can be collectively handled on the personal terminal side. For this reason, even when searching for images from the characteristics of individual images, it is not necessary to manually check the images one by one, and an efficient search corresponding to the characteristics of the individual images can be performed efficiently. It becomes possible.

Further, according to the image search support device according to claim 7, the metadata giving means includes person image metadata giving means for giving metadata to a person image of the image data, and metadata for an event of the image data. And event metadata providing means for assigning metadata to the event of the image data.
This makes it possible to add metadata for each person image, event, and event. Therefore, it is possible to search image data for each person image, event, and event, and it is possible to efficiently perform a leak-free search corresponding to individual image features and abstract search requests.

Further, according to the image search support device of the present invention, the person image metadata assigning means assigns metadata to the person image by assigning people.
As a result, even when a plurality of persons appear in the same image, it is possible to distinguish between the persons, and even when the same person appears in a plurality of images, The person can be identified. For this reason, even when a large number of digital images are stored on the personal terminal side, it is possible to collectively handle images showing the same person on the personal terminal side. Even when searching for images, it is possible to efficiently search without omission corresponding to the characteristics of each image without manually checking the images one by one.

In addition, according to the image search support device of claim 9, the metadata grant request accepting unit includes additional information including date and time information when the image data is photographed in the image data for which the metadata grant request is made. The event metadata adding unit adds metadata to the event of the image data based on the incidental information.
This makes it possible to efficiently extract events from image data, reducing the burden of adding metadata, and efficiently performing leak-free searches that support abstract search requests. It becomes possible.

The image search support device according to claim 10, further comprising reference image storage means for storing a reference image to which metadata is attached, wherein the event metadata assignment means includes the image data and the reference image. On the basis of the comparison result, the metadata attached to the reference image is attached to the event of the image data.
As a result, it is possible to efficiently extract events from the image data, and it is possible to efficiently perform a complete search corresponding to an abstract search request while reducing the burden of adding metadata. It becomes possible.

The image search support device according to claim 11 further includes metadata addition request condition acquisition means for acquiring a metadata addition request condition set for image data for which a request for giving metadata is performed. The metadata-added image providing unit publishes image data for which a request for adding metadata has been made based on the request for adding metadata.
This makes it possible to add metadata that meets the client's request, and to release image data according to the client's request.

According to the image search support system of claim 12, in the image search support system in which the terminal storing the image data and the metadata grant server are connected via a network, the image performed at the terminal A part of processing accompanying the addition of metadata to data is shared by the metadata providing server.
Accordingly, it is possible to assign the metadata to the image data while sharing the processing that is burdensome for the individual to the metadata providing server and sharing the processing unique to the individual with his / her terminal. For this reason, even when a large number of digital images are stored in your terminal, it is possible to reduce the burden of adding metadata while protecting personal privacy, It is possible to efficiently perform a search without omission corresponding to the features and abstract search requests.

  In the image search support system according to claim 13, the terminal includes: a metadata grant request unit that requests the metadata grant server to grant metadata to image data; and the metadata grant server. Metadata-attached image receiving means for receiving image data to which metadata is attached; metadata-changing means for changing metadata attached to the image data; and the image data based on the changed metadata. Image data search means for performing a search, wherein the metadata grant server accepts a request for giving metadata to the image data from the terminal, and metadata in the accepted image data And metadata providing means for providing image data to which the metadata is provided to the terminal. Characterized in that it comprises an applying image providing means.

  This makes it possible to assign metadata on the metadata grant server, eliminating the need to assign metadata on one's own terminal, and knowing the metadata given on the metadata grant server to oneself. It can be changed to an easy name. For this reason, even when a huge number of digital images are stored in your terminal, it is possible to reduce the burden of adding metadata while protecting personal privacy, By making changes, it is possible to easily reach the target image from the characteristics of the individual images, and the terminal can efficiently perform a leak-free search corresponding to the characteristics of individual images and abstract search requests. It becomes possible.

The image search support system according to claim 14, wherein the metadata adding means includes person image metadata adding means for adding metadata to a person image of the image data, and metadata for an event of the image data. And event metadata providing means for assigning metadata to the event of the image data.
This makes it possible to assign metadata for each person image, event, and event at the metadata grant server, and to perform a search without omission corresponding to individual image features and abstract search requests. It becomes possible to carry out efficiently.

Further, according to the image search support system of the fifteenth aspect, the person image metadata assigning means assigns metadata to the person image by assigning persons.
As a result, different metadata can be added to the person image in which different persons are captured by the metadata adding server, and a search without omission corresponding to the characteristics of each image can be efficiently performed on the terminal side. Is possible.

Further, according to the image search support system according to claim 16, the terminal adds additional information including additional information including date and time information when the image data is photographed to the image data for which a request for adding metadata is made. The metadata addition request accepting means further comprises accompanying information acquisition means for acquiring the accompanying information from the terminal, and the event metadata attaching means is configured to generate an event of the image data based on the accompanying information. It is characterized in that metadata is assigned to.
This makes it possible to efficiently extract events from image data on the metadata grant server, reducing the burden of adding metadata, and searching without omissions in response to abstract search requests. Can be efficiently performed on the terminal side.

Further, according to the image search support system according to claim 17, the metadata providing server further includes a reference image storing unit that stores a reference image to which metadata is added, and the event metadata adding unit includes: Based on a comparison result between the image data and the reference image, metadata attached to the reference image is attached to the event of the image data.
As a result, it is possible to efficiently extract events from image data with the metadata grant server, reducing the burden of granting metadata, and searching without omissions in response to abstract search requests. Can be efficiently performed on the terminal side.

Further, according to the image search support system of claim 18, the terminal further includes metadata addition request condition setting means for setting a metadata addition request condition including a disclosure condition of the image data in the image data, The metadata grant server further comprises metadata grant request condition acquisition means for acquiring the metadata grant request condition from the terminal, and the metadata grant image providing means is based on the metadata grant request condition, The image data is disclosed.
This makes it possible to provide metadata that meets the client's request on the metadata providing server side, and to make image data available according to the client's request.

In addition, according to the image search support system of claim 19, the terminal further includes object display means for network-displaying objects related to each other using the changed metadata as an object, and the object display means are the same. At least one of an object changing unit that changes the size of the object according to the quantity of image data to which the metadata is added and a relevance control unit that controls the relevance between objects in association with the time of shooting One is provided.
As a result, it is possible to display a list of a plurality of objects while considering not only the relationship between objects but also the temporal characteristics and amount of information of the objects, thereby improving the search accuracy and the search time. Can be shortened.

  Further, according to the image search method of claim 20, the step of requesting a metadata grant server to give metadata to image data, and the step of giving metadata to the image data requested from the terminal A step of accepting at the metadata granting server, a step of giving metadata to the image data accepted by the metadata granting server, and image data to which the metadata is given by the metadata granting server to the terminal Providing at the terminal, receiving at the terminal the image data provided with metadata at the metadata providing server, changing the metadata attached to the image data at the terminal, Searching for the image data in the terminal based on the changed metadata. To.

  As a result, it is possible to share a part of the process of giving metadata to the image data with the metadata giving server, and to reduce the burden on giving the metadata, while also making the characteristics of each image In addition, it is possible to efficiently perform a leak-free search corresponding to abstract search requests on the terminal side.

In the image search method according to claim 21, the step of assigning metadata to the image data assigns metadata to the person image by sorting person images included in the image data. And a step of assigning metadata given to the reference image to an event of the image data based on a comparison result between the reference image to which metadata is given and the image data. .
This makes it possible to extract human images and events from the image data at the metadata grant server, while reducing the burden of adding metadata, and for individual image features and abstract search requests. Therefore, it is possible to efficiently perform a leak-free search on the terminal side.

In addition, according to the image search method of claim 22, the step of adding incidental information including date and time information when the image data was photographed to the image data for which a request for giving metadata is added at the terminal, Acquiring the additional information added at the terminal at the metadata adding server, and adding the metadata to the image data includes adding metadata to the event of the image data based on the auxiliary information. It is characterized by providing the step which provides.
This makes it possible to efficiently extract events from image data on the metadata grant server, reducing the burden of adding metadata, and searching without omissions in response to abstract search requests. Can be efficiently performed on the terminal side.

The image search method according to claim 23, wherein the terminal sets metadata request conditions including a disclosure condition for the image data at the terminal, and the metadata set at the terminal. The method further comprises the steps of: obtaining a grant request condition at the metadata grant server; and publishing the image data based on the metadata grant request condition.
This makes it possible to provide metadata that meets the client's request on the metadata providing server side, and to make image data available according to the client's request.

In addition, according to the image search method of claim 24, the step of displaying the objects related to each other using the changed metadata as an object, and the quantity of image data to which the same metadata is assigned The method further includes the step of changing the size of the object, and the step of controlling the relationship between the objects in association with the time of shooting.
As a result, it is possible to display a list of a plurality of objects while considering not only the relationship between objects but also the temporal characteristics and amount of information of the objects, thereby improving the search accuracy and the search time. Can be shortened.

According to the image search program of claim 25, the step of giving the metadata to the image data on the network, the step of changing the metadata assigned on the network on the terminal, And causing the computer to execute a step of searching for the image data based on the changed metadata.
As a result, general-purpose processing that is burdensome for individuals can be shared by the metadata grant server, and metadata can be added to image data while processing that is unique to the individual is shared by one's own terminal. It is possible to efficiently perform a search without omission corresponding to the characteristics of individual images and abstract search requests while reducing the burden of data assignment.

  As described above, according to the present invention, it is possible to share a part of the process of giving metadata to image data on the network, and it is possible to reduce the burden of giving metadata while reducing It is possible to efficiently perform a search without omission corresponding to image features and abstract search requests.

Hereinafter, an image search system and an image search method according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of an image search system according to an embodiment of the present invention.
In FIG. 1, a communication network 1 is connected with a metadata grant server device 2, an image providing server device 5, and terminal devices 6 and 8, and a service terminal device 4 is connected to the metadata grant server device 2 and a terminal. A camera 7 is connected to the device 6.

  Here, the metadata providing server device 2 can accept a request for adding metadata to the image data from the terminal device 6, and can reply to the terminal device 6 after adding the metadata to the requested image data. . The service terminal device 4 assists the metadata adding process performed by the metadata adding server device 2. When it is difficult to automatically add metadata by the metadata adding server device 2, Add metadata manually. The terminal device 6 can request the metadata giving server 2 to give metadata to the image data, and can change the metadata given to the image data by the metadata giving server on its own terminal. The image providing server device 5 can store photographed images and publish the photographed images on the Web. The terminal device 8 accesses the image providing server device 5 to view or capture a photographed image released by the image providing server device 5.

  The communication network 1 can be a public communication network that performs IP communication, and may be the Internet. Alternatively, it may be a fixed telephone communication network, a mobile communication network, a LAN, or the like. The terminal devices 6 and 8 may be a notebook personal computer or a desktop personal computer, or may be a mobile phone terminal or a PDA (Personal Data Assistant).

  And the terminal device 6 will request | require the addition of the metadata to image data to the metadata provision server 2, if the image data image | photographed with the camera 7 is taken in. When the metadata giving server 2 receives a request for giving metadata to the image data from the terminal device 6, the metadata giving server 2 gives metadata to the image data, and the image data to which the metadata is given is sent to the terminal device 6. Send back. When the terminal device 6 receives the image data to which the metadata is attached from the metadata assignment server 2, the terminal device 6 can change the metadata attached to the image data to a name that is easily understood by the user such as an individual name or a nickname. . The terminal device 6 can search for image data based on the changed metadata.

  As a result, the terminal device 6 can share a general-purpose process with a large burden to the metadata providing server 2 and can add metadata to the image data while the terminal device 6 shares the process related to personal information. . For this reason, even when a huge number of digital images are stored in the terminal device 6, it is possible to reduce the burden on the addition of metadata while protecting the privacy of the individual, as well as individual images. It is possible to efficiently perform a search without omission corresponding to the features and abstract search requests.

FIG. 2 is a block diagram showing an overview of the metadata providing method according to an embodiment of the present invention.
In FIG. 2, the metadata grant server apparatus 2 includes a metadata grant request receiving unit 21 that accepts a request for giving metadata to image data from the terminal device 6, and a person that grants metadata to a person image of the image data. An image metadata giving unit 23, an event metadata giving unit 24 for giving metadata to an event of image data, an event metadata giving unit 25 for giving metadata to an event of image data, and image data to which metadata is given. A metadata-added image providing unit 28 provided to the terminal device 6 is provided. Further, the reference image 104 is stored in the metadata providing server apparatus 2, and metadata is assigned to the reference image 104.

  The terminal device 6 stores, for example, an image file 101 in which an image photographed by the camera 7 when a family travels to a zoo or abroad is stored. When the terminal device 6 adds the metadata 114 to the image data stored in the image file 101, the terminal device 6 accesses the metadata adding server 2 and assigns the metadata 114 to the image data. To ask. Note that the terminal device 6 can set a metadata addition request condition for designating how to add the metadata 114 when making a request for adding the metadata 114. It should be noted that the metadata grant request condition may include a condition on how to publish such as whether or not the image data is published free of copyright.

  Then, when a request for giving the metadata 114 is made, the metadata giving server 2 accepts a request for giving the metadata 114 to the image data from the terminal device 6 at the metadata giving request receiving unit 21. Save as the raw image archiver 103. Then, when a request for adding the metadata 114 to the image data is received by the metadata grant request accepting unit 21, the metadata 114 that can uniquely identify the person image of the image data is sent to the person image metadata assigning unit 23. Is granted. Here, the person image metadata assigning unit 23 can assign metadata to the person image by sorting the persons.

  Further, when the request for adding the metadata 114 to the image data is received by the metadata grant request accepting unit 21, the event metadata giving unit 24 grants metadata 114 that can uniquely identify the event of the image data. Is done. Here, the event metadata giving unit 24 can give the metadata 114 to the event of the image data based on the incidental information including the date and time information when the image data was taken.

  Further, when the request for adding the metadata 114 to the image data is accepted by the metadata grant request accepting unit 21, the metadata 114 that can uniquely identify the event of the image data is given by the event metadata giving unit 25. Is done. Here, the event metadata adding unit 25 compares the image data with the reference image 104, and can add one or a plurality of metadata 114 included in the reference image having high similarity to the event of the image data. .

  Further, the metadata grant request accepting unit 21 determines that the metadata 114 is difficult to be automatically given by the person image metadata grant unit 23, the event metadata grant unit 24, or the event metadata grant unit 25. The service terminal apparatus 4 is requested to give 114. Then, when the service terminal device 4 receives a request for adding the metadata 114 from the metadata grant request receiving unit 21, the service terminal device 4 manually assigns the metadata 114 on the service terminal device 4, and the image to which the metadata 114 is given. The data is provided to the metadata grant request receiving unit 21. Here, the service terminal device 4 can refer to the reference image 104 stored in the metadata assignment server device 2 when assigning the metadata 114 to the image data that has been requested to be assigned the metadata 114. . Then, when the metadata 114 is added to the image data at the service terminal device 4, the image data to which the metadata 114 is added can be stored in the reference image 104. As a result, the accuracy of metadata given to the reference image 104 can be improved, and metadata can be automatically given under more conditions.

  Then, when the metadata 114 is added to the image data requested from the terminal device 6, the metadata-added image providing unit 28 provides the image data with the metadata 114 to the terminal device 6. When the terminal device 6 receives the image data to which the metadata is attached from the metadata assignment server 2, the terminal device 6 changes the metadata attached to the image data to a name that is easily understood by the user such as an individual name or a nickname. For example, when the metadata 114 is assigned to the person image in the metadata assignment server device 2 and the person is his / her child, the metadata 114 attached to the person image is used as the name of his / her child. It may be changed, and when the metadata 114 is added to the event in the metadata adding server device 2, if the event is the time when the event has been made to the zoo, the event is assigned to the event. The metadata 114 may be changed to a name “zoo”.

  As a result, the metadata 114 can be given by the metadata grant server device 2, and the metadata 114 need not be given by the terminal device 6, and is given by the metadata grant server device 2. The metadata 114 can be changed to a name that can be easily understood by the user such as an individual name or a nickname on the terminal device 6. For this reason, even when an enormous number of image files 101 are stored in one's own terminal, it is possible to reduce the burden on the provision of the metadata 114 while protecting personal privacy, By adding data 114, it is possible to reach the target image from various keywords, and it is possible to efficiently perform a search without omission corresponding to the characteristics of individual images and abstract search requests. Become.

  Also, the metadata-added image providing unit 28 provides the image providing server device 5 with the image data to which the metadata 114 has been added when the request to release image data is in a copyright-free request condition. . When the image providing server device 5 receives the image data to which the metadata 114 is added from the metadata-added image providing unit 28, the image providing server device 5 accumulates the image data in the image archiver 105. Then, the terminal device 8 can browse and capture the captured image stored in the image archiver 105 by accessing the Web 51 server.

As a result, it is possible to easily collect image data taken by various people, and it is possible to distribute the image data to which the metadata 114 has been added, thereby enabling various reuse.
Here, the image data provided with the metadata stored in the image archiver 105 can be provided to the metadata providing server device 2 as the reference image 104. As a result, the metadata providing server apparatus 2 can collect more reference images 104, increase the number of comparison images used for pattern matching, and automatically perform metadata under more conditions. Granting is possible.

  In the above-described embodiment, the method of adding the metadata 114 to the person image, the event, and the event has been described. However, when an object is reflected in the image data, the metadata adding server device 2 The metadata 114 may also be assigned to the object. For example, the metadata adding server device 2 may add the metadata 114 “car” if a car appears in the image data, or add the metadata 114 “dog” if a dog appears in the image data. May be. On the terminal device 6 side, when the metadata 114 “car” is given, if the user has “crown”, the metadata 114 “car” is changed to the metadata 114 “crown”. If the metadata 114 “dog” is given, and the name of the dog is “pochi”, the metadata 114 “dog” is changed to the metadata 114 “pochi”. It may be changed.

FIG. 3 is a block diagram illustrating an example of a metadata providing method according to an embodiment of the present invention.
In FIG. 3, when a captured image 111 is captured by the camera 7, the terminal device 6 captures the captured image 111. Here, the camera 7 can add the supplementary information 112 to the photographed image 111 when photographing the photographed image 111, and the terminal device 6 can capture the photographed image 111 together with the supplementary information 112. The incidental information 112 can include date and time information when the captured image 111 is captured. Further, when the camera 7 has a function of detecting position information, position information indicating the position when the captured image 111 is captured may be added to the incidental information.

  Then, when the terminal device 6 captures the captured image with supplementary information 112 in which the supplementary information 112 is added to the photographed image 111, the terminal device 6 sets the metadata provision request condition 113 to the photographed image 111 to thereby obtain the metadata provision request image. 142 is generated. Here, as the metadata assignment request condition 113, for example, a condition such as assigning metadata to a person image, an event, and an event can be given. Further, the metadata grant request condition 113 may include a condition on how to publish such as whether to publish image data without copyright.

  And the terminal device 6 will request | require the metadata provision server apparatus 2 of provision of the metadata to the picked-up image 111, if the metadata provision request image 142 is produced | generated. Here, for example, when it is requested to add metadata to the four captured images 111a to 111d as the captured images 111, the metadata providing server apparatus 2 selects all the four captured images 111a to 111a. A metadata-added image 110 is generated by adding metadata to 111d. That is, the metadata providing server apparatus 2 extracts person images from the photographed images 111a to 111d by performing image recognition of the photographed images 111a to 111d, and assigns metadata to these person images. For example, when the first person is captured in the captured images 111a to 111c, the metadata 114 that identifies the first person image is added to the captured images 111a to 111c. In addition, when the second person is captured in the captured images 111b and 111d, metadata 114 that identifies the second person image is added to the captured images 111b and 111d. Further, when a third person is captured in the captured image 111d, metadata 114 that identifies the third person image is added to the captured image 111d.

  Further, the metadata providing server apparatus 2 extracts events from the captured images 111a to 111d based on the date and time information included in the incidental information 112, and assigns event metadata 114 to the captured images 111a to 111d. For example, when the first event is shown in the captured images 111a and 111b, the metadata 114 that identifies the first event is added to the captured images 111a and 111b. Further, when the second event is shown in the captured images 111c and 111d, the metadata 114 specifying the second event is added to the captured images 111c and 111d. Here, when event metadata is added to the captured images 111a to 111d, the date and time information added to the captured images 111a to 111d is checked, and if the captured dates are close to each other, it is determined that they are the same event. When the date and time of shooting are different from each other, it is possible to add different metadata 114 for each different event by determining that the events are different.

FIG. 4A is a diagram illustrating an example of an event extraction method of the metadata providing server apparatus 2 of FIG.
In FIG. 4A, the metadata assignment server apparatus 2 can group the captured images based on the date / time information added to the captured images and the number of captured images. Then, a group of photographed images is formed by collecting photographed images whose photographing dates and times are close to each other. For example, if there are captured images that are continuously captured from June 16 to June 17, a group G1 is generated in which these captured images are collected. Further, if there are captured images that are continuously captured in June 18 at a predetermined time from the group G, it is determined that these captured images are different from the events of the group G1, and the captured images are summarized. Group G2 is generated.

Under the above conditions, if the captured image 111a is captured on June 16 and the captured image 111b is captured on June 17, the captured images 111a and 111b are considered to be in the group G1, and the first Metadata 114 that identifies an event can be assigned to each of the captured images 111a and 111b. Also, assuming that the captured images 111c and 111d were captured on June 18, the captured images 111c and 111d are considered to be in the group G2, and the metadata 114 specifying the second event is used as the captured image 111c, 111d can be given respectively.
As a result, even when the captured images 111a and 111b are captured across dates, the metadata 114 of the same event can be added to the captured images 111a and 111b, and an event can be extracted from the captured images 111a to 111d. Can be performed with high accuracy.

  Further, the metadata assignment server apparatus 2 extracts events from the photographed images 111a to 111d by comparing the reference image 104 to which the metadata is imparted and the photographed images 111a to 111d, and those photographed images 111a to 111d. To the event metadata 114. For example, if the reference image 104 to which the metadata “spring” is assigned is similar to the photographed image 111a, the metadata 114 “spring” is assigned to the photographed image 111a. If the reference image 104 to which the metadata “Haruyama” is assigned is similar to the captured image 111b, the metadata 114 “Haruyama” is assigned to the captured image 111b. Further, assuming that the reference image 104 to which the metadata “mountain” is added is similar to the captured image 111c, the metadata 114 “mountain” is added to the captured image 111c. If the reference image 104 to which the metadata “sea” is assigned is similar to the photographed image 111d, the metadata 114 “sea” is assigned to the photographed image 111d.

FIG. 4B is a diagram illustrating an example of an event metadata adding method of the metadata adding server apparatus 2 of FIG.
In FIG. 4B, the metadata providing server device 2 is provided with a reference image storage unit 26 that stores the reference image 104. Then, the metadata providing server apparatus 2 performs pattern matching between the captured image 111 transmitted from the terminal apparatus 6 and the reference image 104 stored in the reference image storage unit 26. Then, the reference image 104 that has obtained a certain score or more is regarded as having high similarity, and the metadata to which the reference image 104 is added is added to the captured image 111.

And the metadata provision server apparatus 2 will provide the metadata provision image 110 to the terminal device 6, if the metadata provision image 110 about the picked-up images 111a-111d is produced | generated.
And the terminal device 6 will generate the change metadata provision image 143 by changing the metadata 114 provided to the picked-up images 111a-111d based on the metadata change information 115, if the metadata provision image 110 is received. In addition, a directory for managing the changed metadata-added image 143 is generated.

  For example, the first person image specified by the metadata 114 is “Midori” of the child, the second person image specified by the metadata 114 is “dad”, and the third person specified by the metadata 114 Is the “mother”, the first event specified in the metadata 114 is “zoo”, and the second event specified in the metadata 114 is “travel to Italy”. In this case, “green” is the first person image, “dad” is the second person image, “mother” is the third person image, “zoo” is the first event, and “zoo” is the first event. The metadata change information 115 “Travel in Italy” is input to the terminal device 6. When the metadata change information 115 is input, the terminal device 6 extracts the metadata 114 assigned by the metadata assignment server device 2 from the captured images 111a to 111c. Then, by changing the metadata 124 changed by the metadata change information 115 to the captured images 111a to 111c, a changed metadata-added image 143 in which the metadata 124 is added to the captured images 111a to 111c is generated.

  As a result, metadata 124 “green”, “zoo”, and “spring” is given to the photographed image 111a, and “green”, “dad”, “zoo”, and “haruyama” are assigned to the photographed image 111b. Metadata 124 is added, metadata 124 “green”, “travel to Italy”, and “mountain” is added to the photographed image 111c, and “dad”, “mother”, “travel to Italy” are added to the photographed image 111d. ”And“ Sea ”metadata 124.

  This makes it possible to assign metadata while sharing functions between the terminal device 6 and the metadata assignment server device 2, and the metadata assignment server device 2 can share the distinction of a person or an event. And the personal name of the user and the place name of the destination can be prevented from leaking outside.

  When the terminal device 6 generates the changed metadata-added image 143, the terminal device 6 performs address management for network-type display, thereby enabling directory-type display and network-type display. That is, the captured image address 116 is managed so that the captured image 111 specified by the object serving as the center of the network type display can be specified by the arbitrary metadata 114 added to the captured image 111. For example, when the network type display is performed around the object “green”, the photographed images 111a to 111c in which “green” is photographed are arbitrary person images, events, or events reflected in the photographed images 111a to 111c. The captured image address 116 is managed so that it can be specified for each event.

  That is, there are three photographed images 111a to 111c in which “Midori” is photographed, one photographed image 111b in which “Midori” is photographed with the father, and two photographed images 111a and 111b in which “Midori” is present in the zoo. One shot image 111c when “Midori” traveled to Italy, one shot image 111a in which “Midori” was photographed in spring, and one shot image 111b in which “Midori” was in Haruyama, There is only one photographed image 111c in which “Midori” is in the mountain. Therefore, the captured image address 116 is managed for each of the captured images 111a to 111c to which the metadata 124 is assigned. Then, by managing the photographed image address 116 for each of the photographed images 111a to 111c to which the metadata 124 “green” is given, it is possible to perform network-type display centering on the object “green”.

FIG. 5 is a diagram showing an example of a display screen of the terminal device 6 of FIG.
In FIG. 5, in the terminal device 6, a directory type display screen 151 and a network type display screen 152 are prepared, and the directory type display screen 151 and the network type display screen 152 can be switched. Here, on the directory type display screen 151, a folder 151 a is displayed for each object such as “Midori”, “Haruyama”, “Travel in Italy”, and double-clicking the folder 151 a displays the preview images of the captured images 111 a to 111 d. Can be migrated.

  On the other hand, on the network type display screen 152, “Dad”, “Mom”, “Travel in Italy”, “Zoo”, “Sea”, “Spring”, “Mountain” and “Haruyama” are centered on the object 152 a “Midori”. Are concentrically arranged. Here, the size of the object may be changed according to the quantity of the captured images 111a to 111d possessed by each object, or the object is linked to the time when the captured images 111a to 111d possessed by each object are captured. You may make it control the presence or absence of the relationship between.

  That is, the objects are arranged concentrically in the order of objects with a large number of captured images in common with “Midori” centering on “Midori”. In addition, in the distance between the object 152a arranged at the center and the object directly connected to the object 152a arranged at the center, the shooting with the metadata 124 having the direct connection to the object 152a arranged at the center added. The larger the number of images, the closer to the object 152a arranged at the center. Note that how many objects having direct or indirect connection with the object 152a arranged at the center can be set on the configuration screen.

  Then, by clicking the object, the object is selected, the color of the object is inverted, and the screen can be searched by double-clicking to display a preview screen. Further, when an object other than “Midori” is brought close to the center of the network type display screen 152 by dragging and dropping, for example, an object “Dad”, the network type display screen 152 centered on the object “Dad” is displayed. Can be made. The color of the object can be changed by setting from the category or right click. In addition, the time can be changed by operating the toolbar displayed in the lower right, and information other than the photograph such as the photographed date can be filtered and displayed. As a result, by moving the toolbar, it is possible to express the presence / absence of an object and the transition of its relevance.

  Further, in the metadata grant request condition 113, for example, when the condition that only the event metadata 114 is given and the captured image 111 is made public is set, the metadata grant server device 2 sets the event metadata 114. The image providing server apparatus 5 is provided with the captured image 111 to which only the image is assigned. When the image providing server device 5 receives the captured image 111 to which only the event metadata 114 is added from the metadata grant request condition 113, the image providing server device 5 displays the captured image 111 to which only the event metadata 114 is added as a public image 130. Publish as

FIG. 6 is a block diagram showing a specific configuration example of the image search system of FIG.
6, the terminal device 6 includes a captured image reception unit 61 with additional information, a metadata addition request condition setting unit 62, a metadata addition request image offer unit 63, a metadata addition image reception unit 64, and a metadata change unit 65. A change metadata-added image storage unit 66 and a display control unit 67 are provided.

  Here, the captured image reception unit 61 with supplementary information captures the captured image 141 with supplementary information in which the supplementary information 112 is added to the captured image 111 from the camera 7. The metadata assignment request condition setting unit 62 generates the metadata assignment request image 142 by setting the metadata assignment request condition 113 in the captured image 111 automatically or manually. Note that the metadata assignment request condition 113 can include a disclosure condition for the captured image 111. The metadata grant request image offer unit 63 requests the metadata grant server 2 to give the metadata 114 to the captured image 111. The metadata addition image receiving unit 64 receives a metadata addition request image 142 in which the metadata 114 is added to the captured image 111 from the metadata addition server 2. The metadata changing unit 65 changes the metadata 114 assigned to the captured image 111. The changed metadata-added image storage unit 66 stores the changed metadata-added image 143 in which the changed metadata 124 is added to the captured image 111. The display control unit 67 manages the address of the stored photographed image 111, displays the object of the photographed image 111 in a directory-type display or a network-type display, and displays a preview of the photographed image 111 related to the selected object. To do. Note that the display control unit 67 changes the size of the object according to the quantity of the captured images 111a to 111d possessed by each object in the network type display, or at the time when the captured images 111a to 111d possessed by each object are captured. In conjunction with this, it is possible to control whether or not there is a relationship between objects.

Note that the photographic image receiving unit 61 with additional information, the metadata addition request condition setting unit 62, the metadata addition request image offer unit 63, the metadata addition image reception unit 64, the metadata change unit 65, and the display control unit 67 This can be realized by causing a computer to execute a program in which these functions are described.
The metadata grant server device 2 includes a metadata grant request accepting unit 21, a metadata grant request image storage unit 22, a person image metadata grant unit 23, an event metadata grant unit 24, an event metadata grant unit 25, and a reference image. A storage unit 26, a metadata-added image storage unit 27, and a metadata-added image providing unit 28 are provided.

  Here, the metadata addition request receiving unit 21 receives a request for adding metadata to the image data from the terminal device 6 and calls the service terminal device 4 when it is difficult to automatically add metadata. The metadata addition request image accumulation unit 22 accumulates the metadata addition request image 142 received by the metadata addition request reception unit 21. The person image metadata adding unit 23 adds metadata 114 to the person image of the photographed image 111 by using an image recognition technique. The person image metadata assigning unit 23 can assign metadata to the person image by assigning people. The event metadata adding unit 24 adds metadata 114 to the event of the captured image 111 by using an image recognition technique. Note that the event metadata adding unit 24 can extract an event from the captured image 111 based on incidental information including date and time information, and add the metadata 114 to the event of the captured image 111. The event metadata giving unit 25 gives the metadata 114 to the event of the captured image 111 by using an image recognition technique. The event metadata adding unit 25 can add one or a plurality of metadata of the reference image 104 having a high similarity stored in the reference image storage unit 26 to the event of the captured image 111. The reference image storage unit 26 stores the reference image 104 to which metadata is added. The metadata-added image storage unit 27 temporarily stores the metadata-added image 110 obtained by adding the metadata 114 to the captured image 111. The metadata-added image providing unit 28 provides the terminal device 6 with the metadata-added image 110 obtained by adding the metadata 114 to the captured image 111. The metadata-added image providing unit 28 can cause the image providing server device 5 to release the captured image 111 in accordance with the disclosure conditions for the captured image 111 set by the metadata-adding request condition setting unit 62.

Note that the metadata grant request accepting unit 21, the person image metadata grant unit 23, the event metadata grant unit 24, the event metadata grant unit 25, and the metadata grant image provision unit 28 have a program in which these functions are described. This can be realized by causing a computer to execute it.
The image providing server device 5 is provided with a Web server 51 and an image storage unit 52. Here, the Web server 51 opens the public image 130 to the user by receiving access from the terminal device 8. The image storage unit 52 stores the public image 130 to which the metadata 114 is added. Here, the image storage unit 52 of the image providing server device 5 is connected to the reference image storage unit 26 of the metadata providing server device 2. The public image 130 stored in the image storage unit 52 can be stored in the reference image storage unit 26 together with metadata. As a result, the metadata providing server apparatus 2 can collect more reference images 104, increase the number of comparison images used for pattern matching, and automatically perform metadata under more conditions. Granting is possible.

  Further, when it is difficult to add the metadata 114 in the metadata adding server device 2, the service terminal device 4 manually assigns the metadata 114 to the captured image 111, and the captured image 111 to which the metadata 114 is added is added to the metadata. This is provided to the data grant request receiving unit 21. Here, the service terminal device 4 can refer to the reference image 104 stored in the reference image storage unit 26 when adding the metadata 114 to the captured image 111. When the service terminal device 4 adds the metadata 114 to the captured image 111, the captured image 111 with the metadata 114 added can be stored in the reference image storage unit 26. Thereby, the precision of the metadata provided to the reference image 104 can be improved, and the automatic assignment of the metadata 114 to the event can be enhanced.

FIG. 7 is a flowchart showing an example of the operation of the terminal device 6 of FIG.
In FIG. 7, the captured image reception unit 61 with supplementary information takes in the captured image 141 with supplementary information from the camera 7 (step A1). Next, the metadata addition request condition setting unit 62 generates the metadata addition request image 142 by automatically setting the metadata addition request condition 113 to the captured image with additional information 141 (step A2). Next, when the metadata grant request image setting unit 62 generates the metadata grant request image 142, the metadata grant request image offer unit 63 adds the metadata 114 to the captured image 111. 2 (Step A3). Next, when accessed from the metadata grant server 2 (step A4), the metadata grant image receiving unit 64 receives the metadata grant image 110 from the metadata grant server 2 (step A5). Next, when there is an access from the user (step A6), the metadata changing unit 65 changes the metadata 114 added to the captured image 111 (step A7). Next, when the metadata 114 is changed by the metadata changing unit 65, the changed metadata-added image storage unit 66 stores the changed metadata-added image 143 (step A8). Next, when there is an access from the user (step A9), the display control unit 67 displays the captured image 111 in a directory type display or a network type display (step A10).

FIG. 8 is a flowchart showing an example of the operation of the metadata assignment server apparatus 2 of FIG.
In FIG. 8, the metadata addition request receiving unit 21 receives a request for adding metadata to image data from the terminal device 6 and calls the service terminal device 4 when it is difficult to automatically add metadata. (Step B1). Next, the metadata grant request image storage unit 22 stores the metadata grant request image 142 received by the metadata grant request accepting unit 21 (step B2). The person image metadata giving unit 23 gives the metadata 114 to the person image of the captured image 111 by using an image recognition technique (step B3). Next, the event metadata giving unit 24 gives the metadata 114 to the event of the captured image 111 by using an image recognition technique (step B4). Next, the event metadata adding unit 25 adds metadata 114 to the event of the captured image 111 by using an image recognition technique (step B5). Next, the metadata-added image accumulating unit 27 acquires the metadata-added image 110 from the person image metadata providing unit 23, the event metadata providing unit 24, the event metadata providing unit 25, or from the service terminal device 4. (Step B6), the metadata-added image 110 is temporarily stored (step B7). Next, the metadata-added image providing unit 28 provides the terminal device 6 with the metadata-added image 110 stored in the metadata-added image storage unit 27 and also sends the captured image 111 to the image providing server device 5 in accordance with the disclosure conditions. Provide (step B8).

  Further, when accessed from the image storage unit 52 of the image providing server device 5 (step B9), the metadata providing server device 2 receives the image sent from the image storage unit 52 (step B9). Step B10). Then, when it is necessary to add metadata to the image sent from the image storage unit 52, the metadata is added by calling the service terminal device 4, and the reference image 104 to which the metadata is added is received ( Step B11). When the reference image storage unit 26 receives the reference image 104 to which metadata is added, the reference image storage unit 26 stores the reference image 104 to which metadata is added (step B12).

FIG. 9 is a sequence chart showing an example of the operation of the image search system of FIG.
In FIG. 9, the captured image reception unit 61 with additional information captures a captured image 141 with additional information from the camera 7 (step C2). Next, the metadata addition request condition setting unit 62 generates the metadata addition request image 142 by automatically setting the metadata addition request condition 113 to the captured image with additional information 141 (step C3). Next, when the metadata grant request image setting unit 62 generates the metadata grant request image 142, the metadata grant request image offer unit 63 adds the metadata 114 to the captured image 111. 2 (step C4).

  Next, the metadata addition request receiving unit 21 receives a request for adding metadata to the image data from the terminal device 6 and calls the service terminal device 4 when it is difficult to automatically add metadata ( Step C5). Next, the metadata assignment server 2 assigns the metadata 114 to the person image, the event, and the event of the photographed image 111 (step C6), and the photographed image 111 to which the metadata 114 is imparted by the service terminal device 4 is added. Receive (step C7). And the metadata provision image provision part 28 provides the metadata provision image 110 to the terminal device 6, and provides the picked-up image 111 to the image provision server apparatus 5 according to publication conditions (step C8).

  Next, the metadata-added image receiving unit 64 receives the metadata-added image 110 from the metadata-adding server 2, and the metadata changing unit 65, when accessed by the user (step C9), displays the captured image 111. The given metadata 114 is changed (step AC10). Next, when the metadata 114 is changed by the metadata changing unit 65, the changed metadata-added image storage unit 66 stores the changed metadata-added image 143 (step C11). Next, when there is an access from the user (step C12), the display control unit 67 displays the captured image 111 in a directory type display or a network type display (step C13).

  On the other hand, when the published image 130 to which the metadata 114 is assigned is sent from the metadata giving server 2, the image providing server device 5 accumulates the published image 130 in the image accumulation unit 52 (Step C <b> 14). When the terminal device 8 accesses the Web server 51 (step C15), the public image 130 stored in the image storage unit 52 is sent to the terminal device 8 (step C16).

  The present invention makes it possible to efficiently perform a leak-free search corresponding to individual image features and abstract search requests while reducing the burden of adding metadata, and is convenient for image reuse. And the demand for image communication systems and image equipment can be increased.

1 is a block diagram illustrating a schematic configuration of an image search system according to an embodiment of the present invention. It is a block diagram which shows the outline | summary of the metadata provision processing method which concerns on one Embodiment of this invention. It is a block diagram which shows an example of the metadata provision processing method which concerns on one Embodiment of this invention. 4A is a diagram showing an example of an event extraction method of the metadata assignment server apparatus 2 of FIG. 3, and FIG. 4B is an event metadata assignment method of the metadata assignment server apparatus 2 of FIG. It is a figure which shows an example. It is a figure which shows an example of the display screen of the terminal device 6 of FIG. It is a block diagram which shows the specific structural example of the image search system of FIG. It is a flowchart which shows an example of operation | movement of the terminal device 6 of FIG. 4 is a flowchart showing an example of the operation of the metadata assignment server apparatus 2 of FIG. 3. It is a sequence chart which shows an example of operation | movement of the image search system of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Communication network 2 Metadata provision server apparatus 4 Service terminal apparatus 5 Image provision server apparatus 6, 8 Terminal apparatus 7 Camera 21 Metadata provision request reception part 22 Metadata provision request image storage part 23 Human image metadata provision part 24 Event meta Data adding unit 25 Event metadata adding unit 26 Reference image storing unit 27 Metadata added image storing unit 28 Metadata added image providing unit 51 Web server 52 Image storing unit 61 Captured image receiving unit with additional information 62 Setting of metadata adding request condition Unit 63 Metadata grant request image offer unit 64 Metadata grant image reception unit 65 Metadata change unit 66 Change metadata grant image storage unit 67 Display control unit 101 Image files 114 and 124 Metadata 103 Raw image archiver 104 Reference image 105 Image Archaeological Bar 110 Metadata added image 111 Photographed image 112 Additional information 113 Metadata addition request condition 114a Person image 114b Event 114c Event 115 Metadata change information 116 Photographed image address 130 Public image 141 Photographed image with additional information 142 Metadata addition request image 143 Change metadata added image 151 Directory type display screen 152 Network type display screen

Claims (25)

Metadata grant requesting means for requesting metadata to be added to image data;
Metadata-attached image receiving means for receiving image data to which metadata is attached in response to the request;
Metadata changing means for changing metadata attached to the image data;
An image search apparatus comprising: image data search means for searching for the image data based on the changed metadata.   2. The image search apparatus according to claim 1, further comprising incidental information adding means for adding incidental information including date and time information when the image data is photographed to image data for which a request for adding metadata is made. A metadata addition request condition setting unit for setting a metadata addition request condition for image data for which a request for adding metadata is performed;
3. The image search apparatus according to claim 1, wherein the request for adding metadata includes a disclosure condition for the image data. Object display means for network-displaying objects related to each other using the changed metadata as an object,
The object display means includes
One of an object changing unit that changes the size of the object according to the quantity of image data to which the same metadata is assigned, and a relevance control unit that controls the relevance between objects in conjunction with the time of shooting. The image search apparatus according to claim 1, further comprising at least one. Metadata grant request accepting means for accepting a request for giving metadata to image data;
Metadata giving means for giving metadata to the received image data;
An image search support apparatus, comprising: metadata added image providing means for providing image data to which the metadata is added to a request source.   6. The image search support apparatus according to claim 5, wherein the metadata providing unit assigns the same metadata to the same person, the same event, or the same event. The metadata giving means is
Person image metadata giving means for giving metadata to a person image of the image data;
Event metadata giving means for giving metadata to the event of the image data;
7. The image search support apparatus according to claim 5, further comprising event metadata adding means for adding metadata to the event of the image data.   8. The image search support apparatus according to claim 7, wherein the person image metadata assigning unit assigns metadata to the person image by performing person distribution. The metadata grant request receiving means
The apparatus further comprises incidental information acquisition means for acquiring incidental information including date and time information when the image data was captured in the image data for which the request for adding the metadata is performed,
8. The image search support apparatus according to claim 7, wherein the event metadata adding unit adds metadata to the event of the image data based on the incidental information. Reference image storage means for storing a reference image to which metadata is attached,
8. The event metadata adding unit adds the metadata added to the reference image to the event of the image data based on a comparison result between the image data and the reference image. The image search support apparatus described. A metadata addition request condition acquisition unit that acquires a metadata addition request condition set for image data for which a request for adding metadata is performed,
11. The metadata-added image providing unit publishes image data for which a request for adding metadata has been made based on the request for adding metadata. The image search support apparatus described. In an image search support system in which a terminal storing image data and a metadata grant server are connected via a network,
An image search support system that causes the metadata grant server to share a part of processing accompanying the grant of metadata to the image data performed at the terminal. The terminal
Metadata grant request means for requesting the metadata grant server to give metadata to image data;
Metadata-attached image receiving means for receiving image data to which metadata is attached in the metadata-attaching server;
Metadata changing means for changing metadata attached to the image data;
Image data search means for searching the image data based on the changed metadata,
The metadata grant server
Metadata grant request accepting means for accepting a request for giving metadata to image data from the terminal;
Metadata giving means for giving metadata to the received image data;
An image search support system comprising: metadata added image providing means for providing the terminal with image data to which the metadata is added. The metadata giving means is
Person image metadata giving means for giving metadata to a person image of the image data;
Event metadata giving means for giving metadata to the event of the image data;
14. The image search support system according to claim 13, further comprising event metadata adding means for adding metadata to the event of the image data.   15. The image search support system according to claim 14, wherein the person image metadata assigning means assigns metadata to the person image by performing person assignment. The terminal further includes incidental information adding means for adding incidental information including date and time information when the image data was taken to image data for which a request for adding metadata is performed,
The metadata grant request accepting means further comprises attendant information obtaining means for obtaining the attendant information from the terminal,
15. The image search support system according to claim 14, wherein the event metadata adding unit adds metadata to the event of the image data based on the incidental information. The metadata grant server further comprises reference image storage means for storing a reference image to which metadata is attached,
The event metadata adding unit adds the metadata given to the reference image to the event of the image data based on a comparison result between the image data and the reference image. The image search support apparatus described. The terminal further includes metadata grant request condition setting means for setting a metadata grant request condition including a disclosure condition of the image data in the image data,
The metadata grant server further comprises metadata grant request condition acquisition means for acquiring the metadata grant request condition from the terminal,
18. The image search support system according to claim 12, wherein the metadata-added image providing unit publishes the image data based on the metadata addition request condition. The terminal further includes an object display means for displaying an object related to each other using the changed metadata as an object.
The object display means includes
One of an object changing unit that changes the size of the object according to the quantity of image data to which the same metadata is assigned, and a relevance control unit that controls the relevance between objects in conjunction with the time of shooting. The image search support system according to claim 12, further comprising at least one. Requesting the metadata grant server to give metadata to the image data;
Receiving at the metadata grant server the grant of metadata to the image data requested from the terminal;
Giving metadata to image data received by the metadata grant server;
Providing the terminal with image data provided with metadata by the metadata providing server;
Receiving, at the terminal, image data to which metadata is attached at the metadata assignment server;
Changing the metadata given to the image data at the terminal;
And a step of searching the image data by the terminal based on the changed metadata. The step of assigning metadata to the image data includes:
Assigning metadata to the person image by sorting the person images included in the image data;
21. The method of adding metadata added to the reference image to an event of the image data based on a comparison result between the reference image to which metadata is added and the image data. The image search method described. Adding incidental information including information on the date and time when the image data was photographed to the image data for which the request for the addition of metadata is performed at the terminal;
Acquiring additional information added by the terminal at the metadata grant server;
The image search method according to claim 20 or 21, wherein the step of assigning metadata to the image data includes the step of assigning metadata to an event of the image data based on the supplementary information. The terminal sets a metadata grant request condition including a disclosure condition of the image data at the terminal;
Acquiring the metadata grant request condition set in the terminal at the metadata grant server;
23. The image search method according to claim 20, further comprising a step of releasing the image data based on the metadata addition request condition. Network-displaying objects related to each other using the changed metadata as an object;
Changing the size of the object according to the quantity of image data to which the same metadata is attached;
The image search method according to any one of claims 20 to 23, further comprising a step of controlling the relationship between objects in conjunction with the time of shooting. Adding metadata to the image data over the network;
Changing metadata given on the network on the terminal;
An image search program comprising: causing a computer to execute a step of searching for the image data based on the changed metadata.