JP2016075992A - Imaging device, information processing device and information processing method thereof, and information processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract a meta data candidate suitable for subjects included in photographed images in short time.SOLUTION: In an imaging device 101, an image feature quantity storage unit 205 is configured to store an image feature quantity by subject, and an image feature quantity acquisition unit 207 is configured to acquire a frequent image feature quantity amount corresponding to a current position of the imaging device 101 from an image information accumulation server 104. A priority comparison object determination unit 209 is configured to compare the frequent image feature quantity with the image feature amount by subject, and determine the image feature quantity by subject serving as a priority comparison object. A subject detection unit 204 is configured to calculate an image feature quantity of the subject included in a photographed image, and a meta data processing unit 203 is configured to compare the image feature quantity of the subject with the image feature quantity of the priority comparison object, and extract a meta data candidate to be given to the image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to information processing for extracting metadata candidates suitable for a subject included in a captured image.

  It has become common to add metadata to captured images. Metadata is information related to data, and is used for efficiently managing and retrieving data. The metadata in the image includes shooting information generated internally by the imaging device at the time of shooting. For example, geographical information indicating a shooting date and a shooting position. Further, not only photographing information but also subject information is given as metadata based on input information by a photographer. The subject information is information such as an object name or a person name of the subject included in the captured image, for example.

  In order to provide subject information as metadata at the time of shooting, it is necessary to input information suitable for the subject for each shooting, which is a complicated operation for the photographer. For this reason, a method has been proposed in which metadata candidates relating to a subject are acquired by some method, and metadata can be given more easily.

  Patent Document 1 discloses a technique for generating a metadata candidate file in advance and storing it in a server. The imaging apparatus presents the metadata candidate file acquired based on the shooting position, the shooting time, and the like to the photographer at the time of shooting. For each subject included in the image, the photographer selects subject information that matches the subject from the list included in the metadata candidate file. The imaging device adds the selected subject information as metadata to the captured image.

  On the other hand, a method for recognizing a subject included in a photographed image and extracting metadata candidates more suitable for the subject has been proposed. For example, the technique of Patent Document 2 searches for a registered image including the same subject as the subject included in the photographed image, using a group of images to which metadata has been assigned and registered in advance as dictionary information. Then, the subject information given to the detected registered image is extracted as a metadata candidate. Whether or not the same subject is included is determined by comparing image feature amounts of the captured image and the registered image. Patent Document 2 discloses that this processing is collectively performed on a group of captured images, but extraction of metadata candidates by subject recognition is also effective when adding metadata for each shooting.

  However, the technique of Patent Document 1 extracts all metadata candidates associated in advance with the shooting position and shooting time. Therefore, metadata candidates unrelated to the subject included in the photographed image are also presented, and a troublesome operation for selecting subject information from a large number of metadata candidates occurs.

  If the metadata candidates are extracted based on subject recognition at the time of shooting as in the technique of Patent Document 2, the troublesomeness of selecting subject information from a large number of metadata candidates is reduced. However, in order to accurately recognize various subjects, it is necessary to prepare a large number of registered images as dictionary information. If the number of registered images increases, it takes time to compare image feature amounts. As a result, it is difficult to extract metadata candidates to be presented within a practical time if metadata is to be assigned for each shooting. The technique of Patent Document 2 aims to reduce the time required for subject recognition by using an image with a small number of pixels as a registered image, but to cope with an increase in processing time accompanying an increase in the number of registered images as dictionary information. Is not disclosed.

JP 2008-257471 A JP 2007-323382 A

  An object of the present invention is to extract metadata candidates suitable for a subject included in a captured image in a short time.

  The present invention has the following configuration as one means for achieving the above object.

  An imaging apparatus according to the present invention is an imaging apparatus that captures an image, and includes first storage means for storing an image feature amount for each subject, and information on frequent image feature amounts corresponding to the current position of the imaging device. An acquisition means for acquiring from the apparatus; a determination means for comparing the frequent image feature quantity and the image feature quantity for each subject to determine the image feature quantity for each subject to be a priority comparison target; and Calculating means for calculating the image feature amount of the included subject, and extraction means for comparing the image feature amount of the subject with the image feature amount of the priority comparison target and extracting a metadata candidate to be given to the image .

  An information processing apparatus according to the present invention includes a receiving unit that receives an image or geographic information from a client device, calculates an image feature amount of a subject included in the received image, and the image feature amount and imaging information of the image Are stored in the storage means in association with each other, classification means for classifying the image feature amounts stored in the storage means into a plurality of sets based on their similarity, and the position indicated by the received geographic information And evaluating the plurality of sets based on shooting positions indicated by shooting information associated with the image feature amounts stored in the storage means, and selecting image feature amounts from the plurality of sets in descending order of evaluation values. Selection means, and transmission means for transmitting the selected image feature quantity as a frequent image feature quantity to the client apparatus that is the transmission source of the geographic information.

  According to the present invention, metadata candidates suitable for a subject included in a captured image can be extracted in a short time.

1 is a block diagram illustrating an example of the configuration of an information processing system configured by an imaging apparatus and an image information storage server of an embodiment. The block diagram explaining the functional structural example of an imaging device and an image information storage server. FIG. 5 is a diagram for explaining an example of information managed in an imaging apparatus. The figure which shows the example of information which the image feature-value storage part of an image information storage server manages. 7 is a flowchart for describing metadata assignment processing in the imaging apparatus. The flowchart explaining the acquisition process of image feature-value. The flowchart explaining the priority comparison target determination processing. The flowchart explaining the extraction process of a metadata candidate. The figure which shows the mode of a detection of a to-be-photographed object. The flowchart explaining the classification process of an image feature-value. The flowchart explaining the selection process of frequent image feature-value.

  Hereinafter, an imaging device, an information processing device functioning as an image information storage server, an information processing method thereof, and an information processing nest system according to an embodiment of the present invention will be described in detail with reference to the drawings. In addition, an Example does not limit this invention concerning a claim, and all the combinations of the structure demonstrated in an Example are not necessarily essential for the solution means of this invention.

[System configuration]
A configuration example of an information processing system constituted by the imaging apparatus 101 and the image information storage server 104 of the embodiment will be described with reference to the block diagram of FIG. The imaging device 101 is connected to the network 103 via an access point (AP) 102 and communicates with the image information storage server 104. FIG. 1 shows a configuration example in which one imaging device 101 is used as a client device and one image information storage server 104 is used as a server device. However, the number is not limited to one.

Imaging Device The imaging device 101 is a digital camera or the like that records captured images on a medium (not shown). The control unit 105 of the imaging apparatus 101 is a main body that controls the operation of each unit constituting the imaging apparatus 101 and executes various processes in the imaging apparatus 101 described later. The storage unit 106 includes a RAM, a ROM, a flash memory, and the like, stores various processing programs executed by the control unit 105 and various data, and functions as a work area when the control unit 105 performs processing.

  The communication unit 107 functions as a wireless communication interface for connecting to the network 103 via the AP 102. The imaging apparatus 101 transmits / receives data to / from the image information storage server 104 via the communication unit 107. The imaging unit 108 includes an imaging device such as a charge imaging device (CCD) or a CMOS sensor, converts light passing through the lens into an electrical signal, and outputs imaging data obtained by analog-digital conversion of the electrical signal.

  The image processing unit 109 performs development processing including demosaicing processing, white balance processing, gamma correction processing, and the like on the imaging data output from the imaging unit 108, and outputs image data of a live view image and a captured image. The image processing unit 109 performs data compression processing such as JPEG on the image data of the captured image. The image data subjected to the data compression processing is stored in the storage unit 106, a medium, or the like by the control unit 105. Further, the image processing unit 109 performs data decompression processing on the image data of the captured image stored in the storage unit 106 or media, and outputs the decompressed image data to the display unit 110.

  The display unit 110 is a liquid crystal monitor (LCD) or the like, and displays a live view image output from the image processing unit 109 at the time of shooting, or displays a shot image stored in the storage unit 106, media, or the like. The display unit 110 is configured as a touch panel in which the operation unit 115 is stacked. A user can operate the operation unit 115 with reference to a user interface (UI) displayed on the display unit 110 to perform various settings and imaging operations of the imaging apparatus 101. The position positioning unit 111 receives a GPS (global positioning system) radio wave and measures the current position of the imaging apparatus 101.

Image Information Storage Server The image information storage server 104 provides a function for storing image data received from a client device. Examples of the client device include a personal computer (PC), a mobile phone, a smartphone, and a tablet that can access the image information storage server 104 via the network 103, in addition to the imaging device 101.

  The control unit 112 of the image information storage server 104 is a main body that controls the operation of each unit constituting the image information storage server 104 and executes various processes in the image information storage server 104 described later. The storage unit 113 includes a RAM, a ROM, a hard disk drive (HDD), a solid state drive (SSD), etc., stores various processing programs executed by the control unit 112 and various data, and the control unit 112 performs processing. Function as a work area. The storage unit 113 functions as an image information storage unit that stores information about image data received by the image information storage server 104.

  The communication unit 114 functions as a communication interface for connecting to the network 103. The image information storage server 104 transmits and receives data to and from the client device including the imaging device 101 via the communication unit 114.

[Function configuration]
A functional configuration example of the imaging device 101 and the image information storage server 104 will be described with reference to the block diagram of FIG. 2 is realized by the control unit 105 of the imaging device 101 and the control unit 112 of the image information storage server 104.

Imaging device The image capturing unit 201 captures an image in accordance with a photographer's operation. The image storage unit 202 stores image data of a captured image generated by the image capturing unit 201 and related information. The subject detection unit 204 detects a subject included in the captured image. The metadata processing unit 203 extracts metadata candidates to be given to the subject included in the photographed image, and records the metadata in the image data of the photographed image in accordance with the photographer's instruction.

  The image feature amount storage unit 205 stores the image feature amount for each subject and the subject information as dictionary information in advance. It is assumed that an image feature amount that well expresses a typical feature of a subject is selected in advance as the image feature amount for each subject used as dictionary information. The image feature amount may be automatically selected from a large number of image groups by machine learning or the like, or may be individually selected by the photographer. The subject information includes a general name, a unique name, a name, a common name, a common name, an attribute, a geographic name indicating the location of the subject, and the like.

  The position information acquisition unit 206 acquires geographic information indicating the current position of the imaging device 101 by GPS or the like. The image feature amount acquisition unit 207 acquires from the image information storage server 104 an image feature amount of a subject that is frequently imaged at the current position, and associates the acquired image feature amount with the geographic information when the image feature amount is acquired. And stored in the image feature amount storage unit 208.

  The priority comparison target determination unit 209 gives priority to a subject having a high degree of similarity with the image feature amount acquired by the image feature amount acquisition unit 207 from among the image feature amounts for each subject stored in the image feature amount storage unit 205. It is determined as a comparison target. The image feature amount for each subject determined as the priority comparison target is preferentially used for comparison with the image feature amount of the subject included in the captured image.

  An example of information managed in the imaging apparatus 101 will be described with reference to FIG. FIG. 3A shows an example of information managed by the image storage unit 202. Shooting information such as shooting position and shooting date / time is managed for each shot image identified by the shot image ID, and the coordinates and images in the shot image are included for each subject identified by the subject ID included in the shot image. Feature quantities, subject information, etc. are managed. Some of these pieces of information may be recorded as attribute information in image data of a captured image such as an Exif area.

  FIG. 3B shows an example of information managed by the image feature amount storage unit 205. For each image feature amount ID, an image feature amount and subject information for each subject are managed as dictionary information. Further, whether or not each image feature amount is determined as a priority comparison target by the priority comparison target determination unit 209 is managed.

  FIG. 3C shows an example of information managed by the image feature amount storage unit 208. The image feature amount storage unit 208 stores geographic information when the image feature amount is acquired, and image feature amounts (hereinafter, frequent image feature amounts) of a subject with high photographing frequency at the position indicated by the geographic information. . The frequent image feature amount is assigned an image feature amount set ID indicating the classification result of the image feature amount based on the level of similarity in the image information storage server 104.

Image Information Storage Server The image receiving unit 210 receives image data from various client devices, and stores the received image data in the image storage unit 211. The subject detection unit 212 detects a subject included in the image represented by the image data stored in the image storage unit 211, and stores the image feature amount of the detected subject in association with shooting information included in the image data. Stored in the unit 213. The image feature quantity classification unit 214 classifies the image feature quantities stored in the image feature quantity storage unit 213 into a plurality of sets based on the height of the similarity.

  It is not necessary to continuously store the image data received by the image receiving unit 210 in the image storage unit 211. That is, the image feature amount of the subject included in the image represented by the image data stored in the image storage unit 211 is stored in the image feature amount storage unit 213 in association with the shooting information, and then the image data is deleted from the image storage unit 211 It is preferable to do.

  The position information receiving unit 215 receives geographic information from the imaging device 101. The image feature quantity selection unit 216 converts an image feature quantity (frequent image feature quantity) frequently taken at the position indicated by the geographic information received by the position information reception unit 215 into information stored in the image feature quantity storage unit 213. Select based on. The image feature amount transmission unit 217 transmits the frequent image feature amount selected by the image feature amount selection unit 216 to the imaging apparatus 101 that is the transmission source of the geographic information.

  FIG. 4 shows an example of information managed by the image feature amount storage unit 213 of the image information storage server 104. For each subject identified by the subject ID included in the image represented by the image data received by the image receiving unit 210 identified by the received image ID, photographing information such as a photographing position and a photographing date and time, and an image feature amount It is managed.

  The image feature amount of each subject is classified into a plurality of image feature amount sets by the image feature amount classification unit 214 based on the similarity. The information managed by the image feature amount storage unit 213 is given an image feature amount ID for identifying a set to which the image feature amount of the subject belongs. Note that image feature amounts to which the same image feature amount set ID is assigned are regarded as image feature amounts of the same or similar subject.

[Processing of imaging device]
With reference to the flowchart of FIG. 5, the process of adding metadata in the imaging apparatus 101 will be described. Note that FIG. 5 shows a process corresponding to one shooting. However, when a plurality of shootings are performed, the processing of steps S703 to S708 described later is repeated, or after shooting, the number of times of shooting is set. The processing from S705 to S708 is repeated.

  The image photographing unit 201 receives a photographing start request from the photographer (S701). The state in which the imaging start request has been received is a state in which, for example, the power of the imaging apparatus 101 is turned on, and the image capturing unit 201 is in a state where it can be captured by, for example, extending a lens. When the photographing is enabled, the image feature amount acquisition unit 207 acquires the image feature amount (frequent image feature amount) of the subject with high photographing frequency at the current position from the image information accumulation server 104, as will be described in detail later (S702). .

  Next, when the photographer presses the shutter button, the image photographing unit 201 performs photographing (S703), and stores the generated photographed image in the image storage unit 202 (S704). In conjunction with the generation of the captured image, the subject detection unit 204 and the metadata processing unit 203 extract metadata candidates for each subject included in the captured image, as will be described in detail later (S705). Then, the metadata processing unit 203 displays information indicating the extracted metadata candidates and a captured image on the UI 218 of the display unit 110 (S706).

  The metadata processing unit 203 accepts selection of metadata candidates to be added to the photographed image by the photographer (S707), and records the selected metadata in the image data of the corresponding photographed image stored in the photographed image storage unit 202. (S708). It should be noted that the photographer can determine that the metadata is not added, and in this case, the metadata recording (S708) is skipped.

Image Feature Amount Acquisition Image feature amount acquisition processing (S702) will be described with reference to the flowchart of FIG. The image feature quantity acquisition unit 207 acquires geographic information (see FIG. 3C) associated with the frequent image feature quantity stored in the image feature quantity storage unit 208 (S801), and receives the current information from the position information acquisition unit 206. Geographic information indicating the position is acquired (S802).

  Next, the image feature quantity acquisition unit 207 calculates the difference between the position indicated by the geographic information associated with the frequent image feature quantity and the current position as the movement distance Dm (S803), and the movement distance Dm exceeds the predetermined value Dth1. It is determined whether or not (S804). When the moving distance exceeds the predetermined value (Dm> Dth1), the image feature quantity acquisition unit 207 transmits geographic information indicating the current position to the image information storage server 104 (S805).

  If the moving distance is equal to or smaller than the predetermined value (Dm ≦ Dth1), the image feature amount acquisition unit 207 ends the image feature amount acquisition process (S702). That is, by requesting an image feature amount from the image information storage server 104 based on the moving distance Dm, an excessive request for requesting frequent image feature amounts can be prevented even when slight movement occurs, and the imaging apparatus 101 and the image information The processing load on the storage server 104 can be reduced.

  When the location information receiving unit 215 of the image information storage server 104 receives geographic information from the imaging device 101 (S811), the image feature quantity selection unit 216, as will be described in detail later, is a subject that is frequently imaged at the location indicated by the geographic information. The image feature amount is selected (S812). Then, the image feature amount transmitting unit 217 transmits the frequent image feature amount to the imaging device 101 (S813).

  When receiving the frequent image feature amount from the image information storage server 104 (S806), the image feature amount acquisition unit 207 stores the received frequent image feature amount in the image feature amount storage unit 208 in association with the transmitted geographic information ( S807). Although the details will be described later, the priority comparison target determination unit 209 performs priority comparison with respect to the subject-specific image feature amounts stored in the image feature amount storage unit 205 based on the frequent image feature amounts stored in the image feature amount storage unit 208. The target is determined (S808).

Determination of priority comparison target The priority comparison target determination processing (S808) will be described with reference to the flowchart of FIG. The priority comparison target determination unit 209 selects an image feature amount to be processed (hereinafter referred to as a first image feature amount) from the subject image feature amounts stored as dictionary information in the image feature amount storage unit 205 (S901). . In addition, an image feature amount to be compared (hereinafter referred to as a second image feature amount) is selected from the frequent image feature amounts stored in the image feature amount storage unit 208 (S902).

  Next, the priority comparison target determination unit 209 calculates the similarity S by comparing the first and second image feature amounts (S903). The similarity S is calculated, for example, by calculating the Euclidean distance between the feature quantities converted into multidimensional vectors. Then, it is determined whether or not the similarity S is greater than or equal to a predetermined value Sth1 (S904). If the similarity is greater than or equal to a predetermined value (S ≧ Sth1), the first image feature amount is determined as a priority comparison target (S905). . If the similarity is less than a predetermined value (S <Sth1), it is determined whether or not all of the frequent image feature values have been selected for comparison (S906). If there is an unselected frequent image feature amount as a comparison target, the process returns to step S902 to update the second image feature amount, and the calculation and determination of similarity S are repeated.

  When it is determined that the first image feature amount is a priority comparison target, or when there are no more frequently selected image feature amounts as comparison targets, the priority comparison target determination unit 209 is stored in the image feature amount storage unit 205. It is determined whether or not all image feature amounts have been selected as processing targets (S907). If there is an unselected image feature amount as a processing target, the process returns to step S901 to update the first image feature amount, and the calculation and determination of the similarity S are repeated.

Metadata candidate extraction The metadata candidate extraction process (S705) will be described with reference to the flowchart of FIG. The subject detection unit 204 detects a subject included in the captured image (S1001). FIG. 9 shows how the subject is detected. For example, the subject detection unit 204 extracts a curve indicating the boundary of the object by edge detection, and uses a rectangular region including the curve as a subject. In the captured image 1101 shown in FIG. 9, two subjects 1102 and 1103 are detected.

  Next, the subject detection unit 204 selects one of the detected subjects (S1002), and calculates an image feature amount of the subject (hereinafter, a third image feature amount) (S1003). When the third image feature amount is calculated, the parameter processing unit 203 compares the third image feature amount with the image feature amount for each subject determined as the priority comparison target (hereinafter, the fourth image feature amount). Then, the similarity S is calculated (S1004). When there are a plurality of image feature amounts determined as priority comparison targets, the parameter processing unit 203 calculates the similarity S for all of the image feature amounts.

  Next, the parameter processing unit 203 determines whether or not there is a fourth image feature amount having a similarity S equal to or greater than a predetermined value Sth2 (S1005), and if there is, the fourth similarity S is the highest. Subject information associated with the image feature quantity is extracted as a metadata candidate (S1006).

  A case is assumed where the similarity to all the fourth image feature amounts is less than a predetermined value (S <Sth2). In such a case, the parameter processing unit 203 compares the third image feature quantity with the image feature quantity (hereinafter referred to as the fifth image feature quantity) for each non-priority comparison target subject that has not been determined as the priority comparison target. Then, the similarity S is calculated (S1007). Then, it is determined whether or not there is a fifth image feature amount having a similarity S equal to or greater than a predetermined value Sth2 (S1008), and if it exists, it is associated with the fifth image feature amount having the highest similarity S. Subject information is extracted as a metadata candidate (S1006).

  Furthermore, it is assumed that the similarity to all the fifth image feature amounts is less than a predetermined value (S <Sth2). In such a case, the parameter processing unit 203 extracts subject information associated with the image feature amount corresponding to the highest similarity S among the calculated similarities S as metadata candidates (S1009). In this case, the extraction result may indicate that there is no metadata candidate.

  Next, the parameter processing unit 203 determines whether metadata candidate extraction processing has been performed for all subjects detected from the captured image (S1010). If there is a subject whose metadata candidate extraction processing has not been completed, the process returns to step S1002, and the next subject is selected, and the metadata candidate extraction processing is repeated.

[Processing of image information storage server]
Image Feature Quantity Storage and Classification The image feature quantity classification process will be described with reference to the flowchart of FIG. In the image information storage server 104, when the image receiving unit 210 receives image data from the client device (S1201), the received image data is stored in the image storage unit 211 (S1202).

  The subject detection unit 212 detects a subject included in the image represented by the image data stored in the image storage unit 211 (S1203). Note that the subject detection process is performed, for example, by the same method as in step S1001. Then, the subject detection unit 212 calculates the image feature amount of the detected subject (S1204), and stores the calculated image feature amount in the image feature amount storage unit 213 in association with the shooting information included in the image data (S1205). ). Note that the calculation and storage of the image feature amount is repeated for the detected subject.

  Next, the image feature quantity classification unit 214 is classified based on the image feature quantity stored in the image feature quantity storage unit 213 (hereinafter referred to as the sixth image feature quantity) and the height of the similarity between the image feature quantities. The similarity S is calculated between the existing image feature quantity sets (S1206). The similarity S is calculated, for example, by calculating the centroid of the image feature amount belonging to the image feature amount set and the Euclidean distance of the sixth image feature amount. When there are a plurality of image feature quantity sets, the image feature quantity classification unit 214 calculates the similarity S for all of the image feature quantity sets.

  Next, the image feature quantity classification unit 214 determines whether or not there is an image feature quantity set having a similarity S equal to or greater than a predetermined value Sth3 (S1207), and if it exists, the image feature having the highest similarity S is found. A sixth image feature quantity is assigned to the quantity set (S1208). If the similarity to all existing image feature quantity sets is less than a predetermined value (S <Sth3), the image feature quantity classification unit 214 assigns the sixth image feature quantity to the new image feature quantity set (S1209). . Note that the classification of the image feature amount is repeated for the detected subject.

Selection of Frequent Image Feature Amount A frequent image feature amount selection process (S812) will be described with reference to the flowchart of FIG. In the image information storage server 104, when the location information receiving unit 215 receives geographic information from the imaging device 101 (S1301), the image feature amount selecting unit 216 includes an image feature amount stored in the image feature amount storage unit 213 (hereinafter referred to as “image feature amount”). The seventh image feature amount) is selected (S1302).

  Next, the image feature quantity selection unit 216 calculates a distance D between the shooting position associated with the seventh image feature quantity and the position indicated by the received geographic information (hereinafter, imaging device position) (S1303). Then, it is determined whether or not the calculated distance D is within a predetermined value Dth2 (S1304). When the distance is equal to or smaller than the predetermined value (D ≦ Dth2), the image feature amount selection unit 216 increments the evaluation value of the image feature amount set to which the seventh image feature amount belongs (S1305). The evaluation value of the image feature quantity set is a count value that represents the number of image feature quantities whose distance D is equal to or less than a predetermined value Dth2 in the image feature quantity set, and is 0 before starting the frequent image feature quantity selection process. Has been reset to

  Next, the image feature quantity selection unit 216 determines whether or not the distance D has been calculated and determined for all the image feature quantities stored in the image feature quantity storage unit 213, in other words, the image feature quantity set evaluation process has been completed. It is determined whether or not (S1306). If the evaluation process has not been completed, the process returns to step S1302, the next image feature amount is selected, and the calculation and determination of the distance D are repeated.

  In this way, the image feature quantity selection unit 216 performs the image feature quantity set evaluation process, and when the evaluation process ends, selects the frequent image feature quantity (S1307). For example, the frequent image feature quantity is selected by selecting a predetermined number of image feature quantity sets in descending order of evaluation values, and selecting a predetermined number of image feature quantities in order from the selected image feature quantity set in which the shooting position is closest to the imaging device position. It is done by selecting. When the selection of the frequent image feature amount is completed, the image feature amount notification unit 217 transmits the frequent image feature amount to the imaging apparatus 101 (S1308).

  When the evaluation processing of the image feature amount set is started after the geographic information is received from the imaging device 101, if the number of image feature amounts stored in the image feature amount storage unit 213 is large, a large amount of time is required for the evaluation processing. This is assumed. Therefore, the image feature amount selection unit 216 sequentially classifies the image feature amounts with respect to the positions, and creates an index of the image feature amounts associated with the positions in advance. Then, when geographic information is received from the imaging device 101, the evaluation process is efficiently performed by calculating and determining the distance D by selecting an image feature amount related to the imaging device position based on the index. can do.

[Modification]
In the above, an example has been described in which the image feature amount storage unit 205 of the imaging apparatus 101 stores the image feature amount for each subject, the subject information, and the like as dictionary information in advance. However, the dictionary information may be classified into client devices, photographers, geographic information, seasons, etc., and stored in the storage unit 113 of the image information storage server 104. The image feature quantity acquisition unit 207 of the imaging device 101 refers to the attribute of the dictionary information stored in the image feature quantity storage unit 205, and detects a change in the current position, a change in the photographer, a change in the season, etc. Information for selecting dictionary information to be acquired is transmitted to the image information storage server 104. Then, from the image information storage server 104, dictionary information that matches the client device, the photographer, geographic information, seasons, and the like can be acquired.

  In the above, when the similarity S between the image feature amount of the subject and the image feature amount for each subject determined as the priority comparison target is less than the predetermined value Sth2, the image feature amount for each subject not determined as the priority comparison target An example of calculating the similarity S has been described. However, in the case of S <Sth2, the processing of steps S1007 to S1009 may be omitted with the extraction result indicating that there is no metadata candidate.

  In the above, a predetermined number of image feature value sets are selected in descending order of evaluation values, image feature values are selected from the selected image feature value sets in order of closest shooting position to the imaging device position, and the imaging frequency at the imaging device position is selected. The example in which the image feature amount corresponding to the subject having a high is selected has been described. Further, the image feature amount may be selected in consideration of the date and time when the image information storage server 104 receives the geographic information. For example, after selecting an image feature amount set based on the evaluation value, selection may be performed in the order of the shooting position and the imaging apparatus position, and the selection may be performed in the order of the reception date and the season of the shooting information. Of course, the selection by distance and the selection by season may be reversed. In addition, for example, spring / summer / autumn / winter or monthly may be used as the classification of seasons.

  In the above description, the example in which the position information acquisition unit 206 acquires geographic information using GPS has been described. However, the position information acquisition unit 206 can also acquire geographic information from the photographer via the UI 218. For example, a map may be displayed on the UI 218 so that the photographer points to the current location, or a zip code, a nearby station name, or a place name may be input to the UI 218.

  In the above description, the example in which the communication unit 107 connects to the network 103 via the AP 102 has been described. However, in addition to the wireless network, it is also possible to connect to the network 103 via a mobile communication network such as a mobile phone line.

[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program It can also be realized by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  DESCRIPTION OF SYMBOLS 101 ... Imaging device, 205 ... Image feature-value storage part, 207 ... Image feature-value acquisition part, 104 ... Image information storage server 104, 209 ... Priority comparison object determination part, 204 ... Subject detection part, 203 ... Metadata processing part

Claims (17)

An imaging device for taking an image,
First storage means for storing image feature values for each subject;
Acquisition means for acquiring a frequent image feature amount corresponding to the current position of the imaging device from the information processing device;
A determination unit that compares the frequent image feature quantity and the image feature quantity for each subject to determine the image feature quantity for each subject that is a priority comparison target;
A calculation means for calculating an image feature amount of a subject included in the photographed image;
An image pickup apparatus comprising: an extracting unit that compares the image feature amount of the subject with the image feature amount of the priority comparison target and extracts metadata candidates to be added to the image. Furthermore, it has means for obtaining geographical information indicating the current position of the imaging device,
2. The acquisition unit according to claim 1, wherein the acquisition unit transmits the geographic information to the information processing apparatus and stores the frequent image feature amount received from the information processing apparatus in a second storage unit in association with the geographic information. Imaging device.   When the difference between the position indicated by the geographic information associated with the frequent image feature quantity stored in the second storage means and the current position exceeds a first predetermined value, the acquiring means exceeds the first predetermined value. 3. The imaging device according to claim 2, wherein the acquisition is performed.   The determining means calculates a similarity between the frequent image feature quantity and the image feature quantity for each subject, and judges the image feature quantity for each subject whose similarity is a second predetermined value or more as the priority comparison target. The imaging device according to any one of claims 1 to 3.   The extraction unit calculates a similarity between the image feature quantity of the subject and the image feature quantity of the priority comparison target, and the similarity is associated with the image feature quantity of the priority comparison target having a third predetermined value or more. 5. The imaging apparatus according to claim 1, wherein subject information is extracted as the metadata candidate.   When there is no image feature quantity of the priority comparison target having the similarity equal to or greater than the third predetermined value, the extraction unit calculates the image feature quantity of the subject and the image feature quantity of the subject of the non-priority comparison target. 6. The imaging apparatus according to claim 5, wherein similarity is calculated, and subject information associated with the non-priority comparison target image feature quantity having the similarity equal to or greater than the third predetermined value is extracted as the metadata candidate. .   When there is no image feature quantity for the priority comparison target and image feature quantity for the non-priority comparison target with the similarity equal to or greater than the third predetermined value, the extraction unit is the highest in the calculated similarity 7. The imaging apparatus according to claim 6, wherein subject information associated with the image feature amount for each subject corresponding to the similarity is extracted as the metadata candidate.   When there is no image feature quantity for the priority comparison target and non-priority comparison target image feature quantity with the similarity equal to or greater than the third predetermined value, the extraction means sets the no metadata candidate as the extraction result. The imaging device according to claim 6.   9. The imaging according to claim 1, wherein, for each subject included in the image, the calculation unit calculates the feature amount, and the extraction unit extracts the metadata candidate. apparatus. Receiving means for receiving images or geographic information from the client device;
Calculating means for calculating an image feature amount of a subject included in the received image, and storing the image feature amount and shooting information of the image in association with each other;
Classification means for classifying the image feature amounts stored in the storage means into a plurality of sets based on their similarity;
The plurality of sets are evaluated based on the position indicated by the received geographic information and the shooting position indicated by the shooting information associated with the image feature amount stored in the storage unit, and the plurality of sets are evaluated in descending order. Selection means for selecting an image feature amount from a set of
An information processing apparatus comprising: a transmission unit configured to transmit the selected image feature quantity as a frequent image feature quantity to a client apparatus that is a transmission source of the geographic information. The selection means includes
When the difference between the position indicated by the geographic information and the shooting position indicated by the shooting information associated with the image feature quantity stored in the storage means is less than or equal to a predetermined value, the evaluation value of the set to which the image feature quantity belongs is incremented. And
Selecting a predetermined number of sets from the plurality of sets in descending order of the evaluation value;
11. The predetermined number of image feature amounts are selected from the selected set in the order in which shooting positions indicated by shooting information associated with image feature amounts belonging to the selected set are closer to positions indicated by the geographic information. Information processing apparatus.   12. The information processing apparatus according to claim 10, wherein the calculation unit calculates and stores the image feature amount for each subject included in the received image. The imaging device according to any one of claims 1 to 9,
An information processing system comprising: the information processing device according to any one of claims 10 to 12. An information processing method for an imaging apparatus that captures an image,
The storage means stores the image feature amount for each subject,
The acquisition unit acquires a frequent image feature amount corresponding to the current position of the imaging device from the information processing device,
A determination unit compares the frequent image feature quantity with the image feature quantity for each subject to determine the image feature quantity for each subject to be a priority comparison target,
The calculating means calculates the image feature amount of the subject included in the captured image,
An information processing method in which an extraction unit compares an image feature amount of the subject and an image feature amount of the priority comparison target, and extracts metadata candidates to be added to the image. The receiving means receives an image or geographic information from the client device,
A calculating unit that calculates an image feature amount of a subject included in the received image, and stores the image feature amount and shooting information of the image in association with each other in a storage unit;
The classifying means classifies the image feature amounts stored in the storage means into a plurality of sets based on their similarity,
The selection unit evaluates the plurality of sets based on the position indicated by the received geographic information and the shooting position indicated by the shooting information associated with the image feature amount stored in the storage unit, and the evaluation value is high. In order, an image feature amount is selected from the plurality of sets,
An information processing method in which a transmission unit transmits the selected image feature quantity as a frequent image feature quantity to a client device that is a transmission source of the geographic information.   10. A program for causing a computer to function as each unit of the imaging device according to claim 1.   13. A program for causing a computer to function as each unit of the information processing apparatus according to claim 10.

Images