JP2013114653A - Information processor, method of controlling the same, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify a predetermined object from an image which is robust in the direction and position of a subject.SOLUTION: A system comprises a first image display device and a second image display device. The first image display device acquires an identifier, identifies an object in an image from first feature information associated with the identifier, and on the basis of the identified object, generates second feature information. When first object identifying means fails to identify the object, the second image display device identifies an object on the basis of the second feature information generated by the first image display device, and displays the object with superposing thereon attribute information associated with the identifier near the object.

Description

  The present invention relates to an information processing apparatus that identifies an object in a captured image.

In recent years, there is an AR (Augmented Reality) technology that combines and displays attribute information of an object in a captured image captured by a camera or the like. For example, there is a technique of combining and displaying attribute information of an object in a captured image based on position information obtained by GPS (Global Positioning System) (Non-patent Document 1).
In such an AR system, it is desired to display information related to the subject in the vicinity of the subject. In other words, when various subjects are included in the photographed image, it is desirable to display the information so that the user can know which subject information is associated with each subject.
In Patent Document 1, a face recognition process of a captured image is performed using face information acquired from a portable information device or the like by an imaging device, and information relating to the specified person is combined with the captured image and displayed. In Patent Document 2, information on a subject that is considered to be an imaging range is acquired based on not only a feature amount of an image obtained from an imaging process but also an imaging position / direction and displayed in a captured image.

http: // sekaicamera. com

JP 2002-305717 A JP2010-118019

However, the conventional technique has a problem that information related to the subject cannot always be displayed in the vicinity of the subject in the captured image. In Patent Document 1, since an object in a captured image is to be specified from feature information such as face image information registered in advance, the captured image is different if the registered image differs from the orientation or position of the object. It is difficult to identify the subject from the inside. In other words, the feature information of the reference face image and the face image at the time of imaging greatly fluctuates due to changes over time and the imaging environment, and it is guaranteed that advanced recognition is not necessarily possible only by using the registered image for recognition processing. I don't mean. In particular, the feature information varies greatly even when the face orientation of the reference image and the face image at the time of imaging is different. In Non-Patent Document 1 and Patent Document 2, information on a subject that appears to be in the imaging range is displayed in the captured image based on the imaging position / direction, so information on the subject in the captured image is not necessarily displayed. No. In addition, even if information about the subject in the captured image is displayed, it does not necessarily specify an accurate region in the captured image of the subject, so the user can determine which subject information the information related to the subject is. It is not always displayed in an understandable form.
The present invention has been made in view of the above-described problems, and an object of the present invention is to improve a recognition rate of a predetermined object from a captured image and display the identified object and predetermined information in association with each other.

  As means for solving the above-mentioned problems, first specifying means for specifying the predetermined object from the captured image using first feature information for specifying the predetermined object in the captured image, Second feature information acquisition for acquiring second feature information for specifying the predetermined object generated from an image in which the other device has successfully specified the predetermined object using the first feature information And a second specifying means for specifying the predetermined object from the captured image using the second feature information when the first specifying means cannot specify the predetermined object; A table that causes the display unit to display the predetermined object specified by the first specifying unit or the second specifying unit in association with information about the predetermined object. To provide an information processing apparatus characterized by comprising a control means.

  According to the present invention, since a predetermined object is specified from a captured image using a plurality of feature information, the recognition rate of the predetermined object can be increased. Therefore, it is possible to improve the accuracy when displaying the predetermined object and the predetermined information in association with each other.

It is a figure which shows the structure of the system in Example 1-2. 1 is a conceptual diagram of a database of a server in Embodiment 1. FIG. 1 is a diagram illustrating a configuration of a digital camera according to Embodiments 1 and 2. FIG. It is a figure which shows the structure of the mobile telephone in Example 1-2. FIG. 6 is a sequence diagram illustrating a shooting process of the digital camera 101 according to the first exemplary embodiment. 6 is a diagram illustrating an example of image data immediately after photographing by the digital camera 101 in Embodiments 1 and 2. FIG. FIG. 6 is a diagram illustrating an example of image data after image synthesis by the digital camera 101 according to the first and second embodiments. FIG. 6 is a sequence diagram illustrating a shooting process of the digital camera in the first embodiment. FIG. 6 is a diagram illustrating an example of image data immediately after shooting by the digital camera in the first and second embodiments. FIG. 6 is a diagram illustrating an example of image data after image synthesis by the digital camera in the first and second embodiments. FIG. 10 is a sequence diagram illustrating a shooting process of the digital camera 101 according to the second embodiment. It is a database conceptual diagram of the server in Example 2. FIG. 10 is a sequence diagram illustrating a shooting process of the digital camera in the second embodiment. 6 is a flowchart illustrating operations of the digital cameras 101 and 102 according to the first and second embodiments.

[Example 1]
Hereinafter, embodiments according to the present invention will be described with reference to the accompanying drawings. FIG. 1 shows a system configuration in this embodiment. In this embodiment, a case where the digital camera 101 and the digital camera 102 capture a person 105 and a person 106 will be described. The person 105 is the owner of the mobile phone 103.

  When the server 104 receives an inquiry including the identification information of the terminal device from the digital cameras 101 and 102, the server 104 transmits attribute information associated with the received identification information held in the database to the digital cameras 101 and 102. Further, the server 104 detects an object associated with the received identification information from the captured image, acquires feature information for specifying from the database, and transmits the feature information to the digital cameras 101 and 102.

  An example of information held in a database managed by the server 104 is shown in FIG. The database includes identification information 201 as an identifier for uniquely identifying the terminal device, an owner name 202 as attribute information corresponding to the terminal device, and feature information 203 used for specifying a person corresponding to the terminal device. Each associated table data is held. The identification information 201 is, for example, a MAC address or a user ID of the owner of the terminal device as an identifier that can uniquely identify the terminal device. The name 202 is an example of attribute information associated with the identification information, and is information that is displayed superimposed on an image captured by the digital camera 101. Note that the attribute information is not limited to a name, and may be a link to an external site such as a URL, or may have a plurality of items such as hobbies and dates of birth. The feature information 203 is information for detecting and specifying an object associated with the identification information from the captured image. In this embodiment, a reference image for performing object specifying processing by image processing is used as feature information. Note that the feature information is not limited to image data, and may be arbitrary feature information used for object specifying processing.

  The digital camera 101 and the digital camera 102 acquire attribute information associated with the identification information of the terminal device received from another terminal device from the server 104, and synthesizes the acquired attribute information and the captured image on the display unit. An information processing apparatus to be displayed. In addition, the digital cameras 101 and 102 associate the acquired attribute information with the object on the captured image and display the combined information on the display unit. Therefore, since information related to the owner of the terminal device is displayed on the display screen of the digital cameras 101 and 102, for example, the user can easily acquire information related to the object in the captured image. Further, since information is displayed in association with the object in the captured image, it is possible to easily identify which object emits what information. The digital cameras 101 and 102 have a wireless communication function in accordance with the IEEE 802.11 series.

  The mobile phone 103 is a terminal device that transmits, as identification information, an identifier that can uniquely identify the device itself periodically or in response to a request from another device. In this embodiment, the mobile phone 103 has a wireless communication function conforming to the IEEE 802.11 series. The identification information transmitted by the mobile phone 103 is an identifier for uniquely determining a terminal device, and is information set uniquely for each device. The identification information may be information for uniquely identifying the user of the terminal device. The identification information is given as an element of an information element of a beacon frame conforming to the IEEE 802.11 series, and is transmitted. Assume that the owner of the mobile phone 103 is a person 105.

  Next, the configuration of the digital cameras 101 and 102 as information processing apparatuses will be described with reference to FIG. The digital cameras 101 and 102 have a CPU (Central Processing Unit). Processing to be described later is realized by the CPU executing a control program to calculate and process information and control each hardware. The wireless communication control unit 301 controls an antenna and a circuit in order to communicate with an external device by transmitting / receiving a wireless signal. Note that the wireless communication control unit 301 controls communication processing conforming to the IEEE 802.11 series. A shutter button 302 is a button for starting shooting. The shutter button 302 detects that the user has pressed the shutter button 302, and the imaging unit 303 controls the start of shooting processing. The imaging unit 303 includes a lens, an imaging element, and A / D conversion. The imaging unit 303 images subject light acquired through the lens, and generates the captured subject light as image data. The display unit 304 performs display control for displaying a captured image and various types of information on a display. The identification information acquisition unit 305 acquires the identification information of the terminal device existing in the communicable range received by the wireless communication control unit 301. The first feature information acquisition unit 306 makes an inquiry to the server 104 based on the identification information acquired by the identification information acquisition unit 305, and is associated with the identification information and registered in advance (first feature information). Information) is acquired from the server 104. The attribute information acquisition unit 307 makes an inquiry to the server 104 based on the identification information acquired by the identification information acquisition unit 305 and acquires attribute information associated with the identification information from the server 104.

  The first specifying unit 308 is hardware or a program for specifying a predetermined object from the image data picked up by the image pickup unit 303 based on the acquired feature information. In the present embodiment, the first specifying unit 308 performs a known object recognition process to detect an object corresponding to the acquired feature information. The synthesizing unit 309 synthesizes the corresponding attribute information and the object in the vicinity of the object specified by the first specifying unit 308 or the second specifying unit 312 on the image data or in an arbitrary place, and combines them. Create data. The feature information generation unit 310 generates new feature information related to the object specified by the first specifying unit 308 or the second specifying unit 312 described later based on the captured image used for specifying. The generated feature information is stored in the storage unit 313.

  The second feature information acquisition unit 311 controls the wireless control unit 301 to make an inquiry to another device based on the identification information acquired by the identification information acquisition unit 305, and is associated with the identification information. The feature information generated by is acquired. The second specifying unit 312 is a program or hardware for specifying a subject from image data obtained from the imaging unit 303 based on the feature information acquired by the second feature information acquiring unit 311. The storage unit 313 is a memory for storing the feature information generated by the feature information generation unit 310 and various types of information.

  Next, the configuration of the mobile phone 103 will be described with reference to FIG. The mobile phone 103 has a wireless communication function conforming to the IEEE 802.11 series. The mobile phone 103 holds identification information that can uniquely identify its own device. Further, the mobile phone 103 has a CPU, and each process described later is realized by the CPU executing a control program to calculate and process information and control each hardware. An RF unit 401 illustrated in FIG. 4 controls an antenna and a circuit for transmitting / receiving a wireless signal to / from another wireless device via a wireless LAN. The identification information transmission unit 402 controls the RF 401 to notify the held identification information periodically or in response to a request from another device. The identification information transmission unit 402 assigns identification information (identifier) as an element of an Information Element of a beacon frame conforming to the IEEE 802.11 series and transmits the information. The mobile phone control unit 403 is connected to a mobile phone communication network by controlling an antenna and a circuit for operating as a mobile phone, and communicates with other devices.

  The operation of the system having the configuration described above will be described. Here, processing when the digital camera 101 and the digital camera 102 capture images will be described. An example in which the digital cameras 101 and 102 photograph the person 105 and the person 106 as shown in FIG. 1 will be described. The imaging unit 303 of the digital camera 101 faces the front of the person 105 and the person 106, and can detect and specify a face. On the other hand, the imaging unit 303 of the digital camera 102 faces the back of the person 105 and the person 106, and cannot capture a face from the front.

  First, the case where the digital camera 101 captures an image will be described with reference to the sequence diagram of FIG. First, the digital camera 101 starts photographing processing upon detecting the operation of the shutter button 302 (S501). A photographed image is shown in FIG. An image 601 is a captured image. In the image 601, a person 105 and a person 106 are captured. The subject image 602 is a subject image corresponding to the person 105 (region in the image 601), and the subject image 603 is a subject image corresponding to the person 106 (region in the image 601). Next, the identification information acquisition unit 305 of the digital camera 101 requests identification information from a terminal device existing within a communicable range. Specifically, the identification information acquisition unit 305 controls the wireless communication control unit 301 to broadcast and transmit a probe request frame conforming to the IEEE 802.11 series (S502). The mobile phone 103 that has received the probe request frame responds with a probe response frame to which its own device identification information is added (S503). When the digital camera 101 receives the probe response frame, the identification information acquisition unit 305 can obtain the identification information of the mobile phone 103. However, at this point, the digital camera 101 cannot recognize which object in the captured image corresponds to the obtained identification information, that is, which person owns the device.

  Subsequently, when the digital camera 101 obtains the identification information acquired by the identification information acquisition unit 305, it uses the first feature information acquisition unit 306 and the attribute information acquisition unit 307 to use the feature information and attributes corresponding to the acquired identification information. Queries the server 104 to obtain information. (S504). That is, based on the identification information acquired from the mobile phone 103, the digital camera 101 requests the server 104 for feature information and attribute information of an object corresponding to the identification information. Upon receiving the inquiry from the digital camera 101, the server 104 searches the database for attribute information and feature information associated with the identification information based on the identification information included in the inquiry (S505). When the server 104 can acquire attribute information and facial feature information associated with the received identification information, the server 104 transmits these pieces of information to the digital camera 101 (S506). When the digital camera 101 receives the attribute information and the feature information (first feature information) from the server 104, the first specifying unit 308 specifies the corresponding object from the image 601 based on the acquired feature information. Try (S507). The first specifying unit 308 compares the obtained feature information with the feature of the face image acquired from the image 601, and specifies that the image 601 (person 105) is a corresponding object. When the digital camera 101 succeeds in identifying an object from the captured image using the first feature information, the feature information generating unit 310 generates new feature information about the person 105 from the image 601 (S508). In the present embodiment, the feature information generation unit 310 generates feature information of an area corresponding to the identified person's clothes (clothes). It is assumed that the feature information regarding clothes is information regarding colors. Note that, as the feature information to be generated, any information used for specifying the clothes may be used as the feature information, not the information regarding the color. The generated clothes feature information is stored in the storage unit when a request from another terminal device is generated (S509).

  The synthesizing unit 309 of the digital camera 101 synthesizes the attribute information obtained in S506 in the vicinity of the identified subject image 602 by the synthesizing unit 309, and the display unit 304 performs display control to display the synthesized image. (S510). An image of the image displayed on the display unit 304 at this time is shown in FIG. The image 701 is an entire image generated by combining the captured image 601 and attribute information and displayed on the display unit 304. The subject image 702 and the subject image 703 correspond to the subject image 602 and the subject image 603 after image composition, respectively. The attribute information image 704 displays the attribute information obtained in S506 as an image, and is displayed in the vicinity (within a predetermined distance) in association with the subject image 702 that is the identified object. Thus, the user of the digital camera 101 can visually see information about the person 105.

  Next, the case where the digital camera 102 captures images will be described with reference to the sequence diagram of FIG. First, the digital camera 102 starts photographing processing upon detection of an operation of the shutter button 302 (S501). The photographed image is shown in FIG. An image 901 is a captured image. In the image 901, a person 105 and a person 106 are captured. The subject image 902 is a subject image corresponding to the person 105, and the subject image 903 is a subject image corresponding to the person 106. Both the person 105 and the person 106 are images from the back.

  Since the processing from S802 to S806 is the same as that from S502 to S506, description thereof will be omitted. S807, like S507, attempts to identify the corresponding object from the image 901 by the first identifying unit based on the first feature information acquired from the server 104. Since the person 105 from the digital camera 102 is on the back side, the characteristic information of the acquired face image of the person 105 does not exist on the image 901, so the specifying process fails. If the digital camera 102 fails to specify an object based on the first feature information, the digital camera 102 tries to specify a person using another feature information. Therefore, the digital camera 102 attempts to acquire feature information (second feature information) generated from the captured image used in the specifying process from surrounding devices that have succeeded in the specifying process by using the second feature information acquiring unit 311. Specifically, the second feature information acquisition unit 311 broadcasts a request for the second feature information to surrounding terminal devices existing in a communicable range via the wireless control unit 301 (S808). At this time, the second feature information acquisition unit adds identification information corresponding to the object for which the specific process has failed, so that it is possible to determine which object is requested for feature information.

  Upon receiving the feature information request from the digital camera 102, the digital camera 101 determines whether the object corresponding to the identification information included in the request has been successfully identified. The digital camera 101 has succeeded in identifying the object corresponding to the identification information of the mobile phone 103 through S507 described above. Therefore, the feature information generated from the subject image 602 (subject image 702) generated in S508 is acquired from the storage unit 313, and the feature information is transmitted to the digital camera 102 (S809). Thereby, even when the digital camera 102 fails in the specific process using the first feature information corresponding to the acquired identification information, the digital camera 102 uses other feature information (second feature information) generated by another device. Can be obtained.

  Upon acquiring the second feature information from the digital camera 101, the digital camera 102 tries to specify the corresponding object from the image 901 by the second specifying unit 312 based on the second feature information (S810). The second specifying unit 312 compares the acquired second feature information with the image feature amount generated from the image 901. The second specifying unit 312 specifies the subject image 902 as a corresponding object when the correlation between the acquired second feature information and the image feature amount of the subject image 902 is high and exceeds a predetermined threshold. Here, the second feature information is feature information related to the color of the clothes of the person 105. Therefore, since the feature information related to the orientation is small as in the face, the specific processing can be performed robustly with respect to the orientation. it can. If the specifying process is successful, the digital camera 102 combines the attribute information obtained in S806 in the vicinity of the specified subject image 902 with the combining unit 309, and displays the generated image on the display unit 304 (S811).

  FIG. 10 shows an image displayed on the display unit 304 at this time. An image 1001 is the entire image displayed on the display unit 304. The subject image 1002 and the subject image 1003 correspond to the subject image 902 and the subject image 903 after image composition, respectively. The attribute information image 904 displays the attribute information obtained in S806 as an image, and displays the attribute information and the subject image 1003 that is the identified object in association with each other.

  As described above, in this embodiment, the digital camera 102 attempts to detect and specify an object from the captured image using the first feature information for specifying the object corresponding to the acquired identifier of the mobile phone 103. Also, new feature information (second feature information) generated from a captured image that has succeeded in identifying an object using the first feature information for a device that succeeded in identifying the object using the first feature information. ) To get. When the identification fails using the first feature information, the digital camera 102 can try to identify the object again using the second feature information. As described above, even when the object cannot be specified from the image from the first registered feature information, the object can be specified by the feature information newly generated by another device.

  In addition, since the second feature information is requested by broadcast to surrounding devices capable of communication, the device that exists near the subject and is likely to have succeeded in identifying the object corresponding to the most recent object is assigned to the device. A request is received. For this reason, the recognition process can be improved because the specific process is performed again using feature information with little temporal change. In addition, the recognition rate can be improved by using feature information that is robust in direction as the second feature information.

  Therefore, since the recognition rate is improved, it is possible to display the object and attribute information related to the object in association with each other, so that it is possible to display in such a way that the user can easily know which information is related to which object.

[Example 2]
The difference between the present embodiment and the first embodiment is the second feature information storage method and the second feature information acquisition method. Specifically, in the first embodiment, the digital camera 102 acquires the second feature information from the digital camera 101 that generated the second feature information. That is, the example in which the device that generated the second feature information is the same as the acquisition destination device is shown. In the present embodiment, an example in which the device that generated the second feature information is not the same as the acquisition destination device is shown. The configuration of the system, the digital cameras 101 and 102, and the mobile phone 103 are shown in FIGS. 1, 3, and 4 as in the first embodiment. Description of points that are the same as those in the first embodiment is omitted here.

  FIG. 11 is a sequence diagram of processing when the digital camera 101 photographs the persons 105 and 106 from the front. Steps S1101 to S1108 are the same as steps S501 to S508 in the first embodiment, and a description thereof will be omitted. When the digital camera 101 newly generates the feature information from the captured image that has been successfully identified, the digital camera 101 associates the generated feature information with the identification information of the mobile phone 103 and transmits it to the server 104 (S1109). The server 104 registers the transmitted feature information in the database based on the received identification information (S1110).

  A conceptual diagram of the database of the server 104 is shown in FIG. Identification information, attribute information (name), first feature information, and second feature information are associated with each person. Further, the second feature information is not necessarily registered.

  After transmitting the generated feature information, the digital camera 101 displays the attribute information near the subject (S1111). The obtained image data is shown in FIG. Since S1111 is the same processing as S510 in the first embodiment, detailed description thereof is omitted.

  FIG. 13 is a flowchart of processing when the digital camera 102 photographs the persons 105 and 106 in the back. Since the processing from S1301 to S1307 is the same as that from S801 to S807, description thereof is omitted. If the digital camera 102 fails in the specific process based on the first feature information in S1307, the digital camera 102 requests the second feature information from the server 104 (S1308). At this time, the identification information of the mobile phone 103 is inquired to the query.

  When the server 104 receives the request for the second feature information corresponding to the identification information of the mobile phone 103, the server 104 searches the database for the second feature information associated with the identification information (S1309). Here, the second feature information transmitted by the digital camera 101 in S1109 can be obtained as a search result. The server 104 returns the searched second feature information to the digital camera 102 as a response (S1310). Thereby, the digital camera 102 can obtain the second feature information. Since the processing of S1311, S1312 is the same as that of S810, S811, description thereof will be omitted. By performing these photographing processes, image data as shown in FIG. 10 can be obtained, the person 105 can be identified even from the back of the person 105, and the attribute information can be visually seen.

  In the second embodiment, the digital camera that can identify the person transmits the feature information generated to the server, and the server registers it. For this reason, even if there is no digital camera that can identify a person in the vicinity at the time of shooting, it can be identified from the image data using the generated feature information by inquiring of the server.

  The operations of the digital cameras 101 and 102 of the first and second embodiments described above will be summarized with reference to the flowchart shown in FIG. First, shooting processing is started by operating the shutter button 302 of the digital cameras 101 and 102 (S1401). Here, the operation of the shutter button 302 includes an instruction of automatic focus control started by a predetermined operation such as half-pressing of the shutter button 302. Subsequently, when the photographing process is started, the digital camera 101 acquires identification information (identifier) from a terminal device existing in a communicable range (S1402). If the identifier cannot be acquired, it is determined not to display the attribute information (S1413), and the process ends.

  When the digital cameras 101 and 102 acquire the identification information of the terminal devices existing within the communicable range, the digital cameras 101 and 102 make a request including the acquired identification information and acquire attribute information and feature information associated with the identification information from the server 104. (S1403). The first specifying unit 308 of the digital cameras 101 and 102 tries to detect an object corresponding to the feature information on the captured image using the acquired feature information (S1404). Subsequently, the digital cameras 101 and 102 determine whether the specific processing in S1404 is successful (S1405). As a result of S1405, if the specific process is successful, the process proceeds to S1411.

  On the other hand, if the identification process fails as a result of S1405, the second feature information acquisition unit 311 requests the second identification information from another device including the identifier acquired in S1403 (S1407). Here, the other apparatus is a digital camera existing in the vicinity in the first embodiment, and the server 104 in the second embodiment. Subsequently, the digital cameras 101 and 102 determine whether or not there is a response from another device requested in S1406 (S1407). As a result of S1407, if there is a response, the process proceeds to S1408. If not, it is determined not to display the attribute information (S1413), and the process ends.

  The second feature information acquisition unit 311 of the digital cameras 101 and 102 acquires second feature information included in a response from another device (S1408). Subsequently, the second specifying unit 312 tries to detect an object corresponding to the feature information on the captured image using the acquired second feature information (S1409). The digital cameras 101 and 102 determine whether the specific processing in S1409 is successful (S1410). As a result of S1410, when the specific process is successful, the process proceeds to S1411. On the other hand, if the result of S1410 is that the specific process has failed, it is determined not to display the attribute information (S1413), and the process ends.

  In step S1411, the feature information generation unit 310 newly generates feature information related to the identified object from the captured image that has been successfully identified. In the first embodiment, the generated feature information is held in the storage unit 313 for transmission when requested by another device. In the second embodiment, the generated feature information is transmitted to the server 104. In S1412, when the object corresponding to the identification information is detected from the captured image and specified, the combining unit 309 combines the attribute information with the detected object in association with the detected object. Then, the display unit 304 displays the composite image (S1412). As described above, even when the object cannot be specified from the image from the first registered feature information, the object can be specified by the feature information newly generated by another device. Then, the user of the digital camera 101 can obtain information related to the object in the captured image by viewing the composite image displayed with the attribute information associated with the object. In addition, the user can easily know which displayed information is associated with which object.

(Other embodiments)
As another configuration, the present invention may specify a person in a moving image and display attribute information regardless of a still image. In that case, you may implement | achieve by processing each flame | frame of a moving image as a still image continuously. Communication related to transmission and acquisition of identification information is not limited to wireless LAN communication conforming to the IEEE802.11 series, but may be Bluetooth (registered trademark), passive / active RFID, or the like. In addition, communication regarding identification information may be simultaneously performed by a plurality of wireless communication interfaces such as a wireless LAN and a passive RFID. Further, the identification information transmission unit may be provided in the digital cameras 101 and 102 so that the identification information is transmitted also in the digital camera.

  In the first and second embodiments, facial feature information is used as the first feature information and clothing feature information is used as the second feature information. However, the present invention does not define which feature information is used. The feature information of clothes may be used as the first feature information, and the feature information of the face may be used as the second feature information. Further, as the first feature information, a plurality of feature information such as facial feature information and clothing feature information may be combined. However, the first feature information is preferably feature information that does not change so much in the medium term, such as a face. On the other hand, since the second feature information is newly generated feature information each time, the second feature information may be feature information having a change such as clothes.

  However, when feature information that changes rapidly is used, it is desirable that the storage unit 313 clears the feature information of the clothing after a certain period of time after storing the feature information of the clothing. Or you may clear at the timing of a certain specific time slot | zone, such as a date changing.

  As other feature information, the feature information may be used from a hairstyle, a hat, a decoration, or the like. Further, the feature information may be generated based on the color or shape under the identified person's face as the second feature information by using the face feature information in the first feature information. The feature information may be generated based on the ratio of the face and the body. The feature information may be calculated from a size ratio with surrounding objects.

  The present invention can also be realized by executing the following processing. Software (program) that realizes the functions of the above-described embodiments is supplied to a system or information processing apparatus via a network or various storage media, and the computer (or CPU, MPU, or the like) of the system or information processing apparatus executes the program. It is a process to read and execute.

101, 102 Digital camera 103 Mobile phone 104 Server 105, 106 People

Claims (11)

An information processing apparatus,
First specifying means for specifying the predetermined object from the captured image using first feature information for specifying the predetermined object in the captured image;
A second second device acquires second feature information for identifying the predetermined object generated from an image in which the predetermined object has been successfully identified using the first feature information. Feature information acquisition means;
A second specifying means for specifying the predetermined object from the captured image using the second feature information when the first specifying means cannot specify the predetermined object;
Display control means for displaying the predetermined object specified by the first specifying means or the second specifying means on the display unit in association with information on the predetermined object;
An information processing apparatus comprising: Identification information acquisition means for acquiring an identifier of the second other device;
Further comprising information acquisition means for acquiring information associated with the identifier acquired by the identification information acquisition means,
The information processing apparatus according to claim 1, further comprising: the information related to the predetermined object is information acquired by the information acquisition unit.   Generating means for generating third characteristic information relating to the predetermined object from the captured image when the predetermined object is successfully identified from the captured image by the first specifying means or the second specifying means; The information processing apparatus according to claim 1, further comprising:   The information processing apparatus according to claim 3, further comprising: a transmission unit that transmits the third feature information generated by the generation unit to a third other device. A fourth other device that transmits the second feature information for specifying the predetermined object associated with the identifier from the captured image, including the identifier acquired by the identification information acquisition means. Further comprising request means for requesting
The information processing apparatus according to claim 2, wherein the second feature information acquisition unit acquires the second feature information from a response of the fourth other device to the request of the request unit.   6. The information processing apparatus according to claim 5, wherein the first other apparatus and the fourth other apparatus are the same apparatus.   The information processing apparatus according to claim 2, wherein the predetermined object is associated with an identifier acquired by the identification information acquisition unit.   The information processing apparatus according to claim 1, further comprising a first feature information acquisition unit configured to acquire the first feature information from an external device.   The information processing apparatus according to claim 1, wherein the first feature information and the second feature information are acquired from different external devices, respectively. A method for controlling an information processing apparatus,
A first specifying step of specifying the predetermined object from the captured image using first feature information for specifying the predetermined object in the captured image;
A second second device acquires second feature information for identifying the predetermined object generated from an image in which the predetermined object has been successfully identified using the first feature information. A feature information acquisition process;
A second specifying step of specifying the predetermined object from the captured image using the second feature information when the first specifying unit cannot specify the predetermined object;
A display control step of associating and displaying the predetermined object specified in the first specific step or the second specific step and information on the predetermined object on a display unit;
A method for controlling an information processing apparatus, comprising:   A program for causing an information processing apparatus to execute the control method according to claim 9.

Images