JP6172353B2 - Terminal apparatus, object identification method, information processing apparatus, and program - Google Patents

Description

  The present invention relates to a terminal device, an object identification method, an information processing device, and a program.

  In recent years, image recognition technology has been advanced, and the position and orientation of an object shown in an input image from a camera can be identified by matching image feature amounts. One application of such object identification is an augmented reality (AR) application. In an AR application, various information (for example, advertisement information, navigation information, or information for a game) is additionally associated with an object in an image showing a building, road, or other object existing in the real world. Can be displayed.

  Patent Document 1 below proposes a feature quantity extraction algorithm for object identification with improved robustness against changes in viewpoints, changes in brightness, and noise. The following Patent Document 2 proposes a feature quantity extraction algorithm called a Random Ferns method, which can be operated at high speed with a lower processing cost.

Japanese Patent No. 4492036 Mustafa Oezuysal, “Fast Keypoint Recognition using Random Ferns”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.32, Nr.3, pp.448-461, March 2010

  As described above, there are various feature amount extraction algorithms for identifying an object shown in an image. However, in general, algorithms that can achieve higher discrimination performance require more processing costs. For this reason, for example, when performing object identification on an apparatus with few processing resources such as a portable terminal, there is a restriction on identification performance (such as the accuracy of identification and the number of objects that can be simultaneously identified). Further, when an image is transferred frame by frame to a server having abundant processing resources to execute object identification, a delay while waiting for a response from the server can hinder the responsiveness of the application.

  Therefore, the present invention is intended to provide a terminal device, an object identification method, an information processing device, and a program that can exhibit higher object identification performance in an apparatus with less processing resources.

  According to an embodiment of the present invention, an image acquisition unit that acquires a captured input image, the input image, and auxiliary information, a feature amount of the input image to identify an object appearing in the input image, and A transmission unit that transmits to a server having a first feature value dictionary to be collated, an object identification result based on the first feature value dictionary, and the auxiliary information, acquired by the server, and the first feature A receiving unit that receives from the server a second feature value dictionary that includes a feature value of a co-occurrence object that is highly likely to co-occur with an object identified based on the amount dictionary; There is provided a terminal device in which the accuracy of the algorithm used for extracting the feature amount included in the feature amount dictionary is different from the accuracy of the algorithm used for extracting the feature amount included in the second feature amount dictionary.

  According to another embodiment of the present invention, a feature amount of the input image is obtained in order to identify the object shown in the input image by acquiring the captured input image and using the input image and the auxiliary information. To the server having the first feature value dictionary to be collated, and obtained by the server according to the object identification result based on the first feature value dictionary and the auxiliary information, and the first feature Receiving from the server a second feature dictionary that includes features for co-occurrence objects that are likely to co-occur with objects identified based on a quantity dictionary, and An object identification method is provided in which the accuracy of an algorithm used to extract a feature amount included in a feature amount dictionary is different from the accuracy of an algorithm used to extract a feature amount included in the second feature amount dictionary.

  According to another embodiment of the present invention, a storage unit for storing a first feature dictionary that is a set of feature quantities of known object images, an input image captured by a terminal device, and auxiliary information are stored. A receiving unit for receiving, an identification unit for identifying an object appearing in the input image by collating the feature amount of the input image with the first feature amount dictionary, an identification result by the identification unit, and the auxiliary information Accordingly, a dictionary acquisition unit that acquires a second feature quantity dictionary including a feature quantity of a co-occurrence object that is highly likely to co-occur with an object identified based on the first feature quantity dictionary, and the dictionary A transmission unit that transmits the second feature value dictionary acquired by the acquisition unit to the terminal device, and the accuracy of an algorithm used for extracting the feature value included in the first feature value dictionary, Features included in the second feature dictionary The accuracy of the algorithm used in the extraction is different, the information processing apparatus is provided.

  A first feature dictionary in which a computer acquires a captured input image, and the input image and auxiliary information are collated with a feature of the input image to identify an object shown in the input image A function of transmitting to a server having an object, an identification result of an object based on the first feature value dictionary, and an object acquired by the server according to the auxiliary information and identified based on the first feature value dictionary, A function of receiving a second feature value dictionary including a feature value of a co-occurrence object having a high possibility of co-occurrence from the server, and realizing a feature amount included in the first feature value dictionary A program in which the accuracy of an algorithm used for extraction differs from the accuracy of an algorithm used for extracting feature amounts included in the second feature amount dictionary.

  As described above, according to the terminal device, the object identification method, the information processing apparatus, and the program according to the present invention, it is possible to exhibit higher object identification performance in an apparatus with few processing resources.

It is explanatory drawing for demonstrating the outline | summary of the system which concerns on one Embodiment. It is explanatory drawing for demonstrating the image which can be displayed on the screen of a terminal device. It is a block diagram which shows an example of the hardware constitutions of the terminal device which concerns on one Embodiment. It is a block diagram which shows an example of a structure of the logical function of the terminal device which concerns on one Embodiment. It is a block diagram which shows an example of the hardware constitutions of the dictionary server which concerns on one Embodiment. It is a block diagram which shows an example of a structure of the logical function of the dictionary server which concerns on one Embodiment. It is explanatory drawing for demonstrating an example of the feature-value dictionary memorize | stored by the dictionary server. It is explanatory drawing for demonstrating the 1st example of the subset of the dictionary acquired by the dictionary server. It is explanatory drawing for demonstrating the 2nd example of the subset of the dictionary acquired by the dictionary server. It is explanatory drawing for demonstrating the 3rd example of the subset of the dictionary acquired by the dictionary server. It is explanatory drawing for demonstrating the 4th example of the subset of the dictionary acquired by the dictionary server. It is explanatory drawing for demonstrating an example of the data memorize | stored by an additional information database. It is a block diagram which shows an example of a structure of the logical function of the dictionary server which concerns on a 1st modification. It is the 1st explanatory view for explaining generation of the feature-value dictionary in the 1st modification. It is the 2nd explanatory view for explaining generation of the feature-value dictionary in the 1st modification. It is a block diagram which shows an example of a structure of the logical function of the dictionary server which concerns on a 2nd modification. It is explanatory drawing for demonstrating an example of the subset of the dictionary acquired in the 2nd modification. It is a flowchart which shows an example of the flow of the process by the terminal device which concerns on one Embodiment. It is a flowchart which shows an example of the flow of the process by the dictionary server which concerns on one Embodiment.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

Further, the “DETAILED DESCRIPTION OF THE INVENTION” will be described in the following order.
1. 1. System overview 2. Configuration example of terminal device according to one embodiment 2-1. Hardware configuration 2-2. 2. Logical configuration 3. Configuration example of dictionary server according to one embodiment 3-1. Hardware configuration 3-2. Logical configuration 3-3. Modified example 4. Flow of processing according to one embodiment 4-1. Processing on terminal side 4-2. Server-side processing Summary

<1. System overview>
FIG. 1 is an explanatory diagram showing an outline of an object identification system to which the technology disclosed in this specification can be applied. Referring to FIG. 1, an object identification system 1 according to an embodiment is shown. The object identification system 1 includes a terminal device 100 and a dictionary server 200.

  The terminal device 100 is a device that identifies an object shown in an image captured by an imaging device. The terminal device 100 may be a mobile terminal carried by a user, such as a smartphone or a PDA (Personal Digital Assistant). The terminal device 100 may be another type of device such as a PC (Personal Computer), a digital home appliance, a game device, or a work robot. The imaging device may be built in the terminal device 100. Instead, the imaging device may be provided outside the terminal device 100 and connected to the terminal device 100 by wire or wirelessly.

  When identifying an object appearing in an image, the terminal device 100 collates the feature amount extracted from the image with a feature amount dictionary that is a set of known feature amounts for one or more objects. Then, the terminal device 100 identifies which object is shown in the image based on a score calculated by matching (hereinafter referred to as a matching score). In the present specification, “high” in the collation score for the feature amount of a known object means that there is a high possibility that the object appears in the input image. For example, if the difference between a known feature quantity and a feature quantity of an input image is close to zero at a specific position and orientation, an object corresponding to the feature quantity is reflected in the input image at that position and orientation. There is a high possibility. This situation is referred to as “high” in this specification (although the evaluation value of the difference itself is small). That is, the terminal device 100 can also identify the position and orientation of the object shown in the image in the image. Various applications using the result of such object identification can be installed in the terminal device 100. In this specification, an example in which an AR application that uses the result of object identification is installed in the terminal device 100 will be mainly described. However, in the terminal device 100, an application having another purpose (for example, monitoring, environment recognition, work support, or the like) may use the object identification result.

  The dictionary server 200 is an information processing device that provides the terminal device 100 with a feature dictionary for object identification. The dictionary server 200 communicates with the terminal device 100 via the network 3. The network 3 may be any kind of network such as the Internet, a provider network or a corporate network. In the present embodiment, the dictionary server 200 receives an image from the terminal device 100. Then, the dictionary server 200 identifies an object shown in the received image, and provides the terminal device 100 with a feature dictionary according to the identification result.

  FIG. 2 is an explanatory diagram for describing an image that can be displayed on the screen of the terminal device 100 in the present embodiment. The image illustrated in FIG. 2 is an AR application image. Referring to FIG. 2, an image showing the building 10 existing in the real space is displayed on the screen of the terminal device 100. Further, additional information 12 is superimposed on the image. The additional information 12 is information indicating the name and rating of a restaurant operating in the building 10. Such additional information is selected based on the result of object identification in the terminal device 100, and is superimposed according to the position of the object in the image. In the present embodiment, a database of additional information superimposed on an image in this way is also provided from the dictionary server 200 to the terminal device 100.

<2. Configuration Example of Terminal Device According to One Embodiment>
[2-1. Hardware configuration]
FIG. 3 is a block diagram illustrating an example of a hardware configuration of the terminal device 100 according to the present embodiment. Referring to FIG. 3, the terminal device 100 includes an imaging unit 102, a sensor unit 104, an input unit 106, a storage unit 108, a display unit 112, a communication unit 114, a bus 118, and a control unit 120.

(Imaging part)
The imaging unit 102 is a camera module that captures an image. The imaging unit 102 generates an input image for object identification by imaging a real space using an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

(Sensor part)
The sensor unit 104 is a sensor group that supports recognition of the position and orientation of the terminal device 100. For example, the sensor unit 104 may include a GPS sensor that receives a GPS (Global Positioning System) signal and measures the latitude, longitude, and altitude of the terminal device 100. In addition, the sensor unit 104 may include a positioning sensor that measures the position of the terminal device 100 based on the strength of the wireless signal received from the wireless access point. In addition, the sensor unit 104 may include a gyro sensor that measures the tilt angle of the terminal device 100, an acceleration sensor that measures triaxial acceleration, or a geomagnetic sensor that measures azimuth. When the terminal device 100 has a position estimation function and a posture estimation function based on image recognition, the sensor unit 104 may be omitted from the configuration of the terminal device 100.

(Input section)
The input unit 106 is an input device that is used by a user to operate the terminal device 100 or input information to the terminal device 100. The input unit 106 may include, for example, a keyboard, a keypad, a mouse, a button, a switch, or a touch panel. The input unit 106 may include a gesture recognition module that recognizes a user's gesture shown in the input image. The input unit 106 may include a line-of-sight detection module that detects a line-of-sight direction of a user wearing a HMD (Head Mounted Display) as a user input.

(Memory part)
The storage unit 108 is configured by a storage medium such as a semiconductor memory or a hard disk, and stores a program and data for processing by the terminal device 100. For example, the storage unit 108 temporarily stores an input image generated by the imaging unit 102 and sensor data measured by the sensor unit 104. The storage unit 108 also stores data received from the dictionary server 200 via the communication unit 114. An example of data received from the dictionary server 200 will be described in detail later.

(Display section)
The display unit 112 is a display module configured by an LCD (Liquid Crystal Display), an OLED (Organic light-Emitting Diode), a CRT (Cathode Ray Tube), or the like. The display unit 112 displays, for example, an input image captured by the imaging unit 102 or an image of an application that uses the result of object identification (for example, an AR application image illustrated in FIG. 2) on the screen. The display unit 112 may be a part of the terminal device 100 or may be provided outside the terminal device 100. Further, the display unit 112 may be an HMD worn by the user.

(Communication Department)
The communication unit 114 is a communication interface that mediates communication between the terminal device 100 and the dictionary server 200. The communication unit 114 supports an arbitrary wireless communication protocol or wired communication protocol, and establishes a communication connection with the dictionary server 200. Thereby, the terminal device 100 can transmit an image to the dictionary server 200 and receive a feature dictionary from the dictionary server 200.

(bus)
The bus 118 connects the imaging unit 102, the sensor unit 104, the input unit 106, the storage unit 108, the display unit 112, the communication unit 114, and the control unit 120 to each other.

(Control part)
The control unit 120 corresponds to a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). The control unit 120 operates various functions of the terminal device 100 described later by executing a program stored in the storage unit 108 or another storage medium.

[2-2. Logical configuration]
4 is a block diagram illustrating an example of a configuration of logical functions realized by the storage unit 108 and the control unit 120 of the terminal device 100 illustrated in FIG. Referring to FIG. 4, the terminal device 100 includes an image acquisition unit 130, a transmission unit 140, a reception unit 150, a dictionary cache 160, an identification unit 170, an additional information cache 180, and a display control unit 190.

(Image acquisition unit)
The image acquisition unit 130 acquires an input image generated by the imaging unit 102. Then, the image acquisition unit 130 sequentially outputs the acquired input image to the transmission unit 140 and the identification unit 170.

(Transmitter)
When a predetermined trigger event is detected, the transmission unit 140 transmits an input image input from the image acquisition unit 130 to the dictionary server 200 via the communication unit 114. As described above, the dictionary server 200 is a server that holds a feature value dictionary that is a set of image feature values for known objects.

The trigger event that triggers transmission of the input image from the transmission unit 140 may be one or more of the following events, for example.
a) Periodic timing arrival: An input image is transmitted in a cycle such as once every n frames or once every t seconds. The period is typically set in advance such that the frequency of transmission of the input image is less than the frequency of object identification by the identification unit 170.
b) User instruction: An input image is transmitted in response to an explicit instruction from the user via the input unit 106.
c) Frame-out of object being tracked: When an object that has been identified as a result of object identification is framed out of the image, an input image is transmitted.
d) New object frame-in: When a new object is detected in the image as a result of object identification, the input image is transmitted. New objects may include known objects that were not present in the image and unknown objects. The known object may be detected using a simple recognition technique such as face recognition. Also, for example, the moving object is recognized from the difference between the current frame and the previous frame (may be a difference after motion compensation is applied), and what kind of object is the recognized moving object. If it cannot be identified, it can be determined that an unknown new object has been framed in. Such detection of the moving object based on the difference image may be performed only when no object is identified in the image by object identification, for example.
e) Movement of the terminal device: When a change in the position or orientation of the terminal device 100 or an increase in speed is detected, an input image is transmitted.
Periodic input image transmission may be employed when it is desirable to continuously identify an object without depending on the content of the image. The transmission of the input image in response to the user instruction can be employed when the user desires to identify or track an object displayed on the screen, for example. The other trigger event is an event assuming that there is a high possibility that a new object will appear in the image. An input image is transmitted in response to these trigger events, and a feature dictionary is provided from the dictionary server 200. It is possible to appropriately identify a new object.

  The transmission unit 140 may transmit auxiliary information that assists the dictionary server 200 to acquire a feature dictionary along with the input image to the dictionary server 200. The auxiliary information may include, for example, at least one of the position (of the terminal device 100 or the imaging device) when the input image is captured, the date and time, and the capability information of the terminal device 100. The position and date / time may be used when filtering the feature dictionary in the dictionary server 200. The capability information of the terminal device 100 can be used when the dictionary server 200 determines the data amount of the feature dictionary to be provided to the terminal device 100. The use of such auxiliary information will be further described later.

(Receiver)
After the input image is transmitted from the transmission unit 140 to the dictionary server 200, the reception unit 150 receives from the dictionary server 200 a feature amount dictionary acquired by the dictionary server 200 according to the object identification result for the input image. To do. The feature amount dictionary received by the receiving unit 150 is a dictionary having a smaller data amount than the feature amount dictionary that the dictionary server 200 has. How the feature dictionary provided to the terminal device 100 is acquired by the dictionary server 200 will be further described later.

  When receiving the feature dictionary, the receiving unit 150 stores the received feature dictionary in the dictionary cache 160. In the present embodiment, an identifier (hereinafter referred to as an object ID) for uniquely identifying an object is associated with each feature quantity included in the feature quantity dictionary. When the reception unit 150 newly receives a feature amount having the same object ID as the feature amount stored in the dictionary cache 160, the reception unit 150 may update the feature amount of the dictionary cache 160 with the newly received feature amount. Good. The receiving unit 150 may attach a reception time stamp to each received feature quantity, and automatically delete from the dictionary cache 160 a feature quantity that has passed a predetermined period from the reception time stamp. Instead, the deletion of the feature value from the dictionary cache 160 may be performed in response to a movement amount of the terminal device 100 or a frame-out from an image of an associated object.

  Further, in the present embodiment, the receiving unit 150 receives from the dictionary server 200 an additional information database acquired by the dictionary server 200 according to the object identification result. The additional information database received by the receiving unit 150 is a database having a smaller data amount than the additional information database that the dictionary server 200 has in advance. The receiving unit 150 stores the received additional information database in the additional information cache 180.

(Dictionary cache)
The dictionary cache 160 stores the feature dictionary received by the receiving unit 150 using the storage unit 108 illustrated in FIG. The feature dictionary stored in the dictionary cache 160 is referred to when the identification unit 170 identifies an object.

(Identification part)
The identification unit 170 extracts the feature amount of the input image input from the image acquisition unit 130, and collates the extracted feature amount with the feature amount dictionary stored in the dictionary cache 160, thereby identifying an object shown in the input image. Identify. The feature amount extraction algorithm used by the identification unit 170 is, for example, the Random Ferns method described in Non-Patent Document 2 or “SURF: Speeded Up Robust Features” (H. Bay, A. Ess, T. Tuytelaars and LVGool, Computer Vision and Image Understanding (CVIU), Vol.110, No.3, pp.346-359, 2008) may be used. These algorithms are lightweight algorithms that can operate at high speed with relatively low processing costs. As a result of the object identification by the identification unit 170, typically, the object ID of the object shown in the input image, and the position and orientation of the object in the input image are derived. Then, the identification unit 170 outputs the object identification result to the display control unit 190.

(Additional information cache)
The additional information cache 180 stores the additional information database received by the receiving unit 150 using the storage unit 108 shown in FIG. The display control unit 190 described below selects additional information to be superimposed on the input image from the additional information database stored in the additional information cache 180.

(Display control unit)
The display control unit 190 acquires additional information related to the object identified by the identification unit 170 from the additional information database stored in the additional information cache 180, and outputs the additional information by superimposing the acquired additional information on the input image. Generate an image. Then, the display control unit 190 outputs the generated output image to the display unit 112.

  The additional information superimposed on the input image may be any information. For example, the additional information superimposed on the input image may be advertisement information and rating information associated with a building shown in the input image as illustrated in FIG. Another example of the additional information will be further described later.

<3. Configuration Example of Dictionary Server According to One Embodiment>
[3-1. Hardware configuration]
FIG. 5 is a block diagram illustrating an example of a hardware configuration of the dictionary server 200 according to the present embodiment. Referring to FIG. 5, the dictionary server 200 includes a storage unit 208, a communication unit 214, a bus 218, and a control unit 220.

(Memory part)
The storage unit 208 is configured by a storage medium such as a semiconductor memory or a hard disk, and stores a program and data for processing by the dictionary server 200. The storage unit 208 may have a richer storage capacity than the storage unit 108 of the terminal device 100. The storage unit 208 stores in advance a feature dictionary and additional information database, which will be described later.

(Communication Department)
The communication unit 214 is a communication interface that mediates communication between the dictionary server 200 and the terminal device 100. The communication unit 214 supports an arbitrary wireless communication protocol or wired communication protocol, and establishes a communication connection with the terminal device 100. Thereby, the dictionary server 200 can receive an image from the terminal device 100 and can transmit the feature dictionary and the additional information database to the terminal device 100.

(bus)
The bus 218 connects the storage unit 208, the communication unit 214, and the control unit 220 to each other.

(Control part)
The control unit 220 corresponds to a processor such as a CPU or a DSP. The control unit 220 may have higher calculation performance than the control unit 120 of the terminal device 100. The control unit 220 operates various functions of the dictionary server 200 to be described later by executing a program stored in the storage unit 208 or another storage medium.

[3-2. Logical configuration]
FIG. 6 is a block diagram showing an example of a configuration of logical functions realized by the storage unit 208 and the control unit 220 of the dictionary server 200 shown in FIG. Referring to FIG. 6, the dictionary server 200 includes a receiving unit 230, a feature value dictionary 240 for the first algorithm (Arg1), a feature value dictionary 242 for the second algorithm (Arg2), an identification unit 250, and a dictionary acquisition unit. 260, an additional information database (DB) 270, an additional information acquisition unit 280, and a transmission unit 290.

(Receiver)
The receiving unit 230 waits for an input image transmitted from the terminal device 100. When receiving the input image via the communication unit 214, the receiving unit 230 outputs the received input image to the identification unit 250. In addition, when the auxiliary information described above is received together with the input image, the reception unit 230 outputs the auxiliary information to the identification unit 250 and the dictionary acquisition unit 260.

(Feature dictionary)
The feature value dictionary (Arg1) 240 and the feature value dictionary (Arg2) 242 are a set of feature values stored in advance by the storage unit 208. Each feature amount of the feature amount dictionary (Arg1) 240 is extracted from a known object image according to the first algorithm. Each feature amount of the feature amount dictionary (Arg2) 242 is extracted from a similar known object image according to the second algorithm. Typically, the first algorithm is a feature amount extraction algorithm that enables object identification with higher accuracy than the second algorithm. On the other hand, the second algorithm is a feature amount extraction algorithm that can be executed at a higher speed than the first algorithm. The first algorithm may be, for example, a feature amount extraction algorithm described in Patent Document 1. Instead, the first algorithm is “Shape Matching and Object Recognition Using Shape Contexts” (S.Belongie, J.Malik, and J.Puzicha, IEEE Trans. Pattern Analysis and Machine Intelligence, vol.2, no.4). , Pp.509--522, April 2002) or “Distinctive image features from scale-invariant keypoints” (DGLowe, Internal Journal of Computer Vision, 60, 2, pp.91--110, January 2004) The algorithm described in) may be used. The second algorithm is a feature amount extraction algorithm (for example, Random Ferns method or SURF method) that is also used when an object is identified by the identification unit 170 of the terminal device 100 described above. In the following description, the first algorithm is referred to as a high-precision algorithm, and the second algorithm is referred to as a lightweight algorithm.

  The feature quantity in the feature quantity dictionary (Arg1) 240 and the feature quantity in the feature quantity dictionary (Arg2) 242 are linked using a common object ID. That is, the feature amount for the same object ID is included in both the feature amount dictionary (Arg1) 240 and the feature amount dictionary (Arg2) 242.

FIG. 7 is an explanatory diagram for describing an example of a feature dictionary stored by the dictionary server 200. Referring to FIG. 7, the feature dictionary (Arg1) 240 stores feature amounts previously extracted from known object images according to a high-precision algorithm for each of a plurality of objects including _eight objects B _{1 to} B _8. Including. Each object is given a name. The feature amount dictionary (Arg2) 242 includes feature amounts extracted in advance according to a lightweight algorithm for each of a plurality of objects that similarly include the _eight objects B _{1 to} B ₈ . The object ID of each object is common between these two feature quantity dictionaries. That is, for example, the feature amount for the object B _{1 in the} feature amount dictionary 242 is the feature amount extracted from the image of the building A, as is the feature amount for the object B _{1 in the} feature amount dictionary 240.

  The feature amount dictionaries 240 and 242 may include additional data without being limited to the example of FIG. In some examples described later, the feature value dictionary 240 includes additional data for supporting efficient acquisition of a feature value dictionary to be provided to the terminal device 100. However, instead of the feature value dictionary 240 (or in addition to the feature value dictionary 240), the feature value dictionary 242 may include such additional data.

(Identification part)
The identifying unit 250 extracts one or more feature quantities of the input image received by the receiving unit 230 according to a high-precision algorithm, and collates the extracted feature quantity with the feature quantity dictionary (Arg1) 240 to thereby display one or more reflected in the input image. Identify the object. Then, the identification unit 250 outputs the object ID and the matching score of the identified object to the dictionary acquisition unit 260 and the additional information acquisition unit 280.

(Dictionary acquisition part)
The dictionary acquisition unit 260 acquires a feature dictionary to be provided to the terminal device 100 according to the identification result by the identification unit 250. The feature value dictionary acquired by the dictionary acquisition unit 260 is a subset of the feature value dictionary (Arg2) 242 having a data amount smaller than that of the feature value dictionary (Arg1) 240 and the feature value dictionary (Arg2) 242 described above. Hereinafter, four examples of dictionary subset acquisition by the dictionary acquisition unit 260 will be described with reference to FIGS.

(1) First Example FIG. 8 is an explanatory diagram for describing a first example of a dictionary subset acquired by the dictionary acquisition unit 260. Referring to FIG. 8, the rank (rank) of the matching score obtained as a result of identification by the identification unit 250 is shown for each object ID of the feature dictionary (Arg1) 240. In the example of FIG. 8, the matching score of the object B ₁ is the highest rank is the first place. Matching score of the object B ₂ is then high, rank is the second largest. The rank of the collation score of the object B ₆ is kth. The dictionary acquisition unit 260 acquires, for example, the feature amounts for the top k objects having such a matching score from the feature amount dictionary (Arg2) 242. Then, the dictionary acquisition unit 260 outputs the feature amount dictionary subset 242 a including the acquired feature amounts to the transmission unit 290 as a feature amount dictionary to be provided to the terminal device 100.

  Note that the amount of data to be included in the feature amount dictionary subset 242a (for example, the number k of feature amounts) is dynamically determined according to the capability information of the terminal device 100 received as auxiliary information from the terminal device 100. Also good. The capability of the terminal device 100 can be expressed by, for example, the number of data that can be processed, the number of cores of a processor, or the memory capacity.

(2) Second Example FIG. 9 is an explanatory diagram for describing a second example of a dictionary subset acquired by the dictionary acquisition unit 260. In the second example, the feature quantity dictionary (Arg1) 240 has data “co-occurrence object” defined in advance in addition to “object ID”, “name”, and “feature quantity” for each object. The “co-occurrence object” represents a list of objects that are highly likely to co-occur with each object. In the present specification, the presence of the second object in the vicinity of the first object is referred to as “co-occurring” between the first object and the second object. In the example of FIG. 9, the co-occurrence objects of the object B ₄ are the object B ₅ and the object B ₉ . This is because there is a high possibility that the object B ₅ (car E) or the object B ₉ (sign I) will appear in the subsequent input image when the object B ₄ (signal D) is identified in the input image. Means that. Using such data, the dictionary acquisition unit 260 can acquire not only the feature amount of the object already shown in the input image but also the feature amount of the object predicted to appear in the subsequent input image. In the example of FIG. 9, the dictionary acquisition unit 260 includes the features of the object B ₅ and the object B ₉ that are predicted to appear in the subsequent input image, in addition to the feature amount of the object B ₄ having the higher matching score. The quantity is acquired from the feature quantity dictionary (Arg2) 242. Then, the dictionary acquisition unit 260 outputs the feature amount dictionary subset 242 b including the acquired feature amount to the transmission unit 290.

(3) Third Example FIG. 10 is an explanatory diagram for describing a third example of a dictionary subset acquired by the dictionary acquisition unit 260. Also in the third example, the dictionary acquisition unit 260 acquires not only the feature amount of the object already shown in the input image but also the feature amount of the object predicted to appear in the subsequent input image. However, in the third example, the object predicted to appear in the subsequent input image is an object that is determined from the position data to be located in the vicinity of the object already shown in the input image. Referring to FIG. 10, the feature dictionary (Arg1) 240 has position data (latitude and longitude, or other coordinate data) for each object. For example, the position of the object B ₁ is X ₁ , the position of the object B ₂ is X ₂ , and the position of the object B ₃ is X ₃ . Among them, the distance between the position X ₁ and position X ₂ is the smaller than the threshold D. Dictionary acquisition unit 260, if the matching score of the object B ₁ is was higher, based on such location data, in addition to the feature amount of the object B _1, the object located in the vicinity of a body B ₁ B _{2 is} acquired from the feature dictionary (Arg2) 242. Then, the dictionary acquisition unit 260 outputs the feature amount dictionary subset 242 c including the acquired feature amount to the transmission unit 290.

  Note that the position data illustrated in FIG. 10 may be used for filtering the feature dictionary. For example, the dictionary acquisition unit 260 may acquire only the feature amount of an object located in the vicinity of the terminal device 100 among the top k objects of the matching score. Further, the identification unit 250 may use only the feature amount of an object located in the vicinity of the terminal device 100 as a target to be compared with the feature amount extracted from the input image. The position of the terminal device 100 can be recognized from auxiliary information received from the terminal device 100.

(4) Fourth Example FIG. 11 is an explanatory diagram for describing a fourth example of a dictionary subset acquired by the dictionary acquisition unit 260. Referring to FIG. 11, the feature dictionary (Arg1) 240 has “illumination condition” data in addition to “object ID”, “name”, and “feature” for each object. The “illumination condition” may be, for example, a category representing an illumination condition when a known object image is captured. Illumination conditions are distinguished from each other by, for example, a condition relating to time such as a time zone in which an image is taken or a season in which an image is taken, or a condition relating to weather. The feature amount dictionary (Arg1) 240 may include a plurality of types of feature amounts that are extracted from images obtained by capturing the same object in situations where the illumination conditions are different from each other. In the example of FIG. 11, the object _{B 2,} illumination conditions L1 (e.g., "morning" or "clear") feature amount corresponding to the illumination condition L2 (e.g., "day" or "cloudy") feature amount corresponding to , And the feature amount corresponding to the illumination condition L3 (for example, “evening” or “rain”) is included in the feature amount dictionary (Arg1) 240. Similarly for the object _{B 3,} feature quantity corresponding to the illumination condition L1, L2 and L3 are included in the feature dictionary (Arg1) 240. As described above, the feature amount dictionary (Arg1) 240 includes a plurality of feature amounts for the same object having different illumination conditions when captured, so that the identification of the object by the identification unit 250 is different in the illumination condition. It becomes difficult to be influenced by the difference in the appearance of the object caused by the. In the example of FIG. 11, for example, when an input image is received that the object B ₂ is reflected, the matching score of the feature quantity corresponding to the illumination conditions L1 and L2 and the feature quantity of the input image although low lighting condition L3 to due to the high matching score with the feature amount corresponding feature quantity for the object B ₂ is included in the subset 242d appropriately feature dictionary.

  Note that the illumination condition data illustrated in FIG. 11 may also be used for filtering the feature dictionary. For example, the dictionary acquisition unit 260 calculates a feature amount corresponding to an illumination condition different from the illumination condition to which the date and time when the input image is captured, from among the feature amounts of the top k objects of the matching score. It may be excluded from the subset 242d. Further, the identification unit 250 may use only the feature amount corresponding to the illumination condition to which the date and time belongs as a target to be compared with the feature amount extracted from the input image. The date and time when the input image is captured can be recognized from the auxiliary information received from the terminal device 100.

(Additional information DB)
The additional information DB 270 is a set of additional information associated with an object existing in the real space. In the AR field, additional information is also called annotation. FIG. 12 is an explanatory diagram for describing an example of data stored in the additional information DB. Referring to FIG. 12, in the additional information DB 270, additional information including two data items “type” and “content” is associated with the object ID of each object. “Type” represents the type of individual additional information. The “content” may be text data, graphic data, image data, or the like as an entity of each additional information. In the example of FIG. 12, it is associated with the advertisement information and the rating information to the object B _1. In addition, advertisement information, alert information, and vehicle type information are associated with the objects B ₂ , B _4, and B ₅ , respectively.

(Additional information acquisition unit)
The additional information acquisition unit 280 acquires additional information to be provided to the terminal device 100 from the additional information DB 270 according to the identification result by the identification unit 250, and generates a subset of the additional information database with a smaller data amount. Then, the additional information acquisition unit 280 outputs the generated subset of the additional information database to the transmission unit 290. The additional information acquisition unit 280 typically acquires, from the additional information DB 270, a set of additional information that has a common object ID and a subset of the feature dictionary acquired by the dictionary acquisition unit 260. That is, the additional information acquisition unit 280 may also acquire a set of additional information corresponding to the top k objects of the matching score from the additional information DB 270. Further, the additional information acquisition unit 280 may further acquire additional information corresponding to an object predicted to appear in the subsequent input image from the additional information DB 270.

(Transmitter)
The transmission unit 290 transmits a subset of the feature dictionary acquired by the dictionary acquisition unit 260 to the terminal device 100 via the communication unit 214. At that time, the transmission unit 290 determines whether or not the identified object includes a new object different from the previously identified object, and only when the new object is identified, May be transmitted to the terminal device 100. Thereby, when the same object is continuously shown in the input image, transmission of the redundant feature dictionary is omitted, and the traffic load is reduced. In addition, the transmission unit 290 transmits a subset of the additional information database generated by the additional information acquisition unit 280 to the terminal device 100. A subset of the side information database may also be sent only when a new object is identified.

[3-3. Modified example]
Next, two modifications of the dictionary server 200 will be described.

(1) First Modification FIG. 13 is a block diagram illustrating an example of a logical function configuration of the dictionary server 200 according to a first modification. Referring to FIG. 13, the dictionary server 200 includes a receiving unit 232, a feature amount dictionary 240 for the high-precision algorithm (Arg1), a feature amount dictionary 242 for the lightweight algorithm (Arg2), an identification unit 252, a dictionary acquisition unit 262, and an addition An information DB 270, an additional information acquisition unit 280, and a transmission unit 290 are included.

  The receiving unit 232 waits for an input image transmitted from the terminal device 100. Then, when receiving the input image via the communication unit 214, the receiving unit 232 outputs the received input image to the identifying unit 252 and the dictionary acquiring unit 262.

  The identifying unit 252 extracts one or more feature quantities of the input image received by the receiving unit 232 according to a high-precision algorithm, and collates the extracted feature quantities with the feature quantity dictionary (Arg1) 240 to thereby display one or more of the input images. Identify the object. The identifying unit 252 identifies the position and orientation of the object shown in the input image. Then, the identification unit 252 outputs the object ID, position, and orientation of the identified object to the dictionary acquisition unit 262. Further, the identification unit 252 outputs the object ID of the identified object to the additional information acquisition unit 280.

  The dictionary acquisition unit 262 generates a feature dictionary to be provided to the terminal device 100 according to the identification result by the identification unit 252. More specifically, the dictionary acquisition unit 262 first recognizes the position of the object identified by the identification unit 252 in the input image, and cuts out a partial image of a region in which each object appears from the input image. And the dictionary acquisition part 262 extracts a feature-value from the cut-out partial image according to a lightweight algorithm. The dictionary acquisition unit 262 associates the object ID input from the identification unit 252 with the feature amount of each object extracted in this manner, and generates a feature amount dictionary for the lightweight algorithm. In this case, the feature dictionary 242 for the lightweight algorithm (Arg2) may be omitted from the configuration of the dictionary server 200. Instead, the dictionary acquisition unit 262 adds a feature amount extracted from the partial image (that is, a feature amount to be additionally learned) to the subset of feature amounts acquired from the feature amount dictionary 242, thereby creating a new A feature dictionary may be generated. The dictionary acquisition unit 262 outputs the feature amount dictionary generated in this way to the transmission unit 290, and causes the transmission unit 290 to transmit the feature amount dictionary to the terminal device 100.

  Further, the dictionary acquisition unit 262 may further generate a variation of the feature amount by changing parameters such as color, brightness, or the degree of blur from the feature amount extracted according to the lightweight algorithm. Variations of these feature quantities can also constitute a new feature quantity dictionary.

14 and 15 are explanatory diagrams for explaining generation of a feature dictionary by the dictionary acquisition unit 262 in the first modification. Referring to FIG. 14, objects B ₁ and B ₄ appearing in the input image Im1 are identified using the feature dictionary 240 and a high-precision algorithm. Then, the dictionary acquisition unit 262, as shown in FIG. 15, cuts out a partial image A2 which is the partial image A1 and the object _{B 4} that the object _{B 1} is reflected reflected from the input image Im1. Then, the dictionary acquisition unit 262 extracts feature amounts from the partial image A1 and the partial image A2 according to the lightweight algorithm. Further, the dictionary acquisition unit 262 generates variations of feature amounts having different parameters such as color or brightness from the extracted feature amounts. Then, the dictionary acquisition unit 262 forms a new feature dictionary 242d to be provided to the terminal device 100 by attaching an object ID to each feature.

  According to the first modification, the feature amount dictionary dynamically generated from the input image by the dictionary server 200 is provided to the terminal device 100. Such a feature dictionary is a feature dictionary with a small amount of data, including feature quantities that are particularly suited to the environment in which the terminal device 100 exists (such as an imaging environment or a lighting environment). Therefore, the terminal device 100 can identify the object shown in the input image and the position and orientation of the object from the subsequent input image with high accuracy and low processing cost.

(2) Second Modification In the example described so far, a subset of the feature dictionary for the lightweight algorithm is provided from the dictionary server 200 to the terminal device 100. However, as in the second modification described in this section, the dictionary server 200 may provide the terminal device 100 with a subset of the feature dictionary for the high-precision algorithm.

  FIG. 16 is a block diagram illustrating an example of a logical function configuration of the dictionary server 200 according to the second modification. Referring to FIG. 16, the dictionary server 200 includes a reception unit 230, a feature dictionary 240 for the high-precision algorithm (Arg1), an identification unit 250, a dictionary acquisition unit 264, an additional information DB 270, an additional information acquisition unit 280, and a transmission unit 290. including.

The dictionary acquisition unit 264 acquires, from the feature amount dictionary (Arg1) 240, a subset of the feature amount dictionary to be provided to the terminal device 100 according to the identification result by the identification unit 250. For example, FIG. 17 shows again the rank of the matching score obtained as a result of identification by the identification unit 250. In the example of FIG. 17, the collation score rank of the object B ₁ is _first , the collation score rank of the object B ₂ is _second , and the collation score rank of the object B ₆ is kth. The dictionary acquisition unit 264 acquires, for example, feature amounts for the top k objects having such a matching score from the feature amount dictionary (Arg1) 240, and forms a subset 240a of the feature amount dictionary including the acquired feature amounts. . Then, the transmitting unit 290 transmits the subset 240a of the feature dictionary to the terminal device 100.

  When the second modification is applied, the identification unit 170 of the terminal device 100 extracts a feature amount from the input image according to a high-precision algorithm, and the extracted feature amount is a subset of the feature amount dictionary provided from the dictionary server 200. To match. In this case, the processing cost of the terminal device 100 required for extracting the feature amount increases compared to the example using the lightweight algorithm. However, the dictionary cache 160 stores not only the entire feature dictionary of the dictionary server 200 but only a subset thereof. Therefore, compared with the case where the terminal device 100 has the entire feature amount dictionary, the processing cost for matching feature amounts in the terminal device 100 and the memory resources consumed can be significantly reduced.

  Here, an example in which the transmission unit 140 of the terminal device 100 transmits an input image to the dictionary server 200 has been mainly described. However, the transmission unit 140 of the terminal device 100 may transmit the feature amount extracted from the input image by the identification unit 170 to the dictionary server 200 instead of transmitting the input image. In this case, the identification unit 250 of the dictionary server 200 can collate the feature amount of the input image received by the reception unit 230 with the feature amount dictionary (Arg1) 240.

<4. Flow of processing according to one embodiment>
[4-1. Terminal processing]
FIG. 18 is a flowchart illustrating an example of the flow of processing by the terminal device 100 according to the present embodiment.

  Referring to FIG. 18, first, the image acquisition unit 130 of the terminal device 100 acquires an input image (step S102). Next, the transmission unit 140 determines whether or not the predetermined trigger event described above (for example, the arrival of periodic timing or a user instruction) is detected (step S104). Here, if a trigger event is not detected, the processing of subsequent steps S106 to S110 is skipped. On the other hand, when the trigger event is detected, the transmission unit 140 transmits the input image (and auxiliary information as necessary) to the dictionary server 200 (step S106). Then, the receiving unit 150 receives the feature dictionary from the dictionary server 200 (step S108). The feature dictionary received here is stored in the dictionary cache 160. The receiving unit 150 receives the additional information DB from the dictionary server 200 (step S110). The additional information DB received here is stored in the additional information cache 180. Next, the identification unit 170 identifies an object shown in the input image using the feature dictionary in the dictionary cache 160 (step S112). Next, the display control unit 190 acquires additional information related to the object identified by the identification unit 170 from the additional information cache 180, and generates an output image by superimposing the acquired additional information on the input image ( Step S114). The position and orientation of the additional information in the input image can be adjusted according to the position and orientation of the object identified by the identification unit 170, for example. Then, the display control unit 190 causes the display unit 112 to display the generated output image (Step S116).

  Such processing is repeated for each of a series of input images acquired by the image acquisition unit 130.

[4-2. Server side processing]
FIG. 19 is a flowchart showing an example of the flow of processing by the dictionary server 200 according to the present embodiment.

  Referring to FIG. 19, first, the receiving unit 230 of the dictionary server 200 waits for reception of an input image from the terminal device 100 (step S202). Then, when the input image is received by the receiving unit 230, the identifying unit 250 extracts a feature amount from the input image according to a high-precision algorithm (step S204). Next, the identifying unit 250 compares the extracted feature quantity of the input image with each feature quantity in the feature dictionary (Arg1) 240, and identifies an object shown in the input image (step S206). If a new object different from the object identified in the previously received input image is identified, the process proceeds to step S210 (step S208). On the other hand, when a new object is not identified, the process of subsequent steps S210-S214 may be skipped. When a new object is identified by the identification unit 250, a subset of the feature value dictionary is acquired (or a new feature value dictionary with a small data amount is generated) according to the identification result (step S210). Next, the additional information acquisition unit 280 acquires a subset of the additional information DB from the additional information DB 270 according to the object identification result by the identification unit 250 (step S212). Next, the transmission unit 290 transmits the subset of the feature dictionary and the subset of the additional information DB to the terminal device 100 (step S214).

  The feature dictionary and the additional information DB provided from the dictionary server 200 to the terminal device 100 by such processing are used for object identification in the terminal device 100.

<5. Summary>
Up to this point, an embodiment of the technology disclosed in the present specification and two modifications have been described in detail with reference to FIGS. According to the above-described embodiment, the feature dictionary used for identifying an object from the input image in the terminal device 100 is the dictionary server 200 that stores in advance a feature dictionary including a richer feature. Provided to the terminal device 100. The feature dictionary provided to the terminal device 100 is a dictionary acquired by the dictionary server 200 in accordance with the object identification result from the input image. Therefore, even if the terminal device 100 with few processing resources does not have an enormous feature amount dictionary in advance, the terminal device 100 identifies an object with higher accuracy by using the feature amount dictionary suitable for the situation where the terminal device 100 is placed. can do.

  Further, according to the present embodiment, the dictionary server 200 can identify an object using a highly accurate feature amount extraction algorithm, and the terminal device 100 can identify an object using a lightweight feature amount extraction algorithm. Therefore, even in the terminal device 100 with few processing resources, an application with object identification that requires real-time performance, such as an AR application, can be operated at high speed with high accuracy.

  Further, according to the present embodiment, a database of additional information that can be superimposed on an image in the AR application is stored in advance by the dictionary server 200, and a subset thereof is provided to the terminal device 100. Additional information provided from the dictionary server 200 to the terminal device 100 is also acquired in accordance with an object identification result from an input image in the dictionary server 200. Accordingly, resources for storing and processing additional information in the terminal device 100 are also saved.

  Further, according to the present embodiment, the feature dictionary provided from the dictionary server 200 to the terminal device 100 is predicted to be reflected not only in the feature amount of the object shown in the latest input image but also in the subsequent input image. The feature amount about the object to be included is included. Therefore, the terminal device 100 can continue to use the feature dictionary once provided from the dictionary server 200 for a certain period of time. Thus, once the feature dictionary is provided, data reception waiting does not occur in the subsequent object identification in the terminal device 100, so that the real-time property of the application operating on the terminal device 100 is improved. Further, since the terminal device 100 does not need to transmit the input image to the dictionary server 200 every frame, the traffic load is also reduced.

  Further, according to the first modification, a new feature amount generated in the dictionary server 200 using the partial image of the input image is provided to the terminal device 100. In this case, as compared with the case where a subset of the feature amount dictionary prepared in advance in a standard environment is provided, the feature amount particularly suited to the environment (such as an imaging environment or a lighting environment) in which the terminal device 100 exists. The terminal device 100 can use the dictionary. Also in the second modified example, it is possible to reduce the processing cost of feature amount matching in the terminal device 100 and the memory resources consumed.

  Note that the above-described object identification technology is not limited to AR or other purpose applications, but also, for example, initialization or calibration of a coordinate system when estimating the position and orientation of the terminal device 100 using SLAM (Simultaneous Localization and Mapping) technology. May be used for. Regarding SLAM technology, refer to “Real-Time Simultaneous Localization and Mapping with a Single Camera” (A. J. Davison, Proceedings of the 9th IEEE International Conference on Computer Vision Volume 2, 2003, pp. 1403-1410).

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Object identification system 100 Terminal device 108 Memory | storage part 130 Image acquisition part 140 Transmission part 150 Reception part 170 Identification part 190 Display control part 200 Dictionary server (information processing apparatus)
230 receiving unit 250 identifying unit 260, 262, 264 dictionary acquiring unit 280 additional information acquiring unit 290 transmitting unit

Claims (10)

An image acquisition unit for acquiring a captured input image;
A transmission unit that transmits the input image and the auxiliary information to a server having a first feature value dictionary that is matched with a feature value of the input image to identify an object reflected in the input image;
Co-occurrence that is highly likely to co-occur with an object that is obtained by the server according to the identification result of the object based on the first feature quantity dictionary and the auxiliary information and identified based on the first feature quantity dictionary A receiving unit that receives a second feature value dictionary including a feature value of an object from the server;
With
A terminal device, wherein the accuracy of an algorithm used for extracting a feature value included in the first feature value dictionary is different from the accuracy of an algorithm used for extracting a feature value included in the second feature value dictionary.   The terminal device according to claim 1, wherein the co-occurrence object includes an object that exists in the vicinity of an object identified based on the first feature dictionary.   The terminal device according to claim 1, wherein the co-occurrence object includes an object that is not reflected in the input image.   The terminal device according to claim 1, wherein the co-occurrence object includes an object that is predicted to be reflected in a subsequent input image.   The terminal device according to claim 1, wherein the auxiliary information includes information on a position where the input image is captured.   The terminal device according to claim 1, wherein the object identified based on the first feature quantity dictionary is an object having a higher collation score based on the first feature quantity dictionary.   The terminal device according to any one of claims 1 to 6, wherein the co-occurrence object includes at least one of a traffic signal, a building, a moving body, a signboard, and a sign. Obtaining a captured input image;
Transmitting the input image and auxiliary information to a server having a first feature dictionary that is matched with a feature of the input image to identify an object shown in the input image;
Co-occurrence that is highly likely to co-occur with an object that is obtained by the server according to the identification result of the object based on the first feature quantity dictionary and the auxiliary information and identified based on the first feature quantity dictionary Receiving a second feature dictionary containing features from the server from the server;
Including
The object identification method, wherein accuracy of an algorithm used for extracting a feature amount included in the first feature amount dictionary is different from accuracy of an algorithm used for extracting a feature amount included in the second feature amount dictionary. A storage unit for storing a first feature quantity dictionary that is a set of feature quantities of known object images;
A receiving unit for receiving an input image captured by the terminal device and auxiliary information;
An identification unit for identifying an object appearing in the input image by comparing the feature amount of the input image with the first feature amount dictionary;
A second feature amount including a feature amount of a co-occurrence object that is highly likely to co-occur with the object identified based on the first feature amount dictionary according to the identification result by the identification unit and the auxiliary information. A dictionary acquisition unit for acquiring a dictionary;
A transmission unit that transmits the second feature dictionary acquired by the dictionary acquisition unit to the terminal device;
With
An information processing apparatus, wherein accuracy of an algorithm used for extracting a feature amount included in the first feature amount dictionary is different from accuracy of an algorithm used for extracting a feature amount included in the second feature amount dictionary. On the computer,
A function to acquire a captured input image;
A function of transmitting the input image and auxiliary information to a server having a first feature dictionary that is matched with a feature of the input image to identify an object appearing in the input image;
Co-occurrence that is highly likely to co-occur with an object that is obtained by the server according to the identification result of the object based on the first feature quantity dictionary and the auxiliary information and identified based on the first feature quantity dictionary A function of receiving, from the server, a second feature value dictionary including a feature value of an object;
Realized,
A program in which the accuracy of an algorithm used to extract a feature amount included in the first feature amount dictionary is different from the accuracy of an algorithm used to extract a feature amount included in the second feature amount dictionary.

Images