JP2007058532A - Information processing system, information processor and method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing system easily and rapidly acquiring desired information. <P>SOLUTION: In this information processing system, a terminal 11 transmits an object image including an image of a retrieval object acquiring related information to an information provision server 13 having a model dictionary and an image recognition engine designated by a user from model dictionaries and image recognition engines registered in an index list received from an index server 12. The information provision server 13 recognizes the retrieval object inside the object image by use of the designated model dictionary and image recognition engine, acquires model information related to the retrieval object from a model dictionary storage part 42, and transmits it to the terminal 11. The information processing system can be applied to an information processing system performing image retrieval. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an information processing system, an information processing apparatus and method, a program, and a recording medium, and more particularly, to an information processing system, an information processing apparatus and method, a program, and a recording medium that search information using an image.

  Conventionally, an image captured by a digital camera of a mobile terminal is transmitted to an image recognition server, the server identifies the type of object present in the image, and searches the database for information on the identified object class. It has been proposed to transmit a result to a portable terminal (see, for example, Patent Document 1).

  In addition, a method for accurately extracting a model stored in advance from an input image has been proposed (see, for example, Patent Document 2). For example, by applying the method described in Patent Document 2 to the invention described in Patent Document 1, it is possible to improve the accuracy of identifying the type of object in the image.

JP 2004-145416 A JP 2004-326693 A

  However, in the inventions described in Patent Documents 1 and 2, as the number of registered images increases, the time for searching for an image that matches the input image becomes longer, or the recognition rate of the image decreases. Therefore, it takes a long time to acquire desired information.

  The present invention has been made in view of such a situation, and makes it possible to acquire desired information easily and quickly.

  An information processing system according to a first aspect of the present invention is an information processing system that searches for information using a plurality of first dictionaries in which object image information related to an object image and related information related to the object are registered. An image acquisition unit that acquires a first input image, and a dictionary determination unit that determines a use dictionary that is the first dictionary used for searching for the related information related to an object in the first input image. And object recognition means for recognizing the object in the first input image based on the object image information registered in the use dictionary, and the object related to the object recognized by the object recognition means And related information acquisition means for acquiring related information from the use dictionary.

  The dictionary determination means may cause the use dictionary to determine the first dictionary designated by the user.

  The image acquisition means further acquires an index image indicating a registration dictionary that is the first dictionary in which the related information related to the object in the first input image is registered, and the index image The detecting means for detecting the registered dictionary shown in FIG. 5 is further provided, and the dictionary determining means can determine the registered dictionary as the used dictionary.

  The image acquisition means further acquires a second input image for identifying a predetermined group to which the first input image belongs, identification image information relating to an identification image for identifying the group, and Registered in the second dictionary registered in association with the index information indicating the first dictionary in which the related information related to the object in the image belonging to the group identified by the identification image is registered. Identification image specifying means for specifying the identification image corresponding to the second input image based on the identification image information, and the index associated with the identification image specified by the identification image specifying means Index information acquisition means for acquiring information from the second dictionary, and the dictionary determination means includes the index information acquisition means. The first dictionary shown in obtained by the obtaining means and the index information can be made to be determined on the dictionary used.

  An information processing method or program according to the first aspect of the present invention is an information processing method for searching for information using a plurality of dictionaries in which object image information related to an object image and related information related to the object are registered, or A program for causing a computer to execute information processing for searching for information using a plurality of dictionaries in which object image information related to an object image and related information related to the object are registered, and controls acquisition of an input image Determining a use dictionary which is the dictionary used for searching for the related information related to the object in the input image, and based on the object image information registered in the use dictionary, the object in the input image Before the related information related to the recognized object Comprising the step of controlling the acquisition from use dictionary.

  The information processing apparatus according to the second aspect of the present invention includes a first image acquisition unit that acquires a first input image, object image information related to an object image, and related information related to the object. A dictionary determining means for determining a use dictionary which is a first dictionary used for searching the related information related to the object in the first input image; and a first apparatus having the use dictionary A first transmission unit configured to transmit information indicating the first input image and the use dictionary to a dictionary holding device; and the object image registered in the use dictionary transmitted from the dictionary holding device. First receiving means for receiving the related information related to the object in the first input image recognized based on the information.

  The apparatus further comprises index list acquisition means for acquiring an index list which is a list of index information indicating the first dictionary and the first device having the first dictionary, and the dictionary determination means includes the index The first dictionary designated by the user from the first dictionaries indicated in the index information in the list may be determined by the use dictionary.

  The index information further indicates an image recognition engine that is hardware or software that performs image recognition using the first dictionary, and the first transmission means further includes the index information in the index list. Information indicating a user-designated image recognition engine that is the image recognition engine designated by the user from among the image recognition engines shown in FIG. 5 is transmitted to the dictionary holding device, and the first receiving means has the dictionary holding The related information related to the object in the first input image recognized using the user-designated image recognition engine based on the object image information registered in the use dictionary and transmitted from a device Can be received.

  The image acquisition means further includes a registration dictionary that is the first dictionary in which the related information related to the object in the first input image is registered, and the first device having the registration dictionary And detecting means for detecting the registration dictionary indicated by the index image and the first device having the registration dictionary, and the dictionary determination means includes the registration dictionary. And the first transmission means transmits information indicating the first input image and the usage dictionary to the dictionary holding device, which is the first device having the registration dictionary. You can make it.

  The index image may be embedded as a watermark in the input image.

  The index image may be an image of an identification code.

  The index image further indicates an image recognition engine that is hardware or software that performs image recognition using the registered dictionary, and the first transmission unit further includes information indicating the image recognition engine in the dictionary. The first receiving means is recognized by the image recognition engine based on the object image information registered in the use dictionary and transmitted by the dictionary holding device. The related information related to the object in the first input image may be received.

  The image acquisition means further acquires a second input image for identifying a predetermined group to which the first input image belongs, and transmits the second input image to the second device. And the related information related to the object in the image belonging to the group identified based on the second input image transmitted from the second device is registered. And a second receiving unit that receives index information indicating the first device having the registration dictionary, and the dictionary determination unit includes the registration dictionary as the use dictionary. And the first transmission means causes the dictionary holding device, which is the first device having the registration dictionary, to transmit information indicating the first input image and the use dictionary. Door can be.

  The index information further indicates an image recognition engine that is hardware or software that performs image recognition using the registered dictionary, and the first transmission means further includes information indicating the image recognition engine in the dictionary. The first receiving means is recognized by the image recognition engine based on the object image information registered in the use dictionary and transmitted by the dictionary holding device. The related information related to the object in the first input image may be received.

  An information processing method or program according to a second aspect of the present invention is a dictionary that controls the acquisition of an input image and stores object image information related to an object image and related information related to the object, Determination of a use dictionary that is a dictionary used for searching for related information related to an object in an input image, and transmission of information indicating the input image and the use dictionary to a dictionary holding device that is a device having the use dictionary And controlling reception of the related information related to the object in the input image recognized based on the object image information registered in the use dictionary and transmitted from the dictionary holding device. Includes steps.

  An information processing apparatus according to a third aspect of the present invention is an information processing apparatus that searches for information using a plurality of dictionaries in which object image information related to an object image and related information related to the object are registered, Based on the object image information registered in the designated dictionary which is the designated dictionary, the receiving means for receiving the input image and the information for designating the dictionary to be used, transmitted from another device, Object recognition means for recognizing an object in the input image, related information acquisition means for acquiring the related information related to the object recognized by the object recognition means from the designated dictionary, and acquired by the related information acquisition means Transmitting means for transmitting the related information to the other device.

  The receiving means further receives information indicating an image recognition engine that is hardware or software for performing image recognition using the designated dictionary, and the object recognition means has the information registered in the designated dictionary. Based on the object image information, the object in the input image can be recognized using the image recognition engine.

  An information processing method or program according to a third aspect of the present invention is an information processing method for retrieving information using a plurality of dictionaries in which object image information related to an image of an object and related information related to the object are registered, or A program for causing a computer to execute information processing for searching for information using a plurality of dictionaries in which object image information related to an object image and related information related to the object are registered, which is transmitted from another device Control the reception of the information specifying the input image and the dictionary to be used, and recognize the object in the input image based on the object image information registered in the specified dictionary which is the specified dictionary , From the specified dictionary of the related information related to the recognized object Controls give, comprising the step of controlling the transmission to the other apparatus of the acquired related information.

  An information processing apparatus according to a fourth aspect of the present invention includes: a receiving unit that receives a second input image for identifying a predetermined group to which the first input image belongs, transmitted from the first apparatus; A first dictionary in which identification image information related to an identification image for identifying a group, object image information related to an image of an object, and related information related to the object are registered, and is identified by the identification image The first dictionary in which related information related to objects in the image belonging to the group is registered, and index information indicating the second device having the first dictionary are registered in association with each other. Identification image specifying means for specifying the identification image corresponding to the second input image based on the identification image information registered in the second dictionary; Including an acquisition unit that acquires the index information associated with the identification image specified from the second dictionary by serial identification image specifying means

  Transmission means for transmitting the index information acquired by the acquisition means to the first device may be further provided.

  The receiving unit further receives the first input image transmitted from the first device, and causes the second device indicated by the index information acquired by the acquiring unit to receive the first input image. Transmission means for transmitting the first input image and information indicating the first dictionary indicated by the index information may be further provided.

  An information processing method or program according to a fourth aspect of the present invention controls reception of a second input image for identifying a predetermined group to which the first input image belongs, transmitted from the first device. A first dictionary in which identification image information related to an identification image for identifying a group, object image information related to an image of an object, and related information related to the object are registered, and is identified by the identification image The first dictionary in which related information related to objects in the image belonging to the group is registered, and the index information indicating the second device having the first dictionary are registered in association with each other. Based on the identification image information registered in the second dictionary, the identification image corresponding to the second input image is identified and identified. Serial comprising the step of controlling the acquisition from the second dictionary of the index information associated with the identification image.

  In the first aspect of the present invention, an input image is obtained, and a dictionary in which object image information related to an object image and related information related to the object are registered, the object being related to the object in the input image A use dictionary that is a dictionary used for searching the related information is determined, and the object in the input image is recognized based on the object image information registered in the use dictionary, and the recognized object is The related information related is obtained from the usage dictionary.

In the second aspect of the present invention, an input image is acquired,
A dictionary in which object image information related to an image of an object and related information related to the object are registered, and a use dictionary that is a dictionary used for searching for the related information related to the object in the input image is determined. The object image information registered in the use dictionary is transmitted from the dictionary holding device to the dictionary holding device, which is a device having the use dictionary, and is transmitted from the dictionary holding device. The related information related to the object in the input image recognized based on the is received.

  In the third aspect of the present invention, an input image transmitted from another device, and a dictionary in which object image information related to the image of the object and related information related to the object are registered, Information specifying a dictionary to be used is received, and an object in the input image is recognized based on the object image information registered in the specified dictionary which is the specified dictionary, and is recognized by the object recognition means. The related information related to the object is acquired from the designated dictionary, and the related information acquired by the related information acquisition means is transmitted to the other device.

  In the fourth aspect of the present invention, a second input image for identifying a predetermined group to which the first input image belongs, received from the first device, is received, and for identifying the group A first dictionary in which identification image information relating to an identification image, object image information relating to an image of an object, and related information relating to the object are registered, and in an image belonging to the group identified by the identification image Registered in the second dictionary in which the first dictionary in which the related information related to the object is registered and the index information indicating the second device having the first dictionary are registered in association with each other The identification image corresponding to the second input image is specified based on the identification image information that has been determined, and is specified by the identification image specifying means. The index information associated with the identification image are acquired from the second dictionary.

  As described above, according to the first to fourth aspects of the present invention, information can be searched based on an input image. Moreover, according to the first to fourth aspects of the present invention, desired information can be acquired easily and quickly.

  Embodiments of the present invention will be described below. Correspondences between constituent elements described in the claims and specific examples in the embodiments of the present invention are exemplified as follows. This description is to confirm that specific examples supporting the invention described in the claims are described in the embodiments of the invention. Therefore, even if there are specific examples that are described in the embodiment of the invention but are not described here as corresponding to the configuration requirements, the specific examples are not included in the configuration. It does not mean that it does not correspond to a requirement. On the contrary, even if a specific example is described here as corresponding to a configuration requirement, this means that the specific example does not correspond to a configuration requirement other than the configuration requirement. not.

  Further, this description does not mean that all the inventions corresponding to the specific examples described in the embodiments of the invention are described in the claims. In other words, this description is an invention corresponding to the specific example described in the embodiment of the invention, and the existence of an invention not described in the claims of this application, that is, in the future, a divisional application will be made. Nor does it deny the existence of an invention added by amendment.

  The information processing system according to the first aspect of the present invention (for example, the information processing system 1 in FIG. 1 or the information processing system 201 in FIG. 14) includes object image information (for example, model image information) relating to an object image and the object. An information processing system for retrieving information using a plurality of first dictionaries (for example, model dictionaries) in which related related information (for example, model information) is registered, and an image for acquiring a first input image Use of acquisition means (for example, the image acquisition unit 72 of FIG. 4 or the image acquisition unit 272 of FIG. 15) and the first dictionary used for searching for the related information related to the object in the first input image. Dictionary determining means for determining a dictionary (for example, the transmission server determining unit 74 in FIG. 4 or the transmission server determining unit 274 in FIG. 15) and the use dictionary are registered. Object recognition means (for example, the image recognition unit 103 in FIG. 5) for recognizing the object in the first input image based on the object image information, and the object recognized by the object recognition means Related information acquisition means (for example, the model information acquisition unit 104 in FIG. 5) for acquiring the related information from the usage dictionary.

  In the information processing system according to the first aspect of the present invention (for example, the information processing system 1 in FIG. 1), the image acquisition means further includes the related information related to the object in the first input image. The image processing apparatus further includes detection means (for example, the index information detection unit 73 in FIG. 4) that acquires an index image indicating the registered dictionary that is the registered first dictionary and detects the registered dictionary indicated in the index image. The dictionary determination means may determine the registered dictionary as the use dictionary.

  In the information processing system according to the first aspect of the present invention (for example, the information processing system 201 in FIG. 14), the image acquisition means further includes a first group for identifying a predetermined group to which the first input image belongs. The identification image information related to the identification image for identifying the group is acquired, and the related information related to the object in the image belonging to the group identified by the identification image is registered. Based on the identification image information registered in a second dictionary (for example, an index dictionary) registered in association with index information indicating the first dictionary, the image corresponding to the second input image Identification image specifying means for specifying an identification image (for example, the image recognition unit 302 in FIG. 16) and the identification image specified by the identification image specifying means Index information acquisition means (for example, the index information acquisition unit 303 in FIG. 16) that acquires the index information associated with the second dictionary from the second dictionary, and the dictionary determination means includes the index information acquisition means. The first dictionary indicated in the acquired index information may be determined as the use dictionary.

  An information processing method or program according to the first aspect of the present invention is a dictionary in which object image information (for example, model image information) related to an image of an object and related information (for example, model information) related to the object are registered. An information processing method for retrieving information using a plurality of (for example, model dictionaries), or object image information (for example, model image information) related to an object image and related information (for example, model information) related to the object. The computer executes information processing for searching for information using a plurality of registered dictionaries (for example, model dictionaries) and controls acquisition of input images in the program (for example, step S4 in FIG. 6 and step S62 in FIG. 12). Alternatively, in step S102 in FIG. 17, the object in the input image is related. The use dictionary which is the dictionary used for searching the related information is determined (for example, step S7 in FIG. 6, step S66 in FIG. 12 or step S107 in FIG. 17), and the object image registered in the use dictionary Based on the information, the object in the input image is recognized (for example, step S43 in FIG. 9 or step S243 in FIG. 22), and the related information related to the recognized object is acquired from the use dictionary. (For example, step S44 in FIG. 9 or step S244 in FIG. 22).

  The information processing apparatus according to the second aspect of the present invention (for example, the service processing unit 64 in FIG. 4 or the service processing unit 264 in FIG. 15) is an image acquisition unit (for example, the image in FIG. 4) that acquires the first input image. The acquisition unit 72 or the image acquisition unit 272 in FIG. 15, the object image information (for example, model image information) related to the image of the object, and the related information (for example, model information) related to the object are registered first. A dictionary determination unit (for example, FIG. 4) that determines a use dictionary that is a first dictionary used for searching the related information related to the object in the first input image. The transmission server determination unit 74 of FIG. 15 or the transmission server determination unit 274 of FIG. 15 and the first device (for example, the information providing server 13 of FIG. 1) having the use dictionary. First transmitting means (for example, the model information requesting unit 75 in FIG. 4 or the model information requesting unit 275 in FIG. 15) that transmits information indicating the first input image and the use dictionary to the holding device, and the dictionary First receiving means for receiving the related information related to the object in the first input image recognized based on the object image information registered in the use dictionary and transmitted from a holding device (For example, the model information receiving unit 76 in FIG. 4 or the model information receiving unit 276 in FIG. 15).

  In the information processing apparatus (for example, the service processing unit 64 in FIG. 4) according to the second aspect of the present invention, the index information indicating the first dictionary and the first apparatus having the first dictionary is stored. Index list acquisition means for acquiring an index list that is a list (for example, index list acquisition unit 71 in FIG. 4), and the dictionary determination means includes the first dictionary indicated by the index information in the index list. The first dictionary designated by the user can be determined as the use dictionary.

  In the information processing apparatus according to the second aspect of the present invention (for example, the service processing unit 64 in FIG. 4), the image acquisition unit further includes the related information related to the object in the first input image. An index image indicating a registration dictionary that is the first dictionary registered and the first device having the registration dictionary is acquired, and the registration dictionary indicated in the index image and the registration dictionary are included. It further includes detection means for detecting the first device (for example, the index information detection unit 73 in FIG. 4), the dictionary determination means determines the registered dictionary as the use dictionary, and the first transmission means includes The information indicating the first input image and the use dictionary can be transmitted to the dictionary holding device, which is the first device having the registration dictionary.

  In the information processing apparatus according to the second aspect of the present invention (for example, the service processing unit 264 in FIG. 15), the image acquisition unit further includes a first group for identifying a predetermined group to which the first input image belongs. 2 is acquired from the second device, and second transmission means (for example, the index information request unit 281 in FIG. 15) that acquires the second input image and transmits the second input image to the second device. A registration dictionary which is the first dictionary in which the related information related to the object in the image belonging to the group identified based on the second input image is registered, and the registration dictionary Second receiving means (for example, index information receiving unit 272 in FIG. 15) for receiving index information indicating the first device, and the dictionary determining means The use dictionary is determined, and the first transmission means transmits the information indicating the first input image and the use dictionary to the dictionary holding device, which is the first device having the registration dictionary. can do.

  The information processing method or program according to the second aspect of the present invention controls acquisition of an input image (for example, step S4 in FIG. 6, step S62 in FIG. 12, or step S102 in FIG. 17), and an object image related to the image of the object. A dictionary (for example, model dictionary) in which information (for example, model image information) and related information (for example, model information) related to the object are registered, and the relationship for the object in the input image A use dictionary which is a dictionary used for information retrieval is determined (for example, step S7 in FIG. 6, step S66 in FIG. 12 or step S107 in FIG. 17), and the dictionary holding device which is a device having the use dictionary is described above. The transmission of information indicating the input image and the use dictionary is controlled (for example, step S8 in FIG. 6, step in FIG. 12). S67 or step S108 of FIG. 17), the related information related to the object in the input image recognized based on the object image information registered in the use dictionary and transmitted from the dictionary holding device. (For example, step S9 in FIG. 6, step S68 in FIG. 12, or step S109 in FIG. 17) is included.

  The information processing apparatus according to the third aspect of the present invention (for example, the model information providing unit 41 in FIG. 5) includes object image information (for example, model image information) related to an object image and related information (for example, model image information). An information processing apparatus that searches for information using a plurality of dictionaries (for example, model dictionaries) registered with model information), and is input from another apparatus (for example, the terminal 11 in FIG. 1). Based on receiving means (for example, the model information request receiving unit 101 in FIG. 5) for receiving an image and information specifying the dictionary to be used, and the object image information registered in the specified dictionary which is the specified dictionary An object recognition means for recognizing an object in the input image (for example, the image recognition unit 103 in FIG. 5), and the object recognition means. The related information acquisition unit (for example, the model information acquisition unit 104 in FIG. 5) that acquires the related information related to the recognized object from the designated dictionary, and the related information acquired by the related information acquisition unit. Transmitting means for transmitting to the other apparatus (for example, the model information transmitting unit 105 in FIG. 5).

  An information processing method or program according to a third aspect of the present invention is a dictionary in which object image information (for example, model image information) related to an object image and related information (for example, model information) related to the object are registered. An information processing method for retrieving information using a plurality of (for example, model dictionaries), or object image information (for example, model image information) related to an object image and related information (for example, model information) related to the object. A program that causes a computer to execute information processing for searching for information using a plurality of registered dictionaries (for example, model dictionaries), and that is input from another device (for example, the terminal 11 in FIG. 1) Controls reception of images and information specifying the dictionary to be used (for example, step S in FIG. 9). 1 or step S241 of FIG. 22), the object in the input image is recognized based on the object image information registered in the designated dictionary which is the designated dictionary (for example, step S43 of FIG. 9 or FIG. 22 step S243), controlling the acquisition of the related information related to the recognized object from the designated dictionary (for example, step S44 of FIG. 9 or step S244 of FIG. 22), and A step of controlling transmission to the other apparatus (for example, step S45 in FIG. 9 or step S245 in FIG. 22).

  The information processing apparatus (for example, the index information providing unit 231 in FIG. 16) according to the fourth aspect of the present invention receives the first input image transmitted from the first apparatus (for example, the terminal 211 in FIG. 14). Receiving means (for example, the receiving unit 301 in FIG. 16) for receiving a second input image for identifying a predetermined group to which it belongs, and identification image information (for example, group identification image information) regarding the identification image for identifying the group ), Object image information (for example, model image information) related to the image of the object, and related information (for example, model information) related to the object, are registered in a first dictionary (for example, model dictionary). The first dictionary in which related information related to objects in the image belonging to the group identified by the identification image is registered; and It is registered in a second dictionary (for example, index dictionary) registered in association with index information indicating the second device (for example, the information providing server 13 in FIG. 14) having the first dictionary. Based on the identification image information, an identification image specifying unit (for example, the image recognition unit 302 in FIG. 16) that specifies the identification image corresponding to the second input image, and the identification image specifying unit specifies the identification image. Acquisition means (for example, the index information acquisition unit 303 in FIG. 16) that acquires the index information associated with the identification image from the second dictionary.

  The information processing apparatus according to the fourth aspect of the present invention further includes a transmission unit (for example, the index information transmission unit 304 in FIG. 16) that transmits the index information acquired by the acquisition unit to the first apparatus. Can be.

  In the information processing device according to the fourth aspect of the present invention, the receiving unit further receives the first input image transmitted from the first device, and the index acquired by the acquiring unit. Transmitting means for transmitting the first input image and information indicating the first dictionary indicated by the index information to the second device indicated by the information (for example, the model information request transfer unit of FIG. 16) 305) can be further included.

  The information processing method or program according to the fourth aspect of the present invention is a first information for identifying a predetermined group to which the first input image belongs, transmitted from the first device (for example, the terminal 211 in FIG. 14). 2 is controlled (for example, step S121 in FIG. 19 or step S221 in FIG. 21), identification image information (for example, group identification image information) regarding an identification image for identifying a group, and an image of an object Is a first dictionary (eg, model dictionary) in which object image information (eg, model image information) and related information related to the object are registered, and belongs to the group identified by the identification image The first dictionary in which related information (for example, model information) related to the object in the image is registered; Based on the identification image information registered in a second dictionary (for example, an index dictionary) registered in association with index information indicating the second device having the first dictionary, the second device The identification image corresponding to the input image is identified (for example, step S122 in FIG. 19 or step S222 in FIG. 21), and the index information associated with the identified identification image is acquired from the second dictionary. (For example, step S123 in FIG. 19 or step S223 in FIG. 21) is included.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of an information processing system to which the present invention is applied. The information processing system 1 is configured to include terminals 11-1 to 11-x, an index server 12, and information providing servers 13-1 to 13-y. Furthermore, the information providing servers 13-1 to 13-y are configured to include model information providing units 41-1 to 41-y and model dictionary storage units 42-1 to 42-y, respectively. The terminals 11-1 to 11-x, the index list providing unit 31, and the model information providing units 41-1 to 41-y are connected to each other via the network 21.

  Hereinafter, when it is not necessary to individually distinguish the terminals 11-1 to 11-x, they are simply referred to as the terminal 11, and when it is not necessary to individually distinguish the information providing servers 13-1 to 13-y, information is simply used. This is referred to as providing server 13. Further, hereinafter, when it is not necessary to distinguish the model information providing units 41-1 to 41-y individually, they are simply referred to as the model information providing unit 41, and the model dictionary storage units 42-1 to 42-y are individually distinguished. When it is not necessary, it is simply referred to as a model dictionary storage unit 42.

  The terminal 11 is configured by, for example, a mobile phone, a personal computer, a PDA (Personal Digital Assistant), and the like. The user can use a predetermined service (hereinafter referred to as an image search service) realized by the information processing system 1 by causing the terminal 11 to execute a predetermined application program. The image search service uses the terminal 11 to transmit an image (hereinafter referred to as an object image) including an image of an object (hereinafter also referred to as a search object) for which related information is to be acquired to the information providing server 13. This is a service in which information related to the search object is searched by the information providing server 13 and the searched information is transmitted to the terminal 11. The search object is a visual object, for example, a tangible object such as a person or an object, or an intangible object such as a landscape.

  When the user starts using the image search service, the terminal 11 requests the index list providing unit 31 of the index server 12 to transmit an index list as necessary (hereinafter referred to as index list request information). Is transmitted via the network 21. The index list providing unit 31 transmits the index list stored in the index list storage unit 32 to the request source terminal 11 via the network 21.

  The index list is a list of index information and has, for example, a data structure shown in FIG. In the example shown in FIG. 2, the index information includes information providing server location information, an image recognition engine identification number, an image recognition engine text, a model dictionary identification number, and a model dictionary text.

  The information providing server position information (hereinafter also simply referred to as position information) has an image recognition engine and a model dictionary identified by an image recognition engine identification number and a model dictionary identification number in the same index information, and the image recognition engine And information (for example, IP address, domain name, etc.) indicating the position on the network 21 of the information providing server 13 that searches for information using the model dictionary is set.

  In the image recognition engine identification number, a number for identifying an image recognition engine that is hardware or software for performing image recognition is set using a model dictionary identified by a model dictionary identification number in the same index information. .

  The image recognition engine text is a character string associated with the image recognition engine identification number. For example, the image recognition engine attribute (for example, name, function, performance, etc.) identified by the image recognition engine identification number A character string indicating a description is set.

  In the model dictionary identification number, a number for identifying a model dictionary described later is set.

  The model dictionary text is a character string associated with the model dictionary identification number, for example, a character string indicating the attribute (for example, name, genre, etc.) and description of the model dictionary identified by the model dictionary identification number. Is set.

  That is, by referring to the index information, the position of the information providing server 13 having each image recognition engine and model dictionary can be specified. In other words, the image recognition engine and model dictionary possessed by each information providing server 13 Can be specified.

  When the terminal 11 requests transmission of information related to the search object, the terminal 11 includes the object image and the image recognition engine identification number and model dictionary identification number of the image recognition engine and model dictionary used for the search. Information requesting transmission of information to be transmitted (hereinafter referred to as model information request information) is transmitted via the network 21 to the model information providing unit 41 of the information providing server 13 having the image recognition engine and model dictionary used for the search. To do.

  The model information providing unit 41 of the information providing server 13 that has received the model information request information searches for information related to the search object using the image recognition engine and the model dictionary specified by the user based on the model information request information. To do.

  Specifically, each model dictionary storage unit 42 stores one or more types of model dictionaries. In the model dictionary, model image information related to images (hereinafter referred to as model images) of various objects (hereinafter referred to as models) and information related to the models (hereinafter referred to as model information) are registered. Yes. The model is a visual object, similar to the search object, and is, for example, a tangible object such as a person or an object, or an intangible object such as a landscape. The model image information includes, for example, the model image itself or information indicating the feature amount of the model extracted from the model image. Model information includes, for example, information indicating model attributes (for example, price, size, descriptive text, etc.), URL (Uniform Resource Locator) of a website from which information related to the model can be obtained, model image, etc. including. Each model is set with a model number for identifying each model.

  In one model dictionary, model image information and model information of models having similar characteristics are registered. Each model dictionary is also provided with one or more image recognition engines specialized to perform image recognition based on model image information registered in the model dictionary. One image recognition engine may correspond to a plurality of model dictionaries.

  FIG. 3 is a diagram illustrating a specific example of the model dictionary stored in the model dictionary storage unit 42. For example, the model dictionary storage unit 42A stores model image information 42A-1 to 42A-3 in which model image information and model information related to a model that is an object in a photograph (image) published in the magazine A are registered. . The model dictionaries 42A-1 to 42A-3 are classified by each issue of the magazine A, and the model dictionary 42A-1 corresponds to the May issue of the magazine A (published in the May issue of the magazine A). The model dictionary 42A-2 is a model dictionary corresponding to the June issue of the magazine A, and the model dictionary 42A- 3 is a model dictionary corresponding to the July issue of Magazine A.

  In addition, for example, the model dictionary storage unit 42B is configured to store model image information and model information related to a model that is an object (for example, a product) in a photograph (image) posted in the catalog B. 1 to 42B-3 are stored. The model dictionaries 42B-1 to 42B-3 are classified according to the genre (for example, food, clothing, electrical appliances, etc.) of products listed in the catalog B, and the model dictionary 42B-1 corresponds to the category A. A model dictionary (model image information and model information relating to objects belonging to category A are registered), model dictionary 42B-2 is a model dictionary corresponding to category B, and model dictionary 42B-3 is category C Is a model dictionary corresponding to

  Further, for example, the model dictionary storage unit 42C stores model image information and model information related to a model that is an object (for example, liquor) provided by a predetermined service C (for example, a liquor delivery service). The dictionaries 42C-1 to 42C-3 are stored. The model dictionaries 42C-1 to 42C-3 are classified by genres (for example, wine, sake, beer, etc.) such as services or products provided by the service C, and the model dictionaries 42C-1 correspond to the category D. (Model image information and model information relating to objects belonging to category D are registered), model dictionary 42C-2 is a model dictionary corresponding to category E, and model dictionary 42C-3 is a category dictionary. A model dictionary corresponding to F.

  Returning to FIG. 1, the model information providing unit 41 is a model dictionary stored in the model dictionary storage unit 42 and registered in the model dictionary identified by the model dictionary identification number included in the model information request information. Based on the model image information, a search object in the object image included in the model information request information is recognized using an image recognition engine identified by an image recognition engine identification number included in the model information request information. In other words, the model information providing unit 41 identifies the model closest to the search object in the object image among the models whose model image information and model information are registered in the model dictionary specified by the model information request information. . The model information providing unit 41 transmits the model information and model number related to the identified model to the requesting terminal 11 via the network 21.

  FIG. 4 is a block diagram illustrating an example of a functional configuration of the terminal 11 of FIG. The terminal 11 is configured to include an image input unit 61, an operation unit 63, a display unit 62, and a service processing unit 64. The service processing unit 64 includes an index list acquisition unit 71, an image acquisition unit 72, an index information detection unit 73, a transmission server determination unit 74, a model information request unit 75, a model information reception unit 76, and a display control unit 77. Configured.

  The image input unit 61 includes, for example, a digital camera, a scanner, various image reproducing devices or image recording / reproducing devices (for example, a DVD (Digital Versatile Disc) player, a hard disk recorder, etc.), and various recording devices (for example, an image recording / reproducing device). , Hard disk drive, semiconductor memory, etc.). The image input unit 61 supplies an object image including the search object image to the image acquisition unit 72. Hereinafter, an example in which the image input unit 61 is a digital camera (for example, when the terminal 11 is a mobile phone, a digital camera provided in the mobile phone) will be described.

  The display unit 62 includes a display such as an LCD (Liquid Crystal Display) and a CRT (Cathode-ray Tube), and displays various types of information as text or images under the control of the display control unit 77.

  The operation unit 63 is a device for the user to perform operations on the terminal 11 and includes, for example, a keyboard, a mouse, a remote controller, a touch panel, a microphone, and the like. The operation content input by the user using the operation unit 63, more precisely, information indicating the operation content is supplied to the transmission server determination unit 74 and the display control unit 77.

  The service processing unit 64 is realized by, for example, a CPU (Central Processing Unit) (not shown) of the terminal 11 executing an application program for using the image search service, and various functions for using the image search service. I will provide a.

  The index list acquisition unit 71 transmits the index list request information to the index server 12 and receives the index list from the index server 12 via the network 21. The index list acquisition unit 71 supplies the received index list to the transmission server determination unit 74 and the display control unit 77.

  The image acquisition unit 72 supplies various images input from the image input unit 61 to the index information detection unit 73, the transmission server determination unit 74, or the display control unit 77 as necessary. For example, the image acquisition unit 72 supplies the object image input from the image input unit 61 to the transmission server determination unit 74 and the display control unit 77. For example, the image acquisition unit 72 supplies an image indicating index information (hereinafter referred to as an index image) input from the image input unit 61 to the index information detection unit 73 and the display control unit 77.

  The index information detection unit 73 detects the index information indicated by the index image by analyzing the index image. The index information detection unit 73 supplies the detected index information to the transmission server determination unit 74 and the display control unit 77.

  The transmission server determination unit 74 acquires information indicating the selection result of the image recognition engine and the model dictionary by the user from the operation unit 63. The transmission server determination unit 74 is the position information of the information providing server 13 having the image recognition engine and model dictionary indicated by the index information detected by the index information detection unit 73 or the image recognition engine and model dictionary specified by the user. Is searched from the index list. The transmission server determination unit 74 includes the position information of the information providing server 13 searched from the index list, the object image, and the image recognition engine identification number and model dictionary identification indicated by the index information detected by the index information detection unit 73. The number, or the image recognition engine identification number and model dictionary identification number of the image recognition engine and model dictionary specified by the user are supplied to the model information requesting unit 75.

  The model information request unit 75 includes the object image, the image recognition engine identification number, and the model dictionary identification number supplied from the transmission server determination unit 74, and requests transmission of model information related to the search object in the object image. The model information request information is transmitted via the network 21 to the information providing server 13 at the position indicated by the position information supplied from the transmission server determination unit 74.

  The model information receiving unit 76 receives model information from the information providing server 13 that has requested transmission of model information via the network 21. The model information receiving unit 76 supplies the received model information to the display control unit 77.

  The display control unit 77 controls the display unit 62 to display various images.

  FIG. 5 is a block diagram illustrating an example of a functional configuration of the model information providing unit 41 of the information providing server 13 of FIG. The model information providing unit 41 includes a model information request receiving unit 101, a model dictionary determining unit 102, an image recognizing unit 103, a model information acquiring unit 104, and a model information transmitting unit 105.

  The model information request receiving unit 101 receives model information request information from the terminal 11 via the network 21 and supplies the received model information request information to the model information determining unit 102.

  The model dictionary determining unit 102 receives the image recognition engine identification number included in the model information request information, the information for instructing the use of the model dictionary and the image recognition engine identified by the model dictionary identification number, and the object image as the image recognition unit 103. To supply.

  The image recognition unit 103 is a model dictionary stored in the model dictionary storage unit 42, and uses the image recognition engine instructed to use based on the model image information registered in the model dictionary instructed to use. To recognize the search object in the object image. In other words, the image recognition unit 103 identifies the model closest to the search object in the object image from among the models whose model image information and model information are registered in the designated model dictionary. The image recognition unit 103 supplies the model number of the identified model and information indicating the used model dictionary to the model information acquisition unit 104.

  The model information acquisition unit 104 acquires model information related to the model specified by the image recognition unit 103 from the model dictionary used by the image recognition unit 103. The model information acquisition unit 104 supplies the acquired model information and model number to the model information transmission unit 105.

  The model information transmission unit 105 transmits the acquired model information and model number to the requesting terminal 11 of the model information via the model network 21 as information related to the search object.

  Next, processing executed by the information processing system 1 will be described with reference to FIGS.

  First, the model information acquisition process executed by the terminal 11 will be described with reference to the flowchart of FIG. This process is started, for example, when the user operates the operation unit 63 to cause the terminal 11 to execute an application program for using the image search service.

  In step S1, the index list acquisition unit 71 requests transmission of an index list. Specifically, the index list acquisition unit 71 transmits the index list request information via the network 21 to the index server 12 in which position information (for example, IP address, URL, etc.) is recorded in advance in the application program. .

  The index server 12 receives the index list request information in step S21 of FIG. 8 to be described later, and in step S22, transmits the index list to the requesting terminal 11 corresponding to the index list request information.

  In step S2, the index list acquisition unit 71 receives the index list. Specifically, the index list acquisition unit 71 receives an index list from the index server 12 via the network 21. The index list acquisition unit 71 supplies the received index list to the transmission server determination unit 74 and the display control unit 77.

  In step S3, the display unit 62 displays the input guidance for the object image. Specifically, the display unit 62 guides the user to input an object image including an image of a search object for which relevant information is desired to be acquired under the control of the display control unit 77 (for example, a guidance that prompts shooting of the search object). Is displayed.

  In step S4, the image acquisition unit 72 acquires an object image. Specifically, for example, the user captures the search object by the image input unit 61 that is a digital camera in accordance with the object image input guidance. The image input unit 61 supplies an object image including the search object image to the image acquisition unit 72. The image acquisition unit 72 supplies the acquired object image to the transmission server determination unit 74 and the display control unit 77.

  For example, when the user wants to acquire information related to a product (search object) published in the May issue of magazine A, the user takes a picture with the image input unit 61. The image input unit 61 supplies an image (object image) taken by the user to the image acquisition unit 72.

  In addition to capturing a search object with a digital camera and acquiring an object image, for example, a photograph of the search object is read by a scanner, or an image including the search object image is recorded on a recording medium. It can also be acquired from various contents (for example, electronic books, video contents, image files, etc.) provided by other information processing apparatuses.

  In step S5, the display unit 62 displays an image recognition engine and model dictionary selection menu. Specifically, the display control unit 77 generates an image for displaying the selection menu based on the index list and the object image. The display control unit 77 supplies the generated image to the display unit 62 and causes the display unit 62 to display a selection menu.

  FIG. 7 is a diagram illustrating an example of a selection menu displayed on the display unit 62. In the example shown in FIG. 7, an object image 131, an image recognition engine selection list 132, a model dictionary selection list 133, and a search button 134 are displayed in the selection menu. The object image 131 is the object image acquired in step S4, and includes the image of the search object 131A.

  The image recognition engine selection list 132 is a list of image recognition engines that can be used by the user. Specifically, the image recognition engine selection list 132 displays a list of text for the image recognition engine indicated in the index information included in the index list acquired in step S2. In the example of FIG. 7, the user recognizes an object in a photo (image) published in the catalog B, an image recognition engine specialized in recognizing an object in a photo (image) published in the magazine A. An image recognition engine specialized in recognition, an image recognition engine specialized in recognition of products provided by a predetermined service C (for example, a liquor delivery service), and a recognition of products of company D The image recognition engine can be selected from the four types of image recognition engines.

  The model dictionary selection list 133 is a model dictionary that can be used by the user, and displays a list of model dictionaries corresponding to the image recognition engine selected by the user from the image recognition engine selection list 132. . Specifically, the model dictionary selection list 133 includes an image recognition engine selected from the image recognition engine selection list 132 among the texts for the model dictionary indicated in the index information included in the index list acquired in step S2. The model dictionary text of the model dictionary corresponding to or a combination thereof is displayed as a list. FIG. 7 shows an example in which an image recognition engine specialized for recognizing an object in a photograph (image) published in magazine A is selected from the image recognition engine selection list 132. The model dictionary for searching for model information is a model dictionary corresponding to the May issue of Magazine A, a model dictionary corresponding to the June issue, a model dictionary corresponding to the July issue, and a May to July issue. Any one of all model dictionaries can be specified.

  In step S6, the transmission server determination unit 74 acquires the selection result of the image recognition engine and the model dictionary. Specifically, when the user selects the image recognition engine and model dictionary to be used from the selection menu by operating the operation unit 63 and then presses the search button 134, the operation unit 63 displays the image Information indicating the selection result of the recognition engine and the model dictionary is supplied to the transmission server determination unit 74.

  For example, when the user inputs an object image obtained by shooting the photograph published in the May issue of magazine A with the image input unit 61 to the image acquisition unit 72, the user selects the image recognition engine by operating the operation unit 63. After selecting the magazine A from the list 132 and selecting the May issue from the model dictionary selection list 133, the search button 134 is pressed. The operation unit 63 is information indicating the selection result of the image recognition engine and the model dictionary, specifically, information indicating the image recognition engine text “magazine A” and the model dictionary text “May issue” selected by the user. To the transmission server determination unit 74.

  In step S7, the transmission server determination unit 74 determines a request destination of model information. The transmission server determination unit 74 determines the image recognition engine and model dictionary (designated by the user) indicated in the acquired selection result as the image recognition engine and model dictionary used for searching for model information. Further, the transmission server determining unit 74 searches the index list for the position information of the information providing server 13 having the image recognition engine and the model dictionary specified by the user, and the information at the position indicated by the searched position information. The providing server 13 is determined as a request destination of model information. The transmission server determination unit 74 uses the object information, the image recognition engine identification number and model dictionary identification number of the image recognition engine and model dictionary specified by the user, and the position information of the information providing server 13 retrieved from the index list as model information. This is supplied to the request unit 75.

  In step S8, the model information requesting unit 75 requests transmission of model information. Specifically, the model information request unit 75 includes the image recognition engine identification number and the model dictionary identification number supplied from the transmission server determination unit 74, and the model information related to the search object in the object image, including the object image. The information providing server 13 at the position indicated by the position information supplied from the transmission server determination unit 74, that is, the information provision having the image recognition engine and the model dictionary specified by the user It transmits to the server 13 via the network 21.

  The information providing server 13 of the model information request destination receives the model information request information in step S41 of FIG. 9 described later, and in step S45, the model information related to the search object and the model information request information corresponding to the model information request information. The model number is transmitted to the terminal 11 that requested the model information.

  In step S9, the model information receiving unit 76 receives model information. Specifically, the model information receiving unit 76 receives model information and a model number related to the search object from the information providing server 13 that has requested transmission of model information via the network 21. The model information receiving unit 76 supplies the received model information and model number to the display control unit 77.

  In step S10, the display unit 62 displays model information, and the model information acquisition process ends. Specifically, the display unit 62 displays the acquired model information based on the control of the display control unit 77. For example, the display unit 62 displays search object attributes, a website from which information related to the search object can be acquired, an image of the search object, and the like.

  Next, an index list providing process executed by the index server 12 requested to transmit the index list will be described in correspondence with the model information acquisition process of FIG. 6 by the terminal 11 with reference to the flowchart of FIG.

  In step S21, the index list providing unit 31 receives the index list request information. Specifically, the index list providing unit 31 receives the index list request information transmitted from the terminal 11 in step S1 of FIG.

  In step S22, the index list providing unit 31 transmits the index list, and the index list providing process ends. Specifically, the index list providing unit 31 transmits the index list stored in the index list storage unit 32 to the terminal 11 that has requested the index list via the network 21.

  Next, with reference to the flowchart of FIG. 9, the model information provision process executed by the information provision server 13 requested to transmit the model information in response to the model information acquisition process of FIG. .

  In step S41, the model information request receiving unit 101 receives model information request information. Specifically, the model information request receiving unit 101 receives the model information request information transmitted from the terminal 11 in step S8 in FIG. The model information request receiving unit 101 supplies the received model information request information to the model dictionary determining unit 102.

  In step S42, the model dictionary determining unit 102 determines a model dictionary to be used. Specifically, the model dictionary determination unit 102 uses a model dictionary and an image used for searching for model information based on a model information identification number (specified by the user) and an image recognition engine identification number included in the model information request information. Determine the recognition engine. The model dictionary determination unit 102 supplies the image recognition unit 103 with information for instructing to search model information using the determined image recognition engine and model dictionary, and the object image.

  In step S43, the image recognition unit 103 recognizes the search object. Specifically, the image recognition unit 103 recognizes a search object in an object image using an image recognition engine specified by the user based on model image information registered in a model dictionary specified by the user. To do. As a result, the model closest to the search object is specified from the models whose model image information and model information are registered in the model dictionary designated by the user. The image recognition unit 103 supplies the model number of the identified model and information indicating the used model dictionary to the model information acquisition unit 104. Note that the image recognition method used at this time is not limited to a specific method, and it is desirable to use a method capable of recognizing a search object at higher speed and more accurately.

  In step S44, the model information acquisition unit 104 acquires model information. Specifically, the model information acquisition unit 104 acquires model information related to the model specified by the search object from the model dictionary used by the image recognition unit 103. The model information acquisition unit 104 supplies the acquired model information and model number to the model information transmission unit 105.

  In step S45, the model information transmission unit 105 transmits the model information, and the model information provision process ends. Specifically, the model information transmission unit 105 transmits the acquired model information and model number as information related to the search object to the terminal 11 that has requested the model information via the network 21.

  As described above, the user simply designates the image recognition engine and the model dictionary used for searching the model information, and transmits the object image including the image of the search object to the information providing server 13. Model information to be acquired. In addition, since model dictionaries for searching for model information are limited and an image recognition engine specialized for each model dictionary is used, searching for model information becomes more accurate and faster. Therefore, the user can acquire desired information more quickly. Further, by appropriately updating the index list, a model dictionary that can acquire the latest information is selected, and the latest information can be acquired.

  If the number of image recognition engines that use each model dictionary is limited to one for each model dictionary, the image recognition engine is also determined once the model dictionary is determined. It is not necessarily included in the information.

  For example, the selection menu displayed in step S5 of FIG. 6 described above may be as shown in FIG. The selection menu shown in FIG. 10 is similar to the selection menu shown in FIG. 7 in that the object image 131 and the search button 134 are displayed, and the image recognition engine selection list 132 and the model dictionary selection list 133 are displayed. Instead, the link destination list 141 is displayed. In each item of the linked list 141, text indicating an image recognition engine and a model dictionary used for searching for model information is displayed, and each text includes an information providing server 13 having the image recognition engine and the model dictionary indicated in the text. A hyperlink indicating the position information is embedded. The user can designate the information providing server 13, the image recognition engine, and the model dictionary used for searching for model information by selecting a desired item in the linked list 141 using the operation unit 63. it can.

  FIG. 11 is a diagram illustrating an example of the data structure of the index information when the selection menu of FIG. 10 is displayed. Compared with the index information shown in FIG. 2, the index information shown in FIG. 11 is common in that it includes information providing server location information, an image recognition engine identification number, and a model dictionary identification number. The difference is that the image recognition engine text and the model dictionary text are not included. In the link character string, text displayed in each item of the link destination list 141 of the selection menu in FIG. 10 is set.

  For example, index information having a data structure shown in FIG. 2, for example, indicating the acquisition destination of model information related to an object in an image published in the magazine or catalog is attached to a cover such as a magazine or catalog. The index image shown may be printed and the index information may be detected from the index image. A model information acquisition process executed by the terminal 11 when the index information is acquired from the index image will be described with reference to the flowchart of FIG. This process is started, for example, when the user operates the operation unit 63 to cause the terminal 11 to execute an application program for using the image search service.

  In step S61, the object image input guidance is displayed on the display unit 62 in the same manner as in step S3 of FIG.

  In step S <b> 62, the image acquisition unit 72 acquires an object image from the image input unit 61 in the same manner as in step S <b> 4 of FIG. 6 described above. The image acquisition unit 72 supplies the acquired object image to the transmission server determination unit 74 and the display control unit 77.

  In step S63, the display unit 62 displays index image input guidance. Specifically, under the control of the display control unit 77, the display unit 62, for example, an index printed on the cover or back cover of a magazine or catalog containing the page where the object image was taken in step S62. A guidance for prompting input of an image (for example, an identification code such as a barcode or a two-dimensional code) is displayed.

  In step S64, the image acquisition unit 72 acquires an index image. Specifically, for example, the user takes an index image using the image input unit 61 that is a digital camera. The image input unit 61 supplies the captured index image to the index information detection unit 73 and the display control unit 77.

  In step S65, the index information detection unit 73 detects index information. Specifically, the index information 73 detects the index information indicated in the index image by analyzing the index image. For example, when the index information has the data structure shown in FIG. 2, information providing server location information, image recognition engine identification number, image recognition engine text, model dictionary identification number, and model dictionary text are detected. The index information detection unit 73 supplies the detected index information to the transmission server determination unit 74 and the display control unit 77.

  The display unit 62 displays, for example, a search screen shown in FIG. 13 under the control of the display control unit 77. In the search screen of FIG. 13, an object image 151, an index image 152, a character string 153 including text for an image recognition engine and text for a model dictionary indicated by index information detected from the index image, and a search button 154 are displayed. Is done.

  In step S66, the transmission server determination unit 74 determines a request destination of model information. Specifically, when the user presses the search button 154 on the search screen in FIG. 13 by operating the operation unit 63, the operation unit 63 transmits information indicating that the search button 154 has been pressed. The data is supplied to the server determination unit 74. The transmission server determining unit 74 determines the image recognition engine and model dictionary indicated in the index information acquired in step S65 as the image recognition engine and model dictionary used for searching for model information. Further, the transmission server determining unit 74 determines the information providing server 13 at the position indicated by the position information in the index information as a request destination of the model information. The transmission server determining unit 74 supplies the model information requesting unit 75 with the object image, the image recognition engine identification number, the model dictionary identification number, and the position information of the information providing server 13 indicated in the index information.

  The processing in steps S67 to S69 is the same as the processing in steps S8 to S10 in FIG. 6 described above, and the description thereof will be omitted because it is repeated.

  Note that the model information providing process performed by the information providing server 13 corresponding to the model information obtaining process of FIG. 12 by the terminal 11 is the same as the model information providing process of FIG. 9 described above, and the description thereof will be repeated. It will be omitted.

  As a result, the model information related to the search object can be easily obtained without the user specifying the image recognition engine and model dictionary used for the search. For example, by capturing an arbitrary page of a catalog and an index image printed on the cover of the catalog and simply pressing the search button 154, information related to objects in the arbitrary page of the catalog is acquired. Can do.

  The index image is not limited to the above-described identification code, and the method is not particularly limited as long as it is an image that can indicate index information. Alternatively, the index information may be detected by recognizing the text content using a text image indicating the index information.

  Further, for example, an index image may be embedded as a watermark in each page of a magazine or a catalog, and the index information may be detected from the embedded watermark. In this case, the index image is included in the object image, the number of times the image is captured can be reduced, and the model information related to the search object can be acquired more easily.

  Further, the index image may be transmitted to another device such as the index server 12, and the index information may be detected by the other device.

  Further, the index image may be captured and analyzed by a dedicated device such as a barcode reader.

  Next, another embodiment of the present invention will be described with reference to FIGS.

  FIG. 14 is a block diagram showing an embodiment of an information processing system to which the present invention is applied. In the figure, portions corresponding to those in FIG. 1 are denoted by the same reference numerals or the last two digits, and the description of the portions having the same processing will be omitted because it will be repeated.

  Compared with the information processing system 1 of FIG. 1, the information processing system 201 is common in that the information providing servers 13-1 to 13-y are included, and instead of the terminals 11-1 to 11-x and the index server 12. , Terminal 211-1 to 211-x and index server 212 are different. The index server 212 is configured to include an index information providing unit 231 and an index dictionary storage unit 232. Further, the terminals 211-1 to 211-x, the index information providing unit 231 and the model information providing units 41-1 to 41-y are connected to each other via the network 21.

  Hereinafter, the terminals 211-1 to 211-x are simply referred to as the terminal 211 when it is not necessary to distinguish them individually.

  The terminal 211 is configured by, for example, a mobile phone, a personal computer, a PDA (Personal Digital Assistant), and the like. A user can use an image search service realized by the information processing system 201 by causing the terminal 211 to execute a predetermined application program.

  The terminal 211 includes an image (hereinafter, referred to as a group identification image) for identifying a group (hereinafter, referred to as an image group) to which the object image (more strictly speaking, an image in the object image) belongs. Index information request information for requesting transmission of index information having the data structure shown in FIG. 2, for example, indicating the acquisition source of the model information related to the object in the image belonging to the image group identified by the image is sent to the network 21. To the index information providing unit 231 of the index server 212.

  The image group is classified by, for example, the source of the image (for example, various books, electronic books, video contents, catalogs, image file types, etc.). For example, an image group of images published in the May issue of magazine A, an image group of images published in catalog C of genre B, and an image group of images included in title D of video content. being classified. All model information related to objects in images belonging to the same image group is registered in the same model dictionary. For example, the model information related to the object in the image belonging to the May magazine image group of magazine A (the image published in the May issue of magazine A) is the same model dictionary (for example, the model dictionary 42A in FIG. 3). -1).

  Note that model information related to objects in images belonging to a plurality of image groups may be registered in one model dictionary. For example, in one model dictionary, model information related to an object in an image belonging to the May magazine image group of magazine A and a model related to an object in an image belonging to the June magazine image group of magazine A Information may be registered.

  The group identification image is, for example, an image representing an image group identified by the group identification image, and is, for example, an image of a book such as a magazine or a catalog or a cover of a document.

  As will be described later with reference to FIG. 19, the index information providing unit 231 obtains index information corresponding to the group identification image from the index information registered in the index dictionary stored in the index dictionary storage unit 232. Search for. The index information providing unit 231 transmits the searched index information to the requesting terminal 211 via the network 21.

  The index dictionary indicates the group identification image information regarding the group identification images of various image groups and the acquisition source of the model information related to the object in the image belonging to the image group identified by the group identification image, for example, as shown in FIG. This is a dictionary in which index information having a data structure is registered in association with each other. The group identification image information includes, for example, information indicating the group identification image itself or a feature amount extracted from the group identification image.

  That is, by referring to the index dictionary, a model dictionary in which model information related to an object in an image belonging to an image group identified by the group identification image is registered, an image recognition engine using the model dictionary, and The information providing server having the model dictionary and the image recognition engine can be specified.

  Further, the terminal 211 includes model images request information for requesting transmission of model information relating to the search object in the object image, including the object image and the group identification image, to the index information providing unit 231 of the index server 212 via the network 21. To send.

  As will be described later with reference to FIG. 21, the index information providing unit 231 obtains index information corresponding to the group identification image from the index information registered in the index dictionary stored in the index dictionary storage unit 232. Search for. The index information providing unit 231 adds the image recognition engine identification number and the model dictionary identification number indicated in the retrieved index information to the model information request information and deletes the group identification image from the model information request information. The index information providing unit 231 transmits the changed model information request information to the model information providing unit 41 of the information providing server 13 at the position indicated by the position information in the searched index information via the network 21. .

  FIG. 15 is a block diagram illustrating an example of a functional configuration of the terminal 211 in FIG. In the figure, parts corresponding to those in FIG. 4 are denoted by the same reference numerals or the last two digits with the same reference numerals, and description of parts having the same processing will be omitted because it will be repeated.

  The terminal 211 is common to the terminal 11 of FIG. 4 in that an image input unit 61, a display unit 62, and an operation unit 63 are provided. A service processing unit 264 is used instead of the service processing unit 64. Different points are provided. Further, the service processing unit 264 includes an image acquisition unit 272, a transmission server determination unit 274, a model information request unit 275, a model information reception unit 276, a display control unit 277, an index information request unit 281 and an index information reception unit 282. Configured to include.

  The operation content input by the user using the operation unit 63, more precisely, information indicating the operation content is supplied to the display control unit 277 or the index information request unit 281.

  The image acquisition unit 272 supplies various images input from the image input unit 61 to the index information detection unit 73, the transmission server determination unit 74, the model information request unit 275, or the display control unit 77 as necessary. To do. For example, the image acquisition unit 272 supplies the object image input from the image input unit 61 to the transmission server determination unit 274, the model information request unit 275, or the display control unit 277. For example, the image acquisition unit 272 supplies the group identification image input from the image input unit 61 to the model information request unit 275 or the index information request unit 281.

  When acquiring the index information from the index information receiving unit 282, the transmission server determining unit 274 models the object image, the image recognition engine identification number indicated in the index information, the model dictionary identification number, and the location information of the information providing server 13 as a model. The information is supplied to the information request unit 275.

  When the model information requesting unit 275 acquires the object image, the image recognition engine identification number, the model dictionary identification number, and the position information of the information providing server 13 from the transmission server determining unit 274, the model information requesting unit 275 The model information request information including the number and the model dictionary identification number and requesting transmission of the model information related to the search object in the object image to the information providing server 13 at the position indicated by the acquired position information. 21 to transmit.

  Further, when the model information requesting unit 275 acquires the object image and the group identification image from the image acquisition unit 272, the model information requesting unit 275 includes the object image and the group identification image and requests model information related to the search object in the object image to be transmitted. The request information is transmitted to the index server 212 via the network 21.

  The index information request unit 281 includes index information request information for requesting transmission of index information indicating an acquisition destination of model information related to an object in an image belonging to a group identified by the group identification image. The data is transmitted to the index server 212 via the network 21.

  The index information receiving unit 282 receives index information from the index server 212 via the network 21. The index information receiving unit 282 supplies the received index information to the transmission server determining unit 274.

  FIG. 16 is a block diagram showing details of the index information providing unit 231 in FIG. The index information providing unit 231 is configured to include a receiving unit 301, an image recognizing unit 302, an index information acquiring unit 303, an index information transmitting unit 304, and a model information request transferring unit 305.

  The receiving unit 301 receives index information request information or model information request information from the terminal 211 via the network 21, and supplies the received index information request information or model information request information to the image recognition unit 302.

  When the image recognition unit 302 acquires the index information request information from the reception unit 301, the group identification image registered in the index dictionary stored in the index dictionary storage unit 232 will be described later with reference to FIG. Based on the information, the group identification image included in the index information request information is recognized. In other words, the image recognition unit 302 identifies a group identification image corresponding to the group identification image transmitted from the terminal 211 among the group identification images whose group identification image information is registered in the index dictionary. The image recognition unit 302 supplies index information request information and information indicating the identified group identification image to the index information acquisition unit 303.

  Further, when the model recognition request information is received from the reception unit 301, the image recognition unit 302, among the group identification images whose group identification image information is registered in the index dictionary, as will be described later with reference to FIG. A group identification image corresponding to the group identification image included in the model information request information is specified. The image recognition unit 302 supplies the model information request information and information indicating the group identification image specified to correspond to the group identification image to the index information acquisition unit 303.

  The index information acquisition unit 303 acquires index information associated with the identified group identification image from the index dictionary stored in the index dictionary storage unit 232. The index information acquisition unit 303 supplies the acquired index information to the index information transmission unit 304 or the model information request transfer unit 305. Further, when the model information request information is acquired from the image recognition unit 302, the index information acquisition unit 303 supplies the model information request information to the model information request transfer unit 305.

  The index information transmission unit 304 transmits the acquired index information to the terminal 211 that requested the index information via the network 21.

  The model information request transfer unit 305 adds the image recognition engine identification number and the model dictionary identification number indicated in the acquired index information to the model information request information in the acquired model information request information. Delete the image. The model information request transfer unit 305 transmits the model information request information whose contents are changed via the network 21 to the information providing server 13 at the position indicated by the position information in the acquired index information.

  Next, processing executed by the information processing system 201 will be described with reference to FIGS. 17 to 19.

  First, the model information acquisition process executed by the terminal 211 will be described with reference to the flowchart of FIG. This process is started when, for example, the user operates the operation unit 63 to cause the terminal 211 to execute an application program for using the image search service.

  In step S101, the object image input guidance is displayed on the display unit 62 in the same manner as in step S3 of FIG.

  In step S102, the image acquisition unit 272 acquires an object image from the image input unit 61 in the same manner as the processing in step S4 of FIG. The image acquisition unit 272 supplies the acquired object image to the transmission server determination unit 274 and the display control unit 277.

  In step S103, the display unit 62 displays group guidance image input guidance. Specifically, under the control of the display control unit 277, for example, the display unit 62 inputs a group identification image that is an image of a cover such as a magazine or a catalog that is the source of the object image acquired in step S102. Display prompting guidance.

  In step S104, the image acquisition unit 272 acquires a group identification image. Specifically, for example, the user photographs a cover such as a magazine or a catalog designated by the input guidance by the image input unit 61 that is a digital camera. The image input unit 61 supplies the captured image (group identification image) to the image acquisition unit 272. The image acquisition unit 272 supplies the acquired group identification image to the index information request unit 281 and the display control unit 277.

  The display unit 62 displays, for example, a search screen shown in FIG. 18 under the control of the display control unit 277. An object image 351, a group identification image 352, and a search button 354 are displayed on the search screen in FIG.

  In step S105, the index information request unit 281 requests transmission of index information. Specifically, when the user presses the search button 353 on the search screen of FIG. 18 by operating the operation unit 63, the operation unit 63 displays information indicating that the search button 353 is pressed as an index. The information is supplied to the information request unit 281. The index information request unit 281 generates index information request information including the acquired group identification image, and transmits the index information request information to the index server 212 via the network 21.

  The index server 212 receives the index information request information in step S121 of FIG. 19 to be described later. In step S124, the index server 212 retrieves the index information searched based on the group identification image included in the index information request information to the requesting terminal 211. Send.

  In step S106, the index information receiving unit 282 receives the index information. Specifically, the index information receiving unit 282 receives index information from the index server 212 via the network 21. The index information receiving unit 282 supplies the received index information to the transmission server determining unit 274.

  In step S107, the transmission server determination unit 274 determines a request destination of model information. Specifically, the transmission server determination unit 274 determines the image recognition engine and model dictionary indicated in the acquired index information as the image recognition engine and model dictionary used for searching for model information. In addition, the transmission server determination unit 274 registers model information related to the information providing server 13 at the position indicated by the position information in the index information, that is, the object in the image belonging to the image group identified by the group identification image. The information providing server 13 having the model dictionary is determined as a model information request destination. The transmission server determining unit 274 supplies the model information requesting unit 275 with the object image, the image recognition engine identification number indicated in the index information, the model dictionary identification number, and the location information of the information providing server 13.

  The processing of steps S108 to S110 is the same as the processing of steps S8 to S10 of FIG. 6 by the terminal 11 described above, and the description thereof will be omitted because it is repeated. Note that the model information and model number related to the search object are acquired by the terminal 211 through the processing of steps S108 to S110.

  Next, an index information provision process executed by the index server 212 corresponding to the model information acquisition process of FIG. 17 by the terminal 211 will be described with reference to the flowchart of FIG.

  In step S121, the receiving unit 301 receives index information request information. Specifically, the receiving unit 301 receives the index information request information transmitted from the terminal 211 in step S105 of FIG. The reception unit 301 supplies the received index information request information to the image recognition unit 302.

  In step S122, the image recognition unit 302 specifies a group identification image. Specifically, the image recognition unit 302 uses the group identification image information registered in the index dictionary based on the group identification image information registered in the index dictionary stored in the index dictionary storage unit 232. Among the images, a group identification image corresponding to the group identification image included in the index information request information is specified. Thereby, the image group identified by the group identification image included in the index information request information is specified. The image recognition unit 302 supplies index information request information and information indicating the identified group identification image to the index information acquisition unit 303. Note that the image recognition method used at this time is not limited to a specific method, and it is desirable to use a method capable of recognizing the group identification image at higher speed and more accurately.

  In step S123, the index information acquisition unit 303 acquires index information. Specifically, the index information acquisition unit 303 acquires index information associated with the group identification image specified in step S122 from the index dictionary stored in the index dictionary storage unit 232. The index information acquisition unit 303 supplies the acquired index information to the index information transmission unit 304.

  In step S124, the index information transmitting unit 304 transmits the index information, and the index information providing process ends. Specifically, the index information transmitting unit 304 transmits the acquired index information to the requesting terminal 211 via the network 21.

  The model information providing process performed by the information providing server 13 corresponding to the model information obtaining process of FIG. 72 by the terminal 211 is the same as the model information providing process of FIG. 9 described above, and the description thereof will be repeated. It will be omitted.

  As described above, the terminal 211 can acquire the model information related to the search object without searching or detecting the index information. Further, for example, as described above, the search object in the object image can be obtained simply by transmitting an object image obtained by photographing an arbitrary page such as a magazine or a catalog, and an index information detection image obtained by photographing the cover of the magazine. Therefore, it is possible to provide a service that is easy for the user to understand and easy to operate.

  For example, a group identification image may be embedded as a watermark in each image. In this case, the group identification image is included in the object image, the number of times the image is captured can be reduced, and the model information related to the search object can be acquired more easily.

  Further, the object image and the group identification image are transmitted to the index server 212, and the information providing server 13 at the position indicated by the position information in the index information searched based on the group identification image, that is, the group identification image is used. The object image is transferred from the index server 212 to the information providing server 13 having the model dictionary in which the model information related to the object in the image belonging to the identified image group is registered, and the model information related to the search object is obtained. You may make it search. An example of processing in this case will be described with reference to FIGS.

  First, the model information acquisition process executed by the terminal 211 will be described with reference to the flowchart of FIG. This process is started when, for example, the user operates the operation unit 63 to cause the terminal 211 to execute an application program for using the image search service.

  In step S201, the object image input guidance is displayed on the display unit 62 in the same manner as in step S101 of FIG.

  In step S <b> 202, the image acquisition unit 272 acquires an object image from the image input unit 61 in the same manner as in step S <b> 102 of FIG. 17 described above. The image acquisition unit 272 supplies the acquired object image to the model information request unit 275 and the display control unit 277.

  In step S203, the input guidance for the group identification image is displayed on the display unit 62 in the same manner as in step S103 of FIG.

  In step S204, the image acquisition unit 272 acquires a group identification image by the same processing as in step S104 of FIG. 17 described above. The image acquisition unit 272 supplies the acquired group identification image to the model information request unit 275 and the display control unit 277. In addition, the above-described search screen shown in FIG. 18 is displayed on the display unit 62.

  In step S205, the model information request unit 275 requests transmission of model information. Specifically, the model information request unit 275 includes object images and group identification images, and sends model information request information for requesting transmission of model information related to search objects in the object images to the index server 212 and the network 21. To send through.

  The index server 212 receives the model information request information in step S221 of FIG. 21 described later, and transfers the model information request information to the information providing server 13 that actually has the model information in step S224. The information providing server 13 to which the model information request information is transferred transmits the requested model information to the requesting terminal 211 in step S245 of FIG.

  In step S206, the model information receiving unit 276 receives model information. Specifically, the model information receiving unit 276 receives model information from the information providing server 13 via the network 21. The model information receiving unit 276 supplies the received model information to the display control unit 277.

  In step S207, the model information is displayed on the display unit 62 as in the above-described process of step S10 of FIG. 6, and the model information acquisition process ends.

  Next, a model information request information transfer process executed by the index server 212 corresponding to the model information acquisition process of FIG. 20 by the terminal 211 will be described with reference to the flowchart of FIG.

  In step S221, the reception unit 301 receives model information request information. Specifically, the receiving unit 301 receives the model information request information transmitted from the terminal 211 in step S205 of FIG. 20 described above via the network 21. The reception unit 301 supplies the received model information request information to the image recognition unit 302.

  In step S222, a group identification image is specified in the same manner as in step S122 of FIG. The image recognition unit 302 supplies the model information request information and information indicating the identified group identification image to the index information acquisition unit 303.

  In step S223, index information is acquired in the same manner as in step S123 of FIG. 19 described above. The index information acquisition unit 303 supplies the model information request information and the acquired index information to the model information request transfer unit 305.

  In step S224, the model information request transfer unit 305 transfers the model information request information, and the model information request information transfer process ends. Specifically, the model information request transfer unit 305 adds the image recognition engine identification number and model dictionary identification number indicated in the acquired index information to the model information request information, and adds model information request information to the model information request information. Delete the group identification image from the request information. The model information request transfer unit 305 transmits the model information request information whose contents are changed via the network 21 to the information providing server 13 at the position indicated by the position information in the acquired index information.

  Next, referring to the flowchart of FIG. 22, in response to the model information request information transfer process of FIG. 21 by the index server 212, the model information provision process executed by the information provision server 13 to which the model information request information has been transferred. Will be explained.

  In step S241, the model information request receiving unit 101 receives model information request information. Specifically, the model information request receiving unit 101 receives the model information request information transferred by the index server 212 in step S224 of FIG. The model information request receiving unit 101 supplies the received model information request information to the model dictionary determining unit 102.

  The processing of steps S242 and S245 is the same as the processing of steps S42 to S45 of FIG. 9 described above, and the description thereof will be omitted because it will be repeated.

  In this way, the terminal 211 does not need to perform processing related to index information, and the processing of the terminal 211 is reduced.

  Note that each of the processes of the information processing system 1 shown in FIG. 1 and the information processing system 201 shown in FIG. 14 may be realized by one apparatus. FIG. 23 is a block diagram showing an embodiment of an information processing apparatus when the information processing system 1 of FIG. 1 is realized by a single apparatus, and FIG. 24 is an information processing system 201 of FIG. It is a block diagram which shows one Embodiment of information processing apparatus at the time of making it implement | achieve by one apparatus.

  23 includes an image input unit 61, a display unit 62, an operation unit 63, a service processing unit 64, an index list providing unit 31 and an index list storage unit 32 of the index server 12, The information providing servers 13-1 to 13-z are configured to include model information providing units 41-1 to 41-y and model dictionary storage units 42-1 to 42-y. The service processing unit 64, the index list providing unit 31, and the model information providing units 41-1 to 41-y are connected to each other via a bus 411.

  The information processing apparatus 421 in FIG. 24 includes an image input unit 61, a display unit 62, an operation unit 63, a service processing unit 264, an index information providing unit 231 and an index dictionary storage unit 232 of the index server 212, and The information providing servers 13-1 to 13-z are configured to include model information providing units 41-1 to 41-y and model dictionary storage units 42-1 to 42-y. The service processing unit 264, the index information providing unit 231, and the model information providing units 41-1 to 41-y are connected to each other via a bus 431.

  In the above description, an example in which only one index server 12 or index server 212 is provided has been described, but a plurality of index servers may be provided.

  Further, in the above description, an example in which the index list is distributed from the index server 12 to the terminal 11 has been shown. However, the index list can be distributed by another method. For example, a recording medium in which an index list is recorded can be distributed and the index list can be input to the terminal 11 via the recording medium.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a network or a recording medium into a general-purpose personal computer or the like.

  FIG. 25 is a diagram illustrating an internal configuration example of a general-purpose personal computer 900. A CPU (Central Processing Unit) 901 executes various processes according to a program stored in a ROM (Read Only Memory) 902 or a program loaded from a recording unit 908 to a RAM (Random Access Memory) 903. The RAM 903 also appropriately stores data necessary for the CPU 901 to execute various processes.

  The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output interface 905 is also connected to the bus 904.

  The input / output interface 905 includes an input unit 906 configured with buttons, switches, a keyboard, or a mouse, an output unit 907 configured with a display such as a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display), and a speaker. Further, a recording unit 908 composed of a hard disk or the like and a communication unit 909 composed of a modem or a terminal adapter are connected. The communication unit 909 performs communication processing via a network including the Internet.

  A drive 910 is connected to the input / output interface 905 as necessary, and a removable medium 911 made up of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted, and a computer program read therefrom is loaded. Are installed in the recording unit 908.

  As shown in FIG. 25, a recording medium that records a program that is installed in a computer and can be executed by the computer is distributed to provide a program to a user separately from the apparatus main body. Recording magnetic disk (including flexible disk), optical disk (including CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc)), magneto-optical disk (MD (Mini-Disc) (registered trademark) )), Or included in the ROM 903 or the recording unit 908 in which a program is provided that is provided to the user in a state of being incorporated in the apparatus main body in advance. Hard disk.

  In the present specification, the step of describing the program stored in the program recording medium is not limited to the processing performed in time series in the order described, but is not necessarily performed in time series. Or the process performed separately is also included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  Furthermore, the embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

It is a block diagram which shows one Embodiment of the information processing system to which this invention is applied. It is a figure which shows the example of the data structure of index information. It is a figure which shows the example of a model dictionary. It is a block diagram which shows the example of a function structure of the terminal of FIG. It is a block diagram which shows the example of a function structure of the model information provision part of FIG. It is a flowchart for demonstrating the model information acquisition process performed with the terminal of FIG. It is a figure which shows the example of a selection menu. It is a flowchart for demonstrating the index list provision process performed by the index server of FIG. It is a flowchart for demonstrating the model information provision process performed by the information provision server of FIG. It is a figure which shows the other example of a selection menu. It is a figure which shows the other example of the data structure of index information. It is a flowchart for demonstrating the other example of the model information acquisition process performed with the terminal of FIG. It is a figure which shows the example of a search screen. It is a block diagram which shows other embodiment of the information processing apparatus to which this invention is applied. It is a block diagram which shows the example of a function structure of the terminal of FIG. It is a block diagram which shows the example of a function structure of the index information provision part of FIG. It is a flowchart for demonstrating the model information acquisition process performed with the terminal of FIG. It is a figure which shows another example on a search screen. It is a flowchart for demonstrating the index information provision process performed by the index server of FIG. It is a flowchart for demonstrating the other example of the model information acquisition process performed with the terminal of FIG. It is a flowchart for demonstrating the model information request | requirement information transfer process performed by the index server of FIG. It is a flowchart for demonstrating the model information provision process performed by the information provision server of FIG. It is a block diagram which shows one Embodiment of the information processing apparatus to which this invention is applied. It is a block diagram which shows other embodiment of the information processing apparatus to which this invention is applied. And FIG. 11 is a block diagram illustrating an example of a configuration of a personal computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Information processing system, 11 Terminal, 12 Index server, 13 Information provision server, 21 Network, 31 Index list provision part, 32 Index list storage part, 41 Model information provision part, 42 Model dictionary storage part, 61 Image input part, 62 Display unit, 63 operation unit, 64 service processing unit, 71 index list acquisition unit, 72 image acquisition unit, 73 index information detection unit, 74 transmission server determination unit, 75 model information request unit, 76 model information reception unit, 77 display control Unit 101 model information request receiving unit 102 model dictionary determining unit 103 image recognition unit 104 model information obtaining unit 105 model information transmitting unit 201 information processing system 211 terminal 212 index server 231 Index information provision unit, 232 Index dictionary storage unit, 264 Service processing unit, 272 Image acquisition unit, 274 Transmission server determination unit, 275 Model information request unit, 276 Model information reception unit, 277 Display control unit, 281 Index information request unit , 282 Index information reception unit, 301 reception unit, 302 image recognition unit, 303 index information acquisition unit, 304 index information transmission unit, 305 model information request transfer unit

Claims (27)

In an information processing system for searching for information using a plurality of first dictionaries in which object image information related to an image of an object and related information related to the object are registered,
Image acquisition means for acquiring a first input image;
Dictionary determining means for determining a use dictionary which is the first dictionary used for searching for the related information related to an object in the first input image;
Object recognition means for recognizing the object in the first input image based on the object image information registered in the use dictionary;
An information processing system comprising: related information acquisition means for acquiring the related information related to the object recognized by the object recognition means from the usage dictionary. The information processing system according to claim 1, wherein the dictionary determining unit determines the first dictionary designated by a user as the use dictionary. The image acquisition means further acquires an index image indicating a registration dictionary that is the first dictionary in which the related information related to the object in the first input image is registered,
A detecting means for detecting the registered dictionary indicated in the index image;
The information processing system according to claim 1, wherein the dictionary determining unit determines the registered dictionary as the use dictionary. The image acquisition means further acquires a second input image for identifying a predetermined group to which the first input image belongs,
Index information indicating the first dictionary in which the identification image information related to the identification image for identifying the group and the related information related to the object in the image belonging to the group identified by the identification image are registered. Identification image specifying means for specifying the identification image corresponding to the second input image based on the identification image information registered in the second dictionary registered in association with
Index information acquisition means for acquiring the index information associated with the identification image specified by the identification image specifying means from the second dictionary;
The information processing system according to claim 1, wherein the dictionary determining unit determines the first dictionary indicated in the index information acquired by the index information acquiring unit as the use dictionary. In an information processing method for retrieving information using a plurality of dictionaries in which object image information related to an image of an object and related information related to the object are registered,
Control the acquisition of input images,
Determining a use dictionary which is the dictionary used for searching for the related information related to an object in the input image;
Recognizing the object in the input image based on the object image information registered in the use dictionary,
An information processing method including a step of controlling acquisition of the related information related to the recognized object from the use dictionary. Information processing for retrieving information using a plurality of dictionaries in which object image information related to an object image and related information related to the object are registered,
An image acquisition control step for controlling the acquisition of the input image;
A dictionary determining step of determining a use dictionary that is the dictionary used for searching for the related information related to an object in the input image;
Recognizing the object in the input image based on the object image information registered in the use dictionary, and an object recognition step;
A program that causes a computer to execute information processing including: a related information acquisition control step that controls acquisition of the related information related to the recognized object from the use dictionary. Image acquisition means for acquiring a first input image;
A first dictionary in which object image information related to an image of an object and related information related to the object are registered, and is used for searching for the related information related to the object in the first input image. Dictionary determining means for determining a use dictionary that is a dictionary of
First transmission means for transmitting information indicating the first input image and the usage dictionary to a dictionary holding device which is a first device having the usage dictionary;
A first receiving of the related information related to the object in the first input image recognized based on the object image information registered in the use dictionary and transmitted from the dictionary holding device. An information processing apparatus including a receiving means. Index list obtaining means for obtaining an index list that is a list of index information indicating the first dictionary and the first device having the first dictionary;
8. The information processing according to claim 7, wherein the dictionary determination unit determines the first dictionary specified by a user from the first dictionary indicated by the index information in the index list as the use dictionary. apparatus. The index information further indicates an image recognition engine that is hardware or software that performs image recognition using the first dictionary;
The first transmission means further includes information indicating a user-designated image recognition engine that is the image recognition engine designated by the user from among the image recognition engines indicated by the index information in the index list. To the holding device,
The first receiving means is the first input recognized by the user-designated image recognition engine based on the object image information registered in the use dictionary and transmitted from the dictionary holding device. The information processing apparatus according to claim 8, wherein the related information related to the object in the image is received. The image acquisition means further includes a registration dictionary that is the first dictionary in which the related information related to the object in the first input image is registered, and the first apparatus having the registration dictionary. Get an index image showing
The registration dictionary shown in the index image, and detecting means for detecting the first device having the registration dictionary,
The dictionary determining means determines the registered dictionary as the use dictionary,
8. The information processing according to claim 7, wherein the first transmission unit transmits information indicating the first input image and the use dictionary to the dictionary holding device which is the first device having the registration dictionary. apparatus. The information processing apparatus according to claim 10, wherein the index image is embedded as a watermark in the first input image. The information processing apparatus according to claim 10, wherein the index image is an image of an identification code. The index image further indicates an image recognition engine that is hardware or software that performs image recognition using the registered dictionary,
The first transmission means further transmits information indicating the image recognition engine to the dictionary holding device,
The first receiving means is the first input image recognized by the image recognition engine based on the object image information registered in the use dictionary and transmitted from the dictionary holding device. The information processing apparatus according to claim 10, wherein the related information related to the object within is received. The image acquisition means further acquires a second input image for identifying a predetermined group to which the first input image belongs,
Second transmission means for transmitting the second input image to a second device;
Registration that is the first dictionary in which the related information related to the object in the image belonging to the group identified based on the second input image transmitted from the second device is registered And a second receiving means for receiving index information indicating the first device having the dictionary and the registration dictionary,
The dictionary determining means determines the registered dictionary as the use dictionary,
8. The information processing according to claim 7, wherein the first transmission unit transmits information indicating the first input image and the use dictionary to the dictionary holding device which is the first device having the registration dictionary. apparatus. The index information further indicates an image recognition engine that is hardware or software that performs image recognition using the registered dictionary,
The first transmission means further transmits information indicating the image recognition engine to the dictionary holding device,
The first receiving means is the first input image recognized by the image recognition engine based on the object image information registered in the use dictionary and transmitted from the dictionary holding device. The information processing apparatus according to claim 14, wherein the related information related to the object within is received. Control the acquisition of input images,
Determining a use dictionary which is a dictionary in which object image information related to an image of an object and related information related to the object are registered, and is a dictionary used for searching the related information related to the object in the input image; ,
Controlling transmission of the input image and information indicating the use dictionary to a dictionary holding device which is a device having the use dictionary;
Information including a step of controlling reception of the related information related to the object in the input image recognized based on the object image information registered in the use dictionary and transmitted from the dictionary holding device Processing method. An image acquisition control step for controlling the acquisition of the input image;
A dictionary in which object image information related to an object image and related information related to the object are registered, and a use dictionary that is a dictionary used for searching for the related information related to the object in the input image is determined. A dictionary decision step;
A transmission control step for controlling transmission of information indicating the input image and the usage dictionary to a dictionary holding device which is a device having the usage dictionary;
A reception control step for controlling reception of the related information related to the object in the input image recognized based on the object image information registered in the use dictionary and transmitted from the dictionary holding device; A program that causes a computer to execute processing including In an information processing apparatus for retrieving information using a plurality of dictionaries in which object image information related to an image of an object and related information related to the object are registered,
Receiving means for receiving an input image and information specifying the dictionary to be used, transmitted from another device;
Object recognition means for recognizing an object in the input image based on the object image information registered in the designated dictionary which is the designated dictionary;
Related information acquisition means for acquiring the related information related to the object recognized by the object recognition means from the designated dictionary;
An information processing apparatus comprising: transmission means for transmitting the related information acquired by the related information acquisition means to the other apparatus. The receiving means further receives information indicating an image recognition engine that is hardware or software that performs image recognition using the designated dictionary,
The information processing apparatus according to claim 18, wherein the object recognition unit recognizes an object in the input image using the image recognition engine based on the object image information registered in the designated dictionary. In an information processing method for retrieving information using a plurality of dictionaries in which object image information related to an image of an object and related information related to the object are registered,
Control the reception of information specifying the input image and the dictionary to be used, transmitted from another device,
Recognizing an object in the input image based on the object image information registered in the designated dictionary which is the designated dictionary;
Controlling acquisition of the related information related to the recognized object from the designated dictionary;
An information processing method including a step of controlling transmission of the acquired related information to the other device. Information processing for retrieving information using a plurality of dictionaries in which object image information related to an object image and related information related to the object are registered,
A reception control step for controlling reception of information specifying an input image and the dictionary to be used, transmitted from another device;
An object recognition step for recognizing an object in the input image based on the object image information registered in the designated dictionary which is the designated dictionary;
A related information acquisition control step for controlling acquisition of the related information related to the recognized object from the designated dictionary;
And a transmission control step for controlling transmission of the acquired related information to the other device. Receiving means for receiving a second input image for identifying a predetermined group to which the first input image belongs, transmitted from the first device;
Identification image information relating to an identification image for identifying a group;
A first dictionary in which object image information related to an image of an object and related information related to the object are registered, and related information related to an object in an image belonging to the group identified by the identification image The identification image information registered in the second dictionary registered in association with the registered first dictionary and the index information indicating the second device having the first dictionary. Based on the identification image specifying means for specifying the identification image corresponding to the second input image,
An information processing apparatus comprising: acquisition means for acquiring the index information associated with the identification image specified by the identification image specifying means from the second dictionary. The information processing apparatus according to claim 22, further comprising a transmission unit that transmits the index information acquired by the acquisition unit to the first device. The receiving means further receives the first input image transmitted from the first device,
Transmitting means for transmitting to the second device indicated by the index information acquired by the acquiring means, information indicating the first input image and the first dictionary indicated by the index information; The information processing apparatus according to claim 22, including the information processing apparatus. Controlling reception of a second input image for identifying a predetermined group to which the first input image belongs, transmitted from the first device;
Identification image information relating to an identification image for identifying a group;
A first dictionary in which object image information related to an image of an object and related information related to the object are registered, and related information related to an object in an image belonging to the group identified by the identification image The identification image information registered in the second dictionary registered in association with the registered first dictionary and the index information indicating the second device having the first dictionary. Based on the identification image corresponding to the second input image,
An information processing method including a step of controlling acquisition of the index information associated with the identified identification image from the second dictionary. A reception control step for controlling reception of a second input image for identifying a predetermined group to which the first input image belongs, transmitted from the first device;
Identification image information relating to an identification image for identifying a group;
A first dictionary in which object image information related to an image of an object and related information related to the object are registered, and related information related to an object in an image belonging to the group identified by the identification image The identification image information registered in the second dictionary registered in association with the registered first dictionary and the index information indicating the second device having the first dictionary. An identification image specifying step for specifying the identification image corresponding to the second input image,
An acquisition control step of controlling acquisition of the index information associated with the identified identification image from the second dictionary.   27. A recording medium on which the program according to claim 6, 17, 21, or 26 is recorded.

Images