JP2018005931A - Information processing system, information processing method, and external system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to easily provide electronic information corresponding to a presented image.SOLUTION: An image search system includes: a storage unit for storing an object and cooperation information while associating the object with the cooperation information; a first reception unit for receiving a picked-up image; and a first transmission unit for transmitting the cooperation information associated with the object to a transmission source of the picked-up image on the basis of the object included in the picked-up image. A terminal device includes: an identification information acquisition unit for acquiring identification information that is information for identifying the terminal devices using the image search system, respectively; an imaging unit for imaging a subject and outputting the picked-up image; a second transmission unit transmitting the picked-up image output from the imaging unit to the image search system; a second reception unit for receiving the cooperation information transmitted from the image search system according to transmission of the picked-up image by the second transmission unit; and a third transmission unit for transmitting the identification information to an access destination when the terminal device executes access via a network on the basis of the cooperation information received by the second reception unit.SELECTED DRAWING: Figure 11

Description

  The present invention relates to an information processing system, an information processing method, and an external system that provide information according to an image.

  Conventionally, information is provided through various media. Various technologies for efficiently providing such information to users have been proposed. For example, an information provision management system for providing highly convenient support information to a user using an image processing technique and a mobile phone terminal has been proposed (for example, Patent Document 1).

  The support information providing system described in Patent Literature 1 includes a management computer, a part image data storage unit, and a help data storage unit. A user uses a mobile phone terminal to photograph a part of a copier that requests help information. The mobile phone terminal transmits a help request including the captured image data to the support information providing system. The management computer specifies the part image data recorded in the part image data storage unit based on the model identifier included in the help request and the captured image data. The management computer extracts support information and transmits help screen data to the mobile phone terminal.

  Also in the image search technique, an MMR (Mixed Media Reality) technique has been proposed as a technique for associating image information on paper media with electronic information (Non-Patent Document 1).

  The MMR technique described in Non-Patent Document 1 aims to connect a document on a paper medium and electronic information. In this MMR technique, there is no need to change the appearance or format of a document on a paper medium, and each link destination can be designated on a printed matter through a camera-equipped mobile phone terminal. In particular, in this MMR technology, even when OCR (Optical Character Recognition) is difficult for an image including text, the arrangement of partial text images (such as pitch and positional relationship) in a text document is used. It is possible to specify the original image.

  In addition, a technique for realizing high-speed search is proposed (Patent Document 2). With the technique of Patent Document 2, the search speed is increased on the premise that an object can be dynamically inserted. For this reason, spatial data is registered, a spatial index in M-tree format having a tree structure is generated in advance for the registered spatial data, and spatial data similar to the spatial data specified by the user is used as the spatial index. And search. In this case, the region of each node constituting the tree structure of the spatial index is made a region including all the spatial data existing below, thereby eliminating the overlap between the node regions and increasing the search speed. .

  Electronic information is suitable for processing and storage. On the other hand, information transmission by presenting images to the user, for example, information transmission using a paper medium such as a newspaper is also used in various situations. In addition, there are also information transmission media that transmit information to users, such as posters and electronic bulletin boards posted on places where an unspecified number of users come and go such as street corners and public facilities, posters and electronic bulletin boards using the walls of high-rise buildings, etc. Diversified. If there is a technology for linking such an information transmission medium and electronic information, it is expected that convenience for the user will be further improved. As an example of linking a paper medium and electronic information, it is conceivable to use a two-dimensional code in which link destination information on the Internet as electronic information is embedded on a paper medium.

  However, when printing a paper medium such as a newspaper, it takes time to determine and print an article or layout, and there is a problem that time constraints are large. In addition, the link destination often changes depending on the situation, but every time the change occurs, it is necessary to reflect the change on the information transmission medium side such as a paper medium or an electronic medium. There is a problem that the cost may increase.

  Moreover, when searching for electronic information using text, it is necessary to prepare an index based on text information in advance, which is troublesome.

  Furthermore, in the case of paper media, in order to save information, it has been conventionally performed to cut out a desired article in a newspaper article or the like and paste the article in a scrapbook or the like. However, such work is troublesome and has the disadvantage that the article on the back side cannot be read. It is also possible to scan newspaper articles and store information as image data. However, even in this case, there is a problem that operations such as scanning and storage are necessary, and it takes time and effort to extract and process only a desired article.

  The present invention has been made in view of the above, and an object of the present invention is to easily provide electronic information corresponding to a presented image.

  In order to solve the above-described problem and achieve the object, the present invention transmits one or more captured images obtained by capturing a medium using a terminal device and receives presentation of information associated with an object included in the medium. An image search system that is used by a user and one or more customers who present information to the user via a medium and that is configured by one or more information processing apparatuses, and an image search function by the image search system is used. An information processing system that provides a search service in cooperation with a terminal device on which a terminal program is installed via communication, and determines an object determined by a customer in response to a request from the customer The association information, which is linked information for accessing information held in an external system different from the information processing system, is stored in association with each other. An identification information acquisition unit realized by a terminal program installed in a terminal device, which is identification information for identifying a storage unit and a terminal device in which the terminal program is installed, and that acquires identification information generated in the information processing system And a first transmission unit that transmits the identification information acquired by the identification information acquisition unit to the image search system when the captured image is transmitted to the image search system and the image search function of the image search system is used. The object stored in the storage unit is searched based on the captured image transmitted from the terminal device for use of the function, and the cooperation information associated with the searched object is transmitted to the terminal device that is the transmission source of the captured image. Log information related to the use of the image search function and at least the image search function used by the first transmitter. A log recording unit for recording log information including the identification information in the image search system, and a terminal device equipped with a terminal program, when executing access via the network based on the cooperation information transmitted by the second transmission unit And a third transmitter for transmitting the identification information generated for the terminal device to the access destination.

  According to the present invention, it is possible to easily provide electronic information corresponding to a presented image.

FIG. 1 is a diagram illustrating a configuration of an example of a search system according to the embodiment. FIG. 2 is a diagram illustrating an example of image data that is a source of index data stored in the DB according to the embodiment. FIG. 3 is a functional block diagram illustrating an example of functions of the image search server according to the embodiment. FIG. 4 is a diagram illustrating an exemplary configuration of a DB according to the embodiment. FIG. 5 is a diagram illustrating an example of the XML code according to the embodiment. FIG. 6 is a flowchart illustrating an example of the operation of the image search server according to the embodiment. FIG. 7 is a flowchart illustrating an example of index collaboration information presentation processing according to the embodiment. FIG. 8 is a flowchart illustrating an example of processing when the user is prompted to perform processing for selecting 1 from a plurality of indexes according to the embodiment. FIG. 9 is a block diagram illustrating a configuration of an example of a terminal device according to the embodiment. FIG. 10 is a functional block diagram illustrating functions of an example of the terminal device according to the embodiment. FIG. 11 is a diagram schematically illustrating a configuration of an example of the information processing system according to the first embodiment. FIG. 12 is a flowchart illustrating generation processing of an example of an application ID according to the embodiment. FIG. 13 is a diagram schematically illustrating a configuration of an example of an information processing system according to the second embodiment. FIG. 14 is a flowchart illustrating a generation process of an example of an application ID according to the second embodiment. FIG. 15 is a diagram schematically illustrating a configuration of an example of an information processing system according to the third embodiment. FIG. 16 is a diagram illustrating a configuration of an example of a log information table according to the third embodiment. FIG. 17 is a diagram illustrating a configuration of an example of a cooperation information table according to the third embodiment. FIG. 18 is a block diagram illustrating an outline of a system that can realize the first application example. FIG. 19 is a diagram illustrating an example of grouping of content for members according to the second application example. FIG. 20 is a flowchart showing an example of processing in the content distribution site according to the third application example. FIG. 21 is a diagram for explaining the fourth application example. FIG. 22 is a flowchart showing an example of processing in the content distribution site according to the fourth application example. FIG. 23 is a diagram illustrating an example of a map used for orienteering. FIG. 24 is a flowchart showing an example of processing in the content distribution site according to the fifth application example.

  Exemplary embodiments of an information processing system and an information processing method will be described below in detail with reference to the accompanying drawings.

(Image search system applicable to the embodiment)
Prior to the description of the embodiment, an image search system applicable to the embodiment will be described in order to facilitate understanding. The image retrieval system extracts an object from a captured image captured by an imaging device, compares the extracted object with an object stored in advance in a database, and extracts an object from the captured image among objects stored in the database. An object having the highest similarity to the extracted object is specified. And the link information of the commercial site previously linked | related with the specified object etc. are shown to a user.

  FIG. 1 shows a configuration of an example of an image search system applicable to the embodiment. In the image search system 200 shown in FIG. 1, the image search server 11 and the terminal device 20 are connected to the network 10. The network 10 is, for example, the Internet, and TCP / IP (Transmission Control Protocol / Internet Protocol) is applied as a communication protocol. The network 10 may be a network cloud that includes a plurality of computers connected to each other via a network and is externally a network group in which only input / output is shown as a black box with the inside hidden.

  A plurality of commercial sites 30 </ b> A and 30 </ b> B are further connected to the network 10. These commercial sites 30 </ b> A and 30 </ b> B can ship products according to orders placed through the network 10.

  The terminal device 20 can communicate with the network 10 by wireless or wired. The terminal device 20 can perform an operation in accordance with a command received via the network 10, and can display an image received via the network 10 on a UI (user interface) including a display device and an input device. Data can be transmitted to the network 10 in accordance with the display or user operation performed on the UI.

  In addition, the terminal device 20 has an imaging function, and can capture an image of a subject in response to an imaging instruction operation on the UI. The terminal device 20 can transmit a captured image obtained by imaging toward the network 10.

  The image search server 11 is connected to a database (DB) 12 and can perform a search process according to the embodiment on the DB 12 in accordance with a request received via the network 10. In the embodiment, each index data for searching the plurality of image data 50A and 50B is stored in the DB 12 in association with the cooperation information. The index data is data obtained by analyzing the binary of the image data, and is data obtained by converting the image data into a structure for search.

  In addition, it is also possible to store image data directly in DB12. The file name and path of the image data may be stored in the DB 12.

  The image search server 11 receives image data via the network 10, searches the DB 12 based on the received image data, and acquires cooperation information corresponding to the image data as a search result. The image search server 11 returns the cooperation information acquired as the search result to the transmission source of the image data.

  FIG. 2 shows an example of the image data 50A and 50B that are the basis of the index data stored in the DB 12. Here, an object included in the image data is defined. An object refers to an image of a main object in an image based on image data, and is an image that is distinguished from a background image of the main object and accompanying information of the main object in the image. It is.

  As an example, it is assumed that the image data 50A and 50B are a content page that is the contents of a catalog magazine and a cover page that is the cover of the catalog magazine. 2A shows an example of image data 50A based on the content page image 130A, and FIG. 2B shows an example of image data 50B based on the cover page image 130B.

  FIG. 2A shows an example in which one object 131 is arranged at the center of the image 130A. In the image 130 </ b> A, a target area 134 including the object 131 is set for the object 131. FIG. 2A shows an example in which the ratio of the size of the object 131 to the size of the target area 134 is small. In the case of a catalog magazine, the purpose of the content page is to introduce the product, so the object 131 that is an image showing the product is displayed larger than the content page.

  FIG. 2B shows an example in which a plurality of objects 132 and 133 are arranged on the image 130B. Here, the object 132 is an image obtained by reducing the object 131 shown in FIG. 2A, and the object 133 is another image. More objects may be arranged on the image 130B. In this example, as the display page summarizes the contents of the catalog magazine, the objects 132 and 133 included in the content page are reduced and displayed as thumbnails. In the image 130B, the target area 135 includes the objects 132 and 133 and is set to cover the entire display page (image 130B).

  The target areas 134 and 135 are used as indices for recognizing the object 131 and the objects 132 and 133, respectively. The target area includes the entire object and can be set to any size smaller than the entire image including the object.

  The linkage information will be described. The cooperation information is, for example, information indicating a function that cooperates with an object included in an image, and more specifically, information including an access method for accessing attribute information of the object. Therefore, it can be considered that the cooperation information is actually stored in the DB 12 in association with the object included in the image.

  As an example, in the object 131 included in the image 130A of the content page, the linkage information includes, for example, the URL (Uniform Resource Locator) of the commercial site 30A that handles the product indicated by the object 131, or the URL of the purchase page that performs purchase settings for the product. It is. Not limited to these, as the linkage information, a telephone number or an e-mail address of a store handling the product may be used. Furthermore, information for downloading a data file including information on the product may be used as linkage information. In the DB 12, a plurality of pieces of cooperation information can be associated with one object.

  As another example, for each object 132 and 133 included in the cover page image 130B, for example, a URL of a site on the network 10 that introduces the catalog magazine itself including the cover page is used as linkage information. Can do.

  FIG. 3 is a functional block diagram illustrating an example of the functions of the image search server 11. The image search server 11 includes a reception unit 100, an extraction unit 101, a calculation unit 102, a search unit 103, a presentation unit 104, and a control unit 110. In FIG. 3, the DB 12 is shown to be included in the image search server 11. The control unit 110 controls the overall operation of the image search server 11. The image search server 11 includes one or more computers.

  The receiving unit 100 receives data transmitted via the network 10. For example, a captured image (captured image 40A) transmitted from the terminal device 20 is received by the receiving unit 100.

  The extraction unit 101 performs image analysis on the captured image 40A received by the reception unit 100, detects feature points from the captured image 40A, and extracts objects included in the captured image 40A. For example, the extraction unit 101 performs binarization processing on the captured image 40A, performs edge detection or normalization processing on the binarized captured image 40A, and detects feature points of the captured image 40A. . An object is represented by this detected feature point.

  The calculation unit 102 compares the feature points of the captured image 40A extracted by the extraction unit 101 with the feature points representing each object searched by each index stored in the DB 12. Based on the comparison result, the similarity between the object included in the captured image 40A and each object searched by each index stored in the DB 12 is calculated. The similarity can be obtained, for example, by comparing the relationship between each feature point extracted from the image for each image. As an example, a feature amount is calculated by quantifying the relationship between feature points, an absolute value of a difference between feature amounts is obtained between images, and when the obtained absolute value of the difference is smaller, the similarity between the images is Suppose it is expensive.

  Based on the similarity calculated by the calculation unit 102, the search unit 103 searches for an image having a high similarity to the object included in the captured image 40A among the objects searched by the indexes stored in the DB 12. . At this time, the search unit 103 outputs, as a search result, one index including an object having the highest similarity to each object included in the captured image 40A from each object searched by each index stored in the DB 12. .

  Not limited to this, the search unit 103 searches a predetermined number of indexes according to the order including objects having high similarity to the objects included in the captured image 40A from the objects searched by the indexes stored in the DB 12. It may be output as a result, or one or a plurality of indexes for searching for objects whose similarity is equal to or greater than a threshold value may be output as a search result. Furthermore, it is conceivable that an index whose similarity is within a predetermined range (such as 5%) from the maximum value of the similarity calculated for the captured image 40A may be output as a search result.

  The presenting unit 104 takes out the cooperation information associated with the index of the search output by the search unit 103 from the DB 12 and presents (transmits) it to the terminal device 20. Therefore, the presentation unit 104 has a function of a transmission unit that transmits cooperation information to the terminal device 20. The cooperation information may be presented by transmitting the cooperation information directly to the terminal device 20, or by generating a Web page including the cooperation information and accessing the Web page toward the terminal device 20 (URL Etc.) may be presented. By accessing this Web page from the terminal device 20, the cooperation information can be acquired.

  Not limited to this, application software (hereinafter abbreviated as an application) that performs a series of operations from capturing the captured image 40A to displaying the presented cooperation information is installed in advance in the terminal device 20 and presented. It is also possible to acquire the cooperation information using this application.

  The image search server 11 is a general computer device (information processing) including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and a storage device such as a hard disk drive. Device). Further, the image search server 11 is not limited to being configured by one computer device, and may be configured to operate in a distributed manner on a plurality of computer devices, for example.

  The reception unit 100, the extraction unit 101, the calculation unit 102, the search unit 103, the presentation unit 104, and the control unit 110 described above are realized by a search program that operates on the CPU. Not limited to this, a part or all of the reception unit 100, the extraction unit 101, the calculation unit 102, the search unit 103, the presentation unit 104, and the control unit 110 may be realized by using separate hardware that cooperates with each other. Good. The search program for executing the search process according to the embodiment is provided by being recorded in a computer-readable recording medium such as a CD or a DVD in an installable or executable format file.

  Furthermore, a search program for executing the search process according to the embodiment may be provided by being stored on a computer connected to a communication network such as the Internet and downloaded via the communication network. Further, a search program for executing the search processing according to the embodiment may be provided or distributed via a communication network such as the Internet.

  The search program for executing the search processing according to the embodiment includes, for example, a module configuration including the above-described units (reception unit 100, extraction unit 101, calculation unit 102, search unit 103, presentation unit 104, and control unit 110). As actual hardware, the CPU reads the search program from the storage device, for example, and executes it to load the above-described units onto the main storage device (for example, RAM), and the respective units are generated on the main storage device. It has become so.

  In such a configuration, the user captures, for example, a catalog magazine page using the terminal device 20, and obtains the captured image 40A and the captured image 40B. In this example, the captured images 40A and 40B are captured images obtained by capturing the content page and the cover page of the catalog magazine, respectively. For example, the user transmits the captured image 40 </ b> A to the image search server 11 via the network 10.

  The captured image 40A is received by the receiving unit 100 of the image search server 11, the feature point of the received captured image 40A is detected by the extracting unit 101, and an object is extracted. Further, the calculation unit 102 calculates the similarity between the object included in the captured image 40A and each object searched by each index stored in the DB 12, based on the feature points detected by the extraction unit 101. Based on this calculation result, the search unit 103 searches the index. Then, the presentation unit 104 extracts the cooperation information from the DB 12 based on the search output of the search unit, and presents it to the terminal device 20 that is the transmission source of the captured image 40A.

  Based on the cooperation information presented to the terminal device 20 from the image search server 11, the user can access the commercial site 30A, for example, and purchase a product indicated by the object included in the captured image 40A.

  Here, for example, it is assumed that an image including an object very similar to the object 131 included in the image 130A in FIG. 2A is captured by the terminal device 20 as the captured image 40A. The DB 12 stores an image 130A including the object 131 and an image 130B including an object 132 obtained by reducing the object 131.

  In this case, as a result of the search based on the similarity calculated from the feature point detection result of the captured image 40A, the image 130B may be searched instead of the image 130A. In this case, the user cannot obtain desired cooperation information. In order to prevent such a situation, in the embodiment, the search unit 103 searches for an image stored in the DB 12 based on the size ratio between the target area (imaging range) and the object.

  This will be described more specifically with reference to FIG. In the image 130A illustrated in FIG. 2A, the object 131 is arranged so as to occupy a large space with respect to the image 130A, and the target area 134 has a size slightly larger than the size of the object 131 with respect to the object 131. Is set. On the other hand, in the image 130B illustrated in FIG. 2B, the object 132 in which the object 131 is reduced is arranged, and the target area 135 is on the entire surface of the image 130B, that is, the size of the object 132. It is set with a sufficiently large size.

  In the image 130 </ b> A, it is assumed that the first ratio of the size of the object 131 to the size of the target area 134 is, for example, a value obtained based on each area is ½. On the other hand, in the image 130B, the value obtained by calculating the second ratio of the size of the object 132 to the size of the target region 135 as an area is, for example, 1/16, which is significantly different from the case of the image 130A. Note that the size ratio is not limited to the method of obtaining based on the area. Such information indicating the size ratio may be obtained in advance and included in an index and stored in the DB 12.

  Furthermore, for the captured image 40A received from the terminal device 20, it is conceivable to obtain a ratio between the size of the object extracted by the extraction unit 101 and the size of the captured image 40A itself. For example, assume that the captured image 40A is an image as shown by an image 130A in FIG. 2A, and the third ratio of the size of the object (referred to as the object 131) to the size of the captured image 40A is, for example, 1/4. .

  In this case, the third ratio is closer to the first ratio among the first ratio and the second ratio described above. Therefore, the search unit 103 outputs an index corresponding to the image 130A as a search result for the captured image 40A. The presentation unit 104 presents the cooperation information associated with the index corresponding to the image 130 </ b> A to the terminal device 20. Thus, by selecting an index based on the ratio of the object size to the image size, a desired index can be selected more accurately.

  FIG. 4 shows an example of the configuration of the DB 12 for realizing such a process of searching for an image based on the object size ratio. FIG. 4A shows an example of the index table 120 for identifying each index data stored in the DB 12. As illustrated in FIG. 4A, each index data is stored in the DB 12 in association with an ID and binary data of the index data.

  Further, the format of the ID is not particularly limited as long as each index data stored in the DB 12 can be identified. When each index data stored in the DB 12 is for searching for an object of each page of a catalog magazine, a value including a page number on which the object is posted can be used as an ID.

  FIG. 4B shows an example of a target area table 121 that associates an image ID with a target area. In this example, the target area is described using XML (Extensible Markup Language), which is a kind of markup language that defines and describes the meaning and structure of data using tags. As will be described later, the description of the linkage information is also embedded in the description of the target area. The XML code in which the target area is described may be stored directly in the field of the target area table 121, or the XML code is described in a separate file, and the file name and path are described in the field. May be.

FIG. 5 shows an example of the XML code 140 shown in FIG. The XML code 140 shown in FIG. 5 is an example of describing a target area of a content page having a catalog magazine. A page is defined by the portion sandwiched between the outermost tags “<page>” and “</ page>”. In each of the portions 141 ₁ and 141 ₂ , one target region is defined by the portion sandwiched between the tags “<hotspot>” and “</ hotspot>”. Thus, a plurality of target areas can be defined for one page.

In the portion 141 ₁ , the first line 142 indicates that the range in the page of the target area is, for example, the coordinates (x1, y1) of the upper left corner by the portion sandwiched between the tags “<area>” and “</ area>”. And the coordinates of the lower right corner (x2, y2).

In each of the portions 143 ₁ and 143 ₂ , _one piece of linkage information is defined by the portion sandwiched between the tags “<link>” and “</ link>”. In this way, a plurality of pieces of cooperation information can be defined for one target area. In each part 143 _1, the tag "<title>" and "</ title>" by a portion held in cooperation information title is defined. The type of the linkage information is defined by the portion sandwiched between the tags “<type>” and “</ type>”. Further, in this example, address information on the network 10 is defined as the linkage information by a portion sandwiched between tags “<URI>” and “</ URI>”.

  As described above, since the XML code 140 defines the target area in the page and the cooperation information in the target area, the function based on the cooperation information associated with the object included in the target area in the page can be executed. it can.

  FIG. 6 is a flowchart illustrating an example of the operation of the image search server 11 according to the embodiment. First, for example, a captured image 40 </ b> A obtained by being imaged by the terminal device 20 is transmitted to the image search server 11 via the network 10. In step S100, in the image search server 11, the receiving unit 100 receives the captured image 40A. In the next step S101, the extraction unit 101 performs image analysis on the received captured image 40A, detects feature points, and extracts objects.

  In the next step S102, the search unit 103 calculates the ratio of the object size to the size of the captured image 40A. Note that the ratio of the size of the object to the size of the target area is calculated in advance for each image stored in the DB 12. Information indicating the ratio of each image stored in the DB 12 can be embedded in the XML code 140 described above.

  In the next step S103, the calculation unit 102 compares the object extracted from the captured image 40A with the object searched by each index stored in the DB 12. More specifically, the calculation unit 102 calculates each degree of similarity between an object included in the captured image 40A and an object searched by each index stored in the DB 12.

  In the next step S104, the search unit 103 determines whether there is an index for searching for an object that matches the object included in the captured image 40A among the indexes stored in the DB 12, based on the similarity calculated in step S103. Determine whether. In this example, among objects searched for in the index included in the DB 12, an object having a similarity equal to or higher than a threshold is determined as an index for searching for an object that matches the object included in the captured image 40A. To do.

  If it is determined in step S104 that there is an index that matches the object, the search unit 103 shifts the process to step S105. In step S <b> 105, the search unit 103 determines whether or not the object matches the plurality of indexes among the indexes stored in the DB 12. If it is determined that only one index matches the object, the search unit 103 uses the one index as a search output and shifts the process to step S107 described later. On the other hand, when it is determined that the objects match with each other in the plurality of indexes, the search unit 103 shifts the processing to step S106.

  In step S106, the search unit 103 obtains the ratio of the size of the plurality of objects searched by each of the plurality of indexes to the size of the target area. The ratios related to the plurality of objects may be obtained in advance and stored in the DB 12. At the same time, the search unit 103 obtains the ratio of the object size to the size of the captured image 40A for the captured image 40A. Then, out of the ratios obtained for each of the plurality of objects, a ratio closest to the ratio obtained for the captured image 40A is obtained, an index for searching for an object corresponding to the obtained ratio is selected as a search output, The process proceeds to step S107.

  In step S <b> 107, the presentation unit 104 retrieves the cooperation information associated with the index of the search output by the search unit 103 from the DB 12. Then, the presentation unit 104 presents the extracted cooperation information to the terminal device 20 that is the transmission source of the captured image 40A. Then, a series of processes according to the flowchart of FIG.

  If it is determined in step S104 described above that there is no matching index, the process proceeds to step S108, and a message indicating that the matching index is not stored in the DB 12 is displayed on the terminal device 20 by the presentation unit 104. To be presented. Then, a series of processes according to the flowchart of FIG.

  FIG. 7 is a flowchart illustrating an example of index linkage information presentation processing in step S107 in FIG. First, in step S200, the presentation unit 104 specifies a target area corresponding to the captured image 40A in the object searched by the search output index of the search unit 103. In the next step S201, the presentation unit 104 refers to the DB 12 and determines whether or not there is cooperation information associated with the identified target area. If it is determined that there is, the presenting unit 104 shifts the process to step S202, and determines whether or not a plurality of pieces of cooperation information are associated with the target region.

  If the presenting unit 104 determines in step S202 that a plurality of pieces of cooperation information are associated with the target area, the presenting unit 104 shifts the process to step S203. In step S203, the presentation unit 104 creates a function list indicating each function based on the plurality of pieces of cooperation information, and presents a selection screen for selecting one function from the function list to the terminal device 20. For example, when a URL and a telephone number are associated as linkage information for the specified target area, a selection screen for selecting a function for accessing the address indicated by the URL and a telephone function for the telephone number Is generated and presented to the terminal device 20. Then, a series of processes according to the flowchart of FIG.

  On the other hand, in step S202, when the presentation unit 104 determines that only one piece of cooperation information is associated with the target area, the presenting unit 104 shifts the process to step S204. In step S <b> 204, a screen for executing the function based on the one cooperation information is generated and presented to the terminal device 20. Then, a series of processes according to the flowchart of FIG.

  If it is determined in step S201 described above that the cooperation information is not associated with the target area specified in step S200, the process proceeds to step S205. In step S <b> 205, the presentation unit 104 presents a predetermined specific screen to the terminal device 20. Then, a series of processes according to the flowchart of FIG.

  In the above description, in step S106, one index is selected from a plurality of indexes on the image search server 11 side, but this is not limited to this example. That is, it is possible to prompt the user to perform a process of selecting one index from a plurality of indexes.

  FIG. 8 is a flowchart illustrating an example of a process in which the user is prompted to perform the process of selecting 1 from a plurality of indexes based on the search output of the search unit 103. If it is determined in step S105 of FIG. 6 that the object included in the captured image 40A matches with the plurality of indexes, the search unit 103 shifts the processing to step S106 'shown in FIG.

  When the process proceeds to step S106 ′, the presentation unit 104 generates a screen for presenting information indicating a plurality of indexes determined to be matched to the user in step S300, and displays the screen on the terminal device 20. To present. For example, it is conceivable that the presentation unit 104 generates a screen for displaying an image of each object searched by each of a plurality of indexes and presents the screen to the terminal device 20.

  The presentation unit 104 waits for reception of a selection result in which one index (object image) is selected from a plurality of indexes (object images) from the terminal device 20 (step S301). Upon receiving from the terminal device 20 that one index has been selected from the plurality of indexes presented in step S300, the presenting unit 104 determines from the plurality of object images that have been determined to match the index in step S302. 1 is selected according to the reception result. And the cooperation information linked | related with the index by which the selected object is searched is taken out from DB12, and is shown with respect to the terminal device 20. FIG.

  Thus, the user can select a desired image more accurately by presenting a plurality of images to the user.

  FIG. 9 shows an exemplary configuration of the terminal device 20. In the terminal device 20 illustrated in FIG. 9, a CPU 402, a ROM 403, a RAM 404, and a display control unit 405 are connected to a bus 401. A storage 407, a data I / F 408, an input unit 409, a communication unit 410, and an imaging unit 411 are connected to the bus 401. The storage 407 is a storage medium capable of storing data in a nonvolatile manner, and is a nonvolatile semiconductor memory such as a flash memory, for example. Not limited to this, a hard disk drive may be used as the storage 407.

  The CPU 402 controls the entire terminal device 20 using the RAM 404 as a work memory in accordance with programs stored in the ROM 403 and the storage 407. The display control unit 405 converts the display control signal generated by the CPU 402 into a signal that can be displayed by the display unit 406 and outputs the converted signal.

  The storage 407 stores programs and various data to be executed by the CPU 402 described above. The data I / F 408 inputs data from the outside. As the data I / F 408, for example, an interface such as USB (Universal Serial Bus) or IEEE 1394 (Institute of Electrical and Electronics Engineers 1394) can be applied.

  The input unit 409 includes an input device that receives a user input and outputs a predetermined control signal. The user can issue an instruction to the terminal device 20 by operating the input device according to the display on the display unit 406, for example. Note that it is preferable that the input device that receives user input is configured integrally with the display unit 406 and configured as a touch panel that outputs a control signal corresponding to the pressed position and transmits the image of the display unit 406.

  The communication unit 410 communicates with the network 10 using a predetermined protocol. The imaging unit 411 performs predetermined processing on an optical system, an imaging device, a control drive circuit for the optical system and the imaging device, and a signal output from the imaging device, and outputs the result as image data. The imaging unit 411 executes functions such as imaging and zooming according to an instruction by a user operation on the input unit 409. The captured image obtained by the imaging unit 411 is transmitted to the communication unit 410 via the bus 401, for example, and is transmitted to the image search server 11 via the network 10 according to a command from the CPU 402.

  In such a configuration, the terminal device 20 can display each information presented by the presentation unit 104 of the image search server 11 using a browser application for browsing a website on the network 10. . Further, the terminal device 20 can transmit a request and data to the image search server 11 using this browser application.

  In this case, in the image search server 11, the presentation unit 104 generates display control information for presenting each piece of information to the terminal device 20 using, for example, HTML (HyperText Markup Language) or various script languages. The file is placed at a predetermined address on the network 10. By notifying the terminal device 20 of this address from the presentation unit 104, the display control information can be accessed from the terminal device 20, and the presentation of each information by the presentation unit 104 can be displayed on the terminal device 20.

  Not limited to this, it is also conceivable to install a terminal program (referred to as an image search terminal application) that performs a series of processes from transmission of a captured image to display of presentation information on the terminal device 20. FIG. 10 is a functional block diagram showing functions of an example of the terminal device 20 in this case. The terminal device 20 includes a reception unit 420, a transmission unit 421, an imaging processing unit 422, an input processing unit 423, a presentation unit 424, an identification information management unit 425, and a control unit 430.

  The imaging processing unit 422 controls the imaging operation by the imaging unit 411 and acquires a captured image. The transmission unit 421 has address information on the network 10 of the image search server 11 in advance, for example, and performs transmission processing to the image search server 11 via the network 10. The captured image acquired by the imaging processing unit 422 is transmitted to the image search server 11 by the transmission unit 421. The receiving unit 420 receives information transmitted from the server. For example, information presented from the presentation unit 104 of the image search server 11 is received by the reception unit 420.

  The input processing unit 423 processes user input to the input unit 409 as hardware. The presentation unit 424 passes a prepared UI image to the display control unit 405 and causes the display unit 406 to display a screen based on the UI image. In addition, the presentation unit 424 generates a display screen based on the information received from the reception unit 420, for example, from the presentation unit 104 of the image search server 11, and displays the display screen on the display unit 406 via the display control unit 405. . Information presentation by the presentation unit 424 can be performed not only by display but also by voice.

  The identification information management unit 425 manages identification information (referred to as an application ID) that is uniquely acquired by the image search terminal application installed in the terminal device 20. Details of the application ID will be described later.

  The control unit 430 controls the overall operation of the terminal device 20. In addition, the control unit 430 can execute a cooperation function based on cooperation information that is presented from the presentation unit 104 of the image search server 11 and that corresponds to the captured image. For example, when a URL is specified in the cooperation information, the control unit 430 can call a browser application installed in the terminal device 20 to access the address indicated by this URL.

  The reception unit 420, the transmission unit 421, the imaging processing unit 422, the input processing unit 423, the presentation unit 424, the identification information management unit 425, and the control unit 430 described above are executed by a terminal program that operates on the CPU 402. Realized. The terminal program is configured to be stored on a computer connected to the network 10 and provided by being downloaded via the network 10. The terminal program may be configured to be provided or distributed via the network 10. It is also conceivable to provide the terminal program by storing it in the ROM 403 in advance.

  The terminal program is not limited to this, and may be provided by being recorded in a computer-readable recording medium such as a CD or a DVD in a file that can be installed or executed. In this case, for example, the terminal program is supplied to the terminal device 20 via an external drive device connected to the data I / F 408.

  The terminal program has a module configuration including, for example, the above-described units (reception unit 420, transmission unit 421, imaging processing unit 422, input processing unit 423, presentation unit 424, identification information management unit 425, and control unit 430). As the actual hardware, the CPU 402 reads the terminal program from the storage 407 and executes it, for example, so that the above-described units are loaded on the main storage device (for example, the RAM 404), and the respective units are generated on the main storage device. It is like that.

(First embodiment)
Next, a first embodiment will be described. FIG. 11 schematically shows a configuration of an example of an information processing system according to the first embodiment. In FIG. 11, the same reference numerals are given to the portions common to FIG. 1 described above, and detailed description thereof is omitted.

  In FIG. 11, an information processing system 1A includes an image search system 200A and a terminal device 20 that are connected to each other via a network. The image search system 200A corresponds to the image search system 200 described above, and includes an image search server 11 and a DB 12. As described above, the image search server 11 includes one or more computers.

  In the DB 12, for example, by the operator of the content distribution site 210 or at the request of the operator, the index data of the object corresponding to the captured image 40 and the cooperation information corresponding to the object are registered in association with each other. It is assumed that The linkage information is, for example, the URL of the content distribution site 210 itself or the URL of each content distributed by accessing the content distribution site 210.

  For example, the terminal device 20 captures a paper medium and transmits the captured image 40 to the image search system 200A. The captured image 40 is received by the image search system 200 </ b> A and supplied to the image search server 11. In the same manner as described above, the image search server 11 extracts an object from the captured image 40, searches the DB 12 based on the extracted object, and acquires a URL corresponding to the object as a search result. The image search server 11 transmits the acquired URL to the terminal device 20 that is the transmission source of the captured image 40.

  The terminal device 20 accesses, for example, the content distribution site 210 according to the URL transmitted from the image search system 200A in response to the transmission of the captured image 40 using the image search terminal application. At this time, the terminal device 20 transmits the application ID managed by the identification information management unit 425 to the content distribution site 210. Note that the terminal device 20 can transmit the application ID to the content distribution site 210 as individual data. Not limited to this, the terminal device 20 may perform transmission of the application ID to the content distribution site 210 by adding the URL to the content distribution site 210 as an argument when accessing the content distribution site 210. .

  Note that the content distribution site 210 corresponds to the commercial site 30A or the commercial site 30B in FIG. 1 and includes one or more computers. Then, the content distribution site 210 includes the position on the network indicated by the URL obtained as a result of the search by the image search system 200A, and presents the content associated with the URL to the terminal device 20 or the like.

  Here, the application ID will be described. As described above, the application ID is identification information that is uniquely acquired for the image search terminal application according to the embodiment that is installed and executed in the terminal device 20. This application ID is associated with the image search terminal application and cannot be changed by the user. The application ID is deleted when the image search terminal application is deleted (uninstalled) from the terminal device 20.

  That is, the application ID is different from identification information such as an email address or account information that is set by the user and can be arbitrarily changed by the user. Further, the application ID has a property different from the identification information that is associated with the terminal device 20 itself and cannot be changed by the user, like the device ID. Furthermore, since the application ID is unique for the image search terminal application installed in the terminal device 20, it can be substantially used as identification information for each terminal device 20.

  The application ID is generated, for example, when the image search terminal application is installed in the terminal device 20, and is set for the terminal device 20. FIG. 12 is a flowchart illustrating generation processing of an example of an application ID according to the embodiment. Here, it is assumed that the image search terminal application is installed in the terminal device 20 by a dedicated installation program (installer).

  The installer is downloaded from a server on the network to the terminal device 20 via the network and stored in the storage 407, for example. The CPU 402 outputs a command for instructing installation execution, for example, in response to a user operation for instructing installation execution to the input unit 409 or automatically with the download of the installer. In response to this command, the CPU 402 expands the installer stored in the storage 407 in the RAM 404 and executes it.

  In FIG. 12, in step S <b> 10, the CPU 402 executes the installer and starts installing the image search terminal application in accordance with the installer's instruction. When installation is started, the CPU 402 generates and acquires an application ID in accordance with an instruction from the installer (step S11). At this time, the installer determines the application ID generation method so that the generated application ID has a unique value.

  The application ID can be generated using, for example, a random number. As an example, the CPU 402 may generate an application ID by combining a random number having a predetermined length and time information indicating the time when the installation process by the installer is started and the time when the installer is downloaded. Not limited to this, the CPU 402 may acquire the time information in units of seconds or less in accordance with an instruction from the installer and use it as an application ID as it is, or generate a sufficiently long random number and use the application ID as it is. Can also be used.

  In step S12, the CPU 402 stores the application ID generated in step S11 in the storage 407 or the like. At this time, it is more preferable that the CPU 402 can conceal the application ID from the user by encrypting the application ID or making it invisible by a predetermined method and storing it in the storage 407.

  Returning to the description of FIG. 11, the content distribution site 210 can distribute the content corresponding to the application ID transmitted from the terminal device 20 to the terminal device 20. Although details will be described later, for example, the content distribution site 210 registers the application ID for a certain application ID at the first access. Then, the content distribution site 210 collates the transmitted application ID based on the registration information, and when the application ID is already registered, the content distribution site 210 displays the content unique to the application ID as a terminal of the transmission source of the application ID. Deliver to the device 20.

  In this way, by transmitting the application ID, which is unique identification information for each image search terminal application, to the URL obtained as a result of the image search, different contents for each terminal device 20 can be obtained for the same search result. It becomes possible to deliver.

  That is, according to the first embodiment, in an information providing technology that presents a desired image to a user using an information medium and further provides related electronic information based on the image, In addition, it is possible to easily provide the user with electronic information corresponding to the terminal using the image, the attribute of the user, and the like.

  In the above description, the content distribution site 210 presents the content corresponding to the application ID transmitted from the terminal device 20 to the terminal device 20, but this is not limited to this example. For example, in the terminal device 20, predetermined registration information can be registered in the image search terminal application, and the registration information can be added to the application ID as a classification number or symbol and transmitted to the content distribution site 210. The content distribution site 210 can present content according to the classification number or symbol added to the application ID in addition to the application ID.

  Furthermore, the content distribution site 210 can also present content according to position information indicating the position of the terminal device 20. For example, the terminal device 20 is further provided with a GPS positioning unit that receives a GPS (Global Positioning System) signal and performs positioning, and the GPS positioning unit acquires information indicating the position at which the image was taken. The terminal device 20 adds the acquired position information to the application ID and transmits it to the content distribution site 210. The content distribution site 210 can present content according to position information added to the application ID in addition to the application ID.

(Second Embodiment)
Next, a second embodiment will be described. In the first embodiment described above, the application ID is generated on the terminal device 20 side, but this is not limited to this example. In the second embodiment, an application ID is generated externally and set in the terminal device 20. As an example, it can be considered that an application ID is generated on a server connected to the terminal device 20 via a network. The application ID generated on the server is transmitted to the terminal device 20 via the network and stored in the storage 407 of the terminal device 20.

  FIG. 13 schematically shows a configuration of an example of an information processing system that generates an application ID on a server according to the second embodiment. 13, the information processing system 1B adds an application ID generation server 13 including an identification information (application ID) generation unit that generates an application ID to the image search system 200A of the information processing system 1A illustrated in FIG. Thus, the image search system 200B is used. The application ID generation server 13 can be configured by a general computer device. At this time, the application ID generation server 13 is not limited to being configured by one computer device, and may be configured to operate in a distributed manner on a plurality of computer devices, for example.

  In this case, when the image search terminal application installed in the terminal device 20 is activated for the first time, installation information indicating that is transmitted from the terminal device 20 to the image search system 200B and supplied to the application ID generation server 13. . The application ID generation server 13 generates an application ID according to the installation information and transmits it to the terminal device 20.

  FIG. 14 is a flowchart of an example illustrating processing when an application ID is generated externally and set in the terminal device 20 according to the second embodiment. In step S20, the CPU 402 executes the installer and starts installing the image search terminal application in accordance with an instruction from the installer. When the installation is completed, in the next step S21, the CPU 402 activates the installed image search terminal application. The CPU 402 may automatically start the image search terminal application following the installation process, or may be started in response to a user operation on the terminal device 20.

  When the image search terminal application is activated, as described above, each unit (reception unit 420, transmission unit 421, imaging processing unit 422, input processing unit 423) that causes the image search terminal application to function on the main storage device (for example, the RAM 404). The presentation unit 424, the identification information management unit 425, and the control unit 430) are generated as modules. The image search terminal application is executed by the CPU 402 operating according to each module.

  When the image search terminal application is activated, the identification information management unit 425 accesses the application ID generation server 13 in accordance with an instruction from the image search terminal application (step S22). This access is the first access after the image search terminal application is installed in the terminal device 20. The identification information management unit 425 requests the application ID generation server 13 to generate an application ID.

  The application ID generation server 13 generates an application ID in response to the application ID generation request from the terminal device 20, and transmits the generated application ID to the transmission source of the application ID generation request.

  As a method for generating the application ID, a method of combining random numbers and time information can be applied as described above. In this case, for example, the time when the application ID generation server 13 receives the application ID generation request from the terminal device 20 can be used as the time information. Not limited to this, the application ID generation server 13 may generate an application ID from only a random number or only time information, or may generate a serial number or the like as an application ID. The application ID generation server 13 can also manage the application ID by associating the generated application ID with the address information of the terminal device 20 or the like.

  In the next step S23, the identification information management unit 425 receives and acquires the application ID transmitted from the application ID generation server 13. In step S24, the identification information management unit 425 stores the received application ID in, for example, the storage 407. At this time, the identification information management unit 425 preferably stores the application ID in the storage 407 by concealing it from the user by encryption or invisibility.

  As described above, the generation of the application ID may be performed outside the terminal device 20. In this case, the application ID generation server 13 can manage the application ID by associating the generated application ID with the address information of the terminal device 20 or the like.

(Third embodiment)
Next, a third embodiment will be described. In the third embodiment, the history of the terminal device 20 accessing the image search server 11 for image search is recorded as log information. FIG. 15 schematically illustrates an exemplary configuration of an information processing system 1C according to the third embodiment. In FIG. 15, the information processing system 1 </ b> C adds the log server 14 that records the access history by the terminal device 20 as log information to the image search system 200 </ b> A of the information processing system 1 </ b> A shown in FIG. 11. The system 200C is used.

  In the information processing system 1C of FIG. 15, when the terminal device 20 transmits the captured image 40 to the image search system 200C, the terminal device 20 transmits the application ID of the image search terminal application of the terminal device 20 in association with the captured image 40. . In the image search system 200C, the log server 14 stores time information indicating the time when the terminal device 20 is accessed to the image search system 200C, the application ID, and information indicating the captured image 40, and stores the information as log information. An accumulating unit. When the device ID for identifying the device of the terminal device 20 can be acquired, the device ID can be recorded in association with the log information. The log server 14 can be configured by a general computer device. At this time, the log server 14 is not limited to being configured by a single computer device, and may be configured to operate in a distributed manner on a plurality of computer devices, for example.

  Here, in the DB 12, as the cooperation information corresponding to the object corresponding to the captured image 40, the URL of the content distributed by accessing the content distribution site 210 and another site 211 different from the content distribution site 210. Is associated with the URL. As an example of the other site 211, a moving image distribution site that distributes moving images can be cited.

  In this case, when the URL of the content to be distributed is selected in the terminal device 20 by accessing the content distribution site 210, the content distribution site 210 can grasp the access, and the access is determined by the application ID. Can be managed for each. However, when the URL of the other site 211 is selected in the terminal device 20, the content distribution site 210 cannot grasp the access of the other site 211 by the terminal device 20, and the access for each application ID. Cannot be managed.

  Therefore, in the third embodiment, the terminal device 20 associates the captured image 40 with the application ID and transmits the image ID to the image search system 200C, and the log server 14 includes information indicating the application ID and the object corresponding to the captured image 40. Are stored as log information. Accordingly, it is possible to grasp both the case where the terminal device 20 accesses the URL of the content distributed by accessing the content distribution site 210 and the case where the terminal device 20 accesses another site 211 based on the application ID.

  FIG. 16 shows an example of the configuration of a log information table in which the log server 14 accumulates log information according to the third embodiment. The log information table illustrated in FIG. 16 records the access date / time, the object ID, and the device information in association with the application ID. The access date / time stores the date / time when the terminal device 20 accesses the image search system 200C by the captured image 40 and the application ID. The object ID is information for identifying the object extracted from the captured image 40. For example, the ID in the index table 120 described with reference to FIG.

  The device information is information for identifying the terminal device 20. As the device information, an IP (Internet Protocol) address or a MAC (Media Access Control) address of the terminal device 20 can be used. Device information can be omitted.

  FIG. 17 shows an example of the configuration of a linkage information table that associates an object ID and link destination information (URL) for each customer according to the third embodiment. In the cooperation information table, the customer ID is identification information for identifying, for example, an owner of data registered in the DB 12. The owner is, for example, an operator or operating organization of the content distribution site 210. Thus, in the cooperation information table, the object ID and the link destination URL are stored in association with each customer ID. Information stored in the linkage information table can be generated using the information in the target area table 121 shown in FIG.

  In the log server 14, the log information is accumulated in association with the application ID in the log information table in response to the access from the terminal device 20. Then, the log information table is searched based on the application ID, the object ID corresponding to the desired application ID is output, the linkage information table is detected based on the object ID, and link destination information for each object ID is obtained. Thereby, for example, the operator of the content distribution site 210 can predict the number of accesses in which the content distribution site 210 and other sites 211 are accessed with a desired application ID.

(First application example of the third embodiment)
Next, a first application example applicable to the third embodiment will be described. In the first application example, the content distribution site 210 performs a check based on an application ID, performs a registration process when the application ID is not registered, and distributes predetermined content when the application ID is registered. is there.

  FIG. 18 shows an overview of a system capable of realizing the first application example. This system includes an image search system 200C, a content distribution site 210, and a media planning system 220. In the example of FIG. 18, log information can be provided to the content distribution site 210 by applying the image search system 200 </ b> C having the log server 14. When it is not necessary to provide log information to the content distribution site 210, the image search system 200A or the image search system 200B may be applied instead of the image search system 200C.

  In FIG. 18, the media planning system 220 includes a planning system 500, a media production system 501, a media creation system 502, and an authoring data registration system 503.

  The planning system 500 plans and searches for an image to be searched by the image search system 200C for distributing the content and a medium on which the search target image is formed, according to the content distributed on the content distribution site 210. Information indicating the target image and medium is output. In addition, the planning system 500 outputs the URL of the content distribution site 210 and the URL of each content distributed by the content distribution site 210.

  The medium production system 501 creates a search target image in a format suitable for the medium based on the search target image output from the planning system 500 and information indicating the medium. The media production system 501 outputs the created search target image to the media creation system 502. Further, the medium production system 501 outputs the created search target image and each URL supplied from the planning system 500 to the authoring data registration system 503.

  The medium creation system 502 forms a search target image on the medium and creates a medium 510 as a search target (imaging target). For example, the medium creation system 502 is a printer device, and prints a search target image on a paper medium to create a search target medium 510 using paper. The medium 510 is not limited to a paper medium. For example, the material of the medium 510 may be other than paper, or an electronic bulletin board on which the medium 510 electronically posts information.

  The authoring data registration system 503 registers the search target image and each URL supplied from the medium creation system 501 in the DB 12 of the image search system 200C. Thereby, for example, the terminal device 20 can obtain a URL corresponding to the search target image by transmitting a captured image obtained by capturing the search target image on the search target medium 510 to the image search system 200C.

  For example, the terminal device 20 captures a search target image on the search target medium 510, and transmits the captured image and the application ID to the image search system 200C. In the image search system 200C, the image search server 11 searches the DB 12 based on the captured image and acquires a URL. This URL is transmitted to the terminal device 20. In the image search system 200C, the log server 14 uses the object ID obtained by the search by the image search server 11, the application ID transmitted from the terminal device 20, and the date and time when the terminal device 20 accesses the image search system 200C. Information is recorded in a log information table.

  When the terminal device 20 accesses a URL transmitted from the image search system 200C, the terminal device 20 transmits an application ID to the URL. Here, it is assumed that the URL transmitted from the image search system 200C indicates the content distribution site 210, and the terminal device 20 transmits the application ID to the content distribution site 210 when accessing the content distribution site 210 according to the URL. .

  In this example, the content distribution site 210 urges the user who has accessed from the terminal device 20 to register as a member, distributes the content for the member to the user registered as a member, that is, the terminal device 20, and registers as a member. A predetermined content is distributed to users who are not.

  The content distribution site 210 includes an application ID check unit 2200, a member authentication unit 2101, content for members 2102, 2102,..., An application ID storage unit 2103, a member information storage unit 2104, and a log information storage unit 2105. Including. Among these, the member information storage unit 2104 stores information on users registered as members in the content distribution site 210. In addition, the application ID storage unit 2103 associates the application ID transmitted from the terminal device 20 used by the user registered as a member for access at the time of member registration with the user information stored in the member information storage unit 2104. Stored.

  The application ID check unit 2100 determines whether the user who has accessed from the terminal device 20 is registered as a member. Specifically, the application ID check unit 2100 determines whether or not the application ID transmitted from the terminal device 20 is stored (recorded) in the application ID storage unit 2103.

  If it is determined that the application ID transmitted from the terminal device 20 is not stored in the application ID storage unit 2103, the application ID check unit 2100 determines that the user is not registered as a member. In this case, the application ID is passed to the member authentication unit 2101, and the member authentication unit 2101 transmits a notification for prompting member registration to the terminal device 20. In response to this notification, when a response indicating membership registration is returned from the terminal device 20, the member authentication unit 2101 performs a predetermined exchange with the terminal device 20 to obtain user information and the like. Is stored in the member information storage unit 2104, and the application ID is stored in the application ID storage unit 2103 in association with the user information. As a result, the user is registered as a member in the content distribution site 210.

  On the other hand, when it is determined that the application ID transmitted from the terminal device 20 is stored in the application ID storage unit 2103, the application ID check unit 2100 determines that the user is registered as a member. In this case, the content distribution site 210 presents the content for members 2102, 2102,.

  In the above description, the content distribution site 210 has been described to present the content for members 2102, 2102,... To the user registered as a member by the member authentication unit 2101 from the next access after the member registration. However, this is not limited to this example. For example, the content distribution site 210 may present the content for members 2102, 2102,... Immediately after the member registration to the user registered as a member.

  In place of the notification for prompting member registration, notification (screen display) may be performed that allows the user to select whether to perform member registration or to input user information that has already been registered (login is performed). For a user who has already registered as a member and has installed the image search terminal application in the terminal device 20 after the member registration, the member registration has already been completed, that is, user information is stored in the member information storage unit 2104. Therefore, there is no need to register new members. Therefore, by performing notification that allows you to select whether to register as a member or enter user information that has already been registered, member registration is performed for unregistered users, and registered user information for registered users. Can be entered. When the user inputs user information as registered user information, the member authentication unit 2101 determines whether or not the input user information is stored in the member information storage unit 2104. For example, the content for members 2102, 2102,... Is presented, and the application ID is stored in the application ID storage unit 2103 in association with the input user information.

  The log information storage unit 2105 stores log information provided from the log server 14 in the image search system 200C. For example, the log information storage unit 2105 accesses the log server 14 at predetermined time intervals, and acquires log information accumulated since the previous access. Not limited to this, the log information can be transmitted from the log server 14 to the log information storage unit 2105. In this case, for example, provision of log information from the log server 14 to the log information storage unit 2105 can be performed every time the log information is recorded in the log server 14.

  The log information storage unit 2105 can also accumulate log information related to access to the content distribution site 210. By analyzing the log information accumulated in the log information storage unit 2105, the content distribution site 210 can obtain statistical information related to access of the application ID such as the number of accesses for each application ID.

  According to the first application example, the content distribution site 210 can determine whether or not the access from the terminal device 20 is the first access based on the application ID transmitted from the terminal device 20. Different contents can be presented to the device 20 according to the determination result. Further, the access state for each application ID can be analyzed based on the log information stored in the log information storage unit 2105, and the content can be selected and presented more flexibly according to the analysis result.

(Second application example to the third embodiment)
Next, a second application example for the third embodiment will be described. In the second application example, the system configuration shown in FIG. 18 can be applied.

  In the second application example, the content distribution site 210 uses the application ID transmitted from the terminal device 20 to register the characteristic information (age, gender, etc.) of the user to be registered when the user is registered as a member in the member authentication unit 2101 in FIG. Register in association. In addition, the content distribution site 210 acquires log information from the log server 14 of the image search system 200 </ b> C and stores it in the log information storage unit 2105. When the content distribution site 210 is accessed from the terminal device 20 using the registered application ID, the content distribution site 210 is based on the user-specific information and log information associated with the application ID, and the user-specific content 2102, 2102,. Is selected and presented to the terminal device 20.

  An example using learning materials such as foreign language learning will be described as the content for members 2102, 2102,... In the second application example. In the learning material, it is preferable to set the content of the learning material according to the level (age, grade, etc.) and achievement level of the user (assumed to be a learner). Therefore, the content distribution site 210 groups, for example, member-oriented contents 2102, 2102,... As learning materials according to, for example, the level of difficulty of the contents.

  FIG. 19 shows an example of grouping of member-oriented contents 2102, 2102,... According to the second application example. In the example of FIG. 19, three classes of “beginner”, “intermediate”, and “advanced” are set according to the difficulty level of the teaching material. Furthermore, levels such as “A”, “B”, “C”,... Are set in each class according to the difficulty level of the teaching material. A URL for presenting a teaching material is associated with each level of each class.

  For example, the medium planning system 220 creates a learning material using a paper medium as the medium 510 and registers the information in the image search system 200C. The user images the medium 510 as the learning material using the terminal device 20, and transmits the captured image and the application ID to the image search system 200C. It should be noted that the learning of the learning material by the terminal device 20 may be a specific page such as a cover of the learning material, or an arbitrary page of the content of the learning material may be the subject.

  The image search system 200 </ b> C searches, for example, the URL of the content distribution site 210 from the DB 12 based on the captured image transmitted from the terminal device 20, and transmits the retrieved URL to the terminal device 20. Further, the image search system 200C associates the application ID and the object ID obtained by the search of the image search server 11 with the date and time information when the terminal device 20 accesses the image search system 200C, and transmits it to the log server 14, Store as log information.

  The terminal device 20 accesses the content distribution site 210 according to the URL transmitted from the image search system 200C, and transmits an application ID to the content distribution site 210. The content distribution site 210 checks the application ID transmitted from the terminal device 20 by the application ID check unit 2100 with respect to the access from the terminal device 20 with reference to the application ID storage unit 2103 and the member information storage unit 2104. It is determined whether or not the application ID is registered as a learner.

  If it is determined that the content is not registered, the content distribution site 210 transmits a notification prompting the terminal device 20 to register and input the user's characteristic information. In response to this notification, when registration approval and characteristic information are transmitted from the terminal device 20 to the content distribution site 210, the content distribution site 210 authenticates the application ID by the member authentication unit 2101, and serves as a learner. sign up. Specifically, the content distribution site 210 associates the application ID with the characteristic information and stores them in the application ID storage unit 2103 and the member information storage unit 2104, respectively. The content distribution site 210 sets a learner class based on, for example, characteristic information.

  On the other hand, when the application ID check unit 2100 determines that the application ID transmitted from the terminal device 20 is registered as a learner, the content distribution site 210 reads the log corresponding to the application ID from the log information storage unit 2105. Search for information. Then, the content distribution site 210 determines the level of the learner within the class set for the learner according to the search result. For example, when there is a predetermined number of accesses to a certain level (URL) by the application ID, the content distribution site 210 shifts the level of the application ID to the next higher level.

  As described above, in the second application example, the content distribution site 210 can present appropriate content for each user to the terminal device 20 to which the application ID is transmitted based on the application ID and the log information. it can. Further, since the content distribution site 210 authenticates with the application ID, it is not necessary to perform a login operation for each access from the terminal device 20 to the content distribution site 210.

(Third application example to the third embodiment)
Next, a third application example for the third embodiment will be described. In the third application example, the system configuration shown in FIG. 18 can be applied.

  The third application example is an example in which access by the same application ID is limited according to the number of accesses. For example, it is conceivable to permit access up to a predetermined number of times within a predetermined period or every predetermined period. Not limited to this, it is conceivable to permit access to the same application ID up to a predetermined number of times. As an example of restricting access to the same application ID, a campaign application using the terminal device 20 can be considered.

  As an example, consider a campaign that is held for a certain period, can be applied up to once per day with the same application ID, and when the date changes, the application restriction is removed and application can be made again. In this campaign, the application results shall be announced immediately after the end of the campaign period.

  FIG. 20 is a flowchart showing an example of processing in the content distribution site 210 according to the third application example in this case. Prior to the processing of this flowchart, the medium planning system 220 creates, for example, a campaign application form as a paper medium as the medium 510 and registers the information in the image search system 200C. The content distribution site 210 waits for access by the application ID from the terminal device 20 (step S30).

  The terminal device 20 images the medium 510 and transmits the captured image and the application ID to the image search system 200C. The image search system 200 </ b> C searches, for example, the URL of the content distribution site 210 from the DB 12 based on the captured image transmitted from the terminal device 20, and transmits the retrieved URL to the terminal device 20. Further, the image search system 200C associates the application ID and the object ID obtained by the search of the image search server 11 with the date and time information when the terminal device 20 accesses the image search system 200C, and transmits it to the log server 14, Store as log information.

  The terminal device 20 accesses the content distribution site 210 according to the URL transmitted from the image search system 200C, and transmits an application ID to the content distribution site 210. When the content distribution site 210 detects an access by the application ID from the terminal device 20, the content distribution site 210 shifts the process to step S <b> 31 and acquires the application ID and date / time information indicating the time of access. The acquired application ID and date / time information are associated with each other and stored in the log information storage unit 2105, for example.

  In the next step S32, the content distribution site 210 determines whether there has been access in the past based on the log information stored in the log information storage unit 2105, based on the application ID detected in step S30. If it is determined that there is no access in the past, the content distribution site 210 shifts the process to step S34 and executes a predetermined application process for the application ID.

  If it is determined in step S32 that the application ID has been accessed in the past, the content distribution site 210 shifts the processing to the next step S33. In step S33, for example, the content distribution site 210 determines whether or not a predetermined time (one day in this example) when the application restriction by the same application ID is released has elapsed since the previous access by the application ID. For example, based on the application ID and date / time information acquired in step S31 and the application ID and date / time information accumulated as log information in the log information storage unit 2105, the content distribution site 210 has received information from the previous access by the application ID. The elapsed time can be determined.

  If it is determined in step S33 that the predetermined time has passed since the previous access, the content distribution site 210 cancels the application restriction for the application ID and shifts the process to step S34.

  If it is determined in step S33 that the predetermined time has not elapsed since the previous access, the content distribution site 210 shifts the process to step S35, and the application to the terminal device 20 by the application ID is being restricted. Send a message to that effect. Then, the process proceeds to step S36.

  In step S36, the content distribution site 210 determines whether or not the campaign satisfies a predetermined condition. For example, when the campaign is executed within a predetermined period, the content distribution site 210 determines whether or not the current date and time is the end date and time of this campaign. If it is determined that the campaign does not satisfy the predetermined condition, the process returns to step S30. On the other hand, if it is determined that the campaign satisfies the predetermined condition, the content distribution site 210 shifts the process to step S37 and transmits a predetermined message indicating that to the terminal device 20.

(Fourth application example of the third embodiment)
Next, a fourth application example to the third embodiment will be described. In the fourth application example, the system configuration shown in FIG. 18 can be applied.

  The fourth application example is an example in which specific content is presented when a plurality of predetermined objects are acquired with one application ID. As such an application example, a stamp rally is conceivable in which a privilege is given when all predetermined points are visited.

  As an example, as shown in FIG. 21, there are stations A to E, a stamp Stmp 600 at station A, a stamp Stmp 601 at station B, a stamp Stmp 602 at station C, and a stamp Stmp 603 at station E. It is assumed that a privilege can be obtained by collecting all the stamp impressions of the stamps Stmp600 to 603. Participants of the stamp rally go to the station where the stamp is provided among the stations A to E, and record the stamp imprint on a predetermined sheet or the like. At this time, the recording order of the stamps Stmp 600 to 603 is arbitrary. When the user finishes recording all of the stamps Stmp 600 to 603, the user can obtain a privilege by bringing the recorded sheet to the reception of the stamp rally.

  FIG. 22 is a flowchart showing an example of processing in the content distribution site 210 according to the fourth application example. Here, the fourth application example will be described by taking the stamp rally described with reference to FIG. 21 as an example. Further, the flowchart of FIG. 22 shows processing for one application ID already registered in the content distribution site 210.

  Prior to the processing of the flowchart of FIG. 22, the medium planning system 220 forms an imprint of each stamp Stmp600 to 603 on a predetermined medium such as a paper medium, and each medium corresponding to the imprint of each stamp Stmp600 to 603 510 is created. Further, in the media planning system 220, the authoring data registration system 503 has different stamp stamps 600 to 603 as objects, and different object identifiers for the same URL (the URL of the content distribution site 210). The URLs # 1 to # 4 to which the argument is added are associated with each other and registered in the DB 12.

  In addition, the content distribution site 210 associates URLs different from each other corresponding to the objects by the stamps of the stamps Stmp 600 to 603 with respect to the URLs # 1 to # 4 to which the argument is added. For example, when the content distribution site 210 is accessed by URL # 1, this access is redirected to the URL associated with the stamp Stmp 600.

  The content distribution site 210 waits for access from the terminal device 20 (step S40).

  The terminal device 20 images the medium 510 (an imprint of each stamp Stmp 600 to 603) at any one of the stations A, B, C, and E, and transmits the captured image and the application ID to the image search system 200C. The image search system 200 </ b> C searches the DB 12 for a URL associated with the captured image based on the captured image transmitted from the terminal device 20. For example, when the captured image transmitted from the terminal device 20 includes an imprinted object with the stamp Stmp 600, the image search system 200C searches the DB 12 for URL # 1 associated with the object. Further, the image search system 200C associates the application ID and the object ID obtained by the search of the image search server 11 with the date and time information when the terminal device 20 accesses the image search system 200C, and transmits it to the log server 14, Store as log information.

  The terminal device 20 accesses the content distribution site 210 according to the URL transmitted from the image search system 200C, and transmits an application ID to the content distribution site 210. When the content distribution site 210 detects an access by the application ID from the terminal device 20, the process proceeds to step S <b> 41, and this access by the application ID is caused by a predetermined object, that is, an object by any stamp stamp 600 to 603. It is determined whether or not there is.

  In this example, an argument is added to the URL registered in the DB 12 of the image search system 200 </ b> C as an object identifier for identifying an object by any stamp Imprint 600 to 603. Therefore, the content distribution site 210 can determine whether the access by the application ID is by a predetermined object based on this argument. If it is determined that the access is not by a predetermined object, the content distribution site 210 returns the process to step S40.

  On the other hand, if it is determined in step S41 that the access is by a predetermined object, the process proceeds to step S42. In step S42, the content distribution site 210 associates and holds the object (object identifier) and the application ID.

  In the next step S43, the content distribution site 210 determines whether or not all predetermined predetermined objects have been prepared based on the object information held in step S42 for the application ID related to the access in step S40. . In this example, the content distribution site 210 determines whether or not the object identifiers of the objects by the stamps Stmp 600 to 603 have been prepared for the application ID. If it is determined that they are not complete, the process returns to step S40.

  On the other hand, if it is determined in step S43 that the object identifiers of the objects with the stamps Stmp600 to 603 have been prepared for the application ID, the content distribution site 210 shifts the process to step S44 and causes the terminal device 20 corresponding to the application ID to For example, a message indicating that collection of a predetermined object has been completed is transmitted.

  In the above description, the stamps Stamp 600 to 603 are described as being different from each other, but this is not limited to this example. For example, all the stamps Stamp 600 to 603 have the same imprint, and it is possible to determine which of the stamps Stamp 600 to 603 has been imaged using the position information of the terminal device 20.

  That is, the terminal device 20 is further provided with a GPS positioning unit that receives a GPS signal and performs positioning. When the terminal device 20 accesses the content distribution site 210 according to the URL corresponding to the captured image, the terminal device 20 transmits a positioning result (for example, latitude and longitude) to the content distribution site 210. Based on the transmitted positioning result and the position information of each stamp Stmp 600 to 603 registered in advance, the content distribution site 210 has accessed the terminal device 20 from which position among the stamps Stmp 600 to 603. Can be determined.

  In the case of this example, the user can notify that the collection has been completed without having to go to the reception place after collecting all the predetermined stamps.

  In the above description, in step S43, it is determined whether or not all predetermined predetermined objects have been prepared. If not, the process returns to step S40. However, this is limited to this example. Not. For example, when a predetermined object is not aligned in step S43, the number of objects indicating how many objects are further aligned may be determined, and a message corresponding to the number of objects may be transmitted. As an example, if it is necessary to align all four objects to complete the collection, if the number of objects is one, send a message indicating that the remaining three objects need to be aligned, etc. A message indicating the current state may be transmitted.

(Fifth application example of the third embodiment)
Next, a fifth application example with respect to the third embodiment will be described. In the fifth application example, the system configuration shown in FIG. 18 can be applied.

  The fifth application example is an example in which specific content is presented when a plurality of predetermined objects are acquired in a predetermined order using one application ID. As such an application example, orienteering may be considered in which all predetermined points are circulated in a predetermined order to arrive at the final destination.

  FIG. 23 shows an example of a map used for orienteering. This map will be distributed to all participants participating in Orienteering and each group of participants immediately before the start. On the map, points to be passed are indicated by numbers indicating the order of passage. In the example of FIG. 23, it is indicated that the user should start from the point “1”, pass through the points “2” to “5” in order, and reach the point “6” that is the final destination. In addition, control flags indicating the respective points “1” to “6” are arranged at the actual points “1” to “6”, and the participants have reached the respective points “1” to “6”. You can know. The participant performs a check at each of the points “1” to “6”, and proves that the points have been added in order based on the check after reaching the point “6” as the final destination, for example.

  FIG. 24 is a flowchart showing an example of processing in the content distribution site 210 according to the fifth application example. Here, the fifth application example will be described by taking the orienteering described with reference to FIG. 23 as an example. Further, the flowchart in FIG. 24 shows processing for one application ID already registered in the content distribution site 210.

  Prior to the processing of the flowchart of FIG. 24, the medium planning system 220 creates, as the medium 510, a control flag on which a different image is printed for each point “1” to “6”, for example. Also, in the media planning system 220, the authoring data registration system 503 uses the control flag images of the points “1” to “6” as objects, and the same URL (the URL of the content distribution site 210). Then, URLs # 1 to # 6 to which arguments indicating the order of the points “1” to “6” are added as object identifiers are associated with each other and registered in the DB 12.

  Further, the content distribution site 210 associates different URLs with the objects corresponding to the points “1” to “6” with respect to the URLs # 1 to # 6 to which the argument is added. For example, when the content distribution site 210 is accessed by URL # 1, this access is redirected to the URL associated with the object corresponding to the point “1”.

  The content distribution site 210 waits for access from the terminal device 20 with the registered application ID of one (step S50). Here, it is assumed that the user (participant) of the terminal device 20 has already departed from the point “1”, passed through the point “2”, and reached the point “3”.

  The terminal device 20 captures the medium 510 (control flag) at the point “3”, and transmits the captured image and the application ID to the image search system 200C. The image search system 200 </ b> C searches the DB 12 for a URL associated with the captured image based on the captured image transmitted from the terminal device 20. For example, when the captured image transmitted from the terminal device 20 includes the control flag object of the point “3”, the image search system 200C searches the DB 12 for the URL # 3 associated with the object. Further, the image search system 200C associates the application ID and the object ID obtained by the search of the image search server 11 with the date and time information when the terminal device 20 accesses the image search system 200C, and transmits it to the log server 14, Store as log information.

  The terminal device 20 accesses the content distribution site 210 according to URL # 3 transmitted from the image search system 200C, and transmits an application ID to the content distribution site 210. When the content distribution site 210 detects an access by the application ID from the terminal device 20, the content distribution site 210 shifts the processing to step S51. In step S51, the content distribution site 210, based on the URL argument when accessed from the terminal device 20, each point “1” to “1” of the object (control flag) included in the captured image captured by the terminal device 20. The order in “6” is acquired and held as the order of the objects related to the application ID.

  Then, in the next step S52, it is determined whether or not the acquired order indicates the last object in each of the points “1” to “6”. If it is determined that the object is the last object, a series of processes related to the application ID according to the flowchart of FIG. 24 is terminated. At this time, a message indicating that may be sent from the content distribution site 210 to the terminal device 20.

  When it is determined in step S52 that the acquired order is not the last object in each of the points “1” to “6”, the content distribution site 210 moves the process to step S53, and for the app ID, The order held in step S51 at the time of access is acquired. Then, in the next step S54, the order at the previous access and the order at the current access are compared to determine whether or not each point has passed in the correct order. For example, when the captured image includes an object with a control flag of point “3”, objects of points “1” and “2”, which are all points that must be acquired before point “3”, are logged. It is determined whether or not it is stored as information.

  As a result of the determination, when it is determined that the points have passed in the correct order, the content distribution site 210 shifts the processing to the next step S55, and holds the order of the objects related to the application ID. Update.

  The process proceeds to the next step S56, and the content distribution site 210 transmits information indicating the next object to the terminal device 20 according to the updated order of the objects. The information indicating the next object may simply be information indicating the order, or the content distribution site 210 holds in advance an image of the object at each point “1” to “6”, and the image corresponding to the next object is displayed on the terminal. Transmission to the device 20 is also conceivable. Then, the process returns to step S50.

  On the other hand, if it is determined in step S54 that the points have not been passed in the correct order, the process proceeds to step S57. In step S57, the content distribution site 210 transmits information indicating the objects in the correct order to the terminal device 20. Then, the process returns to step S50.

  In the above description, the participant has a map of a paper medium or the like, and the content distribution site 210 passes the points “1” to “6” in the correct order with respect to the terminal device 20 corresponding to the application ID. Whether or not there is a notification is not limited to this example. For example, the content distribution site 210 may transmit a map as map data to each participant's terminal device 20.

  For example, at the time of orienteering, map data is distributed from the content distribution site 210 to the terminal device 20 of each participant. The map data distributed to the terminal device 20 initially displays only the starting point “1”. Then, the content distribution site 210 acquires the order of the objects at the points “1” to “6” according to the flowchart of FIG. 24 described above, and the points “1” to “6” in the correct order as a result of the determination in step S54. When it is determined that it has passed, the position of the next passing point is indicated (step S56). As a result of the determination in step S54, when it is determined that the points “1” to “6” have been passed in an incorrect order, the position of the correct passing point is indicated (step S57).

  Furthermore, in the above description, the terminal device 20 corresponding to the application ID is located at any position of each of the points “1” to “6” based on the captured image obtained by capturing the control flag arranged at each of the points “1” to “6”. However, this is not limited to this example. For example, the current position of the terminal device 20 corresponding to the application ID can be determined using the position information of the terminal device 20.

  That is, the terminal device 20 is further provided with a GPS positioning unit that receives a GPS signal and performs positioning. The terminal device 20 transmits a positioning result to the content distribution site 210 when accessing the content distribution site 210 according to the URL corresponding to the captured image. The content distribution site 210 accesses each point “1” based on the transmitted positioning result and the location information of the points “1” to “6” registered in advance by the terminal device 20 corresponding to the application ID. It is possible to determine from which position of “6” to “6”.

  In each of the above-described embodiments, a paper medium is described as an example of a search target (imaging target) medium. However, the medium is not limited to paper, and may be electronic paper, an electronic bulletin board, or the like.

  As described above, a terminal device and an image search system including at least one information processing apparatus (image search server or log server) are connected to a system that implements the present invention via a network. An information processing system that performs this is an example.

  The above-described embodiment is a preferred embodiment of the present invention, but is not limited thereto, and various modifications can be made without departing from the scope of the present invention.

1A, 1B, 1C Information processing system 11 Image search server 12 DB
13 application ID generation server 14 log server 20 terminal device 200A, 200B, 200C image search system 210 content distribution site 220 medium planning system 420 reception unit 421 transmission unit 422 imaging processing unit 423 input processing unit 424 presentation unit 425 identification information management unit 430 Control unit 2100 Application ID check unit 2101 Member authentication unit 2102 Member-oriented content 2103 Application ID storage unit 2104 Member information storage unit 2105 Log information storage unit

JP 2005-107992 A JP 2007-073063 A

Ricoh Technical Report No.33 DECEMBER, 2007, “Mixed Media Reality (MMR): A New Way to Connect Paper and the Electronic World”, [online], [Search May 27, 2013], Internet, <URL : Http://www.ricoh.co.jp/about/company/technology/techreport/33/pdf/A3314.pdf>

In order to solve the above-described problem and achieve the object, the present invention transmits one or more captured images obtained by capturing a medium using a terminal device and receives presentation of information associated with an object included in the medium. An image search system that is used by a user and one or more customers who present information to the user via a medium and that is configured by one or more information processing apparatuses, and an image search function by the image search system is used. anda terminal device application is mounted for the image retrieval system and a terminal device is an information processing system for providing a search in cooperation through a communication service, the image retrieval system, the customer Depending on the request, the object determined by the customer and the information associated with the object are stored in an external system different from the information processing system. A storage unit for storing that the linkage information for accessing information, the association, the captured image acquired by being captured by the terminal device, and identifying the terminal apparatus using the image retrieval system, and the application A first receiving unit that receives application identification information that is associated with the terminal device and cannot be changed by a user of the terminal device, and searches for an object stored in the storage unit based on the captured image received by the first receiving unit , a first transmission unit for transmitting the link information associated with the object that is the transmission source of the terminal apparatus of the captured image, a log information relating to use of the image search function, first in the use of at least the image search function It has a log recording unit that records log information including application identification information received by one receiving unit, a terminal device, the terminal device captures Accordingly, the second transmission unit that transmits the acquired captured image and the acquired application identification information to the image search system, and the cooperation transmitted by the first transmission unit of the image search system in response to the transmission by the second transmission unit when executing a second receiving unit for receiving information, the access via the network based on the link information received by the second receiving unit, the transmitting the application identification information generated for the terminal device to an external system And 3 transmission units.

Hereinafter, embodiments of an information processing system , an information processing method, and an external system will be described in detail with reference to the accompanying drawings.

Claims (9)

One or more users who transmit a captured image obtained by capturing a medium using a terminal device and receive presentation of information associated with an object included in the medium, and present information to the user via the medium An image search system that is used by one or more customers and configured by one or more information processing devices, and the terminal device in which a terminal program for using an image search function by the image search system is installed. An information processing system that provides a search service in cooperation via communication,
In response to a request from the customer, the object determined by the customer, information associated with the object, and linkage information for accessing the information held in an external system different from the information processing system; , And a storage unit for associating and storing,
Identification information for identifying the terminal device on which the terminal program is mounted, and the identification information realized by the terminal program mounted on the terminal device that acquires the identification information generated in the information processing system An acquisition unit;
A first transmission unit that transmits the identification information acquired by the identification information acquisition unit to the image search system when transmitting the captured image to the image search system and using an image search function of the image search system When,
The object stored in the storage unit is searched based on the captured image transmitted from the terminal device in order to use the image search function, and the cooperation information associated with the searched object is transmitted to the captured image. A second transmitter for transmitting to the original terminal device;
A log recording unit for recording in the image search system the log information related to the use of the image search function, the log information including at least the identification information transmitted by the first transmission unit in the use of the image search function When,
In the terminal device equipped with the terminal program, the identification information generated for the terminal device is executed when the access via the network is performed based on the cooperation information transmitted by the second transmission unit. A third transmitter for transmitting to the access destination;
An information processing system comprising: The storage unit can register the access destination to the external system held by the customer as the cooperation information,
The identification information transmitted by the third transmission unit is user information registered in the external system in the external system, and the user information of the user possessing the terminal device that transmits the identification information The information processing system according to claim 1, wherein the information processing system is recorded in association with the information processing system. The external system is a system that provides information on learning materials to member users registered in the external system,
In response to the access by the third transmission unit, the member user associated with the identification information transmitted by the third transmission unit is authenticated in the external system, and the learning material is transmitted to the authenticated member user. The information processing system according to claim 2, wherein information related to the information is presented. The storage unit can register, as the cooperation information, an access destination to the external system that provides a service that permits use up to a predetermined number of times in a predetermined period by the same user,
The number of times of accessing the external system and using the service in a predetermined period is managed based on the identification information of the terminal device accessing the external system based on the cooperation information. The information processing system according to claim 3. The storage unit is an access destination to the external system that provides a stamp rally service, and associates an access destination corresponding to each stamp constituting the stamp rally as the linkage information with the object corresponding to each stamp. Can register,
Based on the identification information of the terminal device accessing the external system, the stamp corresponding to the stamp of 1 is acquired based on the cooperation information, and the stamp rally progressed by the user. The information processing system according to any one of claims 1 to 4, wherein the information is managed. One or more users who transmit a captured image obtained by capturing a medium using a terminal device and receive presentation of information associated with an object included in the medium, and present information to the user via the medium An image search system that is used by one or more customers and configured by one or more information processing devices, and the terminal device in which a terminal program for using an image search function by the image search system is installed. An information processing method by an information processing system that provides a search service in cooperation via communication,
In response to a request from the customer, the object determined by the customer, information associated with the object, and linkage information for accessing the information held in an external system different from the information processing system; , And a storage step for associating and storing in the storage unit;
Identification information for identifying the terminal device on which the terminal program is mounted, and the identification information realized by the terminal program mounted on the terminal device that acquires the identification information generated in the information processing system An acquisition step;
A first transmission step of transmitting the identification information acquired by the identification information acquisition step to the image search system when transmitting the captured image to the image search system and using an image search function of the image search system. When,
The object stored in the storage unit is searched based on the captured image transmitted from the terminal device in order to use the image search function, and the cooperation information associated with the searched object is transmitted to the captured image. A second transmission step of transmitting to the original terminal device;
A log recording step for recording in the image search system log information relating to the use of the image search function, the log information including at least the identification information transmitted in the first transmission step in the use of the image search function When,
In the terminal device equipped with the terminal program, the identification information generated for the terminal device is executed when the access via the network is executed based on the cooperation information transmitted in the second transmission step. A third transmission step for transmitting to the access destination;
An information processing method characterized by comprising: An external system different from the information processing system for providing content in cooperation with the information processing system according to claim 1,
A member information storage unit for storing user information registered as a member user using the external system;
A terminal device possessed by the member user, in which cooperation information indicating an access destination to the external system is obtained by using an image search function of the information processing system based on a captured image obtained by capturing a medium From the identification information generated in the information processing system, the identification information for identifying each of the terminal devices on which a terminal program for using the information processing system is mounted is sent to the external system. An acquisition unit to be acquired with access;
When the user information of the member user associated with the identification information acquired by the acquisition unit is registered in the member information storage unit, content is provided to the terminal device of the member user according to access A providing department;
An external system characterized by comprising: An external system different from the information processing system that provides a service that is allowed to be used up to a predetermined number of times in a predetermined period for the same user in cooperation with the information processing system according to claim 1,
A terminal device in which the image search function of the information processing system is used based on a captured image obtained by capturing an image of the medium and cooperation information indicating an access destination to the external system is acquired. The identification information generated in the information processing system, the identification information identifying the terminal device on which a terminal program for using the information processing system is mounted is acquired with access to the external system An acquisition unit to
A determination unit that determines, based on the identification information acquired by the acquisition unit, whether or not the number of uses of the service used within a predetermined period according to access by the same terminal device is up to a predetermined number;
Based on the determination result by the determination unit, limiting the provision of services according to access from the terminal device, and a limiting unit for notifying the terminal device that it is limited,
An external system characterized by comprising: An external system different from the information processing system that provides a stamp rally service in cooperation with the information processing system according to claim 1,
A management unit that manages the stamps constituting the stamp rally;
In the information processing system, from the terminal device possessed by the user who has acquired the link information indicating the access destination to the external system using the image search function of the information processing system based on the captured image obtained by capturing the medium. An acquisition unit that is generated identification information, and acquires the identification information for identifying the terminal device on which a terminal program for using the information processing system is mounted, with access to the external system;
A first determination unit that determines which stamp corresponds to the cooperation information in response to an access from the terminal device by the cooperation information;
Based on the identification information acquired by the acquisition unit and the determination result by the determination unit, a recording unit that records the progress of the stamp rally indicating which stamps are aligned according to access by the same terminal device When,
Based on the progress recorded by the recording unit, a second determination unit for determining whether or not all the stamps constituting the stamp rally are aligned by the same terminal device;
An external system characterized by comprising:

Images