JP2015038717A - Information processing system, information processing method, terminal device, and application program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily set a function of an application to be used by another application according to the objective of the other application.SOLUTION: An information processing system comprises a function for providing cooperation information determined on the basis of a captured image. An application program controls a computer to function as: a first reception unit which receives specification information specified by a software program different from the application program and also a call request for calling the application program from the software program; and a transmission unit which transmits a captured image of an information medium captured by a terminal device and the specification information to the information providing system. The information providing system comprises an acquisition unit which acquires cooperation information on the basis of the captured image and specification information transmitted by the transmission unit. The terminal device comprises a second reception unit which receives the cooperation information acquired by the information providing system.

Description

  The present invention relates to an information processing system that provides information via a network, an information providing method, a terminal device, and an application program.

  2. Description of the Related Art Conventionally, a system that provides electronic information related to a paper medium to a user browsing the paper medium is known (for example, Patent Document 1).

  According to Patent Document 1, the management server performs feature extraction suitable for image recognition including text on original data on a paper medium, and creates and registers an index using the extracted feature amount. The management server executes a partial search process based on a partial image that the user has captured and transmitted a paper medium with a camera-equipped mobile phone terminal, etc., identifies a page or article corresponding to the partial image, and sets the article information to the user's Send to camera phone.

  Further, in Patent Document 1, the management server can register a URL (Uniform Resource Locator) of the link destination for the article information transmitted to the user's camera-equipped mobile phone terminal. The user can view the linked content together with the article information transmitted to the camera-equipped mobile phone terminal.

  Here, it is conceivable that the user's camera-equipped mobile phone terminal is equipped with application software (hereinafter referred to as an imaging transmission application) in which the imaging function and the transmission function of the captured image to the management server are integrated. . By using this imaging transmission application, the user does not need to specify the URL of the management server or activate the transmission function of the captured image, which is convenient.

  If the functions of the imaging transmission application described above can be used from other applications, it can be expected that convenience will be further improved. For example, it is conceivable to extend the function of the event application for some event so that the function of the imaging transmission application can be used from the event application. The user can obtain content specific to the event based on the captured image by using the function of the imaging transmission application extended in the event application.

  As described above, in order to make the function of the imaging transmission application available from the event application, conventionally, a method of incorporating the functional part of the imaging transmission application into the event application has been generally used. In this method, the user can use the function of the imaging transmission application without impairing the feeling of use of the event application. In addition, since the user can use the function of the imaging transmission application and the event application as one application, a series of operations can be executed smoothly.

  However, in the above-described method, the function of the imaging / transmission application is ported to the event application, so that a long period of time is required for development, man-hours are increased, and costs are increased. In addition, since the development is a scratch development, there is a problem that a long period of time is required from the specification examination to the actual development. In other words, the lengthening of the period from the specification study to the start of development leads to an increase in cost and makes it difficult to respond to a sudden request from a customer. For this reason, there is a risk of development lacking efficiency and flexibility when each application function is linked or integrated.

  The present invention has been made in view of the above, and an object of the present invention is to make it possible to easily set the function of an application used from another application according to the purpose of the other application.

  In order to solve the above-described problems and achieve the object, the present invention includes an information providing system that includes one or more information processing apparatuses and has a function of providing linked information specified based on a captured image, An information processing system including an application program for accessing a providing system and using a function, and a terminal device that is communicably connected to the information providing system via a network and can install the application program. Includes a first receiving unit that receives a call request for calling an application program from a software program together with designation information specified by a software program different from the application program, and a captured image obtained by imaging an information medium in a terminal device The information providing system functions as a transmission unit that transmits the designation information to the information providing system, and the information provision system includes an acquisition unit that obtains cooperation information based on the captured image and the designation information transmitted by the transmission unit. And a second receiving unit that receives the cooperation information acquired by the information providing system.

  According to the present invention, there is an effect that the function of an application used from another application can be easily set according to the purpose of the other application.

FIG. 1 is a diagram illustrating a configuration of an example of a search system according to the embodiment. FIG. 2 is a diagram illustrating an example of image data that is a source of index data stored in the DB according to the embodiment. FIG. 3 is an exemplary functional block diagram illustrating functions of the server according to the embodiment. FIG. 4 is a diagram illustrating an exemplary configuration of a DB according to the embodiment. FIG. 5 is a diagram illustrating an example of the XML code according to the embodiment. FIG. 6 is a flowchart illustrating an example of the operation of the server according to the embodiment. FIG. 7 is a flowchart illustrating an example of index collaboration information presentation processing according to the embodiment. FIG. 8 is a flowchart illustrating an example of processing when the user is prompted to perform processing for selecting 1 from a plurality of indexes according to the embodiment. FIG. 9 is a diagram illustrating a configuration of an example of the information processing system according to the first embodiment. FIG. 10 is a block diagram illustrating a configuration of an example of a terminal device according to the first embodiment. FIG. 11 is a functional block diagram of an example for explaining functions of the terminal device according to the first embodiment. FIG. 12 is a diagram schematically illustrating an example of a configuration of a terminal program according to the first embodiment. FIG. 13 is a diagram schematically illustrating an operation flow when the second execution unit calls the first execution unit according to the first embodiment. FIG. 14 is a functional block diagram illustrating an example of functions of the first execution unit according to the first embodiment. FIG. 15 is a functional block diagram illustrating an example of the function of the second execution unit according to the first embodiment. FIG. 16 is a diagram schematically showing a flow of processing according to the first application example of the first embodiment. FIG. 17 is a diagram schematically showing a flow of processing according to the second application example of the first embodiment. FIG. 18 is a diagram illustrating a configuration of an example of an information processing system according to the second embodiment. FIG. 19 is a functional block diagram illustrating an example of functions of the terminal device according to the second embodiment. FIG. 20 is a diagram for schematically explaining an operation of starting the first execution unit from the browser application according to the second embodiment. FIG. 21 is a diagram for schematically explaining an operation of starting the first execution unit from the browser application according to the second embodiment. FIG. 22 is a diagram for schematically explaining an operation of starting the first execution unit from the browser application according to the second embodiment.

  Hereinafter, embodiments of a terminal device, an information processing program, an information processing method, and an information processing system will be described in detail with reference to the accompanying drawings.

(Image search system applicable to the embodiment)
Prior to the description of the embodiment, an image search system applicable to the embodiment will be described in order to facilitate understanding. The image retrieval system extracts an object from a captured image captured by an imaging device, compares the extracted object with an object stored in advance in a database, and extracts an object from the captured image among objects stored in the database. An object having the highest similarity to the extracted object is specified. And the link information of the commercial site previously linked | related with the specified object etc. are shown to a user.

  FIG. 1 shows a configuration of an example of an image search system applicable to the embodiment. In the image search system 200 shown in FIG. 1, the image search server 11 and the terminal device 20 are connected to the network 10. The network 10 is, for example, the Internet, and TCP / IP (Transmission Control Protocol / Internet Protocol) is applied as a communication protocol. The network 10 may be a network cloud that includes a plurality of computers connected to each other via a network and is externally a network group in which only input / output is shown as a black box with the inside hidden.

  A plurality of commercial sites 30 </ b> A and 30 </ b> B are further connected to the network 10. These commercial sites 30 </ b> A and 30 </ b> B can ship products according to orders placed through the network 10.

  The terminal device 20 can communicate with the network 10 by wireless or wired. The terminal device 20 can perform an operation in accordance with a command received via the network 10, and can display an image received via the network 10 on a UI (user interface) including a display device and an input device. Data can be transmitted to the network 10 in accordance with the display or user operation performed on the UI.

  In addition, the terminal device 20 has an imaging function, and can capture an image of a subject in response to an imaging instruction operation on the UI. The terminal device 20 can transmit a captured image obtained by imaging toward the network 10.

  The image search server 11 is connected to a database (DB) 12 and can perform a search process according to the embodiment on the DB 12 in accordance with a request received via the network 10. In the embodiment, each index data for searching the plurality of image data 50A and 50B is stored in the DB 12 in association with the cooperation information. The index data is data obtained by analyzing the binary of the image data, and is data obtained by converting the image data into a structure for search.

  In addition, it is also possible to store image data directly in DB12. The file name and path of the image data may be stored in the DB 12.

  The image search server 11 receives image data via the network 10, searches the DB 12 based on the received image data, and acquires cooperation information corresponding to the image data as a search result. The image search server 11 returns the cooperation information acquired as the search result to the transmission source of the image data.

  FIG. 2 shows an example of the image data 50A and 50B that are the basis of the index data stored in the DB 12. Here, an object included in the image data is defined. An object refers to an image of a main object in an image based on image data, and is an image that is distinguished from a background image of the main object and accompanying information of the main object in the image. It is.

  As an example, it is assumed that the image data 50A and 50B are a content page that is the contents of a catalog magazine and a cover page that is the cover of the catalog magazine. 2A shows an example of image data 50A based on the content page image 130A, and FIG. 2B shows an example of image data 50B based on the cover page image 130B.

  FIG. 2A shows an example in which one object 131 is arranged at the center of the image 130A. In the image 130 </ b> A, a target area 134 including the object 131 is set for the object 131. FIG. 2A shows an example in which the ratio of the size of the object 131 to the size of the target area 134 is small. In the case of a catalog magazine, the purpose of the content page is to introduce the product, so the object 131 that is an image showing the product is displayed larger than the content page.

  FIG. 2B shows an example in which a plurality of objects 132 and 133 are arranged on the image 130B. Here, the object 132 is an image obtained by reducing the object 131 shown in FIG. 2A, and the object 133 is another image. More objects may be arranged on the image 130B. In this example, as the display page summarizes the contents of the catalog magazine, the objects 132 and 133 included in the content page are reduced and displayed as thumbnails. In the image 130B, the target area 135 includes the objects 132 and 133 and is set to cover the entire display page (image 130B).

  The target areas 134 and 135 are used as indices for recognizing the object 131 and the objects 132 and 133, respectively. The target area includes the entire object and can be set to any size smaller than the entire image including the object.

  The linkage information will be described. The cooperation information is, for example, information indicating a function that cooperates with an object included in an image, and more specifically, information including an access method for accessing attribute information of the object. Therefore, it can be considered that the cooperation information is actually stored in the DB 12 in association with the object included in the image.

  As an example, in the object 131 included in the image 130A of the content page, the linkage information includes, for example, the URL (Uniform Resource Locator) of the commercial site 30A that handles the product indicated by the object 131, or the URL of the purchase page that performs purchase settings for the product. It is. Not limited to these, as the linkage information, a telephone number or an e-mail address of a store handling the product may be used. Furthermore, information for downloading a data file including information on the product may be used as linkage information. In the DB 12, a plurality of pieces of cooperation information can be associated with one object.

  As another example, for each object 132 and 133 included in the cover page image 130B, for example, a URL of a site on the network 10 that introduces the catalog magazine itself including the cover page is used as linkage information. Can do.

  FIG. 3 is a functional block diagram illustrating an example of the functions of the image search server 11. The image search server 11 includes a reception unit 100, an extraction unit 101, a calculation unit 102, a search unit 103, a presentation unit 104, and a control unit 110. In FIG. 3, the DB 12 is shown to be included in the image search server 11. The control unit 110 controls the overall operation of the image search server 11. The image search server 11 includes one or more computers.

  The receiving unit 100 receives data transmitted via the network 10. For example, a captured image (captured image 40A) transmitted from the terminal device 20 is received by the receiving unit 100.

  The extraction unit 101 performs image analysis on the captured image 40A received by the reception unit 100, detects feature points from the captured image 40A, and extracts objects included in the captured image 40A. For example, the extraction unit 101 performs binarization processing on the captured image 40A, performs edge detection or normalization processing on the binarized captured image 40A, and detects feature points of the captured image 40A. . An object is represented by this detected feature point.

  The calculation unit 102 compares the feature points of the captured image 40A extracted by the extraction unit 101 with the feature points representing each object searched by each index stored in the DB 12. Based on the comparison result, the similarity between the object included in the captured image 40A and each object searched by each index stored in the DB 12 is calculated. The similarity can be obtained, for example, by comparing the relationship between each feature point extracted from the image for each image. As an example, a feature amount is calculated by quantifying the relationship between feature points, an absolute value of a difference between feature amounts is obtained between images, and when the obtained absolute value of the difference is smaller, the similarity between the images is Suppose it is expensive.

  Based on the similarity calculated by the calculation unit 102, the search unit 103 searches for an image having a high similarity to the object included in the captured image 40A among the objects searched by the indexes stored in the DB 12. . At this time, the search unit 103 outputs, as a search result, one index including an object having the highest similarity to each object included in the captured image 40A from each object searched by each index stored in the DB 12. .

  Not limited to this, the search unit 103 searches a predetermined number of indexes according to the order including objects having high similarity to the objects included in the captured image 40A from the objects searched by the indexes stored in the DB 12. It may be output as a result, or one or a plurality of indexes for searching for objects whose similarity is equal to or greater than a threshold value may be output as a search result. Furthermore, it is conceivable that an index whose similarity is within a predetermined range (such as 5%) from the maximum value of the similarity calculated for the captured image 40A may be output as a search result.

  The presenting unit 104 takes out the cooperation information associated with the index of the search output by the search unit 103 from the DB 12 and presents (transmits) it to the terminal device 20. Therefore, the presentation unit 104 has a function of a transmission unit that transmits cooperation information to the terminal device 20. The cooperation information may be presented by transmitting the cooperation information directly to the terminal device 20, or by generating a Web page including the cooperation information and accessing the Web page toward the terminal device 20 (URL Etc.) may be presented. By accessing this Web page from the terminal device 20, the cooperation information can be acquired.

  Not limited to this, application software (hereinafter abbreviated as an application) that performs a series of operations from capturing the captured image 40A to displaying the presented cooperation information is installed in advance in the terminal device 20 and presented. It is also possible to acquire the cooperation information using this application.

  The image search server 11 is a general computer device (information processing) including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and a storage device such as a hard disk drive. Device). Further, the image search server 11 is not limited to being configured by one computer device, and may be configured to operate in a distributed manner on a plurality of computer devices, for example.

  The reception unit 100, the extraction unit 101, the calculation unit 102, the search unit 103, the presentation unit 104, and the control unit 110 described above are realized by a search program that operates on the CPU. Not limited to this, a part or all of the reception unit 100, the extraction unit 101, the calculation unit 102, the search unit 103, the presentation unit 104, and the control unit 110 may be realized by using separate hardware that cooperates with each other. Good. The search program for executing the search process according to the embodiment is provided by being recorded in a computer-readable recording medium such as a CD or a DVD in an installable or executable format file.

  Furthermore, a search program for executing the search process according to the embodiment may be provided by being stored on a computer connected to a communication network such as the Internet and downloaded via the communication network. Further, a search program for executing the search processing according to the embodiment may be provided or distributed via a communication network such as the Internet.

  The search program for executing the search processing according to the embodiment includes, for example, a module configuration including the above-described units (reception unit 100, extraction unit 101, calculation unit 102, search unit 103, presentation unit 104, and control unit 110). As actual hardware, the CPU reads the search program from the storage device, for example, and executes it to load the above-described units onto the main storage device (for example, RAM), and the respective units are generated on the main storage device. It has become so.

  In such a configuration, the user captures, for example, a catalog magazine page using the terminal device 20, and obtains the captured image 40A and the captured image 40B. In this example, the captured images 40A and 40B are captured images obtained by capturing the content page and the cover page of the catalog magazine, respectively. For example, the user transmits the captured image 40 </ b> A to the image search server 11 via the network 10.

  The captured image 40A is received by the receiving unit 100 of the image search server 11, the feature point of the received captured image 40A is detected by the extracting unit 101, and an object is extracted. Further, the calculation unit 102 calculates the similarity between the object included in the captured image 40A and each object searched by each index stored in the DB 12, based on the feature points detected by the extraction unit 101. Based on this calculation result, the search unit 103 searches the index. Then, the presentation unit 104 retrieves the cooperation information from the DB 12 based on the search output of the search unit 103 and presents it to the terminal device 20 that is the transmission source of the captured image 40A.

  Based on the cooperation information presented to the terminal device 20 from the image search server 11, the user can access the commercial site 30A, for example, and purchase a product indicated by the object included in the captured image 40A.

  Here, for example, it is assumed that an image including an object very similar to the object 131 included in the image 130A in FIG. 2A is captured by the terminal device 20 as the captured image 40A. The DB 12 stores an image 130A including the object 131 and an image 130B including an object 132 obtained by reducing the object 131.

  In this case, as a result of the search based on the similarity calculated from the feature point detection result of the captured image 40A, the image 130B may be searched instead of the image 130A. In this case, the user cannot obtain desired cooperation information. In order to prevent such a situation, in the embodiment, the search unit 103 searches for an image stored in the DB 12 based on the size ratio between the target area (imaging range) and the object.

  This will be described more specifically with reference to FIG. In the image 130A illustrated in FIG. 2A, the object 131 is arranged so as to occupy a large space with respect to the image 130A, and the target area 134 has a size slightly larger than the size of the object 131 with respect to the object 131. Is set. On the other hand, in the image 130B illustrated in FIG. 2B, the object 132 in which the object 131 is reduced is arranged, and the target area 135 is on the entire surface of the image 130B, that is, the size of the object 132. It is set with a sufficiently large size.

  In the image 130 </ b> A, it is assumed that the first ratio of the size of the object 131 to the size of the target area 134 is, for example, a value obtained based on each area is 1/2. On the other hand, in the image 130B, the value obtained by calculating the second ratio of the size of the object 132 to the size of the target region 135 as an area is, for example, 1/16, which is significantly different from the case of the image 130A. Note that the size ratio is not limited to the method of obtaining based on the area. Such information indicating the size ratio may be obtained in advance and included in an index and stored in the DB 12.

  Furthermore, for the captured image 40A received from the terminal device 20, it is conceivable to obtain a ratio between the size of the object extracted by the extraction unit 101 and the size of the captured image 40A itself. For example, assume that the captured image 40A is an image as shown by an image 130A in FIG. 2A, and the third ratio of the size of the object (referred to as the object 131) to the size of the captured image 40A is, for example, 1/4. .

  In this case, the third ratio is closer to the first ratio among the first ratio and the second ratio described above. Therefore, the search unit 103 outputs an index corresponding to the image 130A as a search result for the captured image 40A. The presentation unit 104 presents the cooperation information associated with the index corresponding to the image 130 </ b> A to the terminal device 20. Thus, by selecting an index based on the ratio of the object size to the image size, a desired index can be selected more accurately.

  FIG. 4 shows an example of the configuration of the DB 12 for realizing such a process of searching for an image based on the object size ratio. FIG. 4A shows an example of the index table 120 for identifying each index data stored in the DB 12. As illustrated in FIG. 4A, each index data is stored in the DB 12 in association with an ID and binary data of the index data.

  Further, the format of the ID is not particularly limited as long as each index data stored in the DB 12 can be identified. When each index data stored in the DB 12 is for searching for an object of each page of a catalog magazine, a value including a page number on which the object is posted can be used as an ID.

  FIG. 4B shows an example of a target area table 121 that associates an image ID with a target area. In this example, the target area is described using XML (Extensible Markup Language), which is a kind of markup language that defines and describes the meaning and structure of data using tags. As will be described later, the description of the linkage information is also embedded in the description of the target area. The XML code in which the target area is described may be stored directly in the field of the target area table 121, or the XML code is described in a separate file, and the file name and path are described in the field. May be.

FIG. 5 shows an example of the XML code 140 shown in FIG. The XML code 140 shown in FIG. 5 is an example of describing a target area of a content page having a catalog magazine. A page is defined by the portion sandwiched between the outermost tags “<page>” and “</ page>”. In each of the portions 141 ₁ and 141 ₂ , one target region is defined by the portion sandwiched between the tags “<hotspot>” and “</ hotspot>”. Thus, a plurality of target areas can be defined for one page.

In the portion 141 ₁ , the first line 142 indicates that the range in the page of the target area is, for example, the coordinates (x1, y1) of the upper left corner by the portion sandwiched between the tags “<area>” and “</ area>”. And the coordinates of the lower right corner (x2, y2).

In each of the portions 143 ₁ and 143 ₂ , _one piece of linkage information is defined by the portion sandwiched between the tags “<link>” and “</ link>”. In this way, a plurality of pieces of cooperation information can be defined for one target area. In each part 143 _1, the tag "<title>" and "</ title>" by a portion held in cooperation information title is defined. The type of the linkage information is defined by the portion sandwiched between the tags “<type>” and “</ type>”. Further, in this example, address information on the network 10 is defined as the linkage information by a portion sandwiched between tags “<URI>” and “</ URI>”.

  As described above, since the XML code 140 defines the target area in the page and the cooperation information in the target area, the function based on the cooperation information associated with the object included in the target area in the page can be executed. it can.

  FIG. 6 is a flowchart illustrating an example of the operation of the image search server 11 according to the embodiment. First, for example, a captured image 40 </ b> A obtained by being imaged by the terminal device 20 is transmitted to the image search server 11 via the network 10. In step S100, in the image search server 11, the receiving unit 100 receives the captured image 40A. In the next step S101, the extraction unit 101 performs image analysis on the received captured image 40A, detects feature points, and extracts objects.

  In the next step S102, the search unit 103 calculates the ratio of the object size to the size of the captured image 40A. Note that the ratio of the size of the object to the size of the target area is calculated in advance for each image stored in the DB 12. Information indicating the ratio of each image stored in the DB 12 can be embedded in the XML code 140 described above.

  In the next step S103, the calculation unit 102 compares the object extracted from the captured image 40A with the object searched by each index stored in the DB 12. More specifically, the calculation unit 102 calculates each degree of similarity between an object included in the captured image 40A and an object searched by each index stored in the DB 12.

  In the next step S104, the search unit 103 determines whether there is an index for searching for an object that matches the object included in the captured image 40A among the indexes stored in the DB 12, based on the similarity calculated in step S103. Determine whether. In this example, among objects searched for in the index included in the DB 12, an object having a similarity equal to or higher than a threshold is determined as an index for searching for an object that matches the object included in the captured image 40A. To do.

  If it is determined in step S104 that there is an index that matches the object, the search unit 103 shifts the process to step S105. In step S <b> 105, the search unit 103 determines whether or not the object matches the plurality of indexes among the indexes stored in the DB 12. If it is determined that only one index matches the object, the search unit 103 uses the one index as a search output and shifts the process to step S107 described later. On the other hand, when it is determined that the objects match with each other in the plurality of indexes, the search unit 103 shifts the processing to step S106.

  In step S106, the search unit 103 obtains the ratio of the size of the plurality of objects searched by each of the plurality of indexes to the size of the target area. The ratios related to the plurality of objects may be obtained in advance and stored in the DB 12. At the same time, the search unit 103 obtains the ratio of the object size to the size of the captured image 40A for the captured image 40A. Then, out of the ratios obtained for each of the plurality of objects, a ratio closest to the ratio obtained for the captured image 40A is obtained, an index for searching for an object corresponding to the obtained ratio is selected as a search output, The process proceeds to step S107.

  In step S <b> 107, the presentation unit 104 retrieves the cooperation information associated with the index of the search output by the search unit 103 from the DB 12. Then, the presentation unit 104 presents the extracted cooperation information to the terminal device 20 that is the transmission source of the captured image 40A. Then, a series of processes according to the flowchart of FIG.

  If it is determined in step S104 described above that there is no matching index, the process proceeds to step S108, and a message indicating that the matching index is not stored in the DB 12 is displayed on the terminal device 20 by the presentation unit 104. Presented to. Then, a series of processes according to the flowchart of FIG.

  FIG. 7 is a flowchart illustrating an example of index linkage information presentation processing in step S107 in FIG. First, in step S200, the presentation unit 104 specifies a target area corresponding to the captured image 40A in the object searched by the search output index of the search unit 103. In the next step S201, the presentation unit 104 refers to the DB 12 and determines whether or not there is cooperation information associated with the identified target area. If it is determined that there is, the presenting unit 104 shifts the process to step S202, and determines whether or not a plurality of pieces of cooperation information are associated with the target region.

  If the presenting unit 104 determines in step S202 that a plurality of pieces of cooperation information are associated with the target area, the presenting unit 104 shifts the process to step S203. In step S203, the presentation unit 104 creates a function list indicating each function based on the plurality of pieces of cooperation information, and presents a selection screen for selecting one function from the function list to the terminal device 20. For example, when a URL and a telephone number are associated as linkage information for the specified target area, a selection screen for selecting a function for accessing the address indicated by the URL and a telephone function for the telephone number Is generated and presented to the terminal device 20. Then, a series of processes according to the flowchart of FIG.

  On the other hand, in step S202, when the presentation unit 104 determines that only one piece of cooperation information is associated with the target area, the presenting unit 104 shifts the process to step S204. In step S <b> 204, a screen for executing the function based on the one cooperation information is generated and presented to the terminal device 20. Then, a series of processes according to the flowchart of FIG.

  If it is determined in step S201 described above that the cooperation information is not associated with the target area specified in step S200, the process proceeds to step S205. In step S <b> 205, the presentation unit 104 presents a predetermined specific screen to the terminal device 20. Then, a series of processes according to the flowchart of FIG.

  In the above description, in step S106, one index is selected from a plurality of indexes on the image search server 11 side, but this is not limited to this example. That is, it is possible to prompt the user to perform a process of selecting one index from a plurality of indexes.

  FIG. 8 is a flowchart illustrating an example of a process in which the user is prompted to perform the process of selecting 1 from a plurality of indexes based on the search output of the search unit 103. If it is determined in step S105 of FIG. 6 that the object included in the captured image 40A matches with the plurality of indexes, the search unit 103 shifts the processing to step S106 'shown in FIG.

  When the process proceeds to step S106 ′, the presentation unit 104 generates a screen for presenting information indicating a plurality of indexes determined to be matched to the user in step S300, and displays the screen on the terminal device 20. To present. For example, it is conceivable that the presentation unit 104 generates a screen for displaying an image of each object searched by each of a plurality of indexes and presents the screen to the terminal device 20.

  The presentation unit 104 waits for reception of a selection result in which one index (object image) is selected from a plurality of indexes (object images) from the terminal device 20 (step S301). When receiving from the terminal device 20 that one index has been selected from the plurality of indexes presented in step S300, the presenting unit 104 determines, in step S302, from the plurality of object images that are determined to match the indexes. 1 is selected according to the reception result. And the cooperation information linked | related with the index by which the selected object is searched is taken out from DB12, and is shown with respect to the terminal device 20. FIG.

  Thus, the user can select a desired image more accurately by presenting a plurality of images to the user.

  In such a configuration, the terminal device 20 can display each information presented by the presentation unit 104 of the image search server 11 using a browser application for browsing a website on the network 10. . Further, the terminal device 20 can transmit a request and data to the image search server 11 using this browser application.

  In this case, in the image search server 11, the presentation unit 104 generates display control information for presenting each piece of information to the terminal device 20 using, for example, HTML (HyperText Markup Language) or various script languages. The file is placed at a predetermined address on the network 10. By notifying the terminal device 20 of this address from the presentation unit 104, the display control information can be accessed from the terminal device 20, and the presentation of each information by the presentation unit 104 can be displayed on the terminal device 20.

  Here, it is highly convenient for the user if a series of processing related to image search from the transmission of the captured image from the terminal device 20 to the display of the presentation information described above can be executed in an integrated manner with one application software. . In the embodiment, a series of processes related to the image search is executed by one application software, and this application software is called an image search application.

(First embodiment)
Next, the information processing system according to the first embodiment will be described. FIG. 9 shows an example of the configuration of the information processing system according to the first embodiment. In FIG. 9, the information processing system 1 includes an image search system 200A and a terminal device 20 that are connected to each other via a network. The image search system 200A corresponds to the image search system 200 described above.

In this example, the image search system 200A includes a plurality of image search servers 11A, 11B, and 11C each including one or more information processing apparatuses. Each of the image search servers 11A, 11B, and 11C is configured by one or more computers. Each image search server 11A, 11B, and 11C can include one or more DBs 12. In the example of FIG. 9, the image search server 11A includes a DB12 of 3 DB 12 a _1, 12A ₂ and 12A _3, the image search server 11B includes a first DB12 of DB 12 b _1, the image search server 11C is DB 12 c ₁ and 12C ₂ 2 DBs 12 are included.

  The terminal device 20 can transmit the captured image 40 and designation information for designating the image search server 11 and the DB 12 to the image search system 200A. The image search system 200 </ b> A selects the image search server 11 and the DB 12 to be used for the search by the captured image 40 based on the designation information transmitted from the terminal device 20.

In the image search system 200A, server names are assigned to the image search servers 11A, 11B, and 11C. In this example, the server names of the image search servers 11A, 11B, and 11C are image search servers # 0, # 1, and # 2, respectively. Similarly, in the image search system 200A, a DB name is assigned to each DB 12A ₁ , 12A ₂ and 12A ₃ , DB 12B ₁ , and DB 12C ₁ and 12C ₂ . In this example, the DB names of the respective DBs 12A ₁ , 12A ₂ and 12A ₃ are DB # 1, # 2 and # 3, the DB name of the DB 12B ₁ is DB # 11, and the DB names of the DB 12C ₁ and 12C ₂ Are DB # 21 and # 22, respectively.

The terminal device 20 adds the server name and the DB name to the captured image 40 and transmits them to the image search system 200A. Image retrieval system 200A according server name and DB name appended to the captured image 40 received, the image search server 11A, and 11B and 11C, each DB 12 a _1, 12A ₂ and 12A _3, DB 12 b _1, and, DB 12 c ₁ And 12C ₂ determine the image search server and DB used for image search.

Hereinafter, the image search servers 11A, 11B, and 11C will be described as the image search server 11 unless they are particularly distinguished. Similarly, DB12A ₁ , 12A ₂ and 12A ₃ , DB12B ₁ , and DB12C ₁ and 12C ₂ will be described as DB 12 unless they are particularly distinguished.

  The terminal device 20 includes the above-described image search application 220 and one or more specific applications that respectively use the functions of the image search application 220. In the example of FIG. 9, three applications 221A, 221B, and 221C are shown as specific applications. In the first embodiment, the image search application 220 is called and used from this specific application. Of course, the terminal device 20 may include only one specific application that calls the image search application 220, or may include a plurality of different specific applications 221A, 221B, and 221C as shown in FIG. The image search application 220 can also be activated and used independently.

  Hereinafter, unless otherwise specified, a plurality of specific applications 221A, 221B, and 221C will be described as representative applications.

  The specific application can pass parameters to the image search application 220 when calling the image search application 220. The parameter includes information indicating the caller, and may further include other values. A URL scheme can be used as the format of such parameters. The URL scheme includes a scheme name and predetermined information in an expression format defined for each scheme, and the scheme name and the predetermined information are described by being separated by a colon “:”. For example, information (application name or the like) indicating the caller is described in the scheme name portion, and other values (variables, commands, or the like) are described in a predetermined format after the colon.

  The image search application 220 can change the behavior according to the parameters passed from the specific application. For example, the image search application 220 can change the display appearance (referred to as skin) on the terminal device 20 of the image search application 220 and the user interface according to the passed parameters.

  Further, for example, the specific application can include the server name and DB name described above in the parameters and pass them to the image search application 220. The image search application 220 adds the server name and DB name included in the parameters passed from the specific application to the captured image 40 and transmits the captured image 40 to the image search system 200A. As a result, the image search system 200A can execute an image search for the same captured image 40 using different image search servers 11 and DB 12, and search for different cooperative information for the unified captured image 40 as a search result. It can be returned to the terminal device 20.

For example, it is assumed that the image search application 220 is called from a specific application by passing the parameter A including the server name “image search server # 0” and the DB name “DB # 1”. In this case, the image search application 220 adds the server name “image search server # 0” and the DB name “DB # 1” to the captured image 40 and transmits them to the image search system 200A. The image search system 200A performs an image search using the DB 12A ₁ by the image search server 11A according to the server name and DB name added to the captured image 40, and obtains the URL of the content distribution site 210A obtained as a search result, for example. The cooperation information shown is transmitted to the terminal device 20.

Further, for example, it is assumed that the image search application 220 is called by the application 221B by passing the parameter B including the server name “image search server # 1” and the DB name “DB # 11”. In this case, the image search application 220 adds the server name “image search server # 1” and the DB name “DB # 11” to the captured image 40 and transmits them to the image search system 200A. The image search system 200A performs an image search using the DB 12B ₁ by the image search server 11B according to the server name and the DB name added to the captured image 40, and obtains, for example, the URL of the content distribution site 210B obtained as a search result. The cooperation information shown is transmitted to the terminal device 20.

  FIG. 10 shows an exemplary configuration of the terminal device 20 according to the first embodiment. In the terminal device 20 illustrated in FIG. 10, a CPU 402, a ROM 403, a RAM 404, and a display control unit 405 are connected to a bus 401. In addition, a storage 407, a data I / F 408, an input unit 409, a communication I / F 410, and an imaging unit 411 are connected to the bus 401. The storage 407 is a storage medium capable of storing data in a nonvolatile manner, and is a nonvolatile semiconductor memory such as a flash memory, for example. Not limited to this, a hard disk drive may be used as the storage 407.

  The CPU 402 controls the entire terminal device 20 using the RAM 404 as a work memory in accordance with programs stored in the ROM 403 and the storage 407. The display control unit 405 converts the display control signal generated by the CPU 402 into a signal that can be displayed by the display device 406 and outputs it.

  The storage 407 stores programs and various data to be executed by the CPU 402 described above. The data I / F 408 inputs data from the outside. As the data I / F 408, for example, an interface such as USB (Universal Serial Bus) or IEEE 1394 (Institute of Electrical and Electronics Engineers 1394) can be applied.

  The input unit 409 includes an input device that receives a user input and outputs a predetermined control signal. The user can issue an instruction to the terminal device 20 by operating the input device in accordance with the display on the display device 406, for example. Note that it is preferable that the input device that accepts user input is configured integrally with the display device 406 and configured as a touch panel that outputs a control signal corresponding to the pressed position and transmits the image of the display device 406.

  The communication I / F 410 communicates with the network 10 using a predetermined protocol. The imaging unit 411 performs predetermined processing on an optical system, an imaging device, a control drive circuit for the optical system and the imaging device, and a signal output from the imaging device, and outputs the result as image data. The imaging unit 411 executes functions such as imaging and zooming according to an instruction by a user operation on the input unit 409. The captured image obtained by the imaging unit 411 is transmitted to the communication I / F 410 via the bus 401, for example, and is transmitted to the image search server 11 via the network 10 according to a command from the CPU 402.

  FIG. 11 is a functional block diagram illustrating an example of functions of the terminal device 20 according to the first embodiment. 11, the terminal device 20 includes a control unit 2001, a storage 2002, a communication unit 2003, and a UI (user interface) unit 2004. Furthermore, the terminal device 20 includes a first execution unit 2200 and a second execution unit 2300.

  The first execution unit 2200 corresponds to the image search application 220, and the second execution unit 2300 corresponds to the specific application. Therefore, the first execution unit 2200 is called by the second execution unit 2300 and activated. Further, when called from the second execution unit 2300, the first execution unit 2200 returns the process (execution right) to the second execution unit 2300 when the process ends.

  The control unit 2001 controls the overall operation of the terminal device 20. The storage 2002 corresponds to the storage 407 described above, and stores a program for operating on the terminal device 20 and various data. The communication unit 2003 controls the communication I / F 410, for example, and performs communication with the outside through a network.

  The UI unit 2004 includes an input unit 2010 and a display unit 2011. The input unit 2010 receives a control signal output by the input unit 409 illustrated in FIG. 10 in response to a user operation. The display unit 2011 generates display information for displaying an execution screen on the display device 406 illustrated in FIG. At this time, the UI unit 2004 associates the position on the execution screen displayed on the display unit 2011 with the control signal according to the contact position with respect to the input unit 409 received by the input unit 2010.

  The control unit 2001, the communication unit 2003, the UI unit 2004, and the first execution unit 2200 and the second execution unit 2300 described above are realized by a plurality of programs operating on the CPU 402. Hereinafter, the plurality of programs are collectively referred to as “terminal programs”. The terminal program may be stored on a computer connected to the network 10 and provided by being downloaded via a network (not shown). The terminal program may be configured to be provided or distributed via a network. A part or all of the terminal program may be stored in the ROM 403 in advance and provided.

  Not limited to this, the terminal program is recorded in a computer-readable recording medium, such as a CD (Compact Disk) or DVD (Digital Versatile Disk), in an installable or executable format file. Also good. In this case, the terminal program recorded on the recording medium is supplied to the terminal device 20 via, for example, an external drive device connected to the data I / F 408. Not limited to this, a terminal program recorded in a recording medium is temporarily stored on another information processing apparatus such as a personal computer, and communication is performed from the other information processing apparatus via the communication I / F 410 or the data I / F 408. May be supplied to the terminal device 20.

  The terminal program has a module configuration including, for example, the above-described units (the control unit 2001, the communication unit 2003 and the UI unit 2004, and the first execution unit 2200 and the second execution unit 2300). For example, the CPU 402 reads out and executes a terminal program from the storage 407, for example, so that the above-described units are loaded on the main storage device (for example, the RAM 404), and the respective units are generated on the main storage device.

  FIG. 12 schematically illustrates an example of a configuration of a terminal program on the terminal device 20 according to the first embodiment. In FIG. 12, the terminal program 2000 includes an application layer 2100, an API layer 2101, an OS layer 2102, and a hardware I / F layer 2103. The application layer 2100 includes application software (hereinafter, abbreviated as an application) that directly has a function of performing a task that the user of the terminal device 20 wants to execute on the terminal device 20. In the example of FIG. 12, the application layer 2100 includes a first execution unit 2200 and a second execution unit 2300.

  The API layer 2101 includes an API (Application Programming Interface) and mediates between the application layer 2100 and an OS layer 2102 to be described later. The OS layer 2102 includes an OS (Operating System) and controls the overall operation of the terminal program 2000. The OS includes various services that can be commonly used by each application in the application layer 2100. For example, the control unit 2001, the communication unit 2003, and the UI unit 2004 are included in the OS layer 2102. A hardware I / F (HW I / F) layer 2103 mediates between the OS layer 2102 and hardware such as the storage 2002 and the communication I / F 410.

  Of the applications included in the application layer 2100, applications that provide basic functions such as a browser application suitable for browsing various data on a network such as the Internet, and a mailer application that exchanges e-mails are provided. In many cases, the terminal device 20 is installed as a standard together with the OS from the beginning.

  Each application included in the application layer 2100 exchanges data and commands with the OS layer 2102 via the API layer 2101. Thereby, each application uses the hardware function of the terminal device 20 and exchanges data and commands between the applications.

  The first execution unit 2200 according to the first embodiment can be loaded on the main storage device by being called from the second execution unit 2300. In addition, the program for configuring the first execution unit 2200 does not initially exist on the terminal device 20 and can be provided from a network or the like when called from the second execution unit 2300.

  The flow of operations when the second execution unit 2300 calls the first execution unit 2200 according to the first embodiment will be schematically described with reference to FIG. Here, it is assumed that the first execution unit 2200 has not been activated yet in the state where the second execution unit 2300 is initially activated and loaded onto the main storage device. That is, the initial application layer 2100 includes the second execution unit 2300, but does not include the first execution unit 2200. In this state, the second execution unit 2300 calls the first execution unit 2200.

  When the second execution unit 2300 is instructed to call the first execution unit 2200 by a user operation, for example, the second execution unit 2300 requests the OS to start the first execution unit 2200 as illustrated in FIG. (Step S10). In response to this request, the OS determines whether or not a program for executing the first execution unit 2200 (referred to as a first execution unit program) has been installed in the terminal device 20. Here, the OS performs this determination based on whether or not the first execution unit program is stored in the storage 2002 (step S11). Not limited to this, the OS may make this determination with reference to a list of applications installed in the terminal device 20 read in advance in the RAM 404 or the like.

  When the OS determines that the first execution unit program is already installed, the OS reads the first execution unit program from the storage 2002, for example, as illustrated in FIG. (Step S12), and the first execution unit 2200 is activated. Then, for example, the OS moves the execution right of the application from the second execution unit 2300 to the first execution unit 2200.

  On the other hand, when the OS determines that the first execution unit program is not yet installed in the terminal device 20, the OS controls the communication I / F 410 as illustrated in FIG. Then, the first execution unit program is acquired from the outside (step S13). The acquired first execution unit program is temporarily stored in the storage 2002, and installation processing is started in a predetermined procedure. When the first execution unit program is installed, the OS activates the first execution unit 2200 through the process of step S12 described above.

  FIG. 14 is a functional block diagram illustrating an example of the function of the first execution unit 2200 according to the first embodiment. 14, the first execution unit 2200 includes a captured image acquisition unit 2201, a transmission unit 2202, a reception unit 2203, a setting unit 2204, a reception unit 2205, a calling unit 2206, a storage unit 2207, and a presentation unit. 2208.

  The captured image acquisition unit 2201 controls the imaging operation by the imaging unit 411 and acquires a captured image. The transmission unit 2202 performs information transmission processing via a network. For example, the transmission unit 2202 has address information on the network of the image search system 200A in advance, and performs transmission processing on the image search system 200A via the network. The captured image acquired by the captured image acquisition unit 2201 is transmitted to the image search system 200A by the transmission unit 2202. The receiving unit 2203 performs reception processing of information transmitted via the network. For example, the receiving unit 2203 receives the cooperation information presented from the presentation unit 104 of the image search server 11 included in the image search system 200A.

  The setting unit 2204 sets the behavior at the time of execution of the first execution unit 2200 according to parameters passed from the outside. For example, the setting unit 2204 can set the appearance (skin) on the display of the activated first execution unit 2200 according to the parameter. For example, the setting unit 2204 may prepare several different skins in advance and select a skin corresponding to the parameter from the prepared skins.

  The skin includes, for example, image data, a drawing command, and layout information. The first execution unit 2200 can have a plurality of data constituting the skin in advance. Not limited to this, the first execution unit 2200 may acquire data constituting the skin from the outside via a network, or may acquire the data from the second execution unit 2300 that calls the first execution unit 2200.

  Further, the setting unit 2204 can set the UI of the first execution unit 2200 according to the parameters. For example, the setting unit 2204 may set the display and layout of each component such as buttons provided on the UI according to the parameters. At this time, the first execution unit 2200 can have a plurality of pieces of information for configuring the UI, such as display and layout of each component provided in the UI, in advance. Not limited to this, the first execution unit 2200 may acquire information for configuring the UI from the outside via a network, or may acquire the information from the second execution unit 2300 that calls the first execution unit 2200. Good.

  Furthermore, the setting unit 2204 can set the image search server 11 and the DB 12 used for image search based on the designated parameters. For example, when “server name” and “DB name” are included in the designated parameters, these “server name” and “DB name” are passed to the transmission unit 2202. The transmission unit 2202 adds these “server name” and “DB name” to the captured image 40 when transmitting the captured image 40 to the image search system 200A.

  The accepting unit 2205 accepts parameters passed from the second execution unit 2300, for example. The accepting unit 2205 passes the accepted parameters to, for example, the setting unit 2204. The calling unit 2206 calls the caller who has called the first execution unit 2200. For example, the calling unit 2206 performs a process of calling the second execution unit 2300 when the first execution unit 2200 called and started by the second execution unit 2300 is terminated.

  The storage unit 2207 stores the content data acquired based on the image search result by the image search system 200A. At this time, the storage unit 2207 can store the content data in association with information indicating the caller included in the parameter passed from the second execution unit 2300. However, the storage unit 2207 may store the content data and information (URL or the like) indicating the acquisition source of the content data in association with each other. In this way, content data can be separated by storing the content data in association with the caller or acquisition source, and the content data can be collected.

The presentation unit 2208 generates presentation information based on the cooperation information received by the reception unit, and passes it to the display unit 2011. As an example, let us consider a case where a plurality of pieces of cooperation information are acquired by an image search based on one captured image 40 transmitted from the terminal device 20 to the image search system 200A. In addition, as shown by the parts 143 ₁ and 143 ₂ in FIG. 5, a plurality of pieces of linkage information can be defined for one object.

  The presenting unit 2208 can filter the plurality of pieces of acquired cooperation information using the parameters received by the receiving unit 2205 and extract predetermined cooperation information. For example, the parameter is configured to include a part or all of one piece of linkage information. The presenting unit 2208 compares the acquired plurality of pieces of cooperation information with parameters, and extracts cooperation information that matches the parameters.

  The presentation unit 2208 can generate presentation information so that, when one piece of cooperation information is extracted from a plurality of pieces of cooperation information, the extracted piece of cooperation information is highlighted in the plurality of pieces of cooperation information. Based on this presentation information, the display unit 2011 generates display information so that the extracted one piece of cooperation information is highlighted with respect to other pieces of cooperation information. The method for performing highlighting is not particularly limited, such as bolding or display color inversion. Not only this but the presentation part 2208 produces | generates presentation information so that only one extracted cooperation information may be displayed in the some cooperation information, when one cooperation information is extracted from several cooperation information. May be.

  FIG. 15 is an exemplary functional block diagram for explaining the function of the second execution unit 2300 according to the first embodiment. In FIG. 15, the second execution unit 2300 includes a processing unit 2301 and a calling unit 2302.

  The processing unit 2301 executes specific processing for the second execution unit 2300. For example, when the second execution unit 2300 is an application for promoting a singer, the processing unit 2301 displays a profile related to the singer, plays a promotion video, displays event information, purchases a concert ticket, and the like. Process. For example, when the second execution unit 2300 is an application for advertising a store, the processing unit 2301 performs processing such as display of store navigation information and product information, and mail order sales of products.

  The calling unit 2302 calls the first execution unit 2200. For example, the calling unit 2302 activates the first execution unit 2200 by performing steps S10 to S13 in FIGS. 13A to 13C in accordance with a user operation or the like. Further, the calling unit 2302 has in advance parameters to be passed to the first execution unit 2200 to be activated. The calling unit 2302 passes this parameter to the first execution unit 2200 when starting the first execution unit 2200. The parameter is transferred from the second execution unit 2300 to the first execution unit 2200 via the OS, for example.

(First application example of the first embodiment)
Next, a first application example of the first embodiment will be described. In the first application example, the first execution unit 2200 called by the second execution unit 2300 acquires the image specified by the second execution unit 2300 as the captured image 40 and performs image search, thereby specifying The content of can be acquired.

  FIG. 16 schematically shows the flow of processing according to the first application example of the first embodiment. Here, it is assumed that the specific application that is the second execution unit 2300 is, for example, a specific application for performing promotion or notification, but the specific application applicable to the first application example is not limited to this example. . When the specific application is activated in the terminal device 20, the top screen 1000 of the specific application is displayed on the display device 406 of the terminal device 20. In this example, button images corresponding to the functions of the specific application are displayed at the bottom of the top screen 1000. The user can execute the function of the specific application by touching the position of the button image corresponding to the desired function with a finger or the like.

  At this point, it is assumed that the image search application 220 serving as the first execution unit 2200 has not yet been activated in the terminal device 20. In the following, the button image is simply referred to as “button”, and the action of touching the position of the button image is referred to as “operating the button”.

  When a predetermined button on the top screen 1000 is operated, the specific application switches the display on the display device 406 to the notification screen 1001 (step S20). In this example, an image search is performed by the image search system 200A based on a captured image 40 obtained by capturing a specified subject, for example, a poster 1200, so that a URL for accessing predetermined content, for example, a privilege video can be acquired. It has become. Here, it is assumed that the content distribution site 210A in FIG. 9 distributes the privilege video.

  On the notification screen 1001, a button 1011 for adding an image search function is arranged. When the button 1011 is operated, the specific application calls the image search application 220. At this time, the terminal device 20 determines whether or not the image search application 220 is installed. For example, the specific application requests the OS to start the image search application 220 as shown in step S10 of FIG. In response to this request, the OS determines whether or not the image search application 220 is installed in step S11 (step S11).

  When it is determined by the terminal device 20 that the image search application 220 is installed, the specific application calls and activates the image search application 220 (step S21). At that time, the specific application passes predetermined parameters to the image search application 220 to be activated. This parameter includes information indicating the specific application such as the name of the specific application, and information on the server name and DB name of the image search server 11 and DB 12 for performing an image search on the captured image 40 obtained by capturing the poster 1200. . Here, as the server name and DB name included in the parameters, a predetermined server name and DB name depending on the specific application are used.

  The parameter further includes information for setting the skin of the execution screen (for example, the image search application screen 1002) when the image search application 220 is activated, and information for setting the UI of the image search application 220. May be included.

  On the other hand, when the terminal device 20 determines that the image search application 220 is not installed, such as when the image search application 220 is used for the first time, a program for installing the image search application 220 is installed on the network or the like. Need to get from outside through. Therefore, in step S22, the display on the display device 406 is switched to the DL screen 1003 for downloading (DL) the program via the network in the terminal device 20 by the specific application.

  The terminal device 20 starts downloading and installing the program in response to a predetermined user operation on the DL screen 1003 (step S23). For example, as illustrated in FIG. 13C, the OS controls the communication I / F 410 to acquire the program from the outside via the network (step S <b> 13), and performs installation on the terminal device 20.

  When the installation of the program is completed, the installed program is automatically executed, and the image search application 220 is activated. When activated, the image search application 220 switches the display on the display device 406 to the image search application screen 1002.

  In the terminal device 20 in which the image search application 220 is activated, by operating a predetermined button arranged on the image search application screen 1002 using the designated poster 1200 as a subject, the image of the poster 1200 is captured. 40 (step S24). The image search application 220 transmits the captured image 40 obtained by capturing the poster 1200 to the image search system 200A. At this time, the image search application 220 adds the server name and DB name included in the parameters passed from the specific application to the captured image 40 and transmits the captured image 40 to the image search system 200A.

  The image search system 200A selects the image search server 11 and the DB 12 according to the server name and DB name added to the captured image 40, and performs an image search based on the captured image 40. The image search system 200A acquires, as a search result, one piece of cooperation information indicating an access destination for the privilege video, for example, and transmits the search result to the terminal device 20 (step S25). This cooperation information is received by the image search application 220 in the terminal device 20.

  The image search application 220 accesses the content distribution site 210A according to the received cooperation information, and acquires a privilege video as the content. The image search application 220 switches the display of the display device 406 to the content display screen 1003 and displays the acquired privilege movie. At this time, it is preferable that the image search application 220 displays information 1004 indicating the caller specific application on the content display screen 1003.

  Note that the image search application 220 can store the content (privilege image) acquired in step S25 using the storage unit 2207 (step S26). The storage unit 2207 associates the content with information indicating the caller, or information indicating the content distribution site 210A from which the privilege video is acquired, and stores the content in the storage 407, for example.

  The image search application 220 changes the display screen of the display device 406 to the content storage screen 1006 in response to a predetermined operation for requesting storage of content on the content display screen 1003 or the image search application screen 1002 after the captured image 40 is acquired. Switch to. The image search application 220 stores the content acquired by the storage unit 2207 in response to a predetermined operation for determining storage of the content on the content storage screen 1006. The saved contents are displayed in a list in the collection display area 1007, for example.

  By operating the return button 1005 on the content display screen 1003, it is possible to return to the caller specific application. More specifically, when the return button 1005 is operated, the calling unit 2206 calls a specific application that is a calling source (step S27). The calling operation at this time is substantially the same as the operation described using steps S10 to S12 in FIGS. 13 (a) and 13 (b).

  As shown in the first application example, the image search application 220 according to the first embodiment acquires specific content according to parameters passed from the specific application at the time of activation by image search based on the captured image 40. be able to. Therefore, the provider side of the specific application can easily perform the function expansion that adds the function of the image search application 220 to the specific application.

(Second application example of the first embodiment)
Next, a second application example of the first embodiment will be described. In the second application example, the first execution unit 2200 called by the second execution unit 2300 obtains the cooperation information acquired by the image search based on the captured image 40 according to the parameters passed from the second execution unit 2300. This is an example of selection.

  FIG. 17 schematically shows a flow of processing according to the second application example of the first embodiment. Here, it is assumed that the specific application that is the second execution unit 2300 is a specific application provided by a certain store for selling merchandise (electronic commerce) via store guidance or a network. The specific application applicable to the second application example is not limited to this example. When the specific application is activated in the terminal device 20, the top screen 1100 of the specific application is displayed on the display device 406 of the terminal device 20. In this example, the store name “XXYYZZ” is displayed at the top of the top screen 1100, and buttons for executing various functions of the specific application are arranged below the store name. Of the buttons, the button 1110 is a button for adding an image search function.

  At this point, it is assumed that the image search application 220, which is the first execution unit, has not been activated in the terminal device 20.

  When the button 1110 is operated on the top screen 1100, the specific application calls the image search application 220. At this time, the terminal device 20 determines whether or not the image search application 220 is installed. When it is determined by the terminal device 20 that the image search application 220 is installed, the specific application calls and activates the image search application 220 (step S30). When activated, the image search application 220 switches the display on the display device 406 to the image search application screen 1101.

  Here, when the specific application calls the image search application 220, the specific application passes predetermined parameters to the image search application 220. This parameter includes information indicating the specific application such as the name of the specific application and server name and DB name information of the image search server 11 and the DB 12 for performing an image search on the captured image 40 taken. At this time, the default server name and DB name in the image search system 200A are used as the server name and DB name.

  The parameter further includes information for setting the skin of the execution screen (for example, the image search application screen 1101) when the image search application 220 is activated, and information for setting the UI of the image search application 220. May be included.

  On the other hand, if it is determined in the terminal device 20 that the image search application 220 is not installed, the specific application displays a display on the display device 406 and a program for installing the image search application 220 in step S31. Switch to the DL screen 1102 for downloading via.

  The specific application starts downloading and installing the program in response to a predetermined user operation on the DL screen 1102 (step S32). When the installation of the program is completed, the installed program is automatically executed, and the image search application 220 is activated (step S30). When activated, the image search application 220 switches the display on the display device 406 to the image search application screen 1101.

  In the second application example, the subject for image search performed by the terminal device 20 in which the image search application 220 is activated is the advertisement medium 1201 for the product handled by the store that provides the specific application. The creator of the advertising medium 1201 registers in advance the image of the product included in the advertising medium 1201 as an object in the image search system 200A (step S50).

  In the case of the second application example, the creator of the advertising medium 1201 is, for example, a manufacturer of a product advertised by the advertising medium 1201, and there is a high possibility that there is no direct relationship with a store, that is, a provider of a specific application. That is, the same product is sold by a plurality of stores, and an advertisement medium 1201 having the same content that introduces the same product by the plurality of stores is used.

  In addition, when an object included in the advertisement medium 1201 is registered in the image search system 200A, a plurality of pieces of cooperation information corresponding to a plurality of shops handling products advertised by the advertisement medium 1201 may be associated with the object. There is. For example, a plurality of pieces of linkage information corresponding to the stores A, B, and C handling the product are registered in the image search system 200A for an object based on an image of one product.

  Each store A, B, and C can register a plurality of pieces of linkage information in association with an object based on an image of one product. For example, it is conceivable to associate and register cooperation information for displaying details of a product and cooperation information for purchasing the product via a network with respect to an object based on an image of one product.

  By operating a predetermined button arranged on the image search application screen 1101 using the advertising medium 1201 as a subject in the terminal device 20, an image of the advertising medium 1201 is acquired as the captured image 40 (step S33). The image search application 220 transmits the captured image 40 obtained by capturing the advertisement medium 1201 to the image search system 200A. At this time, the image search application 220 adds the server name and the DB name included in the parameters passed from the specific application to the captured image 40 and transmits them to the image search system 200A.

  In the second application example, the addition of the server name and the DB name to the captured image 40 can be omitted. When the server name and the DB name are not added to the captured image 40 transmitted from the terminal device 20, the image search system 200A performs an image search using the default image search server 11 and DB 12 in the system.

  The image search system 200 </ b> A selects the image search servers 11 and DB 12 according to the server name and DB name added to the captured image 40, and performs an image search based on the captured image 40. Here, since default values are used for the server name and DB name, the image search system 200A selects the default image search server 11 and DB 12 for the system. Even when addition of the server name and DB name to the captured image 40 is omitted, the image search system 200A selects the default image search server 11 and DB 12 for the system.

  The image search system 200A performs an image search based on the captured image 40 obtained by capturing the advertisement medium 1201. And the cooperation information corresponding to the object contained in the advertising medium 1201 is acquired. Here, as described above, it is assumed that three pieces of linkage information corresponding to the stores A, B, and C are associated with the object, and the image search system 200A determines that the three linkages are the result of the image search. Get information.

  The image search system 200A transmits the three pieces of cooperation information acquired as the search result to the terminal device 20 (step S34). The three pieces of linkage information are received by the image search application 220 in the terminal device 20.

  Upon receiving the cooperation information, the image search application 220 extracts cooperation information corresponding to the specific application that called the image search application 220 from the received cooperation information.

  For example, the information indicating the specific application included in the parameter passed by the specific application to the image search application 220 is created in advance so as to include a part of the cooperation information corresponding to the specific application. As an example, when the cooperation information is a URL, it is conceivable that the domain name of the URL is included in the parameter as information indicating a specific application. In this case, the image search application 220 compares each received cooperation information (URL) with information indicating the specific application included in the parameter, and selects a domain name that matches the information indicating the specific application included in the parameter. The included linkage information is extracted as linkage information corresponding to the specific application. At this time, when there are a plurality of pieces of linkage information having the same domain name and different path names, the plurality of pieces of linkage information can be extracted.

  In this way, by filtering a plurality of pieces of linkage information based on parameters, it is possible to extract linkage information corresponding to a specific application that has called the image search application 220.

  The image search application 220 generates a related information display screen 1102 based on the cooperation information extracted based on the parameters, and displays the related information display screen 1102 on the display device 406. In the related information display screen 1102, buttons for specifying related information related to objects included in the advertisement medium 1201 are arranged in an area 1103. The user can switch the display of the display device 406 to a screen that displays the specified related information by operating a button that specifies desired related information arranged in the area 1103.

  As an example, a plurality of buttons are arranged in the area 1103. The first button among the plurality of buttons specifies, for example, a URL that presents detailed information on the product indicated by the object included in the advertising medium 1201. When the first button is operated, the image search application 220 accesses the URL and switches the display on the display device 406 to a content display screen 1104 that displays product detailed information based on the URL (step S35). .

  In the example of FIG. 17, the image search application 220 accesses the URL by starting up a browser application installed as a standard in the terminal device 20 by passing the URL as a parameter. In this case, the screen by the browser application is the content display screen 1104. Not limited to this, the content display screen 1104 may be displayed using a function inside the image search application 220.

  The second button specifies, for example, a URL for purchasing the product via the network. When the second button is operated, the image search application 220 accesses the URL and switches the display on the display device 406 to the product purchase screen 1105 based on the URL (step S36). In this example, the product purchase screen 1105 can be shifted to the specific application top screen 1100 (step S37).

  For example, the third button stores related information in the terminal device 20 (step S38). For example, the image search application 220 can specify a URL (cooperation information) displayed on the related information display screen 1102 and store the specified URL in the storage unit 2207. At this time, a page with the specified URL can be acquired as an image, for example, and the acquired image can be displayed as a thumbnail image in the collection display area 1107 in association with the URL. The collection display area 1107 displays a list of images and URLs saved so far by the image search application 220.

  In the collection display area 1107, by specifying a thumbnail image by a user operation, it is possible to access a URL associated with the specified thumbnail image. For example, if the designated thumbnail image is associated with the URL that presents the detailed product information, the image search application 220 switches the display on the display device 406 to the content display screen 1104 (step S40). Similarly, if the designated thumbnail image is associated with the URL for purchasing the product, the image search application 220 switches the display on the display device 406 to the product purchase screen 1105 (step S39).

  By operating the return button 1111 on the related information display screen 1102, the specific application is called from the image search application 220, and the process is shifted from the image search application 220 to the specific application. Further, the display on the display device 406 is switched to the specific application top screen 1100 (step S41).

  As shown in the second application example, the image search application 220 according to the first embodiment uses a plurality of pieces of cooperation information acquired from an image search based on the captured image 40 as parameters passed from the specific application at the time of activation. Corresponding linkage information can be extracted. The operation of the specific application for that purpose is simply to call the image search application 220 from the specific application and pass the parameters, so that the function extension for realizing the second application example is easy.

(Second Embodiment)
Next, a second embodiment will be described. FIG. 18 illustrates an exemplary configuration of an information processing system according to the second embodiment. In FIG. 18, the same reference numerals are given to portions common to FIG. 9 described above, and detailed description thereof is omitted.

  As illustrated in FIG. 18, in the information processing system 1 ′ according to the second embodiment, a specific site 240 capable of realizing a function equivalent to that of the specific application in the first embodiment is provided on the network. In the information processing system 1 ′, the specific site 240 is accessed from the terminal device 20. The specific site 240 calls the image search application 220 in the terminal device 20 and passes parameters to the activated image search application 220. The parameter includes at least the URL of the specific site 240 as information indicating the specific site 240 of the caller.

  Since the operation of the image search application 220 to which the parameter is passed is the same as the operation described in the first embodiment, a detailed description thereof is omitted here.

  FIG. 19 is a functional block diagram of an example for explaining functions of the terminal device 20 according to the second embodiment. Note that, in FIG. 19, the same reference numerals are given to portions common to FIG. 11 described above, and detailed description thereof is omitted. As illustrated in FIG. 19, the terminal device 20 calls the first execution unit 2200 using the browser application 2400 instead of mounting the second execution unit 2300 illustrated in FIG. 11.

  The operation of starting the first execution unit 2200 (image search application 220) from the browser application 2400 will be schematically described with reference to FIGS. FIG. 20 schematically illustrates an example of a configuration of a terminal program on the terminal device 20 according to the second embodiment. Note that, in FIG. 20, the same reference numerals are given to the portions common to FIG. 12 described above, and detailed description thereof is omitted.

  20, the terminal program 2000 ′ includes an application layer 2100, an API layer 2101, an OS layer 2102, and a hardware I / F layer 2103. The application layer 2100 includes a browser application 2400 and a first execution unit 2200. .

  FIG. 21 schematically shows an operation flow when the browser application 2400 reads the data of the specific site 240 and calls the first execution unit 2200 according to the second embodiment. FIG. 21 is an example of the operation when the first execution unit program for causing the terminal device 20 to execute the first execution unit 2200 is already installed.

  The browser application 2400 accesses the specific site 240 via the communication I / F 410 according to a predetermined URL. By this access, the page data indicated by the URL is transmitted to the communication I / F 410 via the network (step S20). This data is passed from the communication I / F 410 to the OS via the hardware I / F layer 2103 (step S21), and from the OS to the browser application 2400 via the API layer 2101 (step S22).

  When data of the specific site 240 is read into the browser application 2400, a screen with a page of the specific site 240 is displayed on the display screen of the browser application 2400. This screen is provided with buttons and hyperlinks for calling the first execution unit 2200 and passing parameters.

  When these buttons and hyperlinks are operated and a call to the first execution unit 2200 is instructed, the browser application 2400 activates the first execution unit 2200 with respect to the OS as illustrated in FIG. Is requested (step S23). In response to this request, the OS accesses the storage 2002, for example (step S24). As illustrated in FIG. 21B, the OS reads the first execution unit program from the storage 2002, loads the first execution unit program into the RAM 404 that is the main storage device (step S25), and activates the first execution unit 2200. Then, for example, the OS moves the right to execute the application from the browser application 2400 to the first execution unit 2200.

  FIG. 22 shows that the browser application 2400 stores the data of the specific site 240 when the first execution unit program for executing the first execution unit 2200 according to the second embodiment is not installed in the terminal device 20. A flow of operations when reading and calling the first execution unit 2200 is schematically shown. In FIG. 22, the operations in steps S20 to S22 described above have already been executed, and buttons and hyperlinks for calling the first execution unit 2200 and passing parameters are provided on the display screen of the browser application 2400. It is assumed that the image of the page of the specific site 240 is displayed.

  If the OS determines that the first execution unit program is not yet installed in the terminal device 20 as a result of accessing the storage 2002 in step S24, the OS controls the communication I / F 410 and externally through the network. The first execution unit program is acquired (step S26). The acquired first execution unit program is temporarily stored in the storage 2002, and installation processing is started in a predetermined procedure. When the first execution unit program is installed, the OS activates the first execution unit 2200 by the process of step S25 described above.

  In the above description, when starting the first execution unit 2200 from the browser application 2400, the browser application 2400 passes parameters to the first execution unit 2200 based on the read data of the specific site 240. The parameter can use, for example, the URL of the specific site 240 as information indicating the caller of the first execution unit 2200. When the URL of the specific site 240 is used as information indicating the caller, the browser application 2400 accesses the specific site 240 when returning from the first execution unit 2200. The parameters can include a server name and a DB name used for image search.

  When transmitting the captured image 40 to the image search system 200A, the first execution unit 2200, that is, the image search application 220 adds information included in the passed parameters to the captured image 40, as in the first embodiment. To do. Thereby, the image search application 220 can acquire the cooperation information according to the specific site 240 as a result of the image search. In addition, by passing the cooperation information acquired by the image search application 220 to the browser application 2400, the browser application 2400 can access the content distribution site 210A and the content distribution site 210B corresponding to the specific site 240.

  As described above, according to the second embodiment, the terminal device 20 is used to start the image search application 220 from a site on the network and perform an image search by the image search system 200A based on the captured image 40. Cooperation information corresponding to the site can be acquired. In recent years, Web sites dedicated to events are often created on the Web, and event participants have increased opportunities to participate in events from this Web site. By applying the second embodiment, even when participating in an event from a website, it is possible to easily obtain cooperation information corresponding to the event.

  In the above description, the parameters passed to the image search application 220 are described as being a specific application or a specific site 240 on the network, but this is not limited to this example. For example, a GPS positioning unit that performs positioning by receiving a GPS (Global Positioning System) signal in the terminal device 20 is further provided, and information indicating the position where the captured image 40 is acquired is acquired by the GPS positioning unit. This position information may be added to the parameter and passed to the image search application 220. When accessing the URL indicated by the cooperation information acquired by the image search by the image search system 200A based on the captured image 40, the image search application 220 adds the position information as a parameter of the URL and transmits the URL. Accordingly, it is possible to transmit content corresponding to the position information to the terminal device 20 in the URL indicated in the cooperation information.

  Further, in the above description, according to the parameters passed from the specific application to the image search application 220, the designation of the image search server 11 and DB 12 used for the image search, the selection of cooperation information acquired by the image search, the image search application 220 Although it has been described that skin and UI are set, this is not limited to this example.

  For example, the startup screen of the image search application 220 may be set according to the parameters. As an example, when the image search application 220 is activated, it is conceivable to set whether to display the image search application screen 1101 of FIG. 17 or directly display the camera imaging screen according to the parameters. When the camera imaging screen is directly displayed, for example, it may be possible to return to the specific application top screen 1100 immediately after imaging is completed.

  Further, the imaged medium to be imaged to obtain the captured image 40 is not limited to a paper medium. The imaged medium may be a medium that displays an electronic image, such as electronic paper or signage.

  Further, for example, the cooperation information acquired by the image search may be limited according to the parameter. For example, the image search application 220 restricts the types of linkage information acquired by image search according to the parameters. Further, for example, a language mode applied to the image search application 220 may be set according to the parameters. The language mode is generally set by the OS, but this is forcibly changed according to the parameters.

  The above-described embodiment is a preferred embodiment of the present invention, but is not limited thereto, and various modifications can be made without departing from the scope of the present invention.

1,1 ′ Information processing system 11, 11A, 11B, 11C Image search server 12, 12A ₁ , 12A ₂ , 12A ₃ , 12B ₁ , 12C ₁ , 12C ₂ DB
20 Terminal device 200, 200A Image search system 210A, 210B Content distribution site 220 Image search application 240 Specific site 406 Display device 407, 2002 Storage 410 Communication I / F
2000, 2000 ′ terminal program 2200 first execution unit 2201 captured image acquisition unit 2202 transmission unit 2203 reception unit 2204 setting unit 2205 reception unit 2206 and 2302 calling unit 2207 storage unit 2208 presentation unit 2300 second execution unit 2400 browser application

JP 2010-140250 A

In order to solve the above-described problems and achieve the object, the present invention includes an information providing system that includes one or more information processing apparatuses and has a function of providing linked information specified based on a captured image, An information processing system including an application program for accessing a providing system and using a function, and a terminal device on which a computer is mounted, which is communicably connected to the information providing system via a network and can install the application program a is, the application program, the computer, the application program specified by the different software programs, together with specification information for use in the process of identifying the link information, to receive a call request for calling the application program from the software program The information providing system functions as a first receiving unit, a captured image obtained by capturing an information medium in the terminal device, and a transmitting unit that transmits the specified information to the information providing system. The information providing system includes the captured image and the specified information transmitted by the transmitting unit. The terminal device includes a second receiving unit that receives the cooperation information acquired by the information providing system.

In order to solve the above-described problems and achieve the object, the present invention includes an information providing system that includes one or more information processing apparatuses and has a function of providing linked information specified based on a captured image, An information processing system including an application program for accessing a providing system and using a function, and a terminal device on which a computer is mounted, which is communicably connected to the information providing system via a network and can install the application program In this case, the application program receives a call request for calling the application program from the software program together with designation information that is designated by the software program that is different from the application program and that is used for specifying the linkage information. The information providing system functions as a first receiving unit, a captured image obtained by capturing an information medium in the terminal device, and a transmitting unit that transmits the specified information to the information providing system. The information providing system includes the captured image and the specified information transmitted by the transmitting unit. And the acquisition unit that acquires the specified cooperation information. The terminal device includes a second reception unit that receives the cooperation information acquired by the information providing system.

Claims (22)

An information providing system that includes one or more information processing apparatuses and has a function of providing cooperation information identified based on a captured image; an application program for accessing the information providing system and using the function; An information processing system including a terminal device mounted with a computer, which is communicably connected to an information providing system via a network and can install the application program,
The application program causes the computer to
A first receiving unit for receiving a call request for calling the application program from the software program together with designation information specified by a software program different from the application program;
The terminal device functions as a transmission unit that transmits the captured image obtained by capturing an information medium and the specified information to the information providing system,
The information providing system includes:
An acquisition unit that acquires the cooperation information based on the captured image and the designation information transmitted by the transmission unit;
The terminal device
An information processing system comprising: a second receiving unit that receives the cooperation information acquired by the information providing system. An information providing system that includes one or more information processing apparatuses and has a function of providing cooperation information identified based on a captured image; an application program for accessing the information providing system and using the function; An information processing system including a terminal device on which a computer is installed and capable of installing the application program, which is communicably connected to an information providing system via a network,
The application program causes the computer to
A first receiving unit for receiving a call request for calling the application program from the software program together with designation information specified by a software program different from the application program;
Causing the terminal device to function as a transmission unit that transmits the captured image obtained by capturing an information medium to the information providing system;
The information providing system includes:
An acquisition unit that acquires the cooperation information based on the captured image transmitted by the transmission unit;
The terminal device
A second receiving unit for receiving the cooperation information acquired by the information providing system;
An information processing system comprising: a specifying unit that specifies the cooperation information according to the designation information from the received cooperation information. An information providing system that includes one or more information processing apparatuses and has a function of providing cooperation information identified based on a captured image; an application program for accessing the information providing system and using the function; An information processing system including a terminal device on which a computer is mounted, which is communicably connected to an information providing system via a network and can install the application program,
The application program causes the computer to
A first receiving unit for receiving a call request for calling the application program from the software program together with designation information specified by a software program different from the application program;
Function as a transmission unit that transmits the captured image obtained by imaging the information medium in the terminal device and the information included in the specified information;
The information providing system includes:
An acquisition unit that acquires the cooperation information based on the captured image transmitted by the transmission unit and information included in the designation information;
The terminal device
A second receiving unit for receiving the cooperation information acquired by the information providing system;
An information processing system comprising: a specifying unit that specifies, based on information included in the designation information, the cooperation information corresponding to the information included in the designation information from the received cooperation information. The terminal device
4. The apparatus according to claim 1, further comprising: an installation processing unit that executes an installation process for installing the application program in response to the call request when the application program is not installed. 5. The information processing system described. The terminal device
When the installation of the application program is completed based on the installation process, a transmission processing unit that starts the application program and executes a process for transmitting the captured image and the designation information to the information providing system The information processing system according to claim 4 further provided. The software program is
The information processing system according to claim 1, wherein the information processing system is another application program that can be installed in the terminal device. The software program is a software program stored in a storage device on the network accessed from the terminal device via the network,
The information processing system according to claim 1, wherein the computer functions as a screen display unit that provides a screen for calling the application program from the terminal device to the terminal device. An information providing system that includes one or more information processing apparatuses and has a function of providing cooperation information identified based on a captured image; an application program for accessing the information providing system and using the function; An information providing method for an information processing system including a terminal device that is communicably connected to an information providing system via a network and can install the application program,
A first receiving step of receiving a call request for calling the application program from the software program together with designation information specified by a software program different from the application program;
A transmission step of transmitting the captured image obtained by capturing an information medium in the terminal device and the specified information to the information providing system;
An acquisition step in which the information providing system acquires the linkage information based on the captured image and the designation information transmitted in the transmission step;
An information providing method comprising: a second receiving step in which the terminal device receives the cooperation information acquired by the information providing system. An information providing system that includes one or more information processing apparatuses and has a function of providing cooperation information identified based on a captured image; an application program for accessing the information providing system and using the function; An information providing method for an information processing system including a terminal device capable of installing the application program, which is communicably connected to the information providing system via a network,
A first receiving step of receiving a call request for calling the application program from the software program together with designation information specified by a software program different from the application program;
A transmission step of transmitting the captured image obtained by imaging the information medium in the terminal device to the information providing system;
An acquisition step in which the information providing system acquires the cooperation information based on the captured image transmitted in the transmission step;
A second receiving step in which the terminal device receives the cooperation information acquired by the information providing system;
An information providing method comprising: a specifying step in which the terminal device specifies the cooperation information according to the designation information from the received cooperation information. An information providing system that includes one or more information processing apparatuses and has a function of providing cooperation information identified based on a captured image; an application program for accessing the information providing system and using the function; An information providing method for an information processing system including a terminal device that is communicably connected to an information providing system via a network and can install the application program,
A first receiving step of receiving a call request for calling the application program from the software program together with designation information specified by a software program different from the application program;
A transmission step of transmitting the captured image obtained by imaging the information medium in the terminal device and information included in the designation information;
The information providing system acquires the cooperation information based on the captured image transmitted by the transmission unit and information included in the designation information; and
A second receiving step in which the terminal device receives the cooperation information acquired by the information providing system;
An information providing method comprising: a specifying step in which the terminal device specifies, based on information included in the designation information, the cooperation information corresponding to the information included in the designation information from the received cooperation information.   The installation process step of the said terminal device is further provided with the installation process step which performs the installation process for installing this application program according to the said call request | requirement, when the said application program is not installed. The information provision method according to any one of the above.   When the installation of the application program is completed based on the installation process, the terminal device starts the application program and executes a process for transmitting the captured image and the designation information to the information providing system The information providing method according to claim 11, further comprising a transmission processing step.   The information providing method according to any one of claims 8 to 12, further comprising a step of installing the software program in the terminal device as another application program different from the application program that can be installed in the terminal device.   13. The method according to claim 8, further comprising: accessing, via the network, the software program on the network that provides the terminal device with a screen for calling the application program from the terminal device. The information provision method according to any one of the above. For connecting to an information providing system configured by one or more information processing apparatuses and having a function of providing linked information specified based on a captured image via a network, and accessing the information providing system to use the function A terminal device on which a computer can be installed to install an application program,
The application program causes the computer to
A receiving unit that receives a request for calling the application program from the software program, together with designation information specified by a software program different from the application program,
A transmission unit that transmits the captured image obtained by capturing an information medium and the designation information to the information providing system;
Function as a receiving unit that receives the cooperation information specified by the information providing system based on the captured image and the specified information transmitted by the transmitting unit;
A terminal device comprising the computer. For connecting to an information providing system configured by one or more information processing apparatuses and having a function of providing linked information specified based on a captured image via a network, and accessing the information providing system to use the function A terminal device on which a computer can be installed to install an application program,
The application program causes the computer to
A receiving unit that receives a request for calling the application program from the software program, together with designation information specified by a software program different from the application program,
A transmission unit that transmits the captured image obtained by imaging an information medium to the information providing system;
A receiving unit that receives the cooperation information specified by the information providing system based on the captured image transmitted by the transmitting unit;
The terminal device which functions as a specific part which specifies the cooperation information according to the specification information from the cooperation information received by the receiving part. For connecting to an information providing system configured by one or more information processing apparatuses and having a function of providing linked information specified based on a captured image via a network, and accessing the information providing system to use the function A terminal device on which a computer can be installed to install an application program,
The application program causes the computer to
A receiving unit that receives a request for calling the application program from the software program, together with designation information specified by a software program different from the application program,
A transmission unit that transmits the captured image obtained by imaging an information medium and the information included in the designation information to the information providing system;
A receiving unit that receives the cooperation information specified by the information providing system based on the captured image transmitted by the transmitting unit and information included in the designation information;
A terminal device that causes, based on information included in the designation information, the cooperation information corresponding to the information included in the designation information to function as a specifying unit that identifies from the cooperation information received by the reception unit. An application installed in a computer that is configured by one or more information processing apparatuses and that is connected to an information providing system having a function of providing linked information specified based on a captured image via a network and can access the information providing system A program,
The computer,
A receiving unit that receives a request for calling the application program from the software program, together with designation information specified by a software program different from the application program,
A transmission unit that transmits the captured image obtained by imaging an information medium and the designation information to the information providing system;
An application program that functions as a reception unit that receives the cooperation information specified by the information providing system based on the captured image and the designation information transmitted by the transmission unit. An application installed in a computer that is configured by one or more information processing apparatuses and that is connected to an information providing system having a function of providing linked information specified based on a captured image via a network and can access the information providing system A program,
The computer,
A receiving unit that receives a request for calling the application program from the software program, together with designation information specified by a software program different from the application program,
A transmission unit that transmits the captured image obtained by imaging an information medium to the information providing system;
A receiving unit that receives the cooperation information specified by the information providing system based on the captured image transmitted by the transmitting unit;
An application program that causes the cooperation information corresponding to the designation information to function as a specifying unit that specifies from the cooperation information received by the receiving unit. An application installed in a computer that is configured by one or more information processing apparatuses and that is connected to an information providing system having a function of providing linked information specified based on a captured image via a network and can access the information providing system A program,
The computer,
A receiving unit that receives a request for calling the application program from the software program, together with information specified by a software program different from the application program;
A transmission unit that transmits the captured image obtained by imaging the information medium and information included in the designation information designated from the software program to the information providing system;
A receiving unit that receives the cooperation information specified by the information providing system based on the captured image transmitted by the transmitting unit and information included in the designation information;
An application program that, based on information included in the designation information, causes the cooperation information corresponding to the information included in the designation information to function as a specifying unit that specifies from the cooperation information received by the reception unit. A first application for accessing and using an information providing system comprising a display means and comprising a function for providing linked information specified based on a captured image, which is configured by one or more information processing apparatuses A terminal device capable of mounting a program and a second application program linked to the first application program,
The second application program is a computer,
Functioning as a first display control unit for displaying on the display means a screen capable of selecting use of the function of the first application program;
The first application program causes the computer to
A second display control unit that displays a screen for using the function on the display unit in response to receiving a selection operation using the function of the first application program;
Function as an acquisition unit that acquires the captured image obtained by imaging the information medium in response to a user operation;
The computer;
A terminal device comprising: a display unit configured to display, on the display unit, a screen for presenting the cooperation information according to the requesting computer among the cooperation information specified based on the captured image. A first system for accessing and using an information providing system that includes a computer and a display unit, and that includes a function of providing cooperation information specified based on a captured image, which is configured by one or more information processing apparatuses. And a terminal device information providing method capable of mounting the second application program linked with the first application program,
The first application program is:
The above functions as a first processing unit,
The second application program is:
Causing the computer to function as a second processing unit;
A step in which a second processing unit displays on the display means a screen on which use of the function of the first application program can be selected;
When the first processing unit accepts a selection operation using the function of the first application program, a screen for using the function is displayed based on a request from the second processing unit. Displaying on the display means;
The first processing unit obtaining the captured image obtained by capturing an information medium in response to a user operation;
A step of displaying on the display means a screen for presenting the cooperation information according to the second processing unit of the request source among the cooperation information identified based on the captured image;
An information providing method comprising:

Images