JP2009282864A - System for image recognition information, information server, information providing method, and information providing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To selectively provide a user with a plurality of pieces of information related to an object recognized by image recognition and a service providing method. <P>SOLUTION: A system 1 for image recognition information is a system where a terminal 10 incorporating a camera 12 is connected to an image recognition server 50. The image recognition server includes: an image recognition part 62 for performing recognition processing to an image received from the terminal and outputting recognition data; and a related information retrieval part 63 for retrieving related information data corresponding to the recognition data. The terminal has an information server 100 accessible by the related information data retrieved by the image recognition server. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image recognition service using the Internet and a portable terminal, and more particularly to provision of information related to an image recognized by the service.

  With the widespread use of mobile terminals with built-in digital cameras, and the widespread use of image recognition technology that recognizes what is captured as image data from image data captured by digital cameras. Information services using image recognition technology for such portable terminals have begun.

  Patent Document 1 describes a technique of distributing information related to an object recognized by image recognition used for such a service to a mobile terminal.

US Pat. No. 7,016,532

  Information relating to an object recognized by image recognition can be obtained by a technique such as that disclosed in Patent Document 1. However, even if it is the same thing, the information corresponding to the thing and the service which a user wants to obtain in connection with it are not necessarily one way. There may be a plurality of items of information that the user wants to obtain with respect to the object, and the method of obtaining the information is not limited to one. Depending on the information item, it may be sufficient to display a web page with a specific URL, while the person in charge may need to contact the user directly by telephone or e-mail.

  In addition, when the person in charge contacts the user, it is necessary for the user to communicate the contact method desired by the user, the contactable time, and the like to the person in charge. For example, when a person in charge calls a user, even if a telephone number of a user registered in advance is called, the user may not be available depending on the time zone. In such a case, it is also necessary to inform the person in charge of the time when the user can answer the phone.

  An object of the present invention is to provide an image recognition information system capable of selectively providing a plurality of information and service providing methods related to an object recognized by image recognition to the user and allowing the user to communicate his / her wishes. It is to provide a portable terminal, an image recognition server, an information server, an information providing method, and an information providing program.

  In order to achieve the above object, an image recognition information system according to the present invention is an image recognition information system in which a terminal incorporating a camera and an image recognition server are connected to each other. An image recognition unit that performs recognition processing on a received image and outputs recognition data, and a related information search unit that searches for related information data corresponding to the recognition data (claim 1).

  By having the configuration in which the image recognition server searches for related information data corresponding to the recognition data, a plurality of information and service providing methods related to objects recognized by the image recognition can be selectively provided to the user. Become.

  In this system, if the image recognition server includes a transmission / reception unit that returns the related information data to the terminal (claim 2), the searched related information data can be delivered to the user.

  The exchange of data between the terminal and the image recognition server is such that the terminal transmits an e-mail with an image attached to the image recognition server, and the transmission / reception unit returns an e-mail including related information data to the terminal If this is the case (Claim 3), this system can be used only with standard functions provided on the terminal side.

  On the other hand, if the terminal has an application that collectively transmits images to the image recognition server and displays related information data received from the image recognition server, these processes are performed in a series of flows. The convenience of the user can be improved in that it can be performed (claim 4).

  Here, the related information data searched by the related information search unit is inquireable content information indicating contents that can be inquired by the user of the terminal in relation to the recognition data, and accessible by the user in relation to the inquireable content information. By including the inquiry URL information indicating the URL (Claim 5), the user can access the information corresponding to the inquiryable content by the returned URL information.

  Furthermore, if the related information data includes action type information indicating the type of action that the user can request in response to the inquiry URL information, and an action ID that is an ID uniquely corresponding to the action type information ( (6) The user can request an action desired by the action ID. At this time, the action ID is added as an argument to the inquiry URL information and returned to the terminal (claim 7), which is more desirable because the user can request an action simply by selecting the URL. .

  The terminal may transmit additional information that can be used to select related information data simultaneously with the image, and the related information search unit may select related information data based on the additional information. ).

  The system may further include an information server that can be accessed by the related information data retrieved by the image recognition server by the terminal (claim 9). The information server may include a web server unit that outputs data based on the URL requested from the terminal, and an inquiry handling unit that searches for a method of responding to the inquiry from the user of the terminal corresponding to the URL ( Claim 10), the information server can provide information according to the user's request.

  If the inquiry handling unit transmits a handling request including a handling method to the person in charge (claim 11), the person in charge can also directly contact the user to deal with it. At this time, the URL includes an action ID that is an ID uniquely corresponding to the type of action that can be requested by the user, the inquiry corresponding unit searches for the action type corresponding to the action ID, and includes the action type in the response request. If this is the case (claim 12), the person in charge can take an action as requested by the user.

  Then, the terminal transmits an identification number corresponding to itself to the information server, the inquiry response unit searches for personal information corresponding to the identification number, and can include the personal information in the response request. .

  The present invention not only provides the entire image recognition information system described above, but also provides a terminal, an image recognition server, an information server, an information providing method, and an information providing program that constitute the image recognition information system.

  Since the present invention is configured to search related information data corresponding to recognition data, a plurality of information and service providing methods related to an object recognized by image recognition can be selectively provided to a user. As a result, it is possible to selectively provide the user with a plurality of information and service providing methods related to the objects recognized by the image recognition, and the user can also communicate his / her wishes. An image recognition information system, a portable terminal, an image recognition server, an information server, an information providing method, and an information providing program can be provided.

[First Embodiment]
FIG. 1 is a conceptual diagram showing a configuration of an image recognition information system 1 according to the first embodiment of the present invention. The image recognition information system 1 includes a mobile terminal 10 with a built-in camera that captures an object 160, an image recognition server 50, and an information server 100. The portable terminal 10, the image recognition server 50, and the information server 100 are connected to each other via a public wireless communication network or the Internet, and can perform data communication. As will be described in detail later, the information server 100 transmits the content of the inquiry from the mobile terminal 10 to the person-in-charge terminal 171 and requests the person in charge 170 to take action.

  FIG. 2 is a conceptual diagram showing in more detail the internal configuration of the mobile terminal 10 shown in FIG. The mobile terminal 10 is, for example, a mobile phone terminal, a PHS terminal, a PDA terminal, a notebook personal computer, or the like.

  The mobile terminal 10 includes a control unit 11, a camera 12, a communication module 13, a storage unit 14, and a display 15. The control unit 11 is a microcomputer configured by combining a CPU, a RAM, and an OS, and controls other units and executes an application program. The camera 12 captures the object 160 and outputs the captured image data.

  The communication module 13 performs data communication with the image recognition server 50 or the information server 100 and a voice call with the person in charge 170. The storage unit 14 is a non-volatile storage device such as a hard disk drive or a flash memory that stores application programs executed by the control unit 11, image data captured by the camera 12, data transmitted and received by the communication module 13, and the like. The display 15 displays the result of the application program executed by the control unit 11.

  The control unit 11 executes each functional unit such as the photographing unit 21, the e-mail client 22, and the web browser 23 as an application program. The storage unit 14 stores an individual identification number 24. The photographing unit 21 controls the camera 12 to photograph the object 160, passes the photographed image data to the e-mail client 22, and saves it in the storage unit 14. However, it is also possible to pass the image data to the e-mail client 22 without saving it in the storage unit 14.

  The e-mail client 22 transmits / receives e-mail to / from the image recognition server 50 via the communication module 13. The e-mail client 22 displays the URL described in the body of the transmitted / received e-mail as a clickable one, and when the user clicks the URL, starts the web browser 23 and passes the URL to the URL. Do.

  The web browser 23 is a normal web browser that accesses a website via the communication module 13 and browses a web page. Further, when the web browser 23 is activated by the e-mail client 22 and passed a URL, the web browser 23 performs a process of accessing the web page of the URL. At that time, the individual identification number 24 is given to the web server of the accessed URL. The individual identification number 24 is an identification number uniquely given to each mobile terminal 10, and cannot be forged or altered by the user.

  FIG. 3 is a conceptual diagram showing the internal configuration of the image recognition server 50 shown in FIG. 1 in more detail. The image recognition server 50 is a normal computer device, and includes a control unit 51, a storage unit 52, and a communication interface 53. The control unit 51 is the center of a computer device configured by combining a CPU, a RAM, and an OS, and controls other units and executes application programs. The storage unit 52 is an external storage device configured by a magnetic disk, an optical disk, or the like that stores each data described later. The communication interface 53 is an interface that communicates with the mobile terminal 10.

  The control unit 51 executes each function unit such as the electronic mail transmission / reception unit 61, the image recognition unit 62, and the related information search unit 63 as an application program. The storage unit 52 stores an image recognition database 71 and a related information database 72. In the image recognition database 71, data on the image to be recognized is stored in advance. The related information database 72 will be described later.

  The e-mail transmission / reception unit 61 receives an e-mail from the mobile terminal 10 and passes the image data attached to the e-mail to the image recognition unit 62. The e-mail transmission / reception unit 61 creates an e-mail including information passed from the related information search unit 63 and sends it back to the mobile terminal 10.

  The image recognition unit 62 performs image recognition processing on the image data passed from the e-mail transmission / reception unit 61 using the image recognition database 71, specifies the recognition data corresponding to the image data, and searches for related information. Pass to part 63. Here, the image recognition processing performed by the image recognition unit 62 can be performed using, for example, the technique described in Patent Document 1 described above, or can be performed using other known image recognition techniques. . Therefore, the details of the image recognition process and the data structure of the image recognition database 71 will not be described in detail.

  The recognition data that the image recognition unit 62 passes to the related information search unit 63 is data that indicates what the object 160 is captured as image data. More specifically, for example, the manufacturer name, the product type, the product name, the product number, etc. of the object 160. Among these, what is used as data to be transferred to the related information search unit 63 is arbitrary. Here, the following description will be made assuming that the manufacturer name and the product number of the object 160 are passed to the related information search unit 63.

  The related information search unit 63 searches the related information database 72 for the recognition data passed from the image recognition unit 62, specifies the related information data corresponding to the recognition data, and obtains the recognition data and the related information data. It passes to the e-mail transmission / reception unit 61.

  FIG. 4 is a conceptual diagram showing a data structure of the related information database 72 shown in FIG. The related information database 72 includes a maker name 81, a product number 82, a queryable content 83, a query URL 84, an action type 85, and an action ID 86. The manufacturer name 81 and the product number 82 are recognition data passed from the image recognition unit 62 as described above. The manufacturer name 81 may include various contractors and public organizations related to the object 160 in addition to the manufacturer of the object 160.

  The related information search unit 63 searches the related information database 72 for a queryable content 83 and a query URL 84 corresponding to the manufacturer name 81 and the product number 82. The inquiryable content 83 is a content that allows the user to make an inquiry to the information server 100 for the product indicated by the manufacturer name 81 and the product number 82. The inquiry URL 84 is a URL that can be accessed by the user when making an inquiry about the inquiry content indicated by the inquiryable content 83.

  However, the inquireable content 83 and the inquiry URL 84 corresponding to the manufacturer name 81 and the product number 82 are not limited to one, and a plurality of inquiries may be obtained. The inquiryable contents 83 and the inquiry URL 84 are in one-to-one correspondence in principle. The inquiry URL 84 is not limited to a normal static web page address, but may be a web page address dynamically created by a CGI or the like such as a bulletin board, or a rich content address such as a video or audio. There may be.

  The action type 85 is a type of action that the user can request in response to the inquiry URL 84. There may be a plurality of action types 85 for one inquiry URL 84. The type of action requested by the user corresponding to the action type 85 is, for example, when requesting a callback by telephone, the time zone for performing the callback is “AM”, “PM”, “NIGHT”, “ANYTIME”, etc. It is possible to specify as follows. Of course, it is not limited to this example.

  The action ID 86 corresponds to the action type 85 on a one-to-one basis. However, in the example shown in FIG. 4, when the action type 85 is “None” or “E-mail”, the action ID 86 is blank (it is not always necessary to follow this example). The action ID 86 is included in the URL that the user clicks as described later.

  FIG. 5 is a flowchart showing processing executed by each unit of the image recognition server 50 shown in FIG. When the e-mail transmission / reception unit 61 receives an e-mail attached with image data from the portable terminal 10 (step S201), the e-mail transmission / reception unit 61 passes the image data to the image recognition unit 62 (step S202). The image recognition unit 62 performs image recognition processing on the received image data (step S203), identifies recognition data corresponding to the image data, and passes it to the related information search unit 63 (step S204).

  The related information search unit 63 specifies related information data corresponding to the recognition data passed from the image recognition unit 62 (step S205), and passes the obtained recognition data and related information data to the e-mail transmission / reception unit 61 ( Step S206). The e-mail transmission / reception unit 61 issues additional data to be described later (step S207). The e-mail transmission / reception unit 61 combines the additional data with the recognition data and the related information data passed from the related information search unit 63 to the mail received from the mobile terminal 10. A reply mail is created and transmitted (step S208).

  FIG. 6 is a conceptual diagram illustrating an example of a reply mail 90 to the mobile terminal 10 that the electronic mail transmission / reception unit 61 illustrated in FIG. 3 creates based on data passed to the related information search unit 63. The destination 91 of the reply mail 90 is the address of the mobile terminal 10. In the body 92 of the reply mail 90, the manufacturer name 81 and the product number 82 are described, and further, a plurality of sets of queryable contents 83, a query URL 84, and an action type 85 that are searched for the manufacturer name 81 and the product number 82 are described. The Further, additional data 93 and action ID 86 are added as arguments to the end of each inquiry URL 84.

  The additional data 93 is data issued by the e-mail transmission / reception unit 61 and added to the return to the portable terminal. More specifically, for example, information indicating that the inquiry is made by the image recognition information system 1, an inquiry number, an action requested by the user, and the like. As described above, the action ID 86 is stored in the related information database 72 corresponding to the action type 85.

  In the example shown in FIG. 6, as additional data 93, information “system = shalink” indicating that the inquiry is made by the image recognition information system 1 and an inquiry number “inquiry_no = A000001” issued by the e-mail transmission / reception unit 61. Information is added. For example, when the action type 85 is “callback (morning)”, the action ID 86 corresponding to the action ID 86 is added in the form of “action = callback_am”.

  As described above, the e-mail client 22 incorporated in the mobile terminal 10 displays the URL described in the text of the received e-mail as clickable. Also for the mail of FIG. 6, the inquiry URL 84 including the additional data 93 and the action ID 86 as arguments is displayed so as to be clickable in the electronic mail client 22.

  When the user clicks this clickable URL, the web browser 23 is activated to access the website of the inquiry URL 84, and the additional data 93 and the action ID 86 are passed to the website. At the same time, the web browser 23 also passes the individual identification number 24 to the website. Hereinafter, the description will be continued assuming that the website indicated by the inquiry URL 84 is the information server 100 shown in FIG.

  FIG. 7 is a conceptual diagram showing in more detail the internal configuration of the information server 100 shown in FIG. The information server 100 is a normal computer device, and includes a control unit 101, a storage unit 102, and a communication interface 103. The control unit 101 is the center of a computer device configured by combining a CPU, a RAM, and an OS, and controls other units and executes application programs. The storage unit 102 is an external storage device configured by a magnetic disk, an optical disk, or the like that stores data described later. The communication interface 103 is an interface that communicates with the mobile terminal 10.

  The control unit 101 executes the functional units such as the web server unit 111, the inquiry handling unit 112, the log recording unit 113, and the usage information search unit 114 as application programs. The storage unit 102 also stores an inquiry correspondence database 121, a communication log 122, an action type database 123, and a usage information database 124.

  The web server unit 111 is normal web server software such as Apache, and the web page 111 corresponds to a URL (reference URL 84 in the present embodiment) requested by a client (in the present embodiment, the mobile terminal 10). Display data is returned to the client. As described above, the display data of the web page here is not limited to a normal static web page, but may be a web page dynamically created by CGI or the like, or rich content such as video or audio. Also good.

  The inquiry handling unit 112 searches the inquiry handling database 121 and the action type database 123 to search for a response to the user's inquiry content. The log recording unit 113 records the inquiry from the user and the response to the communication log 122. The usage information search unit 114 searches the usage information database 124 for user personal information corresponding to the individual identification number 24.

  FIG. 8 is a conceptual diagram showing the data structure of the inquiry correspondence database 121 shown in FIG. The inquiry correspondence database 121 stores a manufacturer name 81, a product number 82, an inquiry type 131 corresponding to the inquiry URL 84, a correspondence method 132 that is a method corresponding to the inquiry contents of the user, and a corresponding destination 133.

  For example, if the handling method 132 is “email reply” or “telephone callback”, the handling destination 133 is the email address of the person in charge 170 who requests the email reply or callback. When the handling method 132 is “None”, the handling destination 133 is blank.

  The inquiry handling unit 112 requests the person in charge 170 to deal with the contents of the retrieved handling method 132 and handling destination 133. For the response request to the person in charge 170, for example, an arbitrary means such as an e-mail can be used. When the handling method 132 is “None”, the inquiry handling unit 112 does nothing because it is sufficient for the web server unit 111 to return the display data of the web page corresponding to the query URL 84 to the client.

  The person in charge 170 is a person in charge such as the company in which the information server 100 is installed or a contractor commissioned by the company, and is based on the content of the request from the information server 100 displayed on the person-in-charge terminal 171 shown in FIG. To respond to the user who issued the inquiry.

  FIG. 9 is a conceptual diagram showing a data structure of the usage information database 124 shown in FIG. The usage information database 124 stores personal information corresponding to the user of the mobile terminal 10 corresponding to the individual identification number 24, such as a telephone number 141, a mail address 142, a name 143, and an address 144. It is assumed that the user has registered for use with the service in advance and has previously registered such personal information in the use information database 124. The individual identification number 24 is recorded in the communication log 122 together with the contents using the service.

  FIG. 10 is a conceptual diagram showing the data structure of the action type database 123 shown in FIG. The action type database 123 stores an action ID 151 and an action type 152 corresponding to the action ID 151. The action ID 151 and the action type 152 are contents corresponding to the action ID 86 and the action type 85 of the related information database 72, respectively. However, the action type 152 does not exist for items for which the action ID 151 is not defined.

  FIG. 11 is a flowchart illustrating processing executed by each unit of the information server 100 illustrated in FIG. When the web server unit 111 receives access from the portable terminal 10 (step S251), the web page corresponding to the inquiry URL 84 is displayed (step S252), and at the same time, the inquiry URL 84, additional data 93 received from the portable terminal 10, and The action ID 86 is passed to the inquiry handling unit 112 (step S253), and the individual identification number 24 is passed to the usage information search unit 114 (step S254).

  The inquiry handling unit 112 searches for the inquiry type 131, the handling method 132, and the handling destination 133 corresponding to the inquiry URL 84 (step S255). If the searched response method 132 is “None” (step S256: YES), the log recording unit 113 records the processing content in the communication log 122 as it is (step S262), and the process is ended.

  If the response method 132 is other than “None” (step S256: NO), the usage information search unit 114 searches the usage information database 124 for personal information of the user using the individual identification number 24 as a key (step S257). In response, the inquiry handling unit 112 determines whether the handling method 132 is “telephone callback” or “e-mail reply” (step S258), and the telephone number 141 or the mail address 142 of the retrieved telephone number 141 is determined. Either of them is posted in the body of the response request to the person in charge 170 and a response request is created (step S259 or 260). If the handling method 132 is “telephone callback”, the telephone number 141 is described. If the handling method 132 is “email reply”, the mail address 142 is entered.

  At that time, the inquiry handling unit 112 searches the action type database 123 using the action ID 86 as a key as necessary, and searches for the action type 152 corresponding to the action ID 86 (151).

  The response request created as described above is transmitted as an e-mail from the inquiry response unit 112 to the person in charge 170 (step S261), and the log recording unit 113 records the processing content in the communication log 122 (step S262), and the process is terminated. Incidentally, the e-mail address of the person in charge 170 who is the transmission destination of the response request is described in the response destination 133.

  FIG. 12 is a conceptual diagram showing an example of a response request 180 that the inquiry handling unit 112 shown in FIG. This response request 180 is transmitted by an e-mail addressed to the person in charge 170. The address 181 of the response request 180 is the e-mail address of the person in charge 170 described in the response destination 133 searched by the inquiry response unit 112.

  In the body 182 of the response request 180, a search is made using the manufacturer name 81, product number 82, inquiry type 131, and usage information database 124 using the individual identification number 24 as a key. The telephone number 141 (or mail address 142), name 143, address 144, action type 152, etc. are described.

  The person in charge 170 who receives this response request 180 by the person in charge terminal 171 looks at the inquiry type 131 and the action type 152, and makes a call to the telephone number 141 described in the text 182, or sets the mail address 142. By replying to an e-mail, the user can be contacted and hear more details from the user. At that time, by using the action type 152, it is possible to reduce the number of cases where the user is absent even if a call is made, and to contact the user more reliably.

  At the same time, since the contents of the use of the service by the user are recorded in the communication log 122, when this service is developed for a fee, the user can be charged appropriately using the individual identification number 24 as a key. If necessary, the image recognition server 50 and the information server 100 communicate with each other, and using the additional data 93 and the like, the user corresponding to the individual identification number 24 grasps the history of using the service, and billing is performed. It can also be used.

[Second Embodiment]
Similar to the image recognition information system 1 according to the first embodiment of the present invention shown in FIG. 1, the image recognition information system according to the second embodiment of the present invention has a built-in camera for photographing the object 160. Mobile terminal 10b, image recognition server 50, and information server 100. Except that the portable terminal 10 is replaced with a portable terminal 10b described later, the image recognition server 50 and the information server 100 have almost the same internal configuration and operation as the first embodiment of the present invention. Therefore, illustration of the entire system configuration is omitted.

  FIG. 13 is a conceptual diagram showing in more detail the internal configuration of the mobile terminal 10b according to the second embodiment of the present invention. The mobile terminal 10b has the same hardware configuration as that of the mobile terminal 10 according to the first embodiment of the present invention, but the control unit 11 executes the image recognition information client 322 as an application program. The same elements and functional parts other than this are given the same names and reference numerals, and the description thereof is omitted.

  The image recognition information client 322 is obtained by integrating the functions of the photographing unit 21, the e-mail client 22, and the web browser 23 in the first embodiment into one application program. Therefore, the user does not need to create an e-mail attached with image data obtained by photographing the object 160, and immediately transmits the image data to the image recognition server 50 to recognize the object 160, related information data, and Additional data can be viewed. Data exchange between the portable terminal 10b and the image recognition server 50 may or may not be performed using electronic mail.

  The image recognition information client 322 displays the recognition data, the related information data, and the additional data in the form shown in FIG. 6 (or any other form), and accesses the information server 100 based on the user's operation. A telephone call back or e-mail reply by the person in charge 170 can be received.

  The image recognition information client 322 holds a user ID 324, correspondence method information 325 and contact information 326 in the storage unit 14, and the user ID 324, correspondence method information 325, contact information together with the inquiry URL 84 and the additional data 93. 326 and action information 327 are passed to the information server 100. Note that the action type 85 and the action ID 86 in the first embodiment may not be used here.

  At that time, the image recognition information client 322 allows the user to input a desired response method, contact information, contactable time, etc., and transmits response method information 325, contact information 326, and action information 327 based on the input contents. can do. In particular, when the request for the handling method and the contact information is not input, the image recognition information client 322 does not transmit the handling method information 325, the contact information 326, and the action information 327, but instead corresponds to the action type 85. The action ID 86 may be transmitted in the same manner as in the first embodiment.

  For example, when the user wishes to respond by e-mail, the handling method information 325 is “e-mail reply”, the contact information 326 is an e-mail address to which the e-mail is replied, and the action information 327 is blank. When the user wishes to respond by telephone, the response method information 325 is “telephone callback”, the contact information 326 is a telephone number that is the callback destination, the action information 327 is a time when telephone contact is possible, and the like.

  Here, the handling method information 325 can designate a handling method that does not match the handling method 132 registered in the inquiry handling database 121 in advance. Further, the telephone number and mail address specified by the contact information 326 can specify a different contact from the telephone number 141 and mail address 142 registered in advance in the usage information database 124. Of course, they need not be the telephone number or mail address of the portable terminal 10b. In addition, as the action information 327, it is possible to specify in detail the time when telephone communication is possible.

  The user ID 324 is a unique ID issued when the user downloads the image recognition information client 322 and installs it on the mobile terminal 10b. Instead of the individual identification number 24 in the first embodiment, this user ID 324 is used for user identification and billing processing. The download source of the image recognition information client 322 and the issuer of the user ID 324 may be the image recognition server 50, the information server 100, or other computers.

  FIG. 14 is a conceptual diagram showing an example of a response request 180 that the inquiry handling unit 112 issues to the person in charge 170 in the present embodiment. When the information server 100 receives the handling method information 325 and the contact information 326 from the portable terminal 10b, the information server 100 and the telephone number 141 (or mail address 142) in the first embodiment shown in FIG. Instead of the action type 152, the handling method information 325, the contact information 326, and the action information 327 are posted in the body 182 of the handling request 180. Other than this, the operation is the same as that of the first embodiment, so the same elements are denoted by the same reference numerals and description thereof is omitted.

  In this way, for example, if you want to be contacted by telephone or e-mail other than the mobile terminal 10b, or if you want to be contacted by e-mail because you cannot be answered immediately, you can specify the contact details and specify the service. Can receive. Of course, when the user does not want to transmit the handling method information 325, the contact information 326, and the action information 327, it is possible not to transmit. In this case, a contact from the person in charge 170 is sent to the contact registered in advance in the usage information database 124 by telephone or e-mail based on the content specified by the action type 85.

[Other embodiments]
In the first and second embodiments described so far, the mobile terminal 10 or 10b has a GPS (Global Positioning System) function, and this GPS function causes the image recognition server 50 to store the current position of the user of the mobile terminal. The information may be transmitted simultaneously with the image data obtained by photographing the object 160. The image recognition server 50 can further narrow down the inquiryable contents 83 and the action types 85 that can be searched by the user based on the image recognition result by the received current position information. For example, a person in charge at a sales office closest to the current position of the user can be called back.

  In addition to the current position information, the mobile terminal 10 or 10b sends any additional information (for example, mobile terminal type, current time, etc.) that can be used to select the queryable content 83 and the action type 85 to the image recognition server 50. Can be transmitted simultaneously with the image data.

  Although the present invention has been described with the specific embodiments shown in the drawings, the present invention is not limited to the embodiments shown in the drawings, and is known so far as long as the effects of the present invention are achieved. It goes without saying that any configuration can be adopted.

  It can be used for information services for mobile terminals with built-in cameras. There are no particular restrictions on the type of information provided and the service content.

It is a conceptual diagram which shows the structure of the image recognition information system which concerns on the 1st Embodiment of this invention. It is a conceptual diagram which shows in more detail the structure inside the portable terminal shown in FIG. It is a conceptual diagram which shows in more detail the structure inside the image recognition server shown in FIG. It is a conceptual diagram which shows the data structure of the related information database shown in FIG. It is a flowchart showing the process which each part of the image recognition server shown in FIG. 3 performs. It is a conceptual diagram which shows an example of the reply mail to the portable terminal which the electronic mail transmission / reception part shown in FIG. 3 produces based on the data passed to the related information search part. It is a conceptual diagram which shows the structure inside the information server shown in FIG. 1 in detail. It is a conceptual diagram which shows the data structure of the inquiry corresponding | compatible database shown in FIG. It is a conceptual diagram which shows the data structure of the utilization information database shown in FIG. It is a conceptual diagram which shows the data structure of the action kind database shown in FIG. It is a flowchart showing the process which each part of the information server shown in FIG. 7 performs. It is a conceptual diagram which shows an example of the response request which the inquiry response part shown in FIG. It is a conceptual diagram which shows in more detail the structure inside the portable terminal which concerns on the 2nd Embodiment of this invention. It is a conceptual diagram which shows an example of the response request which an inquiry response part makes to a person in charge in this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image recognition information system 10, 10b Portable terminal 11 Control part 12 Camera 13 Communication module 14 Storage part 15 Display 21 Imaging part 22 E-mail client 23 Web browser 50 Image recognition server 51 Control part 52 Storage part 53 Communication interface 61 E-mail transmission / reception Unit 62 Image recognition unit 63 Related information retrieval unit 71 Image recognition database 72 Related information database 100 Information server 101 Control unit 102 Storage unit 103 Communication interface 111 Web server unit 112 Inquiry handling unit 113 Log recording unit 114 Usage information retrieval unit 121 Inquiry correspondence Database 122 Communication log 123 Action type database 124 Usage information database 160 Object 170 Person in charge 171 Person in charge terminal 322 Recognition information client 324 user ID
325 Response method information 326 Contact information 327 Action information

An object of the present invention is to selectively provide a user with a plurality of information and service providing methods related to an object recognized by image recognition, and the user can transmit his / her wishes and obtain the information as it is. An image recognition information system, an image recognition server, an information server, an information providing method, and an information providing program are provided.

In order to achieve the above object, an image recognition information system according to the present invention includes an image recognition server having a terminal incorporating a camera and an image recognition unit that performs recognition processing on an image received from the terminal and outputs recognition data. And an information server interconnected with a terminal and an information server connected to each other, wherein the image recognition server includes a related information search unit for searching for a plurality of related information data corresponding to the recognition data. The terminal can access the information server by the related information data, and the related information data indicates the contents that can be inquired by the user of the terminal in relation to the recognition data, and the inquireable contents Inquiry URL information indicating the URL accessible by the user in relation to the information, and the type of action that can be requested by the user in response to the inquiry URL information Action type information and an action ID that is an ID uniquely corresponding to the action type information, the information server searches for the action type corresponding to the action ID, and sends a response request including the action type to the person in charge terminal Inquiry responding section to be included (claim 1).

By having the configuration in which the image recognition server searches for related information data corresponding to the recognition data, a plurality of information and service providing methods related to objects recognized by the image recognition can be selectively provided to the user. Become. Further, the user has the inquiry URL information returned from the image recognition server, the action type information indicating the type of action that the user can request in response to the inquiry URL information, and the ID uniquely corresponding to the action type information. The information according to the contents that can be inquired can be accessed by the action ID, and the desired action can be requested by the action ID. Then, the person in charge can directly contact the user and take action according to the action type corresponding to the action ID.

At this time, the action ID is added as an argument to the inquiry URL information and returned to the terminal (claim 5 ), which is more desirable because the user can request an action simply by selecting the URL. .

The information server has a web server unit that outputs data based on the URL requested from the terminal and passes the action ID to the inquiry corresponding unit . The URL includes the action ID, and the inquiry unit corresponds to the action ID. If the action type is searched (claim 7 ), the information server can provide information in accordance with the user's request.

Then, the terminal transmits the identification number corresponding to its own to the information server, inquiries unit searches the personal information corresponding to the identification number, it is also possible to include the personal information in the corresponding request (claim 8) .

The present invention not only provides the entire image recognition information system described above, but also provides an image recognition server, an information server, an information providing method, and an information providing program that constitute the image recognition information system.

  To achieve the above object, an image recognition information system according to the present invention includes a terminal incorporating a camera, and an image recognition unit that receives an image output from the terminal, performs recognition processing on the image, and outputs recognition data. An image recognition information system comprising an image recognition server having information and an information server connected to a terminal, wherein the image recognition server stores a related information database in which related information data is stored in advance, and a plurality of related information corresponding to the recognition data And a related information search unit for searching the information data from the related information database, and the terminal has a function of accessing the information server based on the related information data and simultaneously transmitting an identification number corresponding to itself to the information server. , Related information data can be queried with inquireable content information showing the content that can be queried by the user of the terminal in relation to the recognition data Reference URL information indicating a URL accessible by the user in relation to the content information, action type information indicating the type of action that the user can request in response to the reference URL information, and an ID uniquely corresponding to the action type information A web server unit that outputs data based on the URL requested from the terminal to the terminal, an action type database that stores in advance an action type corresponding to the action ID, and an identification A usage information database in which personal information corresponding to the number is stored in advance, a usage information search unit that searches the personal information corresponding to the identification number from the usage information database, and an action type corresponding to the action ID from the action type database An inquiry response unit, and the web server unit The action ID included in L is output to the inquiry response unit, and the identification number received from the terminal is output to the usage information search unit. The inquiry response unit includes the searched action type and personal information. Is transmitted to the person-in-charge terminal (claim 1).

The present invention not only provides the entire image recognition information system described above, but also provides an information server, an information providing method, and an information providing program constituting the image recognition information system.

Claims (24)

An image recognition information system in which a terminal incorporating a camera and an image recognition server are connected to each other,
The image recognition server is
An image recognition unit that performs recognition processing on the image received from the terminal and outputs recognition data;
An image recognition information system comprising a related information search unit for searching related information data corresponding to the recognition data.   The image recognition information system according to claim 1, wherein the image recognition server includes a transmission / reception unit that returns related information data searched by the related information search unit to the terminal. The terminal sends an email with the image attached to the image recognition server;
The image recognition information system according to claim 2, wherein the transmission / reception unit returns an e-mail including the related information data to the terminal.   The terminal has a function of executing an application for collectively performing transmission of the image to the image recognition server and display of related information data received from the image recognition server. Item 3. The image recognition information system according to Item 2.   The related information data includes inquireable content information indicating contents that can be inquired by the user of the terminal in relation to the recognition data, and an inquiry URL in which the user can access URL in relation to the inquireable contents information. The image recognition information system according to claim 1, further comprising information.   The related information data includes action type information indicating an action type that a user can request in response to the inquiry URL information, and an action ID that is an ID uniquely corresponding to the action type information. The image recognition information system according to claim 5.   The image recognition information system according to claim 6, wherein the image recognition server includes a transmission / reception unit that adds the action ID as an argument to the inquiry URL information and sends it back to the terminal. The terminal transmits additional information that can be used to select the related information data simultaneously with the image;
The image recognition information system according to claim 1, wherein the related information search unit selects the related information data based on the additional information. Having an information server interconnected with the terminal;
The image recognition information system according to claim 1, wherein the terminal can access the information server by the related information data searched by the image recognition server. The information server is
A web server unit that outputs data based on the URL requested from the terminal;
The image recognition information system according to claim 9, further comprising: an inquiry corresponding unit that searches for a method for responding to an inquiry from a user of the terminal corresponding to the URL.   The image recognition information system according to claim 10, wherein the inquiry handling unit transmits a handling request including the handling method to a person in charge. The URL includes an action ID that is an ID uniquely corresponding to the type of action that the user can request,
12. The image recognition information system according to claim 11, wherein the inquiry handling unit searches for an action type corresponding to the action ID, and includes the action type in the handling request. The terminal sends an identification number corresponding to itself to the information server;
12. The image recognition information system according to claim 11, wherein the inquiry handling unit searches for personal information corresponding to the identification number, and includes the personal information in the handling request. A terminal that includes a camera and an execution control unit and is interconnected with an image recognition server and an information server,
The execution control unit is
A photographing unit for photographing an image by the camera;
A client that transmits the photographed image to the image recognition server and displays a URL included in related information data returned from the image recognition server in a selectable manner;
A terminal that executes an application having a function as a browser for accessing the information server indicated by the URL when the URL is selected.   The terminal according to claim 14, wherein the application controls transmission of an identification number corresponding to the application to the information server.   The terminal according to claim 14, wherein the application causes a user to input a desired handling method and controls transmission of information about the handling method to the information server. An image recognition server connected to a terminal with a built-in camera,
An image recognition unit that performs recognition processing on the image received from the terminal and outputs recognition data;
An image recognition server comprising a related information search unit for searching related information data corresponding to the recognition data. An information server interconnected with a camera-equipped terminal,
A web server unit that outputs data based on the URL requested from the terminal;
An inquiry server that searches for a method for responding to an inquiry from a user of the terminal corresponding to the URL. In a system in which a terminal incorporating a camera and an image recognition server are connected to each other, the image recognition server provides information related to an image taken by the camera to the terminal,
The terminal transmitting the image to the image recognition server;
The image recognition server performing recognition processing on the received image to output recognition data and searching for related information data corresponding to the recognition data;
The image recognition server includes a step of returning the related information data to the terminal.   20. The information providing method according to claim 19, further comprising the step of accessing the information server by the related information data returned from the image recognition server. The information server searching for a method for responding to an inquiry from a user of the terminal corresponding to the related information data;
The information providing method according to claim 20, further comprising: a step of transmitting a response request including the response method to a person in charge. A computer that has a built-in camera and controls a terminal interconnected with an image recognition server and an information server.
A process of taking an image with the camera;
Processing for transmitting the captured image to the image recognition server;
Processing for selectively displaying a URL included in the related information data returned from the image recognition server;
An information providing program for executing a process of accessing the information server indicated by the URL when the URL is selected. To the image recognition server that is connected to the terminal with built-in camera,
Processing for performing recognition processing on the image received from the terminal and outputting recognition data;
An information providing program for executing a process of retrieving related information data corresponding to the recognition data. To the information server that is mutually connected to the terminal with a built-in camera,
A processing unit for outputting data based on the URL requested from the terminal;
And a process for searching for a method for responding to an inquiry from a user of the terminal corresponding to the URL.

Images