JP2014086040A - Information processing system, information processing method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide information regarding image data acquired by photography or the like.SOLUTION: Image data, and association data in a plurality of languages, associated with the image data is registered in an image database. When receiving retrieval information including language information and reference image data being image data used for retrieval, a management server retrieves image data registered in the image database, similar to the reference image data, and acquires association data specified by language information included in the retrieval information from among pieces of association data associated with the similarity image data. Then, the association data is transmitted to a portable information terminal.

Description

  The present invention relates to a technique for providing information related to image data.

  Conventionally, a similar image retrieval technique for retrieving image data similar to certain image data from an image database is known. In this image retrieval technique, the feature amount of the reference image is compared with the feature amount of the image data registered in the image database, and it is determined whether or not the image data are similar.

  As a similar image search technique, for example, Patent Document 1 discloses the accuracy of similar image search by using both the similarity based on the feature amount of an image and search processing using text data obtained by OCR processing of image data. A technique for enhancing the above is disclosed.

Japanese Patent Laid-Open No. 2002-7413

  For example, when traveling abroad, there are cases where a restaurant or the like recognizes that it is a restaurant, but does not know what food it is offering or its price range. If you are in Japan, you can easily search the Internet using the name of the store where the signboard is written as a keyword to obtain information about the restaurant. In some cases, it is not easy to perform a search or the like.

  Therefore, using the similar image search technique, for example, when shooting image data such as a store sign (including a logo or the like) is transmitted to the management server, the image data of the store sign managed by the management server If related information such as menus and coupons associated with each other can be obtained in the user's native language, such inconvenience is somewhat eliminated.

  Furthermore, if there is no related image data associated with the transmitted image data, information obtained from the signboard image data, for example, information such as the store name described on the signboard can be obtained in the native language of the traveler. It is useful for travelers.

  Therefore, an object of the present invention is to provide a technique that makes it possible to provide information associated with image data by performing a search using the image data as a key.

  In order to achieve the above object, an information processing apparatus of the present invention is an information processing apparatus that is communicably connected to a portable information terminal, and corresponds to image data and related information of the image data in a plurality of languages. Storage means for storing information, receiving means for receiving reference image data used for searching image data stored in the storage means, and display language information from the portable information terminal; and receiving means for receiving the received image data. Performing a search process using reference image data; determining means for determining whether image data similar to the reference image data is stored in the storage means; and an image similar to the reference image data by the determination means When it is determined that data is stored, the word specified by the display language information among the related information of the similar image data stored in the storage unit Obtaining means for obtaining the relevant information of the related information acquired by said acquiring means, characterized in that it comprises a first transmission means for transmitting to said portable information terminal.

  In order to achieve the above object, an information processing method of the present invention includes storage means for storing image data and related information of the image data in a plurality of languages in association with each other, and is communicably connected to a portable information terminal. An information processing method performed by the information processing apparatus to be received, the receiving step for receiving, from the portable information terminal, reference image data used for searching for image data stored in the storage means and display language information; Performing a search process using the reference image data received in the receiving step, determining whether image data similar to the reference image data is stored in the storage unit, and When it is determined that image data similar to the reference image data is stored, of the related information of the similar image data stored in the storage unit An acquisition step of acquiring related information of a language specified by the display language information, and a first transmission step of transmitting the related information acquired by the acquisition means to the portable information terminal. It is characterized by.

  In order to achieve the above object, a computer program of the present invention comprises storage means for storing image data and related information of the image data in a plurality of languages in association with each other, and is communicably connected to a portable information terminal. An information processing apparatus that receives reference image data and display language information used for searching image data stored in the storage means from the portable information terminal; and a reference image received by the receiving means Performing a search process using data; determining means for determining whether image data similar to the reference image data is stored in the storage means; and image data similar to the reference image data in the determining means If it is determined that the information is stored, the display language information identifies the related information of the similar image data stored in the storage unit. Obtaining means for obtaining the relevant information of the language, the related information acquired by said acquiring means, characterized in that to function as a first transmission means for transmitting to said portable information terminal.

  According to the present invention, it is possible to provide information associated with image data by performing a search using the image data as a key.

It is a figure which shows an example of the system configuration | structure of the information processing system of this invention. 2 is a diagram illustrating an example of a hardware configuration of a management server 101. FIG. 2 is a diagram illustrating an example of a hardware configuration of a portable information terminal 102. FIG. It is a figure which shows an example of a function structure of the management server 101 and the portable information terminal 102. FIG. 4 is a flowchart illustrating an example of an initial setting process performed by a CPU 201 of the management server 101. 10 is a flowchart illustrating an example of related information acquisition request processing performed by a CPU 301 of the portable information terminal 102. It is a figure which shows an example of the image search and related information acquisition process performed by CPU201 of the management server 101. FIG. It is a figure which shows an example of the related information browsing process performed by CPU301 of the management server 101, and CPU301 of the portable information terminal 102. FIG. 6 is a diagram illustrating an example of a configuration of a search result display screen 900. FIG. It is a figure which shows an example of a data structure of image DB (database) 409 of FIG.

  Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating an example of a system configuration of an information processing system according to the present invention. As shown in FIG. 1, the information processing system of the present invention includes a management server 101, a plurality of portable information terminals 102, a communication network 103, and the like.

  The management server 101 stores and manages related information in association with image data in a data table, and is transmitted from the portable information terminal 102 in response to an image data related information acquisition request transmitted from the portable information terminal 102. It has a function of searching for image data stored and managed in a data table similar to the image data and transmitting information registered as related information of similar image data to the portable information terminal 102. At that time, related information is provided in a form that matches the language setting in the portable information terminal 102. That is, if the language setting of the portable information terminal 102 is English, the related information in English is transmitted to the portable information terminal.

  In addition, it has a function of performing character recognition processing on image data transmitted from the portable information terminal 102 and translating the text information obtained as a result to the language selected in the language setting of the portable information terminal. . The text information obtained as a result is transmitted to the portable information terminal 102.

  The portable information terminal 102 has a camera function and can generate image data by photographing a subject using the camera. In addition, the image processing apparatus is a device that makes an acquisition request for related information of image data to the management server 102 by transmitting, to the management server 101, image data obtained by shooting with a camera. For example, a smartphone or a tablet terminal.

  The communication network 103 is a network such as the Internet for connecting the management server 101 and the portable information terminal 102 so that they can communicate with each other. The connection form may be wired or wireless.

  The above is an example of the system configuration of the information processing system of the present invention.

  Next, an example of the hardware configuration of the information processing apparatus applicable to the management server 101 in FIG. 1 will be described with reference to FIG.

  In FIG. 2, reference numeral 201 denotes a CPU that comprehensively controls each device and controller connected to the system bus 204. Further, the ROM 203 or the external memory 211 has a BIOS (Basic Input / Output System), an operating system program (hereinafter referred to as OS), which is a control program of the CPU 201, and a function that is executed by the management server 101 to be described later. Various programs to be stored are stored.

  A RAM 202 functions as a main memory, work area, and the like for the CPU 201. The CPU 201 implements various operations by loading a program necessary for execution of processing from the ROM 203 or the external memory 211 into the RAM 202 and executing the loaded program.

  An input controller 205 controls input from an input device 209 such as a keyboard and a pointing device. A video controller 206 controls display on a display device such as the display device 210. These are used by the operator as needed.

  A memory controller 207 is connected to the hard disk (HD), flexible disk (FD), or PCMCIA card slot for storing a boot program, various applications, font data, user files, editing files, various data, etc. via an adapter. The access to the external memory 211 such as a compact flash (registered trademark) memory is controlled.

  A communication I / F (interface) controller 208 is connected to and communicates with an external device via a network (for example, the LAN 107 shown in FIG. 1), and executes communication control processing on the network. For example, communication using TCP / IP is possible.

  Note that the CPU 201 enables display on the display device 210 by executing outline font rasterization processing on a display information area in the RAM 202, for example. Further, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the display device 210.

  A program for causing the CPU 201 to function as each unit shown in FIG. 4 to be described later is recorded in the external memory 211. Then, the CPU 301 loads the RAM 202 as necessary and executes the program. The CPU 201 executes processing of each step shown in a flowchart described later by executing control according to the program. Furthermore, definition files and various information tables used when executing the program are also stored in the external memory 211, and a detailed description thereof will be described later.

  Next, an example of a hardware configuration of the portable information terminal 102 in FIG. 1 will be described with reference to FIG.

  In FIG. 3, reference numeral 301 denotes a CPU that comprehensively controls various devices to be described later connected to the system bus 310. The ROM 303 and the external memory 309 store various programs and data necessary for realizing the functions executed by the BIOS (Basic Input / Output System), the operating system (OS), and the portable information terminal 102, which are control programs of the CPU 301. Is remembered.

  Reference numeral 302 denotes a RAM that functions as a main memory, work area, and the like for the CPU 301. The CPU 301 implements various operations by loading programs and the like necessary for executing various processes described later from the ROM 303 and the external memory 309 to the RAM 302 and executing the loaded programs.

  The photographing unit 304 includes a photographing lens, an image sensor, a circuit that converts an image signal obtained by the image sensor into a predetermined format, and the like. The communication unit 305 is connected to and communicates with an external device such as the management server 101 via a communication network (for example, the communication network 103 in FIG. 1), and executes communication control processing in the network. For example, communication using TCP / IP is possible.

  The display unit 306 is composed of a liquid crystal display or the like, and can be displayed on the display unit 306 by the CPU 301 executing, for example, an outline font rasterizing process on a display information area in the RAM 302.

  The operation unit 307 is used to input various instructions to the CPU 301 and includes various buttons and a touch panel sheet attached to the display unit 306. When an instruction to press the operation screen of the system displayed on the display unit 306 is received, the operation unit 307 transmits the position information to the CPU 301 when pressed.

  The audio input unit 308 is composed of a microphone or the like, and converts the input audio into an audio signal. The external memory 309 is a storage device that stores a program and data for the portable information terminal 102 to execute processing of each step shown in the flowchart described later.

  A program for causing the CPU 301 to function as each unit shown in FIG. 4 to be described later is recorded in the external memory 309. The CPU 301 loads the RAM 302 as necessary and executes the program. The CPU 301 executes processing of each step shown in the flowchart described later by executing control according to the program. Further, definition files and various information tables used when executing the program are also stored in the external memory 309.

  The above is the description of an example of the hardware configuration of the portable information terminal 102 in FIG.

  Next, functional configurations of the management server 101 and the portable information terminal 102 will be described with reference to FIG. As shown in FIG. 4, the management server 101 includes a setting storage unit 401, a transmission / reception unit 402, a feature amount extraction unit 403, an image search unit 404, an OCR unit 405, a translation unit 406, and the like. The portable information terminal 102 includes a setting storage unit 411, a position information acquisition unit 412, an information display unit 413, a transmission / reception unit 414, a photographing unit 415, and the like.

  First, each unit included in the management server 101 will be described. When the image search unit 404 searches the image data registered in the image database 409 for image data similar to the image data received from the portable information terminal 102 by the transmission / reception unit 402, the setting storage unit 401 is similar. This is a functional unit that stores setting information such as a setting value to be determined as an image and an OCR dictionary in which language to perform OCR processing on image data received by the transmission / reception unit 402.

  The transmission / reception unit 402 is a functional unit for performing data communication with the portable information terminal 102. For example, the image data transmitted from the portable information terminal 102 or the related information acquisition request for the image data registered in the image database 409 is received, the image data related information or the image data For example, text information of a translation word obtained by performing OCR processing on data, translation processing on text information obtained as a result of the OCR processing, or the like is transmitted to the portable information terminal 102.

  The feature amount extraction unit 403 is a functional unit that extracts the feature amount of the image data received by the transmission / reception unit 402 from the portable information terminal 102. The image search unit 404 is a functional unit that searches for image data registered in the image database 409 that is similar to the image data received by the transmission / reception unit 402 and acquires related information.

  The OCR unit 405 is a functional unit that performs text recognition processing (OCR processing) on the image data received by the transmission / reception unit 402 and generates text data. The translation unit 406 is a functional unit that translates the text data obtained by the OCR unit 405 into another language. The translation unit 406 performs translation processing using information registered in the translation dictionary DB 407 when performing translation processing.

  Next, each unit included in the portable information terminal 102 will be described. The setting storage unit 411 is a functional unit that stores setting information such as OS language settings. The position information acquisition unit 412 is a functional unit that acquires position information indicating a position where the portable information terminal 102 exists. For example, it is realized by a GPS function or the like.

  The information display unit 413 is a functional unit that displays various types of information on the display unit. For example, the related information of the image data transmitted from the management server 101 is displayed on the display unit. The photographing unit 414 is a functional unit that uses the photographing unit 304 to photograph a subject and generate image data.

  The transmission / reception unit 415 is a functional unit that transmits / receives various data to / from the management server 101 using the communication unit 305. For example, when a request for acquiring related information of image data is made, the image data, OS language information, and location information of the portable information terminal 102 are transmitted to the management server 101, or the image data transmitted from the management server 101 is related. Get information.

  The above is the description of the functional configurations of the management server 101 and the portable information terminal 102.

  Next, an example of an initial setting process performed by the CPU 201 of the management server 101 will be described with reference to FIG. This process is performed based on control by a program for causing the CPU 201 of the management server 101 to function as a setting storage unit.

  The CPU 201 of the management server 101 translates the text information obtained as a result of the OCR processing of the priority of the dictionary (OCR dictionary) used when performing OCR processing on the image data, which is input according to the operation instruction of the system administrator. The priority of the dictionary (translation dictionary) used when performing the search, how much the image data used as the reference for the search and the image data in the image database are similar to each other, whether to make the image search means 405 determine a similar image The input of the set value is accepted.

  For example, as the priority of the OCR dictionary, the dictionary of the language used in the country / region indicated by the position information acquired by the position information acquisition unit 412 of the portable information terminal 102 is set as the highest priority dictionary, Priorities of languages (eg English, Japanese, Chinese) are further set. As the translation dictionary priority, a dictionary for translation into the language selected in the language setting of the OS of the portable information terminal 102 is set as the highest priority dictionary, and a specific language (for example, English, Japanese, Further set priority for Chinese.

  The similarity setting value can be set to a value between 0% and 100%. The larger this value is, the more the CPU 201 of the management server 101 uses only similar image data to the reference image data. It will be judged as an image.

  Thereafter, after the processing from step S501 to step S503 is completed, the CPU 201 of the management server 101 stores various setting information received in these steps in the external memory 211 (step S504).

  The above is an example of the initial setting process performed by the CPU 201 of the management server 101.

  Next, an example of related information acquisition request processing performed by the CPU 301 of the portable information terminal 102 will be described with reference to FIG.

  The CPU 301 of the portable information terminal 102 uses the photographing unit 305 to take a photograph of a subject (for example, a store sign) and generates image data (step S601). Further, the language information selected in the OS language setting is acquired (step S602).

  Then, the CPU 301 of the portable information terminal 102 determines whether the position information of the portable information terminal 102 can be acquired (step S603). If it is determined that the position information can be acquired (YES), the process proceeds to step S604. The position information is acquired, and search information is created using the image data obtained in step S601, the language information obtained in step S602, and the position information obtained in step S604 (step S605). On the other hand, if it is determined in step S604 that the position information cannot be acquired (NO), the CPU 301 of the portable information terminal 102 advances the process to step S606, and the image data obtained in step S601, Search information is created using the language information acquired in step S602. At this time, if there is position information acquired in advance, search information may be created using the position information.

  The CPU 301 of the portable information terminal 102 transmits the created search information to the management server 101 after the process of step S605 or step S606 is completed (step S607).

  The above is an example of the related information acquisition request process performed by the CPU 301 of the portable information terminal 102.

  Next, an example of image search / related information acquisition processing performed by the CPU 201 of the management server 101 will be described with reference to FIG.

  The CPU 201 of the management server 101 receives the search information transmitted from the portable information terminal 102 and stores it in the RAM 202 (or the external memory 211) (step S701). Thereafter, the feature amount of the image data included in the received search information is extracted (step S702). The feature amount is obtained by replacing information such as brightness, color components, and contours of image data with comparable data such as numerical values. Thereafter, the CPU 201 of the management server 101 searches for image data similar to the image data (reference image data) received in step S701 among the image data registered in the image database 409 (step S703). At this time, image data search processing is performed using the feature amount extracted in step S702.

  Here, an example of the data configuration of the image DB (database) 409 shown in FIG. 4 will be described with reference to FIG. The image DB 409 is stored in the external memory 211 of the management server 101, for example.

  As shown in FIG. 10, the image DB 409 includes data items such as an ID 1001, an image 1002, a feature amount 1003, and related information 1004.

  An ID 1001 is a data item in which identification information used to uniquely identify image data registered in the image DB 409 is registered. An image 1002 is a data item in which an image is registered. The feature amount 1003 is a data item in which the feature amount of the image registered in the image 1002 is registered. This feature amount is information used when searching for similar images.

  The related information 1004 is a data item in which information related to image data is registered. In this embodiment, as the information related to the image data, menu image data (some character images are included) of menus in various languages (Japanese, English, Chinese, etc.) provided in restaurants are used. Text data is registered. In addition, although not shown, thumbnail images of image data registered in the image data 1002 are also registered.

  The above is the description of the data configuration of the image DB 409.

  Returning to the description of FIG. After the similar image search in step S703 is completed, the management server 101 obtains image data (similar image data) whose similarity is equal to or greater than a numerical value set as a similarity that is a criterion for determining similarity as a result of the search process. Is registered in the image DB 409 (step S704).

  If it is determined in the determination process in step S704 that similar image data is registered in the image DB 409 (YES), the CPU 201 of the management server 101 advances the process to step S705, and the similar image registered in the image DB 409. Get a thumbnail image of the data. Note that if a plurality of image data similar to the image data included in the search information is registered in the image database, thumbnail images of all similar image data are acquired.

  Then, language information included in the search information is acquired (step S706), and related information of similar image data is acquired according to the acquired language information (step S707). If a plurality of image data similar to the image data included in the search information is registered in the image database, the related information of the most similar image data is acquired. Then, search result information for displaying thumbnail images of similar images and related information is created (step S708).

  On the other hand, if it is determined in the determination process in step S704 that similar image data is not registered in the image DB 409 (NO), the CPU 201 of the management server 101 advances the process to step S709, and the search information includes position information. To determine whether or not

  If it is determined in step S709 that the position information is included (YES), the process proceeds to step S710, and the language of the OCR dictionary used for OCR is determined according to the position information. For example, if the position information is information indicating Japan, the Japanese OCR dictionary is determined as the dictionary used for OCR, and if the position information is information indicating the inside of Korea, the Korean OCR dictionary is determined as the dictionary used for OCR. On the other hand, if it is determined that the position information is not included (NO), the language of the OCR dictionary used for the OCR is determined in accordance with a preset priority (step S711).

  After determining the language of the OCR dictionary in step S710 or step S711, the CPU 201 of the management server 101 executes OCR processing on the image data included in the search information using the determined OCR dictionary (step S712), and the OCR result A text is acquired (step S713).

  Thereafter, a translation process is performed on the OCR result text in order to translate the OCR result text acquired in step S713 into the language indicated by the language information included in the search information (step S714). Since the language information included in the search information is often a language such as a native language that is normally used by the user who uses the portable information terminal, in the present invention, in order to provide easy-to-understand information for the user, Such translation processing is performed. Then, a translation result text is generated.

  Then, search result information for displaying the OCR result text and the translation result text is created. Since the result of the translation process naturally depends on the ORC result text obtained as a result of the OCR process, in order to make the user recognize whether the translation result is appropriate, the search result information including the OCR result text is created. Yes.

  After the processing in step S708 or step S715 is completed, the search result information created in those steps is transmitted to the portable information terminal 102 that has made the search request (step S716).

  The above is an example of the image search / related information acquisition process performed by the CPU 201 of the management server 101 performed by the CPU 201 of the management server 101.

  Next, an example of related information browsing processing performed by the CPU 201 of the management server 101 and the CPU 301 of the portable information terminal 102 will be described with reference to FIG.

  When receiving the search result information transmitted from the management server 101 (step S801), the CPU 301 of the portable information terminal 102 displays the search result display screen 900 shown in FIG. 9 on the display unit 306 using the received search result information. (Step S802).

  Here, an example of the configuration of the search result display screen 900 displayed on the display unit 306 of the portable information terminal 102 in step S802 of FIG. 8 will be described with reference to FIG.

  The thumbnail image display unit 901 is a display unit that displays thumbnail images of similar image data. Link information for acquiring related information is set in the thumbnail image displayed on the thumbnail image display unit, and when the CPU 301 of the portable information terminal 102 receives a touch instruction for the thumbnail image, the thumbnail image is displayed. A request for acquiring related information of the image data (similar image) indicated by is transmitted to the management server 101.

  The related information (image) display unit 902 is a display unit that displays related information (image) included in the search result information received from the management server 101. The text information display unit 903 is a display unit that displays related text, OCR result text, and translation result text included in the search result information received from the management server 101. By checking the information displayed on these display units, the user can check information on restaurants and the like specified from the signboard taken by the user. Further, it is possible to partially select text displayed on the text information display unit 903 and perform a Web search using the information.

  The above is an example of the configuration of the search result display screen 900.

  Returning to the description of FIG. When the CPU 301 of the portable information terminal 102 receives the touch instruction for the thumbnail image displayed on the thumbnail image display unit 901 after displaying the search result screen 900 on the display unit 306 (YES in step S803), the touch instruction is given. An image ID is identified from the thumbnail image (step S804), and the image ID and language information set as a display language by the OS are transmitted to the management server 101 (step S805).

  When the CPU 201 of the management server 101 receives the image ID and language information transmitted from the portable information terminal 102 (step S806), the CPU 201 corresponds to the received language information among the related information of the image data specified by the received image ID. The related information obtained is acquired from the image DB 409 (step S807). Then, the acquired related information is transmitted to the portable information terminal (step S808).

  When receiving the related information transmitted from the management server 101 (step S809), the CPU 301 of the portable information terminal 102 updates the display of the search result display screen using the related information (step S810). Thereafter, the process returns to step S803.

  When the CPU 301 of the portable information terminal 102 receives a Web search instruction using the search result character string displayed on the text information display unit 903 (YES in step S811), the CPU 301 starts a Web browser application (step S812). Under the control of the Web browser application, a search request using the character string selected by the text information display unit 903 as a search condition is transmitted to a search server (not shown) (step S813). When the search result is received, information obtained from the search server as a result of the search is displayed on the display screen of the browser application (step S814). Thereafter, the process proceeds to step S803.

  In addition, when the CPU 301 of the portable information terminal 102 receives an end instruction (YES in step S815), the process shown in FIG. When other processing is accepted (in this case, NO is determined in step S815), the process proceeds to step S816, other instructed processes are executed, and the process proceeds to step S803.

  The above is an example of the related information browsing process performed by the CPU 201 of the management server 101 and the CPU 301 of the portable information terminal 102.

  As described above, according to the present invention, information related to image data is set as a display language or the like in the language setting of the apparatus that has performed the image capturing process using the image data generated by the image capturing process. It is possible to obtain in the language. This makes it possible to easily obtain information related to the restaurant by taking a picture of the signboard of the restaurant, for example, in an area using other languages such as overseas. However, it is possible to easily acquire information required by itself.

  In the above description, restaurant information acquisition has been described as an example, but it goes without saying that the present invention can be applied to other than this.

  Although the embodiment has been described in detail above, the present invention can take an embodiment as a method, a program, a recording medium, or the like.

  The program according to the present invention is a program that allows a computer to execute (read) each processing method, and the recording medium according to the present invention stores a program that allows the computer to execute each processing method.

  The program in the present invention may be a program for each processing method of each device.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by performing reading.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program is actually It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the processing and the processing is included.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or It goes without saying that the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network by a communication program, the system or apparatus can enjoy the effects of the present invention.

  In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

101 management server 102 portable information terminal 103 communication network 201 CPU
202 RAM
203 ROM
204 System Bus 205 Input Controller 206 Video Controller 207 Memory Controller 208 Communication Interface (I / F) Controller 209 Input Device 210 Display Device 211 External Memory 301 CPU
302 RAM
303 ROM
304 Imaging unit 305 Communication unit 306 Display unit 307 Operation unit 308 Audio input unit 309 External storage device 310 System bus

Claims (9)

An information processing apparatus that is communicably connected to a portable information terminal,
Storage means for associating and storing image data and related information of the image data in a plurality of languages;
Receiving means for receiving, from the portable information terminal, reference image data used for searching image data stored in the storage means, and display language information;
A determination unit that performs a search process using the reference image data received by the reception unit, and determines whether image data similar to the reference image data is stored in the storage unit;
When the determination unit determines that image data similar to the reference image data is stored, the display language information among the related information of the similar image data stored in the storage unit An acquisition means for acquiring related information of a specified language;
First transmission means for transmitting the related information acquired by the acquisition means to the portable information terminal;
An information processing apparatus comprising: A character recognition unit that performs character recognition processing on the reference image data when the determination unit determines that image data similar to the reference image data is not stored in the storage unit;
The information processing apparatus according to claim 1, further comprising: a second transmission unit configured to transmit text information generated by performing character recognition processing by the character recognition unit to the portable information terminal. The receiving means further acquires position information of the portable information terminal,
The translation means for translating the character recognition result text information generated as a result of the character recognition processing by the character recognition means into a language used in an area specified by the position information. The information processing apparatus described in 1. The storage means stores feature data of the image data,
A calculation unit for calculating feature amount data of the reference image data received by the reception unit;
The determination unit determines whether image data similar to the reference image data is stored in the storage unit using feature amount data. The information processing apparatus described. Further comprising setting means for setting a similarity used for determining whether the image data stored in the storage means is similar to the reference image data;
The determination means stores image data similar to the reference image data in the storage means depending on the presence or absence of image data having a similarity to the reference image data equal to or higher than the similarity set by the setting means. 5. The information processing apparatus according to claim 1, wherein the information processing apparatus determines whether the information is present. When there are a plurality of image data similar to the reference image data in the storage means, the first transmission means transmits related information of the image data most similar to the reference image data to the portable information terminal. The information processing apparatus according to any one of claims 1 to 5, wherein: The information processing apparatus according to claim 6, wherein the first transmission unit further transmits information for acquiring related information other than the image data most similar to the reference image data to the portable information terminal. . An information processing method performed by an information processing apparatus including a storage unit that stores image data and related information of the image data in a plurality of languages in association with each other, and is communicably connected to a portable information terminal,
A receiving step for receiving, from the portable information terminal, reference image data used for searching for image data stored in the storage means and display language information;
Performing a search process using the reference image data received in the reception step, and determining whether image data similar to the reference image data is stored in the storage unit;
When it is determined in the determination step that image data similar to the reference image data is stored, the display language information among the related information of the similar image data stored in the storage unit An acquisition step of acquiring related information of a specified language;
A first transmission step of transmitting the related information acquired by the acquisition means to the portable information terminal;
An information processing method comprising: A storage unit that stores image data and related information of the image data in a plurality of languages in association with each other, and stores an information processing apparatus that is communicably connected to the portable information terminal from the portable information terminal to the storage unit Receiving means for receiving reference image data and display language information used for searching the image data being processed;
A determination unit that performs a search process using the reference image data received by the reception unit, and determines whether image data similar to the reference image data is stored in the storage unit;
When the determination unit determines that image data similar to the reference image data is stored, the display language information among the related information of the similar image data stored in the storage unit An acquisition means for acquiring related information of a specified language;
A computer program for causing the related information acquired by the acquiring means to function as a first transmitting means for transmitting to the portable information terminal.

Images