JP2013122627A - Information processing terminal, server device, system, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To acquire distribution information according to information based on a specified part in a display image.SOLUTION: An information processing terminal comprises: an image acquisition unit 301 for acquiring image data and an image identifier corresponding to the image data; a display unit 6 for displaying an image according to the acquired image data on a screen; an operation reception unit 281 for receiving an operation for specifying a part of the image displayed on the screen; an information acquisition unit for acquiring predetermined information from the part of image specified by the received operation; a transmission unit for transmitting the predetermined information acquired by the information acquisition unit and the image identifier acquired by the image acquisition unit 301, to a server; a reception unit for receiving action information including link destination information indicating a link destination of the predetermined information and processing information corresponding to the link destination information, from the server; and a control unit for controlling the information processing terminal on the basis of the received link destination information and processing information.

Description

  The present invention relates to an information processing terminal, and more particularly to an information processing terminal that communicates with an external device based on a processing result of image data.

  A user's portable terminal has a limit in the memory capacity which can be mounted. Therefore, the necessary information is received from the server when necessary to cope with the limitation.

  As a method for distributing information from a server to a portable terminal, for example, there is a method described in Patent Document 1 (Japanese Patent Laid-Open No. 2005-107978). In Patent Document 1, a user photographs a subject with a portable terminal having a photographing function, transmits the photographed image to the server side, recognizes the photographed image on the server side, and transmits data corresponding to the recognition information to the portable terminal. To do.

  In Patent Document 2 (Japanese Patent Laid-Open No. 2001-184141) and Patent Document 3 (Japanese Patent Laid-Open No. 2003-177986), an information terminal device carried by a user reads a URI described in a printed matter with a scanner, and has a built-in browser. The content is acquired via the Internet according to the URI.

JP 2005-107978 A JP 2001-184141 A JP 2003-177986 A

  In the above-mentioned patent document 1, since an external server is requested to search for information based on the entire image, it is not possible to meet the user's desire to acquire only information related to a part of the image, characters, and the like. In Patent Documents 1 and 2, since the terminal searches for an external server using a URL, the URL needs to be printed on a printed matter, and the external server uses an image that does not have URL information. Cannot be searched. Therefore, the user's desire to receive distribution of information related to a part of the character string (name, product name, store name, etc. not limited to URL) of the printed matter cannot be satisfied.

  Therefore, an object of the present invention is to provide an information processing terminal capable of transmitting information based on a specified part of displayed images and acquiring information distributed in response to the transmission.

  According to an aspect of the present invention, an information processing terminal that communicates with a server includes an image acquisition unit for acquiring image data and an image identifier corresponding to the image data, and an image according to the image data acquired by the image acquisition unit. Predetermined information from a display unit for displaying on the screen, an operation receiving unit for receiving an operation for specifying a part of the image displayed on the screen, and a part of the image specified by the operation received by the operation receiving unit An information acquisition unit for acquiring information, a predetermined information acquired by the information acquisition unit, a transmission unit for transmitting the image identifier acquired by the image acquisition unit to the server, and link destination information indicating a link destination of the predetermined information And the action information including the processing information corresponding to the link destination information from the server, the information terminal based on the link destination information and the processing information received by the receiving unit And a control unit for controlling.

  Preferably, the information acquisition unit extracts a character string by recognizing a part of the image designated by the operation received by the operation reception unit, and a character acquisition unit for acquiring the extracted character string as predetermined information Including.

  Preferably, the information acquisition unit includes a position acquisition unit for acquiring, as the predetermined information, the position in the image displayed on the screen of a part of the image specified by the operation received by the operation reception unit.

  Preferably, when the receiving unit receives a plurality of action information, the control unit controls the information processing terminal based on the link destination information and the processing information of the action information selected from the plurality of action information.

Preferably, the image data indicates electronic book data.
Preferably, a transmission part transmits the predetermined information which the information acquisition part acquired, the image identifier which the image acquisition part acquired, and the terminal information regarding an information processing terminal to a server.

Preferably, the terminal information indicates an attribute of the information processing terminal.
Preferably, the terminal information indicates an attribute of the user of the information processing terminal.

  According to another aspect of the present invention, a server device that communicates with the information processing terminal includes an information storage unit that stores predetermined information, an image identifier, and action information in association with each other, and a predetermined information and an image identifier from the information processing terminal. And a server receiving unit for receiving the information, and a reading unit for reading the corresponding action information from the information storage unit based on the received predetermined information and the image identifier. Send to the information processing terminal.

  According to still another aspect of the present invention, a system including an information processing terminal and a server device that communicates with the information processing terminal is provided.

  An information processing terminal of the system includes: an image acquisition unit for acquiring image data and an image identifier corresponding to the image data; a display unit for displaying an image according to the image data acquired by the image acquisition unit on a screen; An operation accepting unit for accepting an operation for designating a part of the image displayed on the screen, and an information obtaining unit for obtaining predetermined information from the part of the image designated by the operation accepted by the operation accepting unit; Corresponding to the transmission unit for transmitting the predetermined information acquired by the information acquisition unit and the image identifier acquired by the image acquisition unit to the server device, the link destination information indicating the link destination of the predetermined information, and the link destination information A receiving unit for receiving action information including processing information to be processed from the server device, and a control unit for controlling the information processing terminal based on the link destination information and processing information received by the receiving unit. No.

  A server device of the system includes an information storage unit that stores predetermined information, an image identifier, and action information in association with each other, a server reception unit that receives the predetermined information and the image identifier from the information processing terminal, and the received predetermined unit Based on the information and the image identifier, a reading unit for reading the corresponding action information from the information storage unit is provided, and the read action information is transmitted to the information processing terminal.

  When the further another situation of this invention is followed, the program for making a computer provided with a processor perform an information processing method is provided.

  The information processing method executed by the program includes a step in which an image acquisition unit of a computer acquires image data and an image identifier corresponding to the image data, and a display unit of the computer displays an image according to the acquired image data on a screen. A step of displaying, a step of accepting an operation for specifying a part of an image displayed on the screen by an operation accepting unit of the computer, and a part of the information designated by an operation accepted by the information acquiring unit of the computer A step of acquiring predetermined information from the image; a step of transmitting the acquired predetermined information and the acquired image identifier to the server via a communication line; and a link of the predetermined information by the receiving unit of the computer. Action information including link destination information indicating the destination and processing information corresponding to the link destination information is passed. Comprising the steps of: receiving from the server via the line, the control unit of the computer, and controlling the operation of the computer based on the received link information and the process information.

  According to the present invention, when the information processing terminal transmits information based on a specified part of the displayed image, the information distributed can be acquired in response to the transmission.

It is a system configuration figure concerning an embodiment of the invention. It is an external view of the terminal which concerns on embodiment of this invention. It is a figure which shows notionally the operation | movement of the system which concerns on embodiment of this invention. It is a function block diagram of the terminal which concerns on embodiment of this invention. It is a figure for demonstrating the example of the content of the memory | storage part of the terminal which concerns on embodiment of this invention. It is a function block diagram of the reference server which concerns on embodiment of this invention. It is a processing flowchart concerning an embodiment of the invention. It is a figure which shows the reference request which concerns on embodiment of this invention. It is a figure which shows the response which concerns on embodiment of this invention. It is a figure which shows 1st action DB which concerns on embodiment of this invention. It is a figure which shows the data of the information acquisition destination in 1st action DB which concerns on embodiment of this invention. It is a figure which shows 2nd action DB which concerns on embodiment of this invention. It is a figure which shows terminal information DB which concerns on embodiment of this invention. It is a lineblock diagram of PC concerning an embodiment of the invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<System configuration>
Referring to FIG. 1, the system according to the present embodiment provides an electronic book distribution server 300 for distributing an electronic book, a plurality of information processing terminals, a reference server 200, and information provision provided by distributing various types of information. A server 400 is provided. In the present embodiment, the information providing server 400 is provided as a function of the cloud (abbreviation of cloud computing). These communicate via the communication network NT which consists of various wired and wireless communication lines including the Internet. The plurality of information processing terminals include a terminal 100 that can be carried by a user and a PC (abbreviation of Personal Computer) 100A that is fixedly installed. The terminal 100 and the PC 100A have a function of distributing an electronic book from the electronic book distribution server 300 and browsing the distributed electronic book. Further, the terminal 100 and the PC 100A communicate with the reference server 200 based on the information of the distributed electronic book, communicate with the information providing server 400 according to the received information by the communication, and operate according to the received information.

  In this embodiment, for simplicity of explanation, the terminal 100 is illustrated as an information processing terminal to which an electronic book is distributed from the electronic book distribution server 300, but the PC 100A also has the same processing function as the terminal 100. ing.

  In the present embodiment, the electronic book is composed of image data. For example, it is composed of image data which is bitmap data obtained by optically reading a paper surface of a book as a publication beforehand with an image scanner. The electronic book may be text data or a combination of image data and text data.

  The terminal 100 includes a processor such as a CPU 2, a storage unit 3 for storing programs and data including a nonvolatile memory, a temporary storage unit 4 used as a work area and a temporary storage area for data, and a user for inputting information. A recording medium 11 such as an operation unit 5, a display unit 6, a telephone network communication unit 7, a communication unit 9, and a CD-ROM (Compact Disc Read Only Memory) connected to the antenna 8 are detachably mounted. Position detection for detecting the current position of the terminal 100 using an I / F (interface) 10 for accessing data of the recorded recording medium 11, a timer 12 for measuring time, and a GPS (Global Positioning System) The unit 13 is provided.

  Here, the terminal 100 has a telephone function but a portable terminal that does not have a telephone function (for example, a PDA (Personal Digital Assistant) having a connection function of the communication network NT, and an electronic book browsing having a connection function of the communication network NT. Terminal).

  The communication unit 9 is connected to the Internet by a wireless local area network (LAN) or a wired LAN according to standards such as IEEE (Institute of Electrical and Electronic Engineers) 802.11a / b / g / n. The telephone network communication unit 7 of the terminal 100 connects to the Internet via a mobile phone network or the like.

  The operation unit 5 has a function for accepting an information / instruction input operation by a user, and includes a button, a key, a trackball, a touch panel, and the like.

  The reference server 200 includes a CPU (Central Processing Unit) 201, a storage unit 205, a temporary storage unit (not shown), a communication unit 205 for communicating with an external device via the communication network NT, and a user operation. An input unit 203 including keys and buttons and an output unit 204 such as a display are provided.

  The storage unit 205 includes various storage devices such as a hard disk, a ROM (Read Only Memory), and a RAM (Random Access Memory). The storage unit 205 includes a first action management DB (hereinafter, “DB” is an abbreviation of database) 210, a second action management DB 230, and a terminal information DB 260 in which information related to the terminal 100 communicating with the reference server 200 is registered in advance. Stored.

  Further, the information providing server 400 includes a CPU 401, a storage unit 405, a temporary storage unit (not shown), a communication unit 402 for communicating with an external device via the communication network NT, and a key / button for receiving a user operation. And an output unit 404 such as a display.

  The storage unit 405 includes various storage devices such as a hard disk, ROM, and RAM. Information to be distributed to the external device via the communication network NT is stored in the storage unit 405 in advance.

  Although the configuration of the electronic book distribution server 300 is not shown, it has the same configuration as the information providing server 400. The electronic book distribution server 300 stores an electronic book in advance in a storage unit, and distributes the electronic book stored in the storage unit to an external device such as the terminal 100 via the communication network NT.

<Appearance of terminal>
Referring to FIG. 2, terminal 100 includes touch panel input device 5A and key / trackball unit 5B so that the user can operate from the outside. The touch panel input device 5A includes an operation unit 5 and a display unit 6 integrally. The touch panel input device 5A detects a position where an operation such as a touch on the screen of the display unit 6 is performed, and inputs information based on the detected position.

<Conceptual operation of the system>
FIG. 3 is a diagram conceptually showing the operation of the system according to the present embodiment. Referring to FIG. 3, terminal 100 communicates with information providing server 400, reference server 200, and electronic book distribution server 300. The content holder computer manages electronic books using a predetermined storage area of the cloud, that is, the information providing server 400. The electronic book distribution request from the computer of the content holder is transmitted to the electronic book distribution server 300 via the information providing server 400. The electronic book stored in advance in the information providing server 400 is transmitted to the electronic book distribution server 300 together with the distribution request. The information providing server 400 includes any server on the cloud, and is not limited to a server owned by the content holder.

The electronic book distribution server 300 is managed by an electronic book sales service company.
The electronic book distribution server 300 sells (distributes) the electronic book in response to a request from the terminal 100. The sold electronic book is stored in the storage unit 3 of the terminal 100.

  The storage unit 3 stores reference server address information 36 of the reference server 200 in advance. The reference server address information 36 indicates a URL (abbreviation of Uniform Resource Locator) for identifying the reference server 200 (more specifically, the storage unit 3) for communication via the communication network NT.

  The reference server 200 stores a first action management DB 210, a second action management DB 230, and a terminal information DB 260 in advance.

  The electronic book viewer that manages the electronic book browsing function on the terminal 100 displays the image of the electronic book 37 on the screen of the display unit 6. When the user touches a desired part of the screen of the display unit 6 in the display state (step S1), the CPU 2 acquires predetermined information from the image of the touch part (step S2).

  When the CPU 2 transmits a reference request including the acquired predetermined information to the reference server 200 (step S3), the reference server 200 searches the database in the storage unit 205 based on the received reference request. Based on the search result, the reference server 200 transmits, as a response, action information including the information acquisition destination indicating the link destination of the information corresponding to the reference request and the action definition to the requesting terminal 100 (step S4).

  When receiving the response, the terminal 100 executes an action based on the action information, and transmits an information request based on the action execution result (step S5).

  The information providing server 400 performs processing based on the information request, and transmits a response based on the processing result (including information to be serviced according to the request) to the requesting terminal 100 (step S6).

  The terminal 100 outputs the distribution information included in the response received from the information providing server 400 in a procedure based on the action definition.

<Functional configuration>
A functional configuration of the terminal 100 will be described with reference to FIG. Each function is realized using a program executed by the CPU 2 or a combination of a program and a circuit.

  Referring to FIG. 4, CPU 2 includes communication processing unit 29 for generating a packet for communication via telephone network communication unit 7, request generation unit 26 for generating and outputting a request to an external device, communication For the response processing unit 27 that processes the response received via the processing unit 29, the action execution unit 28 that operates based on the action information of the received response, the information acquisition unit 30 that acquires information from the outside, and the storage unit 3 A storage unit 31 for storing information and data, a reading unit 32 for reading information and data from the storage unit 3, an input / output unit 20, and a designated information acquisition unit 23.

  The request generator 26 includes a reference request unit 263 that generates various DB reference requests addressed to the reference server 200 and an information request unit 262 that generates an information request that requests information distribution to the information providing server 400. The input / output unit 20 receives an operation of the user via the operation unit 5 and outputs an instruction according to the operation, an operation receiving unit 281 for displaying an electronic book 37 on the display unit 6, and a display processing unit 282. , And a display information holding unit 283 having one type of memory function for holding information for specifying a partial image currently displayed on the electronic book 37. The electronic book processed by the display processing unit 282 may be either the electronic book 37 acquired via the information acquisition unit 30 or the electronic book 37 read from the storage unit 3 by the reading unit 32. Here, in order to simplify the explanation, it is assumed that the electronic book 37 read from the storage unit 3 is displayed.

  The information acquisition unit 30 includes an image acquisition unit 301. The image acquisition unit 301 corresponds to, for example, an electronic book authoring tool that is a function for generating an electronic book from a book on a paper medium. The image acquisition unit 301 optically reads a book made of a paper medium, thereby acquiring an electronic book 37 including image data that is a bitmap and an image identifier such as a book ID, a book version number, and a page number. The acquired electronic book 37 is stored in the storage unit 3 via the storage unit 31. Note that the electronic book 37 is mainly distributed from the electronic book distribution server 300, but as other acquisition paths, the electronic book 37 is acquired by the image acquisition unit 301 or via the communication processing unit 29 with the electronic book distribution server 300. Received from a different external device or supplied via the recording medium 11.

  The designation information acquisition unit 23 includes a character acquisition unit 231 that acquires a character string being displayed on the display unit 6 by the display processing unit 282 in accordance with a command 284 based on a user operation received by the operation reception unit 281, and an operation on the operated image. A position acquisition unit 232 that acquires a position is included.

  This character acquisition and position acquisition will be described with reference to FIG. Here, the display processing unit 282 performs processing so that the electronic book 37 is displayed in units of pages. The display information holding unit 283 acquires the bitmap data of the displayed page image, the page number that is the identification information of the page, and the identification data 371 of the electronic book 37 from the display processing unit 282, and acquires these Remember. Accordingly, the display information holding unit 283 holds the bitmap data, page number, and identification data 371 of the currently displayed page image. Of these, the bitmap data and the page number are updated every time the displayed page is updated.

  Now, assuming that the user touches the part 5C of the display area (see FIG. 2), the operation reception unit 281 acquires position information indicating the position of the touched part 5C on the display area. When the position information is acquired, the input / output unit 20 receives the instruction including the page number read from the display information holding unit 283, the bitmap data, the acquired position information, and the identification data 371 as the specified information. To the unit 23. When the position mode is designated by the operation unit 5, the position obtaining unit 232 of the designation information obtaining unit 23 determines the range (rectangular shape) of the portion 5C in the bitmap data of the displayed page of the portion 5C from the position information of the command. A coordinate 51C indicating the upper left of the region and a lower right coordinate 52C: see FIG. 2) are acquired by a predetermined conversion formula.

  Even when text based on text data is displayed instead of an image, position information based on coordinates can be acquired by the same operation as in the case of an image.

  On the other hand, when the character mode is designated by the operation unit 5, the character obtaining unit 231 of the designation information obtaining unit 23 is a portion in the bitmap data of the displayed page corresponding to the coordinate position obtained by the position obtaining unit 232. A character string is extracted by character recognition processing of the 5C partial bitmap data. The extracted character string includes one or more characters, symbols, numbers, alphabets, Roman characters, and the like.

  Note that even when text based on text data is displayed instead of an image, a character string can be extracted by the same operation as in the case of an image.

  The designation information acquisition unit 23 outputs the designation information 285 including the character string or coordinate position acquired as described above, the page number included in the input command 284, and the identification data 371 to the request generation unit 26.

  With reference to FIG. 5, the example of the content of the memory | storage part 3 which concerns on this Embodiment is demonstrated. The storage unit 3 includes an area for storing the reference server address information 36 and one or more electronic books 37. The electronic book 37 includes identification data 371 for uniquely identifying the electronic book 37 and image data 372 that is bitmap data. Instead of the image data 372, text data may be used, or a combination of image data and text data may be used.

  Next, the functional configuration of the reference server 200 will be described with reference to FIG. The illustrated function is realized by a program executed by the CPU 201 or a combination of a program and a circuit. The CPU 201 includes a communication processing unit 591 for communication processing via the communication unit 202, a request processing unit 592 for receiving and processing a reference request from the terminal 100 via the communication processing unit 591, and a storage unit based on the reference request A search processing unit 593 for searching for 205, and a response generation unit 594 for generating a response based on the search result of the search processing unit 593 and transmitting the response via the communication processing unit 591.

<Configuration of communication packet>
FIG. 7 shows a flowchart according to the present embodiment, and FIGS. 8 and 9 show an example of the packet configuration of the reference request 50R exchanged in the flowchart and a response 50S which is a response to the reference request 50R. The data packet of the reference request 50R is generated by the reference request unit 263, and the data packet of the response 50S is generated by the response generation unit 594 of the reference server 200.

  Referring to FIG. 8A, the data packet of the reference request 50R includes a header 501 including a request code 501A, information for identifying the transmission source of the reference request 50R (address information of the terminal 100) 502, a reference request It includes transmission destination information (reference server address information 36) 503 indicating a transmission destination of 50R, and a main body 504. Note that the address information of the terminal 100 is stored in the storage unit 3 in advance.

  Referring to FIG. 8B, the main body unit 504 includes an image identifier 505 for identifying an image being displayed on the terminal 100, data 513 indicating a character recognition result by the character acquisition unit 231, and a position acquisition unit 232. Data 514 indicating the acquired coordinate information, book purchase date data 515 of the electronic book 37 being displayed, terminal ID data 516 for identifying the terminal 100, terminal attribute data 517, terminal operation date / time data 518, and Various data such as terminal operation location data 519 that is the current position of the terminal 100 acquired by the position detection unit 13 can be included. Data 515 to 518 are stored in advance in the storage unit 3, and some or all of these data in the storage unit 3 can be rewritten via the operation unit 5. The data 513 may be character string data extracted from text-format electronic book data.

  The image identifier 505 includes a book ID 510 for identifying the electronic book 37 of the displayed image, book version number data 511 of the electronic book 37, and page number data 512 of the displayed image. The reference request unit 263 acquires the page number data 512 and the book ID 510 from the page number and identification data 371 of the designation information 285 input from the designation information acquisition unit 23. Here, in order to simplify the description, it is assumed that the image identifier 505 includes a book ID 510 and page number data 512.

  Further, the reference request unit 263 acquires the character recognition result data 513 and the coordinate information data 514 from the character string and the coordinate position of the designation information 285 input from the designation information acquisition unit 23.

  Referring to FIG. 9A, the data packet of response 50S includes a header 501, information 502 for specifying the transmission source of response 50S, transmission destination information 503, and main body 504. The header 501 includes a response code 501B indicating that it is a response to the reference request 50R. The response generation unit 594 stores the transmission destination information 503 of the reference request 50R as the transmission source information 502, and stores the transmission source information 502 of the reference request 50R as the transmission destination information 503. The main body 504 shows a response to the reference request 50R. The response includes information on the action definition 530 and the information acquisition destination 531 (see FIG. 9B).

  Although data packets are used for communication here, communication may be performed in units of frames.

<Database configuration>
10 and 11 show a configuration example of the first action management DB 210, FIG. 12 shows a configuration example of the second action management DB 230, and FIG. 13 shows a configuration example of the terminal information DB 260.

  Referring to FIG. 10, the first action management DB 210 stores a plurality of records 219 for managing action information 267.

  The action information 267 includes an action definition 296 and data 297 indicating an information acquisition destination that is an information distribution source to the terminal 100. In the present embodiment, the action definition 296 can be executed collectively by the CPU 2 such as a shell script for causing the terminal 100 to perform a predetermined operation (output, communication, storage, information processing, etc.) regarding information distribution. Suppose we show a command group of various forms. The action definition is not limited to an executable command group as long as it is a command given to cause the terminal 100 to perform a predetermined operation.

  A record 219 includes a set of data 292, 293, 294, and 295 indicating the book ID of the electronic book, the page number, the coordinate range in the page, and the character string displayed in the coordinate range, and action information 267 in association with the set. including. The action information 267 includes action definition 296 and data 297 indicating an information acquisition destination. As long as these data are associated with each other, the storage format is not limited to the record format.

  The information acquisition destination of the data 297 in FIG. 10 is indicated by a link number. The information indicated by the link number is shown in FIG. In FIG. 11, data 2972 for specifying an external device to communicate with an external device (for example, information providing server 400) that is an information acquisition destination is shown corresponding to each of the plurality of link number data 297. . As shown in the figure, the data 2972 indicates link destination information such as a URL and a telephone number.

  FIG. 12 shows an example of the contents of the second action management DB 230. Referring to FIG. 12, the second action management DB 230 can store a plurality of records 229. The record 229 includes data 241 indicating terminal attributes unique to the terminal 100, data 242 indicating the operation period of the terminal 100, data 243 indicating the operation location of the terminal 100, and the date when the electronic book 37 is distributed and stored in the storage unit 3 Purchase date data 244 indicating the electronic book 37, data 245 for identifying the electronic book 37, data 246 indicating the publication number of the electronic book 37, page number data 247 of the electronic book 37, and a predetermined portion on the bitmap data of the page Corresponding to the set of the coordinate data 248, the character string data 249 displayed in the portion, the number of action definitions corresponding to the action information, the data 250 and 251 indicating the action definition, and the information acquisition destination data 252.

  In the present embodiment, the “terminal attribute” includes attributes of the terminal 100 itself (OS (Operating System) type / version, manufacturer name of the terminal 100, a model identifier such as a model number named by the manufacturer), and the terminal 100. And the user's attributes (age, age / minority, gender, etc.) and contract conditions when using the terminal 100 (for example, browsing under 18 years old, etc.). The “terminal attribute” indicates more detailed information regarding the terminal attached to the terminal 100 as described above.

  FIG. 13 shows an example of the terminal information DB 260. A plurality of records 261 are stored in the terminal information DB 260. The record 261 is provided corresponding to each of the terminals 100 communicating with the reference server 200 via the communication network NT. The record 261 includes terminal ID data 264 for uniquely identifying the corresponding terminal 100 and terminal attribute data 265 of the terminal 100.

  In the present embodiment, the reference server 200 can generate the first action management DB 210, the second action management DB 230, and the terminal information DB 260 based on information input by the operator from the input unit 203. The generated various DBs can be edited (deletion / addition / change of record, etc.) using input information from the input unit 203. Accordingly, the action information 267 can be changed as appropriate according to a change in the destination of the information providing server 400, a change in information registered in the information providing server 400, and the like. Further, by communicating with each terminal 100, it can be generated and updated in the terminal information DB 260.

<Processing flowchart>
An operation in which the terminal 100 transmits the reference request 50R and receives the distribution information from the information providing server 400 will be described with reference to the flowchart of FIG. It is assumed that the reference server 200 stores a first action management DB 210, a second action management DB 230, and a terminal information DB 260 in advance. Further, it is assumed that the electronic book 37 and the reference server address information 36 are stored in the storage unit 3 of the terminal 100 in advance.

  Now, on the terminal 100 carried by the user, an image of the desired electronic book 37 is displayed (see FIG. 2). In this state, when the user touches the part 5C in FIG. 2 (step ST3), the input / output unit 20 detects the touch and outputs a command 284 according to the detection result to the specified information acquisition unit 23.

  In the designation information acquisition unit 23, the position acquisition unit 232 or the character acquisition unit 231 indicates the coordinates of the touched portion 5C (the coordinates 51C and 52C in FIG. 2) or the character string (the singer name in FIG. 2) based on the command 284. The character string “A”) is acquired. The designation information acquisition unit 23 generates designation information 285 including the acquired information and outputs it to the request generation unit 26 (step ST5).

  In the request generation unit 26, the reference request unit 263 generates a reference request 50 </ b> R based on the designation information 285 from the designation information acquisition unit 23 and outputs it to the communication processing unit 29. The communication processing unit 29 transmits the reference request 50R to the reference server 200 (step ST7).

  In the reference server 200, the communication processing unit 591 receives the reference request 50R from the terminal 100 (step RT3). The request processing unit 592 analyzes the received reference request 50R. In the analysis, when it is determined that the request code 501A is included, the type of data stored in the main body 504 of the reference request 50R is determined. A search request 270 including the determined data type and main body 504 is generated and output to the search processing unit 593 (step RT5).

  When the search request 270 is input, the search processing unit 593 determines a DB to be searched based on the data type of the search request. In this case, it is assumed that the DB to be searched is determined to be the first action management DB 210 based on the data type of the search request 270.

  The search processing unit 593 searches the determined first action management DB 210 based on the main body 504 of the search request 270 (step RT7). As a result of the search, the action information 267 is read from the record 219 including the image identifier 505 of the main body 504 and the data 292 to 295 matching the data 513 or 514, and is output to the response generation unit 594.

  The response generation unit 594 generates a response 50S addressed to the request source terminal 100 that stores the action information 267 from the search processing unit 593 in the main body unit 504, and outputs the response 50S to the communication processing unit 591. The communication processing unit 591 transmits the response 50S to the requesting terminal 100 (step RT11).

  The terminal 100 receives the response 50S of the reference request 50R described above via the communication processing unit 29 (step ST9). The communication processing unit 29 outputs the received response 50S to the response processing unit 27. The response processing unit 27 analyzes the main body unit 504 based on the response code 501B of the header 501 and outputs action information 267 included therein to the action execution unit 28 (step ST11).

  The action execution unit 28 executes an action based on the given action information (step ST13). Here, “execute an action” indicates that the command of the action definition 530 of the action information 267 is executed.

  The action execution unit 28 outputs a command to each unit based on the result of the action execution. Here, it is assumed that a command is output to the information request unit 262 based on the action execution result. This command includes the information acquisition destination 531 of the main body 504 of the response 50S.

  The information request unit 262 generates an information request 51R based on the information acquisition destination 531 included in the input command from the action execution unit 28, and transmits the information request 51R via the communication processing unit 29. As a result, the information request 51R is transmitted to the external device that is the information acquisition destination indicated by the information acquisition destination 531 of the response 50S received by the terminal 100 (step ST15). Here, it is assumed that the information is transmitted to the information providing server 400.

  Upon receiving the information request 51R (step PT3), the information providing server 400 searches for information from a predetermined storage area based on the information acquisition destination 531 of the received information request 51R (step PT7), and sends a response 51S including the searched information. It transmits to the requesting terminal 100 (step PT9).

  In terminal 100, communication processing unit 29 receives response 51S from information providing server 400 (step ST15). The received response 51S is analyzed by the response processing unit 27, and the requested information (distribution information) is extracted from the response 51S by the analysis, and is given to the action execution unit 28 (step ST17).

  When the distribution information is input, the action execution unit 28 executes an action according to the action definition 530 received in step ST9 using the input distribution information (step ST19). The distribution information may be stored in the storage unit 3 and then the action may be executed.

  In step ST19, the action execution unit 28 executes the action as follows. For example, the part 5C in FIG. 2 is touched on the terminal 100, and a reference request 50R (book ID “magazine A”, page number “3” and character string “A” representing the singer name) is sent to the reference server 200). Assume that it is sent.

  In this case, in the reference server 200, the first action management DB 210 is searched, and by the search, action information 267 (action definition 296 (command group of “music streaming playback” and “image display”) and information acquisition destination data 297 are obtained. ("Http: // music site \ AAAdl.html" and "http: // music site \ AAA.jpg" of links 3 and 4) are acquired and transmitted to the requesting terminal 100 as a response 51S. Accordingly, in the terminal 100, the action executing unit 28 requests the information providing server 400 to distribute information by executing an action according to the received action information 267, and the distributed information (music data, image data) is requested. The action according to the action definition 530 is executed. As a result, the music data of the distribution information is streamed and played, and the music is output via a speaker (not shown), and the image data is specified by the action definition on the screen of the display unit 6 via the display processing unit 282. An image is displayed in a manner.

  In this way, the user simply acquires the information associated with the partial image of the part 5C from the information providing server 400 by simply touching the desired part 5C of the display image, and outputs the acquired information in a manner according to the action definition. be able to.

  If there are a plurality of action definitions 296 acquired as described above, such as “music streaming playback” and “image display”, a plurality of action definitions 296 are output to the display unit 6. The user may select an action definition 296 desired to be executed. By selecting, when the user desires to distribute only the music data, the music data can be received and reproduced.

  In the above example, the character string “A” representing the singer name of the portion 5C is displayed, but the present invention is not limited to this. For example, when the character string “product number” is displayed in the part 5 </ b> C, the terminal 100 can acquire action information 267 for displaying a product purchase menu from the reference server 200. As a result, “product purchase menu screen information” is received from the information providing server 400 using the action information 267, and the menu screen for product purchase is displayed on the display unit 6 by the action definition 296 of the action information 267. can do.

  When the character string “store name” is displayed in the part 5C, the terminal 100 can acquire action information 267 for displaying information on the advertisement of the store from the reference server 200. it can. Accordingly, “advertisement advertisement information” can be received from the information providing server 400 using the action information 267, and the advertisement screen can be displayed on the display unit 6 by using the action definition 296 of the action information 267.

(Other embodiments)
In the embodiment described above, the reference server 200 is searched based on the image identifier 505 of the reference request 50R and the character string data 513 of the recognition result or the coordinate data 514 of the part 5C. It is not limited to. In the present embodiment, a case where terminal ID data 516 is used as search data will be described.

  The reference server 200 receives the reference request 50 </ b> R in which the main body 504 stores the terminal ID data 516. The search processing unit 593 searches the terminal information DB 260 based on the data 516 of the main body unit 504. As a result of the search, terminal attribute data 265 corresponding to data 264 indicating the terminal ID of data 516 is read.

  Subsequently, based on the terminal attribute of the data 265, the image identifier 505 of the reference request 50R, and the character string data 513 of the recognition result or the coordinate data 514 of the part 5C, the search processing unit 593 matches the data 241 with these data. 245 and 247 and a record 229 having data 248 or 249 are extracted from the second action management DB 230. Then, a response 50S including the action information 267 (the action definition number data 250, the action definition data 251 and the information acquisition destination data 252) of the extracted record 229 is transmitted to the requesting terminal 100, and the action information 267 is transmitted. The based action is executed. Therefore, based on the attribute of the requesting terminal 100, the information distributed from the information providing server 400 to the terminal 100 can be made different, and the action definition executed by the action execution unit 28 can be made different.

  A case where a user attribute is used as an attribute of the terminal 100 will be described. Here, in the second action management DB 230 of FIG. 12, the data acquisition destination data 252 for acquiring the information that is allowed to be presented to the adult in the record 229 whose terminal attribute of the data 241 is “0x06bc”. Is stored, and the record 229 having the terminal attribute “0x06bca” is assumed to store data 252 that does not include an information acquisition destination for acquiring information that is prohibited from being presented to a minor.

  In this case, when the terminal attribute data 265 obtained by searching the terminal information DB 260 indicates “0x06bc”, the second action management DB 230 obtains information that is allowed to be presented as an adult. Only the record 229 in which the data 252 of the information acquisition destination is stored can be extracted.

  Therefore, the requesting terminal 100 can receive only information that is allowed to be browsed from the information providing server 400 according to the attribute (whether the user is an adult or a minor). An action definition for presenting can be received.

  Here, the reference server 200 manages the attribute for each terminal 100 using the terminal information DB 260, but the attribute data 517 may be included in the main body 504 of the reference request 50R. In that case, terminal attribute management using the terminal information DB 260 by the reference server 200 can be omitted.

(Modification 1)
The reference server 200 creates two types of the second action management DB 230 and the terminal information DB 260 that are to be searched on holidays and those to be searched on weekdays, and stores them in the storage unit 205. . If it is determined that the date and time indicated by the data 518 of the main body 504 of the reference request 50R indicates a holiday, the DB for holiday is searched for the reference request 50R, and the DB for weekday is searched when it is determined that the request is received on weekdays. To do. As a result, even when the same part 5C is touched on the requesting terminal 100, the information delivered from the information providing server 400 can be different on holidays and weekdays, and the action execution unit The action definition executed by 28 can be made different.

  Therefore, when making an inquiry about an electronic book, a message indicating that the inquiry center is closed on holidays can be presented, and a telephone number of the inquiry center can be presented on weekdays.

(Modification 2)
The reference server 200 may store different records 219 and 231 for each publication number of the electronic book for the first action management DB 210 and the second action management DB 230.

  For example, the first action management DB 210 or the second action management DB 230 is searched using the publication number of the data 511 of the main body 504 of the reference request 50R. Thereby, even when the same part 5C is touched on the requesting terminal 100, the information delivered from the information providing server 400 can be made different according to the publication number of the electronic book, and the action The action definition executed by the execution unit 28 can be varied.

  In place of the publication number, the date when the terminal 100 purchased the electronic book 37 may be used.

  As a result, information indicating that some privilege can be received can be presented only to the user who purchased the electronic book 37 during the campaign period or the user who purchased the latest version of the electronic book 37.

(Modification 3)
The reference server 200 may store different records 219 and 231 for each area to which the current position of the terminal 100 belongs for the first action management DB 210 and the second action management DB 230.

  For example, the first action management DB 210 or the second action management DB 230 is searched using the operation location of the data 519 of the main body 504 of the reference request 50R. Thereby, a record corresponding to the area corresponding to the operation place can be extracted. Even if the same part 5C is touched with respect to the requesting terminal 100, the information providing server 400 uses the terminal 100 for each area to which the current location where the user is browsing the electronic book 37 belongs. Information to be distributed can be made different, and action definitions executed by the action execution unit 28 can be made different.

  Thereby, the information limited when the user browses the electronic book in the specific area can be distributed.

(Modification 4)
When the reference server 200 generates the first action management DB 210, the reference server 200 generates the records 219 that match the character strings indicated by the data 295, and at the time of searching, the reference server 200 generates a plurality of links linked by one search process. The record 219 may be extracted.

  When the book ID 510 is “magazine A” and the data 513 is searched using the character string “A” representing the singer name, according to the first action management DB 210 of FIG. 10, the linked records R1 and R2 are stored. Extracted. In response 50S, action information 267 of record R1 is transmitted to requesting terminal 100. At this time, the response 50S may include an action definition for presenting recommendation information for the book ID (magazine B) indicated by the data 292 of the linked record R2.

  Thereby, the user can obtain recommendation information of another electronic book in which information about the singer named “A” is described.

(Modification 5)
Assume that the user browses an electronic book whose record 219 is not registered in the first action management DB 210 of FIG. When the electronic book includes a character string (“A”) of “singer name A” and the user touches the character string of “singer name A” of the electronic book, a reference request is made to the reference server 200. 50R (character string “A” indicating the singer name) is transmitted.

  In this case, the reference server 200 searches the first action management DB 210 based on the character string “A”. As a result of the search, action information 267 (action definition 296 (“music streaming playback” and “image display” command group) and information acquisition destination data 297 (links 3 and 4 “http: // music site ¥ AAAdl.html”) And “http: // music site \ AAA.jpg”) are acquired and transmitted as a response 51S to the requesting terminal 100. Accordingly, in the terminal 100, the action execution unit 28 receives the received action information. By executing the action according to H.267, the information providing server 400 is requested to distribute information, and the action according to the action definition 530 is executed using the distributed information (music data, image data). The music data of the information is streamed and played, and the speed is not shown. With music is output via the mosquitoes, the image data is the image displayed in the manner specified by the action definition on the screen of the display unit 6 through the display processor 282.

  Thus, by touching the character string “A” representing “singer name A”, information originally intended for the electronic book being browsed by the user is registered in the first action management DB 210. Can be obtained from the information of the electronic book, and information can be shared among users.

(Modification 6)
The reference server 200 may switch the action information 267 to be transmitted to the terminal 100 based on the attribute of the terminal 100. For example, the action definition 296 may be switched depending on the OS type / version of the terminal 100. Thereby, even if the OS installed for each terminal 100 is different, each terminal 100 can receive an action definition executable by its own OS.

(Effects of this embodiment)
As described above, when displaying image data of a bitmap, the terminal 100 touches a partial image on the screen, and based on the information (character string, coordinate position, etc.) acquired from the partial image, the information The information acquisition destination 531 and the action definition 530 such as the information providing server 400 that distributes the related information can be acquired.

  As a result, since the display screen is small, the characters of the image are crushed, so even if the product image in the electronic book can be confirmed but the product number cannot be read, the portion 5C of the product number is touched. Only the information acquisition destination 531 and the action definition 530 can be acquired.

  If an electronic book is associated with a URL or a hyperlink among electronic books, information on the corresponding server can be browsed by clicking the displayed URL. However, the page information of the electronic book is not displayed as in this embodiment. In the case of an image, in order to access the server, it is necessary to set link information in a specific region of the image, which complicates the data generation procedure of the electronic book. In addition, since link information is individually set for each electronic book, the link information cannot be changed after the electronic book is sold to the user. In the present embodiment, the information acquisition destination data 297 that is link information is collectively managed by the reference server 200 and updated by the reference server 200, so there is no need to set link information in the electronic book. Also, the link information can be easily updated in a batch.

(Still another embodiment)
FIG. 14 shows a hardware configuration of PC 100A according to the present embodiment.

  The information processing method in terminal 100 described above can also be provided as a program. Such a program can be recorded on a computer-readable recording medium such as a flexible disk, a CD-ROM, a ROM, a RAM, and a memory card attached to the computer and provided as a program product. Alternatively, the program can be provided by being recorded on a recording medium such as a hard disk built in the computer. A program can also be provided by downloading via a network. For example, in the configuration of FIG. 1, the program can be supplied to the terminal 100 having the CPU 2 and having the function of a computer using the recording medium 11. The CPU 2 reads and executes a program stored in the recording medium 11 via the I / F 10. The terminal 100 can also store the program in a predetermined area of the storage unit 3 that fluidly carries the program by downloading via the communication network NT.

  The program is also provided to the PC 100A in a similar manner. FIG. 14 shows a configuration example of the PC 100A. Referring to FIG. 14, PC 100A includes a processing unit 601, a display 602 including a CRT (cathode ray tube) / liquid crystal display, a keyboard 603, and a mouse 604. The processing unit 601 is mounted with a CPU 605 for centrally controlling the PC 100A itself, a memory 606 including ROM or RAM, a fixed disk 607, and an FD (flexible disk) 612 detachably mounted. An FD driving device 611 that accesses the FD 612, a CD-ROM (Compact Disc Read Only Memory) 614 is detachably mounted, a CD-ROM driving device 613 that accesses the mounted CD-ROM 614, a communication network NT, A communication interface 609 for communication connection with the PC 100A is included. These units are connected for communication via a bus.

  The PC 100A may be provided with a magnetic tape device in which a cassette type magnetic tape is detachably mounted to access the magnetic tape.

  A program can be provided to the PC 100 </ b> A in FIG. 14 using the FD 612 and the CD-ROM 614. The program can also be stored in the memory 606 or the fixed disk 607 in the PC 100A by downloading via the communication network NT.

  As described above, terminal 100 and PC 100A are supplied with a program product for implementing the method according to the above-described embodiment using a computer. The program product includes the program itself and a recording medium on which the program is recorded.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  13 position detection unit, 20 input / output unit, 23 designation information acquisition unit, 26 request generation unit, 27 response processing unit, 28 action execution unit, 29,591 communication processing unit, 30 information acquisition unit, 31 storage unit, 32 reading unit 36 reference server address information, 37 electronic book, 50R reference request, 50S, 51S response, 51R information request, 100 terminal, 200 reference server, 210 first action management DB, 230 second action management DB, 231 character acquisition unit, 232 position acquisition unit, 262 information request unit, 263 reference request unit, 267 action information, 270 search request, 281 operation reception unit, 282 display processing unit, 283 display information holding unit, 284 command, 285 designation information, 296, 530 action Definition, 300 e-book distribution Trust server, 301 image acquisition unit, 400 information providing server, 592 request processing unit, 593 search processing unit, 594 response generation unit.

Claims (11)

An information processing terminal that communicates with a server,
An image acquisition unit for acquiring image data and an image identifier corresponding to the image data;
A display unit for displaying an image according to the image data acquired by the image acquisition unit on a screen;
An operation accepting unit for accepting an operation for designating a part of the image displayed on the screen;
An information acquisition unit for acquiring predetermined information from the partial image specified by the operation received by the operation reception unit;
A transmission unit for transmitting the predetermined information acquired by the information acquisition unit and the image identifier acquired by the image acquisition unit to the server;
A receiving unit for receiving action information including link destination information indicating a link destination of the predetermined information and processing information corresponding to the link destination information;
An information processing terminal comprising: a control unit that controls the information processing terminal based on the link destination information and the processing information received by the receiving unit. The information acquisition unit
The character acquisition part for extracting a character string by recognizing the said one part image designated by operation which the said operation reception part receives, and acquiring the extracted character string as said predetermined information is included. Information processing terminal described in 1. The information acquisition unit
The position acquisition part for acquiring the position in the image currently displayed on the said screen of the said one part image designated by the operation which the said operation reception part receives as said predetermined information. The information processing terminal described.   When the receiving unit receives a plurality of action information, the control unit controls the information processing terminal based on link destination information and processing information of action information selected from the plurality of action information. Item 4. The information processing terminal according to any one of Items 1 to 3.   The information processing terminal according to claim 1, wherein the image data indicates electronic book data. The transmitter is
6. The device according to claim 1, wherein the predetermined information acquired by the information acquisition unit, the image identifier acquired by the image acquisition unit, and terminal information related to the information processing terminal are transmitted to the server. Information processing terminal.   The information processing terminal according to claim 6, wherein the terminal information indicates an attribute of the information processing terminal.   The information processing terminal according to claim 6 or 7, wherein the terminal information indicates an attribute of a user of the information processing terminal. A server device that communicates with the information processing terminal according to claim 1,
An information storage unit that stores the predetermined information, the image identifier, and the action information in association with each other;
A server receiver for receiving the predetermined information and the image identifier from the information processing terminal;
A reading unit for reading the corresponding action information from the information storage unit based on the received predetermined information and the image identifier;
A server device that transmits the read action information to the information processing terminal. A system comprising an information processing terminal and a server device communicating with the information processing terminal,
The information processing terminal
An image acquisition unit for acquiring image data and an image identifier corresponding to the image data;
A display unit for displaying an image according to the image data acquired by the image acquisition unit on a screen;
An operation accepting unit for accepting an operation for designating a part of the image displayed on the screen;
An information acquisition unit for acquiring predetermined information from the partial image specified by the operation received by the operation reception unit;
A transmission unit for transmitting the predetermined information acquired by the information acquisition unit and the image identifier acquired by the image acquisition unit to the server device;
A receiving unit for receiving action information including link destination information indicating a link destination of the predetermined information and processing information corresponding to the link destination information;
A control unit that controls the information processing terminal based on the link destination information and the processing information received by the receiving unit,
The server device
An information storage unit that stores the predetermined information, the image identifier, and the action information in association with each other;
A server receiver for receiving the predetermined information and the image identifier from the information processing terminal;
A reading unit for reading the corresponding action information from the information storage unit based on the received predetermined information and the image identifier;
A system for transmitting the read action information to the information processing terminal. A program for causing a computer including a processor to execute an information processing method,
The information processing method includes:
An image acquisition unit of the computer acquiring image data and an image identifier corresponding to the image data;
The display unit of the computer displaying an image according to the acquired image data on a screen;
A step of accepting an operation of designating a part of an image displayed on the screen by an operation accepting unit of the computer;
The information acquisition unit of the computer acquiring predetermined information from the partial image designated by the operation received in the receiving step;
The reference unit of the computer transmits the acquired predetermined information and the acquired image identifier to a server via a communication line;
A step of receiving action information including link destination information indicating a link destination of the predetermined information and processing information corresponding to the link destination information from the server via a communication line;
And a step of controlling the operation of the computer based on the received link destination information and the processing information.

Images