JP2011077708A - Relay server device and communication method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a load of a user at the time of realizing multi modal using a plurality of client terminals. <P>SOLUTION: A relay server device provided between a server and a plurality of clients has: a management portion associating a log-in ID input from each client with a session ID identifying each client to be held; a Web server portion relaying a request from a first client to the server and returning a response corresponding to the request received from the server to the first client; and a distribution portion transmitting the response to the second client when the log-in ID corresponding to the session ID identifying the second client held in the management portion is the same as the log-in ID corresponding to the session ID identifying the first client held in the management portion. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a server device and a communication method for providing multimodal.

  In recent years, proposals or practical applications of multimodal systems have increased. In a multimodal system, a user uses a plurality of client terminals, and each terminal interacts with the server system in the same or different communication mode (for example, visual, auditory, etc.). For example, a user communicates with a server using a plurality of client terminals (for example, a telephone, a TV receiver, etc.).

  FIG. 1 is a diagram illustrating an example of a technique for realizing multimodal. Here, a system will be described in which when a user orders a product (pizza in the example shown in FIG. 1) by telephone, a picture of the product or the details of the order are displayed on a television screen.

  The user can operate the telephone set 101 and the television receiver (hereinafter simply referred to as “TV”) 102. Both the telephone 101 and the television 102 are connected to the Internet. A pizzeria order server 103 is also connected to the Internet.

  When ordering a pizza, the user first calls the pizzeria using the telephone 101. At this time, it is assumed that a protocol capable of cooperating with the Internet (for example, SIP (Session Initiation Protocol)) is used. In addition, when the user wants to see an image of a pizza photo when placing an order, he or she informs the person in charge of the pizza shop verbally. Then, the person in charge at the pizzeria requests the IP address of the image transmission destination from the telephone set 101 by giving an instruction to the order server 103. At this time, a request message is transmitted using SIP. In response to this request, the telephone 101 notifies the order server 103 of the IP address of the television 102. When the order server 103 transmits image data to the television 102, a pizza image is displayed on the television 102.

  However, this method requires the television 102 to have a global IP address. For this reason, for example, when NAT (Network Address Translation) is provided on the client side and a local address is assigned to the television 102, image data cannot be transmitted from the order server 103 to the television 102. .

  FIG. 2 is a diagram illustrating another technique for realizing multimodal. In this system, the location of the server is notified from the first terminal (telephone 101) to the second terminal (television 102) using SIP. That is, the order server 103 notifies the telephone 101 of a URL for accessing the image. Then, the telephone set 101 notifies the TV 102 of this URL using SIP. Then, the television 102 acquires an image by connecting to this URL using HTTP. As a result, the requested image is displayed on the television 102.

  According to this method, even if a global IP address is not assigned to the television 102, the requested image is displayed on the television 102, but there are the following problems. That is, in this system, the television 102 needs to support asynchronous communication such as SIP. However, many televisions already shipped do not support SIP. In this method, the IP address of the television 102 needs to be set or registered in advance in the telephone 101, but this setting / registration work is not always easy for many users.

  As a related technique, an electronic shopping system that secures security has been proposed. This system groups multiple user terminals, displays a common display screen, and authenticates group members individually when logging in and making payments to the online shopping site, and a proxy server that mediates between the online shopping site An authentication server. The authentication server further authenticates the members of the group individually, and assigns predetermined authentication completion information to the members who are normally authenticated. The proxy server uses the authentication completion information of each member of the group to mediate with the online shopping site, and can identify each member in the online shopping server.

JP 2006-171829 A

  In the above-described multimodal system, the client terminal needs to have special software or protocol installed. That is, the user needs to prepare a client terminal in which special software or protocol is implemented in order to realize multimodal. Alternatively, the user needs to perform various settings / registrations at the client terminal.

  An object of the present invention is to reduce a burden on a user when realizing multimodal using a plurality of client terminals.

  A relay server device according to one aspect of the present invention is provided between a server and a plurality of clients, and includes a management unit that holds a login ID input from each client and a session ID that identifies each client in association with each other. Relaying a request from the first client to the server, and returning a response corresponding to the request received from the server to the first client; and a second server held in the management unit When the login ID corresponding to the session ID for identifying the second client is the same as the login ID corresponding to the session ID for identifying the first client held in the management unit, the response is A distribution unit for transmitting to the second client;

  According to the configuration or method disclosed in the present application, the burden on the user when realizing multimodal using a plurality of client terminals is reduced.

It is a figure which shows an example of the technique which implement | achieves multimodal. It is a figure which shows the other technique which implement | achieves multimodal. It is a figure explaining the method of implement | achieving multimodal using a relay server. It is a figure which shows the structure of the multimodal system of embodiment. It is a figure which shows an example of a portal screen. It is a flowchart of the method of implement | achieving multimodal in 1st Embodiment. It is an Example of the table which manages login ID and session ID. It is another Example of a management table. It is a flowchart (the 1) of the method of implement | achieving multimodal in 2nd Embodiment. It is a flowchart (the 2) of the method of implement | achieving multimodal in 2nd Embodiment. It is a figure explaining the method to inject | pour a script into an HTML document. It is an Example of an event issue script. It is an Example of an event reception script. It is an example of a display of the gadget implemented in each client terminal. It is a figure which shows the hardware constitutions of a relay server.

  FIG. 3 is a diagram for explaining a method of realizing multimodal using a relay server. In FIG. 3, a telephone 1 and a television receiver (hereinafter simply referred to as “TV”) 2 are client terminals used by a user. Here, it is assumed that the telephone 1 and the television 2 are each equipped with a browser. In the following example, it is assumed that the user orders a pizza using the telephone 1 and the television 2. The order server 3 executes a process corresponding to the order from the client and provides a service. The relay server 4 is provided between the client (telephone 1 and television 2) and the server (order server 3), and operates as a portal server. In the system shown in FIG. 3, multimodal is realized by the following procedure.

  (1) The user logs in to the relay server 4 by inputting a login ID using the browser of the telephone 1.

  (2) The user logs in to the relay server 4 by inputting a login ID using the browser of the television 2. The user inputs the same login ID using the telephone 1 and the television 2. The login ID input using each client terminal is managed by a management table.

  (3) A connection for voice call is set up between the telephone 1 and the order server 3 using SIP. Using this connection, the user requests a pizza image, for example, verbally.

  (4) The order server 3 notifies the telephone 1 of a URL (Uniform Resource Locator) of a gadget that provides a pizza image (hereinafter, “pizza image gadget”). The gadget is not particularly limited, but is an application program distributed to the client terminal, and may be called a widget. The gadget may include a program component that is executed on the Web. Furthermore, the gadget is arranged, for example, on the desktop of the client terminal.

  (5) The browser of the telephone 1 accesses the notified URL and acquires the pizza image gadget. At this time, communication between the telephone 1 and the order server 3 is performed via the relay server 4.

  (6) A pizza order page is displayed on the browser of the telephone 1 by the pizza image gadget.

  (7) The relay server 4 refers to the management table described above and checks whether the same login ID as the login ID input using the telephone 1 is registered. In this example, the same login ID is input using the telephone 1 and the television 2. In this case, the relay server 4 asynchronously notifies the TV 2 of the URL of the pizza image gadget.

  (8) The browser of the television 2 accesses the notified URL and acquires the pizza image gadget. At this time, communication between the television 2 and the order server 3 is performed via the relay server 4.

  (9) A pizza order page is displayed on the browser of the television 2 by the pizza image gadget. At this time, content different from the telephone 1 may be displayed on the browser of the television 2. For example, a larger pizza image or more detailed order information is displayed on the browser of the television 2. Thereafter, the user can order a desired pizza while viewing the pizza image displayed on the browser of the television 2.

  As described above, in the method shown in FIG. 3, if the login IDs input using the telephone 1 and the television 2 are the same, the relay server 4 transmits the pizza image gadget to the telephone 1. The TV 2 is notified of the URL of the pizza image gadget. Therefore, the pizza image is displayed on the browser of the television 2 by the pizza image gadget. According to this method, direct communication is not performed between the telephone 1 and the television 2.

<First Embodiment>
FIG. 4 is a diagram illustrating a configuration of the multimodal system according to the embodiment. The system configuration shown in FIG. 4 is basically common to the first embodiment and the second embodiment described later. Hereinafter, it is assumed that a user orders a product (here, pizza) using the telephone 1 and the television 2.

  The telephone 1 and the television 2 are client terminals used by the user. The telephone 1 and the television 2 are each connected to the Internet. Note that the telephone 1 and the television 2 may be directly connected to the Internet, or may be connected to the Internet via NAT or the like.

  The telephone 1 is, for example, a mobile terminal or a mobile phone, and a browser 11 that browses pages on the Internet by HTTP communication is installed. Further, the telephone 1 includes a call unit 12 and a reception unit 13. The call unit 12 has a function of making and receiving calls by SIP, and terminates a connection for a voice call. The receiving unit 13 receives an asynchronous communication packet using SIP.

The television 2 is a receiver connected to the Internet. The television 2 is also equipped with a browser 21 for browsing pages on the Internet by HTTP communication.
The order server 3 is connected to the Internet, executes a corresponding process in accordance with an order from a client, and provides a service. The order server 3 includes a SIP server unit, and transfers a request using SIP to the call center terminal 5 in an unused state. The call center terminal 5 is connected to the order server 3 and has a function of controlling transmission / reception with the client terminal in accordance with an instruction input by the operator. The order server 3 includes a Web server unit, receives an HTTP request, and returns a corresponding response. Further, the pizza image gadget 31 is stored in the memory provided in the order server 3. And the order server 3 delivers the pizza image gadget 31 according to the request | requirement from a client.

  The relay server 4 is provided between the client (telephone 1 and television 2) and the server (order server 3), and operates as a portal server for the telephone 1 and television 2 in this example. Note that the relay server 4 may also be connected to the Internet via NAT or the like. In this case, the telephone 1, the television 2, and the relay server 4 may be arranged in a NAT private network.

  The relay server 4 includes a Web server unit 41, a distribution unit 42, a management unit 43, and a script injection unit 44. The Web server unit 41 receives an HTTP request and acquires content corresponding to the request as necessary. In this example, the Web server unit 41 acquires a gadget (pizza image gadget 31) corresponding to the HTTP request. The distribution unit 42 transmits the content acquired by the Web server unit 41 to the client. The management unit 43 manages a session of a client that accesses the relay server 4. The script injection unit 44 injects one or a plurality of scripts prepared in advance into the content distributed to the client by the Web server unit 41 or the distribution unit 42. This script is executed in the client terminal that has received the content. The script injection unit 44 is used in the second embodiment.

  In the system configured as described above, the pizza image gadget 31 is distributed to the browser 11 of the telephone 1 and the browser 21 of the television 2. Then, the pizza image gadget 31 is arranged in the portal screen of the browsers 11 and 21.

  FIG. 5 is a diagram illustrating an example of a portal screen. The portal screen is one web page. A gadget acquired from a server on the Internet is usually placed in this portal screen. In the example shown in FIG. 4, portal screens are displayed by the browsers 11 and 21, respectively, and a pizza image gadget 31 is arranged in each portal screen. A plurality of gadgets can be arranged in the portal screen. In FIG. 5, three gadgets are arranged in the portal screen.

FIG. 6 is a flowchart showing a method for realizing multimodal in the first embodiment.
In steps S1 to S2, before starting communication, the user uses the browsers 11 and 21 of all client terminals (in this example, the telephone 1 and the television 2) that want to realize multimodal, and uses the same login ID. To log in to the relay server (Web portal server) 4. Login using the browser 11 may be performed first, or login using the browser 21 may be performed first. At this time, for example, the user inputs a URL indicating the relay server 4 in each of the browsers 11 and 21 by operating a keypad or a remote control input device. Alternatively, the URL of the relay server 4 registered in advance may be selected. Thereby, an HTTP request for login is transmitted from each browser 11, 21 to the relay server 4.

  The user can input the login ID using an input device provided in the client terminal. In addition, when the client terminal has a function of saving the login ID input earlier, the login ID input earlier may be read out and used.

  The relay server 4 performs user authentication by a method such as password input, and identifies the login ID of the user. Further, the relay server 4 returns the portal screen to the browsers 11 and 21 as an HTTP response corresponding to the received request. In the portal screen, for example, as described with reference to FIG. 5, one or a plurality of gadgets can be arranged in one page. At this time, a portal application that virtually realizes asynchronous communication may be downloaded from the relay server 4 to the browsers 11 and 21 using HTTP. In this case, this portal application is arranged in a portal screen on the browsers 11 and 21, for example. The portal application is a program (for example, JavaScript) that can be executed in the browser.

  Asynchronous communication performed between the relay server 4 and the browsers 11 and 21 is, for example, Comet. In this case, the browsers 11 and 21 set an HTTP connection with the relay server 4 using a command such as JavaScript's XMLHttpRequest, and wait for an HTTP response to be returned. Further, the relay server 4 generates a session ID for uniquely identifying each client browser by using a cookie or the like. Thereby, after the login is successful, a session ID is given to the request generated from the browsers 11 and 21, and the relay server 4 transmits a gadget or the like to the browsers 11 and 21 using the session ID. It becomes possible.

  If the login is successful, the management unit 43 of the relay server 4 manages the login ID and the session ID in association with each other. The login ID and session ID are managed in a table format as shown in FIG. 7, for example. Here, the first line of the management table shown in FIG. 7 shows a state in which login is performed with the login ID “user001” using the browser 11 of the telephone 1, and “ab93jdh4sb” is generated as the session ID at that time. Represents. The second row of the management table represents a state where login is performed with the login ID “user001” using the browser 21 of the television 2 and “dk4jdsnskk” is generated as the session ID at that time. That is, in the example illustrated in FIG. 7, the user logs in with the same login ID using the browsers 11 and 21. The information managed by the management unit 43 is deleted, for example, when the session is logged out or when communication for the session is not performed for a certain period of time.

  In step S <b> 3, the user uses the telephone 1 to call a pizzeria call center. At this time, a call is made from the call unit 12 of the telephone 1 to the order server 3. Then, in step S4, the order server 3 transfers the call from the telephone 1 to the call center terminal 5 in an unused state. At this time, an operator of the call center terminal 5 (that is, a person in charge at a pizzeria) accepts an incoming call. As a result, a call between the user and the person in charge at the call center is established, and an order can be made.

  When a user wants to see a photograph of a product (here, pizza), he / she notifies the person in charge of the call center, for example. Then, the person in charge instructs the image transmission to the telephone 1 by operating the call center terminal 5 in step S5. At this time, an image transmission instruction is given to the order server 3.

  In step S6, when the order server 3 receives the above-described image transmission instruction, the order server 3 notifies the telephone 1 of a URL for displaying an image using the SIP message. This URL represents the storage location of the pizza image gadget 31 shown in FIG. 4, for example.

  In step S <b> 7, the receiving unit 13 of the telephone 1 receives the SIP message transmitted from the order server 3. Then, the reception unit 13 notifies the browser 11 of the URL included in the received SIP message. That is, the browser 11 acquires the URL of the pizza image gadget 31. At this time, a portal screen is displayed on the browser 11 and a portal application for asynchronous communication received from the relay server 4 is installed.

  In step S8, the portal application of the telephone 1 notifies the relay server 4 of the URL acquired from the order server 3 by an HTTP request. At this time, this URL is transmitted to the relay server 4 together with a cookie or the like as a parameter indicating an access destination. Note that the notification from the telephone 1 to the relay server 4 is received by the Web server unit 41.

  In step S9, the Web server unit 41 of the relay server 4 accesses the order server 3 by HTTP using the URL notified from the telephone 1. Then, the order server 3 transmits the HTML document of the accessed content to the relay server 4 as a response corresponding to the HTTP request. Thereby, the relay server 4 receives the pizza image gadget 31. Then, the Web server unit 41 returns the pizza image gadget 31 acquired from the order server 3 to the portal application of the telephone 1 that is the transmission source of the HTTP request.

  In step S <b> 10, the portal application of the telephone 1 adds the received gadget to the portal screen displayed on the browser 11. That is, the pizza image gadget 31 is added to the portal screen of the browser 11. At this time, an HTML document including a pizza image is rendered, and the generated image is displayed on the browser 11.

  In step S <b> 11, the relay server 4 extracts the session ID included in the cookie generated for communication with the telephone 1. Then, the management unit 43 refers to the management table shown in FIG. 7 and identifies the login ID corresponding to the extracted session ID. Further, the management unit 43 searches whether or not the same login ID as the specified login ID is registered in the management table. That is, the relay server 4 confirms whether the user of the telephone 1 is logged in simultaneously using another client terminal.

  In the example illustrated in FIGS. 4 and 7, first, the session ID “ab93jdh4sb” generated for the communication with the browser 11 is extracted. Then, the first line of the management table is referred to, and the login ID “user001” corresponding to this session ID is specified. Furthermore, other client terminals that have logged in with “user001” are searched. As a result, the session ID “dk4jdsnskk” is specified from the second row of the management table. That is, it is determined that the browser 11 of the telephone 1 and the browser 21 of the television 2 are logged in.

  In step S <b> 12, the distribution unit 42 of the relay server 4 transmits the HTML document of the pizza image gadget 31 asynchronously to the browser 21 of the television 2 logged in with the same login ID. That is, the pizza image gadget 31 is distributed from the relay server 4 to the browsers 11 and 21. When Comet is used as an asynchronous communication method for transmitting a gadget or the like from the relay server 4 to the television 2, the HTTP session established when the login is performed using the browser 21 is held. Then, the relay server 4 transmits an HTML document or the like to the television 2 as an HTTP response at a desired timing.

  In step S <b> 13, the portal application of the browser 21 of the television 2 receives the HTTP response by asynchronous communication from the relay server 4. This HTTP response includes a pizza image gadget 31 (HTML document). Then, the portal application of the television 2 adds the received gadget to the portal screen displayed on the browser 21. That is, the pizza image gadget 31 is added to the portal screen of the browser 21. At this time, an HTML document including a pizza image is rendered, and the generated image is displayed on the browser 21.

  Thus, in the multimodal system of the embodiment, the same gadget is distributed almost simultaneously to a plurality of browsers logged in with the same login ID. In the example shown in FIG. 4, the browser 11 of the telephone set 1 is a client terminal (first client terminal) operated by a user, and the client terminal (second client terminal) used simultaneously with the first client. A pizza image gadget 31 is displayed on the browser 21 of the television 2. At this time, if software (for example, a browser) for browsing pages on the Internet is installed on the television 2, the television 2 can display an image of the gadget. That is, in an environment in which a browser is installed in each client terminal, it is not necessary for the user to perform difficult operations or settings in order to realize multimodal, and a communication protocol such as SIP is provided for the second client terminal. There is no need to implement it. Furthermore, it is not necessary to previously register the IP address of the second client terminal in the first client terminal.

  In the above example, the pizza image gadget 31 is distributed to the browser 11 of the telephone 1 and the browser 12 of the television 2, but the HTML documents distributed to the browsers 11 and 12 may be different. For example, when the display capability of each client terminal is different, the relay server 4 may transmit an HTML document edited according to the display capability of each client terminal.

  FIG. 8 shows another example of the management table provided in the management unit 43 of the relay server 4. In this management table, information indicating the capability of each client terminal is managed together with the login ID and the session ID. It is assumed that the user-agent information is included in the login information transmitted from each client terminal to the relay server 4. In this case, the relay server 4 can specify the model of the client terminal and its capability based on the received User-Agent information. Then, when the gadget is transmitted, the editing unit included in the relay server 4 edits the size of the transmission image and the text data according to the capabilities of the client terminal (image resolution, communication speed, etc.). According to this configuration, appropriate information is displayed on each client terminal.

  Alternatively, the relay server 4 may transmit the same HTML document to each client terminal (telephone 1, television 2). At this time, each client terminal may extract and display only a displayable portion from the received HTML document.

<Second Embodiment>
As described above, the system configuration of the second embodiment is basically the same as that of the first embodiment. That is, the system of the second embodiment also includes a relay server 4 between a client (telephone 1 and television 2) and a server (order server 3), as shown in FIG.

  However, in the second embodiment, a script is injected into an HTML document distributed to each client. The script is injected by the script injection unit 44 of the relay server 4. Then, by executing the script in each client, an event occurring in one client is notified to other clients, and a plurality of clients operate in synchronization or cooperation.

  9 and 10 are flowcharts illustrating a method for realizing multimodal in the second embodiment. Note that the processes in steps S1 to S8, S10 to S11, and S13 are the same as those in the first embodiment, and a description thereof will be omitted.

  In step S8, the portal application of the telephone 1 notifies the URL notified from the order server 3 to the browser 11 to the relay server 4 using an HTTP request. Then, when the Web server unit 41 receives this HTTP request, the processes after step S21 are executed.

  In step S21, the Web server unit 41 of the relay server 4 accesses the order server 3 with the URL notified by the HTTP request. Then, the order server 3 transmits the HTML document of the accessed content to the relay server 4 as a response corresponding to the HTTP request. Thereby, the relay server 4 receives the HTML document of the pizza image gadget 31.

  In steps S22 to S23, the script injection unit 44 of the relay server 4 inserts an event issuance script and / or an event reception script into the HTML document acquired from the order server 3. The event issuing script and the event receiving script are JavaScripts created in advance.

  FIG. 11 is a diagram illustrating a method for injecting a script into an HTML document. When providing the HTML document to the client, the script injection unit 44, for example, injects an event issue script and an event reception script at the end of the body part. However, the position where the script is injected is not limited to the end of the body part. For example, the script may be injected at the beginning of the body part.

  In step S <b> 22, the script injection unit 44 creates an event issuing HTML document by injecting the event issuing script into the HTML document acquired from the order server 3. In step S23, the script injection unit 44 generates an event reception HTML document by injecting the event reception script into the HTML document acquired from the order server 3.

  12 and 13 are examples of an event issue script and an event reception script, respectively. By executing these one set of scripts, a user operation for pressing an arrow key in one client terminal can be shared by a plurality of client terminals.

  In the event issuing script shown in FIG. 12, the description A includes an instruction for calling the function sendKey using as an argument the key code corresponding to the key pressed by the user. The description B includes an instruction for acquiring the definition key code of the function sendKey. The description C includes a command for connecting to the relay server 4 using XMLHttpRequest and transmitting event information (key code, event type, gadget type) to the connection destination. In this example, “iwc / InterWidgetCommunicationServlet (relay server 4)” is designated as the transmission destination of the event information. The event type is “sendKey” and the gadget type is “PizzaShop”.

  In the event reception script shown in FIG. 13, description D includes an instruction for executing the processing of description E when the gadget type specified in the reception information is “PizzaShop”. Therefore, when the event information generated by the event issuing script shown in FIG. 12 is given, the process of the description E of the event reception script is executed. The description E is called when an event is received via Comet, and includes a command for controlling the variable move according to the notified key code. In this example, the variable move is decremented by 1 when the key code is “38 (up arrow key)”, and the variable move is incremented by 1 when the key code is “40 (down arrow key)”. Is done.

  The description F includes an instruction for acquiring the position of a radio button selected from a plurality of radio buttons provided by the distributed gadget. The description G includes an instruction for changing the selection position of the radio button in accordance with the value of the variable move.

  Returning to FIG. In step S24, the Web server unit 41 of the relay server 4 transmits the event issuing HTML document to the transmission source of the HTTP request received in step S8. Here, the transmission source of the HTTP request is the portal application of the telephone set 1. The event issuing HTML document is a pizza image gadget 31 into which the event issuing script shown in FIG. 12 is injected. That is, the pizza image gadget 31 in which the event issuing script is injected is transmitted to the portal application of the telephone 1. As a result, the pizza image gadget 31 is added to the portal screen of the browser 11 in step S10.

  In step S25, the distribution unit 42 of the relay server 4 transmits the event receiving HTML document based on the session ID specified in step S11. In this example, the browser 21 of the television 2 is specified in step S11. Further, the event reception HTML document is a pizza image gadget 31 in which an event reception script shown in FIG. 13 is injected. That is, the pizza image gadget 31 in which the event reception script is injected is transmitted to the portal application of the television 21. As a result, the pizza image gadget 31 is added to the portal screen of the browser 21 in step S13.

  Thus, also in the second embodiment, the pizza image gadget 31 is displayed on the browser 11 of the telephone 1 and the browser 21 of the television 2. However, in the second embodiment, an event issue / reception script is injected into the pizza image gadget 31.

  Steps S31 to S35 are procedures executed after the event issuance / reception script is injected and the gadget is distributed. That is, in step S <b> 31, the user uses the function provided by the pizza image gadget 31 delivered to the telephone 1 by operating the input key of the telephone 1. In step S32, in the telephone 1, the event issuing script detects the event that has occurred and notifies the relay server 4 of the event content. The detected event is, for example, a key input operation by the user. In this case, the event issuing script notifies the relay server 4 of the key code for identifying the key pressed by the user, the event type, and the gadget type as event information. In this example, the event issuance script transmits event information to the relay server 4 using XMLHttpRequest.

  In step S33, the relay server 4 searches for a client terminal to which the same login ID as that input at the XMLHttpRequest transmission source terminal is input. At this time, the management table of the management unit 43 is referred to, and the television 2 is specified in the example shown in FIG. The search procedure in step S33 is basically the same as that in step S11.

  In step S34, the distribution unit 42 transmits event information to the client terminal specified in step S33. At this time, the distribution unit 42 transmits the event information to the television 2 by an asynchronous communication method such as Comet. This event information is received by the portal application of the television 2.

  In step S35, the portal application of the television 2 identifies the gadget indicated by the gadget type included in the event information, and notifies the gadget of the event information. When this event information is received, the event reception script injected into the specified gadget reproduces the event that occurred in the telephone set 1 according to the event information. That is, the user operation performed on the telephone 1 via the relay server 4 and the operation of the gadget executed on the television 2 are synchronized. Therefore, the user can substantially operate the television 2 by operating the telephone 1.

  FIG. 14 is a display example of a pizza image gadget 31 installed in the browser 11 of the telephone 1 and the browser 21 of the television 2. In the browser 11, the names of a plurality of pizzas (aaa pizza, bbb pizza, ccc pizza) sold by the pizzeria and radio buttons for selecting each pizza are displayed. The browser 21 displays a plurality of pizza images and radio buttons for selecting each pizza. In the browsers 11 and 21, radio buttons for selecting pizza are arranged in the vertical direction on the screen.

  The user orders pizza with reference to the pizza image gadget 31 installed in the browsers 11 and 21. In this case, for example, the user selects a desired pizza on the browser 11 by operating the upward arrow key and the downward arrow key of the telephone 1. If it does so, the event issuing script inject | poured into the pizza image gadget 31 in the telephone 1 will detect the key instruct | indicated by the user, and will notify it to the relay server 4 as event information. Here, it is assumed that the user has operated the down arrow key. In this case, the browser 11 of the telephone 1 changes from the state where the aaa pizza is selected to the state where the bbb pizza is selected.

  As shown in FIG. 7, the management unit 43 of the relay server 4 manages the correspondence between the login ID and the session ID. Therefore, the relay server 4 transmits the event information received from the telephone 1 to the television 2. Then, in the television 2, the event reception script injected into the pizza image gadget 31 performs the same processing as when the downward arrow key provided in the input device for operating the television 2 is instructed. That is, the browser 21 of the television 2 also changes from the state where the aaa pizza is selected to the state where the bbb pizza is selected, similarly to the browser 11 of the telephone 1.

  When the user again operates the down arrow key of the telephone 1 following the above state, ccc pizza is selected in the browsers 11 and 21, respectively. Further, when the user operates the down arrow key of the telephone 1, the pizza image gadget screen is scrolled on the browsers 11 and 21, respectively. For example, ddd pizza is displayed instead of aaa pizza.

  As described above, in the second embodiment, an event occurring in the first client terminal (here, the telephone 1) is notified to the second client terminal (here, the television 2), and the first and first Synchronization operation is realized between the two client terminals. Therefore, if the first client terminal is operated without using the keyboard or remote control input device of the second client terminal, it is possible to substantially operate both the first and second client terminals. Become.

  The relay server 4 may inject both the event issue script and the event reception script into the HTML document when transmitting the HTML document to each client terminal. However, the relay server 4 may transmit only one of the event issue script or the event reception script to the client terminal. For example, in the above-described example, only the event issuing script may be injected into the HTML document transmitted to the request transmission source (in this case, the telephone 1) in step S8. Further, only the event reception script may be injected into the HTML document transmitted to the client terminal (in this case, the television 2) searched in step S11.

<Other embodiments>
In the above example, the pizza image gadget 31 is stored in a memory area provided in the order server 3, but the present invention is not limited to this. That is, the pizza image gadget 31 may be provided from a server device or a storage area different from the order server 3.

  In the above-described example, the relay server 4 transmits the pizza image gadget 31 to the transmission source of the HTTP request and one other client terminal (TV 2), but the present invention is not limited to this. That is, the relay server 4 can transmit the pizza image gadget 31 to all client terminals that have logged in with the same login ID.

  In the above example, the gadget is transmitted to the client terminal, but the present invention is not limited to this. That is, for example, a Web application may be transmitted to the client terminal.

  In the above example, the gadget is transmitted to the client terminal that has logged in using the same login ID, but the present invention is not limited to this. That is, for example, the relay server 4 transmits the gadget / Web application to the client terminal to which the same user ID (including information for directly or indirectly identifying the user of the client terminal) is input. Also good. The login ID is generally information determined by a user at the time of initial registration, and can be used as information for indirectly identifying the user.

<Hardware configuration of relay server 4>
FIG. 15 is a diagram illustrating a hardware configuration of the relay server 4. In FIG. 4, the CPU 51 causes the relay server to execute each processing procedure by executing a multimodal providing program using the memory 53. The storage device 52 is, for example, a hard disk, and stores a multimodal providing program. The storage device 52 may be an external recording device. The memory 53 is a semiconductor memory, for example, and includes a RAM area and a ROM area.

  The reading device 54 accesses the portable recording medium 55 according to an instruction from the CPU 51. The portable recording medium 55 includes, for example, a semiconductor device, a medium in which information is input / output by a magnetic action, and a medium in which information is input / output by an optical action. The communication interface 56 transmits / receives data via a network in accordance with instructions from the CPU 51. The input / output device 57 corresponds to, for example, a device that receives an instruction from a user.

The multimodal providing program according to the first and second embodiments is provided in the following form, for example.
(1) Installed in advance in the storage device 52.
(2) Provided by the portable recording medium 55.
(3) Download from the program server 60.

  And the relay server 4 concerning 1st and 2nd embodiment is implement | achieved by running a multimodal provision program with the computer of the said structure. That is, a part or all of the Web server unit 41, the distribution unit 42, the management unit 43, and the script injection unit 44 is realized by executing the multimodal providing program on the computer having the above configuration.

The following supplementary notes are further disclosed with respect to the embodiments including the above examples.
(Appendix 1)
A relay server device provided between a server and a plurality of clients,
A management unit that holds a login ID input from each client in association with a session ID for identifying each client;
A Web server unit that relays a request from the first client to the server and returns a response corresponding to the request received from the server to the first client;
The login ID corresponding to the session ID for identifying the second client held in the management unit was the same as the login ID corresponding to the session ID for identifying the first client held in the management unit. Sometimes the delivery unit sends the response to the second client,
A relay server device.
(Appendix 2)
The relay server device according to attachment 1, wherein
The response transmitted to the first client is injected with a first script that causes the relay server to transmit event information representing an event that has occurred in the first client, and is transmitted to the second client. A script injection unit that injects a second script that causes the second client to execute processing corresponding to the event information when the second client receives event information from the relay server in response. ,
The relay server device, wherein the distribution unit further transmits the event information to the second client when the event information is received from the first client based on the first script.
(Appendix 3)
The relay server device according to attachment 2, wherein
When the distribution unit receives the event information from the first client, the distribution unit identifies the client identified by the session ID corresponding to the same login ID as the login ID corresponding to the session ID for identifying the first client. In contrast, a relay server device that transmits the event information.
(Appendix 4)
The relay server device according to attachment 1, wherein
The relay server device further comprising an editing unit that edits a response transmitted to at least one of the first or second client in accordance with the capability of the first or second client.
(Appendix 5)
A communication method using a relay server device provided between a server and a plurality of clients,
The relay server device is
Relaying a request from the first client to the server and returning a response corresponding to the request received from the server to the first client;
Based on a table in which a login ID input from each client is associated with a session ID for identifying each client, a login ID corresponding to a session ID for identifying the second client When it is determined that the login ID corresponding to the session ID to be identified is the same, the response is transmitted to the second client.
A communication method characterized by the above.
(Appendix 6)
The communication method according to attachment 5, wherein
The relay server device injects a first script that causes the relay server to transmit event information representing an event that has occurred in the first client, in the response transmitted to the first client, and the second script When the second client receives event information from the relay server, a second script that causes the second client to execute processing corresponding to the event information is injected into a response transmitted to the client. ,
When the relay server device receives the event information from the first client based on the first script, the event information is transmitted to the second client.
(Appendix 7)
In the relay server device provided between the server and multiple clients,
Relaying a request from a first client to the server and returning a response corresponding to the request received from the server to the first client;
Based on a table in which a login ID input from each client is associated with a session ID for identifying each client, a login ID corresponding to a session ID for identifying the second client Sending the response to the second client when it is determined that it is the same as the login ID corresponding to the identifying session ID;
Communication program that executes
(Appendix 8)
A communication program according to appendix 7,
In addition to the relay server device,
The response transmitted to the first client is injected with a first script that causes the relay server to transmit event information representing an event that has occurred in the first client, and is transmitted to the second client. Injecting a second script into the response to cause the second client to execute processing corresponding to the event information when the second client receives event information from the relay server;
Transmitting the event information to the second client when the event information is received from the first client based on the first script;
Communication program that executes

DESCRIPTION OF SYMBOLS 1 Telephone 2 Television 3 Order server 4 Relay server 5 Call center terminal 11, 21 Browser 31 Pizza image gadget 41 Web server 42 Distribution part 43 Management part 44 Script injection part

Claims (4)

A relay server device provided between a server and a plurality of clients,
A management unit that holds a login ID input from each client in association with a session ID for identifying each client;
A Web server unit that relays a request from the first client to the server and returns a response corresponding to the request received from the server to the first client;
The login ID corresponding to the session ID for identifying the second client held in the management unit was the same as the login ID corresponding to the session ID for identifying the first client held in the management unit. Sometimes the delivery unit sends the response to the second client,
A relay server device. The relay server device according to claim 1,
The response transmitted to the first client is injected with a first script that causes the relay server to transmit event information representing an event that has occurred in the first client, and is transmitted to the second client. A script injection unit that injects a second script that causes the second client to execute processing corresponding to the event information when the second client receives event information from the relay server in response. ,
The relay server device, wherein the distribution unit further transmits the event information to the second client when the event information is received from the first client based on the first script. A communication method using a relay server device provided between a server and a plurality of clients,
The relay server device is
Relaying a request from the first client to the server and returning a response corresponding to the request received from the server to the first client;
Based on a table in which a login ID input from each client is associated with a session ID for identifying each client, a login ID corresponding to a session ID for identifying the second client When it is determined that the login ID corresponding to the session ID to be identified is the same, the response is transmitted to the second client.
A communication method characterized by the above. In the relay server device provided between the server and multiple clients,
Relaying a request from a first client to the server and returning a response corresponding to the request received from the server to the first client;
Based on a table in which a login ID input from each client is associated with a session ID for identifying each client, a login ID corresponding to a session ID for identifying the second client Sending the response to the second client when it is determined that it is the same as the login ID corresponding to the identifying session ID;
Communication program that executes