JP5838687B2 - Relay communication system and relay server - Google Patents

Description

  The present invention relates to a relay communication system having a plurality of relay servers that can communicate with each other, a plurality of client terminals, and a LAN that connects the client terminals to the relay server, and further relates to a relay server used in the relay communication system.

  Conventionally, a remote maintenance system for monitoring and maintaining a terminal such as an electronic device installed in a home or office from a remote location has been put into practical use. By using the remote maintenance system, a maintenance worker's service person performs maintenance work on the terminal without going to the place where the terminal is installed.

  Conventionally, there has been proposed an image processing apparatus capable of remotely operating a computer terminal while confirming the contents of the screen of the computer terminal connected via a predetermined network (see, for example, Patent Document 1).

JP 2006-59083 A

  The remote control system disclosed in Patent Document 1 is operated only between the image processing apparatus and one computer terminal. That is, in the remote operation disclosed in Patent Document 1, the content of the remote operation is shared only between the remote operation terminal and the operated terminal having a one-to-one relationship. Therefore, conventionally, when the operation side terminal remotely operates the operated side terminal, even if the other terminal has a request to acquire the contents of the remote operation, such request It was difficult to meet.

  An object of the present invention is that a relay communication system having a plurality of relay servers and a plurality of client terminals that can communicate with each other allows a terminal other than the terminal to be remotely operated and a terminal other than the terminal to be remotely operated to acquire the contents of the remote operation. It is in.

  Hereinafter, a plurality of modes will be described as means for solving the problems. These aspects can be arbitrarily combined as necessary.

A relay communication system according to an aspect of the present invention includes a first network, a second network, a first relay server, a second relay server, a first client terminal, a second client terminal, and a third client terminal. And comprising. The first relay server is connected to the first network. The second relay server is connected to the second network and can communicate with the first relay server via the third network. The first client terminal and the second client terminal are connected to the first relay server via the first network. The third client terminal is connected to the second relay server via the second network.
The first relay server and the second relay server have an information sharing unit that shares the relay server information including the activation information of the relay server itself and the activation information and connection information of the client terminal connected to the relay server itself.
The second relay server has a second operation control unit. The second operation control unit is a request to remotely operate the first client terminal selected with reference to the relay server information from the third client terminal that is an operation client terminal that performs remote operation, If the first remote operation request including the information of the first client terminal is received, the first relay server to which the first client terminal is connected is specified, the first remote operation request is relayed to the first relay server, The first client terminal by the third client terminal established based on the first remote operation request by establishing the first operation session with the three client terminals, establishing the second operation session with the first relay server The operation event and the image data as the operation data of the first remote operation are relayed between the first operation session and the second operation session.
The first relay server includes a first operation control unit and a monitoring control unit. If the first operation control unit receives the first remote operation request from the second relay server, the first operation control unit relays the first remote operation request to the first client terminal and establishes a third operation session with the first client terminal. Then, the operation data of the first remote operation performed based on the first remote operation request is relayed between the second operation session and the third operation session, and the first data including information on the first client terminal and the third client terminal is included. Create remote control information. If the monitoring control unit receives a monitoring request for monitoring the first remote operation selected based on the first remote operation information from the second client terminal when the first remote operation is being executed, the monitoring control unit receives the second client terminal. Monitoring that establishes a monitoring session for monitoring the first remote operation with the server and transmits image data relayed to the second relay server in the second operation session also to the second client terminal via the monitoring session And a control unit.
Here, the second client terminal other than the third client terminal and the first client terminal can monitor the content of the first remote operation performed on the first client terminal by the third client terminal via the monitoring session. .

The operation event relates to the first remote operation received from the user via the operation input unit of the third client terminal , and the first remote operation information includes a screen between the first client terminal and the third client terminal. Image information for setting information in common may be created and further included.

The relay communication system may further include a fourth client terminal connected to the first relay server via the first network.
When the first relay server receives a second remote operation request from the second relay server to the fourth client terminal by the third client terminal, the first operation control unit relays the second remote operation request to the fourth client terminal. The fourth operation session is established with the fourth client terminal, the fifth operation session is established with the second relay server, the fourth operation session, the fifth operation session, and the third client terminal. And the operation data of the second remote operation to the fourth client terminal by the third client terminal executed via the sixth operation session established between the second relay server and the second relay server may be relayed. If the monitoring control unit further receives a monitoring request for monitoring the second remote operation from the second client terminal, the monitoring control unit may transmit the operation content of the second remote operation to the second client terminal via the monitoring session.

The relay server according to another aspect of the present invention can communicate with the second relay server connected to the second network, is connected to the first network, and is connected to the first client terminal via the first network. It functions as a first relay server. The relay server includes an information sharing unit, an operation control unit, and a monitoring control unit .
The information sharing unit shares the relay server information including the activation information of the relay server, the activation information of the client terminal connected to the relay server, and the connection information with the second relay server. The operation control unit is a request to remotely operate the first client terminal selected with reference to the relay server information from the third client terminal connected to the second relay server that is an operation client terminal that performs remote operation. If a first remote operation request including information on the third client terminal and the first client terminal is received, the following operation is performed. That is, the operation control unit relays the first remote operation request to the first client terminal, establishes the second operation session with the second relay server, and establishes the third operation session with the first client terminal. The first operation session is performed based on the first remote operation request via the second operation session, the third operation session, and the first operation session established between the third client terminal and the second relay server. The operation event and the image data as the operation data of the first remote operation to the first client terminal by the three client terminals are relayed between the second operation session and the third operation session, and the first client terminal and the third client terminal First remote operation information including information is created. If the monitoring control unit receives a monitoring request for monitoring the first remote operation selected based on the first remote operation information from the second client terminal when the first remote operation is being executed, the monitoring control unit receives the second client terminal. A monitoring session for monitoring the first remote operation is established between the image data and the image data relayed to the second relay server in the second operation session, and is also transmitted to the second client terminal via the monitoring session.

  A relay communication system according to still another aspect of the present invention includes a first network, a second network, a third network, a first relay server, a second relay server, a third relay server, and a first client. A terminal, a third client terminal, and a fifth client terminal. The first relay server is connected to the first network. The second relay server is connected to the second network. The third relay server is connected to the third network. The first client terminal is connected to the first relay server via the first network. The third client terminal is connected to the second relay server via the second network. The fifth client terminal is connected to the third relay server via the third network. Information sharing in which the first relay server, the second relay server, and the third relay server share the relay server information including the start information of the relay server itself and the start information and connection information of the client terminal connected to the relay server itself Part. The second relay server has a second operation control unit. The second operation control unit is a request to remotely operate the first client terminal selected with reference to the relay server information from a third client terminal which is an operation client terminal that performs remote operation. If the first remote operation request including the information of the first client terminal is received, the first relay server to which the first client terminal is connected is specified, the first remote operation request is relayed to the first relay server, The first client by the third client terminal is established based on the first remote operation request by establishing the first operation session with the third client terminal, establishing the second operation session with the first relay server Relay operation events and image data as operation data of the first remote operation to the terminal between the first operation session and the second operation session. . The first relay server includes a first operation control unit and a monitoring control unit. If the first operation control unit receives the first remote operation request from the second relay server, the first operation control unit relays the first remote operation request to the first client terminal and establishes a third operation session with the first client terminal. Relay the operation data of the first remote operation performed based on the first remote operation request between the second operation session and the third operation session, and include information of the first client terminal and the third client terminal First remote operation information is created. When the first remote operation is being executed, the monitoring control unit receives a monitoring request for monitoring the first remote operation selected based on the first remote operation information from the fifth client terminal through the third relay server. Establishing a monitoring session for monitoring the first remote operation with the fifth client terminal, and transferring the image data relayed to the second relay server in the second operation session via the monitoring session to the fifth client terminal Also send to.

The operation event relates to the first remote operation received from the user via the operation input unit of the third client terminal , and the first remote operation information includes a screen between the first client terminal and the third client terminal. Image information for setting information in common may be created and further included.

  The relay communication system may further include a fourth client terminal connected to the first relay server via the first network. When the first relay server receives a second remote operation request from the second relay server to the fourth client terminal by the third client terminal, the first operation control unit relays the second remote operation request to the fourth client terminal. The fourth operation session is established with the fourth client terminal, the fifth operation session is established with the second relay server, the fourth operation session, the fifth operation session, and the third client terminal. And the operation data of the second remote operation to the fourth client terminal by the third client terminal executed via the sixth operation session established between the second relay server and the second relay server. If a monitoring request for monitoring the second remote operation is further received from the client terminal through the third relay server, the operation of the second remote operation is performed via the monitoring session. To send to the fifth client terminal.

The first relay server according to still another aspect of the present invention can be connected to the second relay server and the third relay server via the WAN, and can be connected to the first client terminal and the second client terminal via the LAN. The first relay server includes a first operation session establishment unit, a second operation session establishment unit, an operation relay unit, a monitoring session establishment unit, and a monitoring control unit. The first operation session establishing unit establishes a first operation session with the first client terminal. The second operation session establishing unit establishes a second operation session with the second relay server. The operation relay unit relays operation data including operation events and image data between the first operation session and the second operation session. The monitoring session establishment unit establishes a monitoring session with the second client terminal or the third relay server. The monitoring control unit also relays the image data relayed between the first operation session and the second operation session to the monitoring session.

  According to the present invention, the contents of the remote operation performed in the first relay group can be monitored by the monitoring client terminal other than the operation client terminal and the operated client terminal via the monitoring session.

The schematic diagram which shows the whole structure of a relay communication system. The block diagram which shows the structure of a 1st relay server. The block diagram which shows the structure of a 1st client terminal. The figure which shows schematic structure of relay group information. The figure which shows the detailed structure of relay group information. The figure which shows schematic structure of relay server information. The figure which shows the detailed structure of relay server information. The figure which shows schematic structure of remote operation information. The flowchart (the 1) which shows the operation process of an operation client terminal. The flowchart (the 2) which shows the operation process of an operation client terminal. The flowchart which shows the process of a to-be-operated client terminal. The figure which shows an example of the established operation session and the monitoring session. The flowchart which shows the monitoring process of a monitoring client terminal. The flowchart which shows the monitoring process of the monitoring control relay server.

1. Outline of this Embodiment In the relay communication system according to this embodiment, client terminals can communicate with each other across a WAN via a plurality of relay servers.

2. Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows the overall configuration of a relay communication system. The relay communication system includes a first LAN 1, a second LAN 2, a third LAN 3, and a WAN 4. The first LAN 1, the second LAN 2 and the third LAN 3 are small-scale networks constructed remotely. The WAN 4 is a large-scale network such as the Internet.
In this embodiment, as will be described in detail later, the first relay server A of the first LAN 1, the second relay server B of the second LAN 2, and the third relay server C of the third LAN 3 are the first relay as a relay group. A group is formed.

  The first LAN 1 has a fourth LAN 11 and a fifth LAN 13, and the fourth LAN 11 and the fifth LAN 13 are connected to each other by a first general-purpose router 15. In the fourth LAN 11, the first relay server A and the first client terminal A1 are connected to each other. In the fifth LAN 13, the second client terminal A2 and the first communication device 17 are connected to each other.

In the second LAN 2, the second relay server B and the third client terminal B1 are connected to each other.
The third LAN 3 has a sixth LAN 19, a seventh LAN 21, and an eighth LAN 23. The sixth LAN 19 and the seventh LAN 21 are connected by a second general-purpose router 25, and the sixth LAN 19 and the eighth LAN 23 are connected by a third general-purpose router 27. The third relay server C belongs to the sixth LAN 19. A fourth client terminal C1 is connected to the seventh LAN 21. A fifth client terminal C2 is connected to the eighth LAN 23.

  The first client terminal A1, the second client terminal A2, the third client terminal B1, the fourth client terminal C1, and the fifth client terminal C2 are, for example, personal computers. The first communication device 17 is also a personal computer, for example.

  The first relay server A, the second relay server B, and the third relay server C are connected to each other of the first client terminal A1, the second client terminal A2, the third client terminal B1, the fourth client terminal C1, and the fifth client terminal C2. Relay communication between them. The WAN 4 relays communication among the first relay server A, the second relay server B, and the third relay server C. The communication protocol among the first relay server A, the second relay server B, and the third relay server C is not particularly limited.

3. Components of the relay server The relay server is connected not only to the LAN but also to the WAN, can communicate with each client terminal connected to the same LAN, and communicates with a relay server arranged in another LAN. It is possible. Therefore, each relay server is given a global IP address in addition to the private IP address.
FIG. 2 shows components of the first relay server A. The first relay server A includes a network interface 121, a LAN control unit 122, a WAN control unit 123, a control unit 124, and a database storage unit 125. Further, the first relay server A may include a display unit 126 and an operation input unit 127.
The network interface 121 executes communication with the terminals in the LAN 1 using the private IP address, and further executes communication with the WAN 4 using the global IP address.
The LAN control unit 122 is a processing unit that controls various communications performed with terminals in the LAN 1 via the network interface 121, and controls communication processing according to a predetermined protocol.
The WAN control unit 123 is a processing unit that controls various communications performed with a terminal connected to the WAN 4 via the network interface 121, and controls communication processing according to a predetermined protocol.

The control unit 124 is, for example, a CPU having control and calculation functions, and can execute various processes using a loaded program.
The control unit 124 of this embodiment includes an information sharing unit 131, an operation control unit 132, and a monitoring control unit 133.

The information sharing unit 131 creates and updates relay group information and relay server information described later.
In addition, the information sharing unit 131 performs processing for sharing relay group information, relay server information, and remote operation information, which will be described later, with relay servers and client terminals in the relay group. Specifically, the information sharing unit 131 shares the created and updated relay group information with the relay servers in the relay group and the client terminals connected to the relay server, and stores them in the relay group information storage unit 141 (described later). To do. The information sharing unit 131 shares the created and updated relay server information with the relay servers in the relay group and the client terminals connected to the relay servers, and stores them in the relay server information storage unit 142 (described later). Further, the information sharing unit 131 shares the remote operation information with the relay server and the client terminal in the relay group, and stores them in the remote operation information storage unit 143 (described later).

  The operation control unit 132 establishes an operation session for performing a remote operation performed in the relay group in response to a remote operation request, and controls the remote operation. The remote operation request is a request for remote operation output from a client terminal (hereinafter referred to as operation client terminal) that performs remote operation, and includes information on the operation client terminal and a remotely operated client terminal (hereinafter referred to as client to be operated). Terminal) information. The information on the operation client terminal and the operated client terminal is information for identifying each client terminal, and in this embodiment, is the account name of each client terminal.

Specifically, when the operation control unit 132 receives a remote operation request, the operation control unit 132 refers to the relay server information, relays the remote operation request to the operated client terminal, and establishes an operation session for executing the remote operation. Perform the process. For example, when the operation control unit 132 receives a remote operation request from an operation client terminal connected to the relay server itself, the operation control unit 132 refers to the relay server information and specifies the relay server to which the operated client terminal is connected. And relay the remote operation request. In addition, the operation control unit 132 establishes an operation session with the relay server to which the operated client terminal is connected. On the other hand, when the operated client terminal is a client terminal connected to the relay server itself, the operation control unit 132 relays the remote operation request to the operated client terminal, and determines the operated client terminal and the operated client terminal based on the relay server information. Establish an operation session during
In addition, the operation control unit 132 relays operation data related to remote operation performed between the operation client terminal and the operated client terminal via the operation session. The operation data includes an operation event and image data described later. Further, the operation control unit 132 creates or updates the remote operation information, details of which will be described later.

  The monitoring control unit 133 controls the monitoring of the remote operation by establishing a monitoring session for monitoring the remote operation performed in the relay group in response to the monitoring request. The monitoring request is output from a client terminal (hereinafter referred to as a monitoring client terminal) that monitors a remote operation executed between the operation client terminal and the operated client terminal, unlike the operation client terminal and the operated client terminal. Is a remote control monitoring request. The monitoring request includes information on the monitoring client terminal and information for identifying the remote operation to be monitored. The monitoring client terminal information is information for identifying the monitoring client terminal, and in this embodiment is the account name of the monitoring client terminal. The remote operation identification information to be monitored is information for specifying the remote operation being executed in the relay group, and is an arbitrary ID in the present embodiment.

For example, if a monitoring request is received when a monitoring client terminal is connected to the relay server itself, the monitoring control unit 133 performs control to establish a monitoring session with the monitoring client terminal. Further, the monitoring control unit 133 refers to the relay server information, relays a monitoring request to the monitoring control relay server, and performs control to establish a monitoring session with the monitoring control relay server. On the other hand, if the monitoring control unit 133 accepts a monitoring request when the relay server itself is a relay server to which the operated client terminal is connected, the monitoring client terminal is connected via the relay server to which the monitoring client terminal is connected. Control to establish a monitoring session with the connected relay server. Hereinafter, the relay server to which the operated client terminal is connected and which receives the monitoring request and controls the monitoring of the remote operation is referred to as a monitoring control relay server. After the monitoring session is established, the monitoring control unit 133 transmits the operation content of the remote operation performed between the operation client terminal and the operated client terminal to the monitoring session. Specifically, the monitoring control unit 133 copies image data indicating the operation contents of the remote operation being executed between the operation client terminal and the operated client terminal, and transmits the copied image data to the monitoring session.
In addition, the monitoring control unit 133 updates the remote operation information by adding information of the monitoring client terminal, and stores the updated remote operation information in the remote operation information storage unit 143.

  The database storage unit 125 is, for example, a hard disk or a nonvolatile RAM, and can store various data. The database storage unit 125 includes a relay group information storage unit 141, a relay server information storage unit 142, and a remote operation information storage unit 143. Details of the information stored in the relay group information storage unit 141, the relay server information storage unit 142, and the remote operation information storage unit 143 will be described later.

The display unit 126 includes a display device such as a liquid crystal display panel. The display unit 126 presents various information related to relay group information, relay server information, remote operation information, and the like to the user.
The operation input unit 127 includes a hard key and a mouse for inputting various instructions to the first relay server A.

  The constituent elements of the second relay server B and the third relay server C are the same as those of the first relay server A, and thus the description thereof is omitted.

4). Components of Client Terminal A client terminal is a terminal that can be directly operated by a user. The client terminal is, for example, a personal computer used for daily work by a user. Each client terminal is assigned a private IP address that is uniquely managed within the same LAN.

  FIG. 3 shows components of the first client terminal A1. The first client terminal A1 includes a LAN interface 221, a control unit 222, a database storage unit 223, a display unit 224, and an operation input unit 225.

  The LAN interface 221 can execute communication with the first relay server A and other terminals in the first LAN 1 using a private IP address.

  The control unit 222 is, for example, a CPU having control and calculation functions, and can execute various processes using a loaded program. The control unit 222 of this embodiment includes an information sharing unit 231, an operation control unit 232, a monitoring control unit 233, an image setting unit 234, and a client communication control unit 235.

  The information sharing unit 231 performs processing for sharing the relay group information, the relay server information, and the remote operation information with the relay server and the client terminal in the relay group. Specifically, the information sharing unit 231 shares the created and updated relay group information with the relay servers in the relay group and the client terminals connected to the relay server, and stores them in the relay group information storage unit 241 (described later). To do. The information sharing unit 231 shares the created and updated relay server information with the relay servers in the relay group and the client terminals connected to the relay servers, and stores them in the relay server information storage unit 242 (described later). Further, the information sharing unit 231 shares the created and updated remote operation information with the relay server and the client terminal in the relay group, and stores them in the remote operation information storage unit 243 (described later).

The operation control unit 232 establishes an operation session for performing a remote operation performed in the relay group in response to the remote operation request, and controls the remote operation.
The operation control unit 232 of the operation client terminal first receives a remote operation request from the user. Thereafter, the operation control unit 232 transmits a remote operation request to the operated client terminal via the relay server, and further controls to establish an operation session with the operated client terminal via the relay server. .
After the operation session is established, the operation control unit 232 performs control to receive an operation event from the user and output the received operation event to the operation session. The operation event is a user operation via the operation input unit 225, and is, for example, cursor movement, mouse input, or keyboard input via a mouse and a keyboard.
Further, when the operation control unit 232 receives the image data in which the operation event is processed from the operated client terminal, the operation control unit 232 outputs the received image data to the display unit 224 for display.

  When the operation control unit 232 of the operated client terminal receives a remote operation request, the operation control unit 232 controls to establish an operation session with the operation client terminal via the relay server. If an operation event is received from the operation client terminal, the operation control unit 232 causes the received operation event to be executed in the system of the operated client terminal, and operates image data reflecting the operation event via the operation session. Output to the client terminal.

The monitoring control unit 233 controls the monitoring of the remote operation by establishing a monitoring session for monitoring the remote operation performed in the relay group in response to the monitoring request.
Specifically, the monitoring control unit 233 receives from the user a monitoring request related to the remote operation performed in the relay group selected with reference to the remote operation information by the user. Thereafter, based on the relay server information, the monitoring control unit 233 identifies the monitoring control relay server that is the relay server to which the remotely operated client terminal that is requested to be monitored is connected. Subsequently, the monitoring control unit 233 transmits a monitoring request to the monitoring control relay server, and further controls to establish a monitoring session with the monitoring control relay server.
In addition, when the monitoring control unit 233 receives image data from the monitoring control relay server via the monitoring session, the monitoring control unit 233 outputs the received image data to the display unit 224 for display.

  The image setting unit 234 of the operation client terminal and the operated client terminal determines image information of the operation client terminal and the operated client terminal when performing a remote operation, and uses the determined image information as the operation client terminal and the operated client terminal. Perform processing to set. The image information includes screen size, color information, resolution, brightness, and the like. Specifically, the image setting unit 234 exchanges screen information between the operation client terminal and the operated client terminal by handshake processing so that the screen information between the operation client terminal and the operated client terminal becomes common. Determine image information. Then, the image setting unit 234 sets image information of the operation client terminal and the operated client terminal based on the determined image information. As described above, the screen information between the operation client terminal and the operated client terminal is set in common, thereby preventing unclear image data from being displayed on the display unit 224 of the operation client terminal. In the present embodiment, the image setting unit 234 of the operation control unit of each client terminal is set to match the image information of the other client terminals with the image information indicating low capability among the image information of the operation client terminal and the operated client terminal. Is done.

  The client communication control unit 235 processes communication packets and controls various communications according to protocols such as TCP / IP, UDP, and SIP performed via the LAN interface 221.

  The database storage unit 223 is, for example, a hard disk or a nonvolatile RAM, and can store various data. The database storage unit 223 includes a relay group information storage unit 241, a relay server information storage unit 242, and a remote operation information storage unit 243. Details of the information stored in the relay group information storage unit 241, the relay server information storage unit 242, and the remote operation information storage unit 243 will be described later.

The display unit 224 includes a display device such as a liquid crystal display panel. The display unit 224 presents various information related to relay group information, relay server information, and remote operation information to the user. In addition, when the terminal is an operation client terminal, the display unit 224 presents an operation window related to remote operation to the user. On the other hand, the display unit 224 of the operated client terminal presents an operated window related to remote operation to the user. Further, the display unit 224 of the monitoring client terminal presents a monitoring window related to remote operation to the user.
The operation input unit 225 includes a hard key and a mouse for inputting various instructions to the first client terminal A1.
Since the second client terminal A2, the third client terminal B1, the fourth client terminal C1, and the fifth client terminal C2 are the same as the first client terminal A1, the description thereof is omitted.

5. Relay Group Information The relay group information 20 will be described with reference to FIGS. 4 and 5. FIG. 4 is a diagram illustrating a schematic configuration of relay group information. FIG. 5 is a diagram illustrating a detailed configuration of relay group information. The relay group information 20 is information indicating an outline of each relay group in the relay communication system. FIG. 4 shows that the first relay group includes a first relay server A, a second relay server B, and a third relay server C.

As shown in FIG. 5, the relay group information 20 includes upper information 201 and lower information 202.
The upper information 201 is information about the first relay group itself. “Group id” indicates identification information of the relay group. “Lastmod” indicates the latest update time of the relay group information. “Name” indicates the name of the relay group.

  The lower information 202 is information about the first relay server A, the second relay server B, and the third relay server C. “Site id” indicates identification information of the relay server.

  The relay group information 20 is shared among the first relay server A, the second relay server B, and the third relay server C, and stored in the relay group information storage unit 141 of each relay server. Further, the relay group information 20 is shared between the relay server and the client terminal, and stored in the relay group information storage unit 241 of each client terminal.

6). Relay Server Information The relay server information 30 will be described with reference to FIGS. FIG. 6 shows a schematic configuration of the relay server information. FIG. 7 shows a detailed configuration of the relay server information 30. The relay server information 30 is information indicating an outline of the relay server and the client terminal that constitute the relay communication system.

  As shown in FIG. 6, in the relay server information 30, the first client terminal A1 and the second client terminal A2 are connected to the first relay server A, and the third client terminal B1 is connected to the second relay server B. The information that the fourth client terminal C1 and the fifth client terminal C2 are connected to the third relay server C is shown.

  As illustrated in FIG. 7, the relay server information 30 includes upper information 301-1, 301-2, and 301-3 and lower information 302-1, 302-2, and 302-3.

  The upper information 301-1, 301-2, 301-3 is information about the relay server. “Site id” indicates identification information of the relay server. “Name” indicates the name of the relay server. “Stat” indicates information about whether or not the relay server is activated.

  The lower information 302-1, 302-2, 302-3 is information about the client terminal. “Div” indicates a department name of the client terminal. “Group” indicates identification information of the relay group to which the client terminal belongs. “Id” indicates identification information of the client terminal. “Name” indicates the name of the client terminal. “Site” indicates identification information of a relay server that is a logon destination when the client terminal is logged on.

  The relay server information 30 is shared among the first relay server A, the second relay server B, and the third relay server C, and is stored in the relay server information storage unit 142 of each relay server. The relay server information 30 is shared between the relay server and the client terminal, and stored in the relay server information storage unit 242 of each client terminal.

  When the relay server is activated, “stat” of the upper information 301-1, 301-2, 301-3 is “active”. When the relay server is not activated, “stat” is blank. Thereby, information about whether or not the relay server is activated is shared as the entire relay communication system.

When the client terminal is logged on to the relay server, the identification information of the relay server that is the logon destination of the client terminal is described in “site” of the lower information 302-1, 302-2, 302-3. When the client terminal is not logged on to the relay server, “site” is blank. Thereby, information about whether or not the client terminal is logged on to the relay server is shared as the entire relay communication system.
7). Remote operation information The remote operation information is information including identification information of the operation client terminal and the operated client terminal, and identification information for identifying the remote operation. If there is a monitoring client terminal that monitors the remote operation, identification information of the monitoring client terminal is further added to the remote operation information. In this embodiment, the identification information of each client terminal is an account name. However, the identification information of each client terminal is not limited to this example as long as the information can identify each client terminal. An arbitrary ID is used as remote operation identification information. However, identification information other than an arbitrary ID may be used as long as remote operation can be identified. The remote operation information may further include image information of the operation client terminal and the operated client terminal. The image information has the screen size, color information, resolution, brightness, and the like as described above, and may be referred to in the image information determination process between the operation client terminal and the operated client terminal.

  The remote operation information will be described with reference to FIG. FIG. 8 shows a schematic configuration of the remote operation information 40. In the remote operation information 40, the account name of the third client terminal B1 is stored as the operation client terminal, and the account name of the fifth client terminal C2 is stored as the operated client terminal. The remote operation information 40 stores the account name of the second client terminal A2 as the monitoring client terminal. Further, “31” is stored as identification information for identifying the remote operation between the third client terminal B1 and the fifth client terminal C2. The third client terminal B1 and the fifth client terminal C2 are associated with image information for each client terminal.

The remote operation information 40 is created by the relay server to which the operated client terminal is connected at the timing when the operation session is established and the handshake process is performed between the operating client terminal and the operated client terminal based on the remote operation request information. Updated.
The remote operation information 40 is updated by adding the account name of the monitoring client terminal by the monitoring control relay server based on the monitoring request at the timing when the monitoring session is established. The remote operation information 40 is blank in the monitoring terminal column of the remote operation information 40 in a state where the remote operation is not monitored.
Since the relay server to which the operated client terminal is connected is also a monitoring control relay server, creation and update of the remote operation information 40 are performed by the same relay server.

8). Information Sharing in Relay Communication System In the relay communication system, setting of the relay group as the first relay group and information sharing in the relay group are performed as follows.
First, as an initial setting of the first relay group, each relay server of the first relay server A, the second relay server B, and the third relay server C creates an account for a client terminal connected to the relay server itself. The relay server information is created and stored in the relay server information storage unit.
Thereafter, if any relay server requests group construction of the first relay group of the relay communication system, each relay server creates relay group information and stores it in the relay group information storage unit. Next, the relay server exchanges relay server information between the relay servers constituting the first relay group, and combines the exchanged relay server information with the relay server information stored in the relay server itself as new relay server information. save. As a result, common relay server information is held among the first relay server A, the second relay server B, and the third relay server C.
In such a relay communication system, when the increase / decrease state and connection state of the LAN and the client terminal change, when one relay server recognizes the state change, the relay group information and the relay server are determined according to the contents. Update information immediately.

  Then, the one relay server immediately notifies the other relay servers described in the relay group information and the relay server information that the relay group information and the relay server information are updated. The one relay server immediately notifies the updated relay server information to the client terminal connected to the relay server itself.

  However, even if another relay server is described in the relay group information and relay server information, if one relay server determines that another relay server is not connected, it immediately notifies the other relay server. Never do.

  As a result, the information about the increase / decrease state and connection state of the LAN and client terminals is shared in real time between the relay server configuring the first relay group and the client terminals connected to the relay server.

9. Data transmission / reception in the relay communication system After the information on the increase / decrease state and connection state of the LAN and the client terminal is shared in the relay communication system as described above, the user who uses the client terminal designates another client terminal for communication When doing so, data is transmitted and received as follows.

  Specifically, the client terminal transmits transmission data including the client terminal information of the designation destination, the client terminal information as the transmission source, and the data to be transmitted to the relay server to which the client terminal itself is connected. The relay server that has received the transmission data refers to the relay server information and confirms to which relay server in the relay group the specified client terminal is subordinate. Further, the relay server confirms whether the relay server under which the designated client terminal is under control is in an activated state and whether the designated client terminal is in a logon state. Specifically, whether or not the relay server is in an active state can be determined by confirming that the upper information “stat” of the relay server information is set to “active”. Further, it is determined whether or not the client terminal is in the logon state by checking whether or not the identification information of the relay server that is the logon destination of the client terminal is described in the lower information “site” of the relay server information.

  If the activation status of the relay server and the logon status of the client terminal can be confirmed, the received transmission data is relayed and transmitted to the relay server under which the designated client terminal is subordinate. The relay server that has received the transmission data relays transmission data to a designated client terminal under its control.

10. Remote operation processing Next, in the relay communication system as described above, in the state where information on the increase / decrease state and connection state of the LAN and the client terminal is shared, the operation client terminal is operated with respect to the operated client terminal belonging to the same relay group The process of performing remote control will be described.
In the following, the first relay server A, the second relay server B, the third relay server C, the second client terminal A2 connected to the first relay server A, and the second relay constituting the first relay group Between the third client terminal B1 connected to the server B and the fourth client terminal C1 and the fifth client terminal C2 connected to the third relay server C, the relay server information 30 shown in FIGS. A description will be given by taking as an example a process in which the third client terminal B1 remotely controls the fifth client terminal C2 in a shared state.

  Whether the user of the operation client terminal operated the operated client terminal with the mouse or keyboard of the operated client terminal by performing an input operation using the mouse and keyboard in the operation window displayed on the display unit 224 Can be operated as follows.

10-1. Processing of Client Terminal Performing Remote Operation First, processing of the third client terminal B1 performing remote operation will be described with reference to FIGS. 9A and 9B. 9A and 9B are flowcharts showing the operation process of the operation client terminal.

  The operation control unit 32 of the third client terminal B1 accepts a search for the operated client terminal (step S101). Specifically, the relay server information 30 stored in the relay server information storage unit 242 is displayed on the display unit 224 in response to a remote operation instruction from the user. The user selects an operated client terminal to be remotely operated from among the client terminals indicated in the relay server information 30 displayed on the display unit 224.

If the user selects the fifth client terminal C2 to be remotely operated via the operation input unit 225 (Yes in step S103), the operation control unit 232 of the third client terminal B1 sends to the second relay server B. A remote operation request is transmitted (step S105). The transmitted remote operation request is transmitted to the fifth client terminal C2 via the second relay server B and the third relay server C.
Thereafter, if an operation permission response is received from the fifth client terminal C2 via the second relay server B (Yes in step S107), the operation control unit 232 of the third client terminal B1 communicates with the second relay server B. The control for establishing the first operation session is performed (step S109).
After that, a second operation session is established between the second relay server B and the third relay server C, and a third operation session is established between the third relay server C and the fifth client terminal C2.

  On the other hand, when the operation permission response is not received from the fifth client terminal C2 via the second relay server B and the third relay server C (No in step S107), the operation control unit 232 of the third client terminal B1 performs remote operation. Terminate the process.

After the first to third operation sessions are established, the image setting unit 234 of the third client terminal B1 executes a handshake process with the image setting unit 234 of the fifth client terminal C2 (step S111) and transmits it. Image information setting processing is performed (step S113).
Subsequently, the operation control unit 232 of the third client terminal B1 receives the image data displayed on the display unit 224 of the fifth client terminal C2 from the fifth client terminal C2 (step S115). Subsequently, the operation control unit 232 of the third client terminal B1 outputs the received image data to the display unit 224 of the third client terminal B1, and displays it as an operation window (step S117).

Thereafter, if an operation event occurs on the operation window via the mouse or the keyboard (Yes in step S119), the operation control unit 232 of the third client terminal B1 transmits the generated operation event via the first operation session. It transmits to the second relay server B (step S121). The transmission event transmitted at this time is relayed by the second relay server B and the third relay server C and transmitted to the fifth client terminal C2.
If the operation control unit 232 of the third client terminal B1 receives a remote operation end instruction from the user (Yes in step S123), the remote control end instruction is transmitted to the second relay server B, and the first operation session is transmitted. Disconnect (step S125). Further, the operation control unit 232 of the third client terminal B1 closes the operation window displayed on the display unit 224. At this time, the remote operation end instruction is transmitted to the fifth client terminal C2 via the second relay server B and the third relay server, and the second operation session and the third operation session are similarly disconnected.

  On the other hand, if the operation control unit 232 of the third client terminal B1 does not receive a remote operation end instruction from the user (No in step S123), the operation control unit 232 returns to step S115 again to perform image data reception processing. At this time, when image difference information (described later) is received as image data, the third client terminal B1 creates a composite image of the latest received image data and image difference information, for example, and displays the operation window. Update.

10-2. Processing of Relay Server Connected with Operation Client Terminal Processing of the second relay server B connected with the third client terminal B1 that performs remote operation will be described.
If a remote operation request is received from the operation control unit 232 of the third client terminal B1 (see step S105 in FIG. 9A), the operation control unit 132 of the second relay server B refers to the relay server information and receives the request. After specifying the relay server to which the operation client terminal is connected, the remote operation request is relayed. Specifically, the operation control unit 132 of the second relay server B specifies the third relay server C to which the fifth client terminal C2 is connected based on the relay server information, and relays the remote operation request to the third relay server C. To do. If an operation permission response is transmitted from the fifth client terminal C2, the operation control unit 132 of the second relay server B establishes a second operation session with the identified third relay server C.
Thereafter, when a remote operation is performed between the third client terminal B1 and the fifth client terminal C2, the operation control unit 132 of the second relay server B relays the operation event and the image data.

10-3. Processing of Remotely Operated Client Terminal Next, processing of the fifth client terminal C2 that is the operated client terminal will be described. FIG. 10 is a flowchart showing processing of the operated client terminal.
The operation control unit 232 of the fifth client terminal C2 receives the remote operation request from the third client terminal B1 relayed by the second relay server B and the third relay server C (step S131), and receives the fifth client terminal B1. Is in a state in which remote control can be accepted (step S133).

If the remote operation can be accepted (Yes in step S133), the operation control unit 232 of the fifth client terminal C2 transmits a remote operation permission response to the third relay server C (step S135). The transmitted remote control permission response is then transmitted to the third client terminal B1 via the third relay server C and the second relay server B.
On the other hand, if it is determined that the remote operation is not acceptable (No in step S133), the operation control unit 232 of the fifth client terminal C2 sends an error response via the third relay server C and the second relay server B. To the third client terminal B1 (step S137), and the process is terminated.

The operation control unit 232 of the fifth client terminal C2 performs control for establishing a third operation session with the third relay server C under the control of the third relay server C (step S139).
If the third operation session is established, the operation control unit 232 of the fifth client terminal C2 next performs handshake processing with the third client terminal B1 (step S141), and performs setting processing of image information to be transmitted. This is performed (step S143).

  After the image information is set, the operation control unit 232 of the fifth client terminal C2 receives the target displayed on the display unit 224 of the fifth client terminal C2 via the third relay server C and the second relay server B. The image data of the operation window is transmitted to the third client terminal B1 based on the set image information (step S145).

  Thereafter, when an operation event is received from the third client terminal B1 (step S147), the operation control unit 232 of the fifth client terminal C2 executes a system process corresponding to the operation event (step S149). Next, the operation control unit 232 of the fifth client terminal C2 transmits the image difference information as image data to the third client terminal B1 (step S151). For example, only part of the image data already shared between the fifth client terminal C2 and the third client terminal B1 may be updated by an operation event. In this case, the operation control unit 232 of the fifth client terminal C2 transmits only the image difference information including the drawing position difference of the image to be the difference as image data to the third client terminal B1. As described above, when the system processing for updating a part of the image data in the operated window is performed in accordance with the operation event, only the image difference information is transmitted, and thus the fifth client terminal C2 to the third client terminal B1. The amount of data transferred to can be reduced.

Subsequently, when there is a screen update spontaneously on the operated window side, such as update of clock information (Yes in step S153), the process returns to step S151, and the operation control unit 232 of the fifth client terminal C2 3 Send the image difference information to the client terminal B1.
Thereafter, if there is no update of the screen in the operated window (No in step S153) and an instruction to end the remote operation is received from the third client terminal B1 (Yes in step S155), the operation control unit of the fifth client terminal C2 In step S232, the remote control process is terminated. If an instruction to end the remote operation is not received from the third client terminal B1 (No in step S155), the process by the fifth client terminal C2 returns to step S147 again.

10-4. Processing of Relay Server Connected with Operated Client Terminal Processing of the third relay server C connected with the fifth client terminal C2 that is the operated client terminal will be described.
First, when receiving the remote operation request from the second relay server B, the operation control unit 132 of the third relay server C relays the remote operation request to the fifth client terminal C2 that is the operated client terminal. Further, when receiving the remote operation permission response, the operation control unit 132 of the third relay server C establishes a third operation session with the fifth client terminal C2.
Furthermore, when a remote operation is performed between the third client terminal B1 and the fifth client terminal C2, the third relay server C relays the operation event and the image data. The third relay server C creates and updates remote operation information and stores it in the remote operation information storage unit 143 of the third relay server C, and further stores the remote operation information in the other relay servers in the first relay group. Process to share with client terminal.

  Specifically, when the third relay server C returns a permission response to the remote operation request between the third client terminal B1 and the fifth client terminal C2, the first to third operation sessions are established and the image is obtained by handshake processing. The remote operation information is recorded at the timing when the information is exchanged, and the remote operation information 40 (see FIG. 8) is created. The created remote operation information 40 is shared within the first relay group, similarly to the relay group information and the relay server information. At this time, since the remote operation is not monitored, the monitoring client terminal column of the remote operation information 40 is blank.

11. Remote operation monitoring processing Next, when other client terminals are remotely operating the client terminals belonging to the relay group as described above, the other client terminals belonging to the same relay group monitor the remote operation. Processing to be performed will be described.

  Hereinafter, referring to FIG. 11, when the third client terminal B1 remotely operates the fifth client terminal C2, the second client terminal A2 performs remote operation of the fifth client terminal C2 by the third client terminal B1. An example of monitoring will be described. FIG. 11 shows an example of the established operation session and monitoring session. As shown in FIG. 11, between the third client terminal B1 and the fifth client terminal C2, the first operation session 401A, the second operation session 401B, and the third operation are performed by the processes shown in FIGS. 9A, 9B, and 10. A session 401C is established.

11-1. Monitoring Process of Monitoring Client Terminal First, the process of the second client terminal A2, which is a monitoring client terminal, will be further described with reference to FIG. FIG. 12 is a flowchart showing the monitoring process of the monitoring client terminal.
First, the second client terminal A2 accepts a search for remote operation information (step S161). Specifically, the remote operation information 40 stored in the remote operation information storage unit 243 is displayed on the display unit 224 according to a user instruction. The user searches the remote operation information 40 displayed on the display unit 224 and selects a remote operation to be monitored.

If the selection of the remote operation “31” is accepted as the monitoring target (Yes in step S163), the monitoring control unit 233 of the second client terminal A2 connects to the first relay server A to which the second client terminal A2 is connected. A monitoring request related to the selected remote operation “31” is transmitted (step S165). Thereafter, in the first relay server A that has received the monitoring request, the relay server information 30 stored in the relay server information 142 is referred to, and the fifth client terminal C2 that is the operated client terminal of the remote operation “31” The connected relay server is identified. In this example, the third relay server C is specified as the relay server to which the fifth client terminal C2 is connected, and the monitoring request is relayed by the first relay server A.
Next, if the monitoring permission response of the remote operation “31” is received from the third relay server C (Yes in step S167), the first relay server A is connected between the second client terminal A2 and the third relay server C. A monitoring session 501 is established through the communication (step S171). Specifically, a monitoring session 501A is established between the second client terminal A2 and the first relay server A, and a monitoring session 501B is established between the first relay server A and the third relay server C. Here, when the monitoring permission response of the fifth client terminal C2 is not received from the third relay server C (No in Step S167), the monitoring control unit 233 of the second client terminal A2 executes error processing (Step S167). S169), the process ends.

  After the monitoring session 501 is established, the monitoring control unit 233 of the second client terminal A2 receives image data relating to the operated window of the fifth client terminal C2 from the third relay server C (step S173). Subsequently, the monitoring control unit 233 of the second client terminal A2 outputs the received image data to the display unit 224 of the second client terminal A2 and displays it as a monitoring window (step S175).

  Thereafter, if a monitoring end instruction is received from the user (Yes in step S177), the monitoring control unit 233 of the second client terminal A2 disconnects the monitoring session 501A with the first relay server A (step S181), and the monitoring is performed. The process ends. At this time, the monitoring session 501B is also disconnected. Or, if the monitoring end instruction is not received from the user (No in step S177), if the remote operation end command is received from the third relay server C (Yes in step S179), the monitoring control of the second client terminal A2 The unit 233 disconnects the monitoring session 501A with the third relay server C (step S181), and ends the monitoring process. At this time, the monitoring session 501B is also disconnected. If the monitoring session 501 is disconnected, the monitoring control unit 233 of the second client terminal A2 closes the monitoring window displayed on the display unit 224.

  On the other hand, when the remote operation end command is not received from the third relay server C (No in step S179), the monitoring process by the monitoring control unit 233 of the second client terminal A2 returns to step S173.

11-2. Processing of Monitoring Control Relay Server Next, processing of the monitoring control relay server to which the operated client terminal whose remote operation is being monitored is connected will be further described with reference to FIG. FIG. 13 is a flowchart showing the monitoring process of the monitoring control relay server.

  The monitoring control unit 133 of the third relay server C first determines whether a monitoring request has been received from the monitoring client terminal (step S191). Here, the monitoring control unit 133 receives a monitoring request for the remote operation “31” between the third client terminal B1 and the fifth client terminal C2 via the first relay server A from the second client terminal A2.

  When the requested monitoring can be permitted (Yes in step S193), the monitoring control unit 133 of the third relay server C transmits a permission response to the second client terminal A2 via the first relay server A ( Step SS195). Here, if the requested monitoring cannot be permitted (No in step S193), the monitoring control unit 133 of the third relay server C returns an error response to the second client terminal A2 via the first relay server A. (Step SS197) and the monitoring process is terminated.

  When the permission response is transmitted, the monitoring control unit 133 of the third relay server C next establishes a monitoring session 501B with the first relay server A (step S199). At this time, a monitoring session 501A is established between the first relay server A and the second client terminal A2. Further, the monitoring control unit 133 of the third relay server C adds the information of the second client terminal A2 as the monitoring client terminal to the remote operation “31” of the remote operation information 40 stored in the remote operation information storage unit 143. Update to do. The remote operation information 40 updated thereafter is shared between the relay server in the first relay group and the client terminals connected to the relay server.

  Thereafter, the monitoring control unit 133 of the third relay server C transmits an instruction to newly acquire screen data to the fifth client terminal C2 (step S201). As described above, there is a case where image data is already shared between the fifth client terminal C2 and the third client terminal B1, and only a part of the shared image data is updated according to the operation event. In this case, only the image difference information including the drawing position difference of the image to be the difference as image data is transmitted from the fifth client terminal C2 to the third client terminal B1. However, the second client terminal A2 that starts monitoring does not have image data shared by the fifth client terminal C2 and the third client terminal B1. Therefore, the image difference information alone is not sufficient to display the monitoring window on the display unit 224 of the second client terminal C2. Accordingly, the monitoring control unit 133 of the third relay server C transmits an instruction to newly acquire all new image data displayed in the operated window to the fifth client terminal C2.

After acquiring the new screen data from the fifth client terminal C2, the monitoring control unit 133 of the third relay server C transmits the newly acquired image data to the second client terminal A2 via the monitoring session 401 (step S203). .
Thereafter, the monitoring controller 133 of the third relay server C receives the image difference information as image data from the fifth client terminal C2 in response to the occurrence of the operation event, and copies the received image difference information to the second client terminal. A copy of the image difference information is transmitted to A2 (step S205).

If the monitoring end instruction is received from the second client terminal A2 (Yes in Step S207), the monitoring control unit 133 of the third relay server C disconnects the monitoring session 501B (Step S213). At this time, the monitoring session 501A is also disconnected. On the other hand, when the monitoring end instruction is not received from the second client terminal A2 (No in step S207), and the remote operation end instruction of the fifth client terminal C2 is received from the third client terminal B1. (Yes in step S209), the monitoring control unit 133 of the third relay server C notifies the end of the remote operation to the second client terminal A2 via the first relay server A (step S211). Specifically, the monitoring control unit 133 of the third relay server C transmits a remote operation end command to the second client terminal A2 via the first relay server A. Thereafter, the monitoring control unit 133 of the third relay server C disconnects the monitoring session 501B (step S213) and ends the process. At this time, the monitoring session 501A is also disconnected. In addition, the monitoring control unit 133 of the third relay server C closes the monitoring window displayed on the display unit 243.
If an instruction to end the remote operation of the fifth client terminal C2 is not received from the third client terminal B1 in step S211 (No in step S211), the processing by the second client terminal A2 returns to step S205.

12 Advantages of this embodiment According to the relay communication system of this embodiment, the contents of the remote operation performed in the first relay group are transmitted to the other than the third client terminal B1 and the fifth client terminal C2 via the monitoring session 501. The second client terminal A2 can monitor.
Further, when the second client terminal A2 makes a monitoring request, the first relay server A to which the second client terminal A2 is connected is referred to by referring to the relay server information 30 shared by the first relay group. It can be easily determined that a monitoring request may be made to the relay server C.
Furthermore, the image data relating to the remote operation between the second client terminal A2 and the third client terminal B1 is output by the third relay server C to the second client terminal A2. Therefore, the second client terminal A2 can monitor the remote operation without imposing any burden on the fifth client terminal C2.

13. Other Embodiments Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the scope of the invention. In particular, a plurality of embodiments and modifications described in this specification can be arbitrarily combined as necessary.

  (A) In the above embodiment, the monitoring control relay server has been described with an example in which the operated client terminal is connected to the relay server. However, the monitoring control relay server is not limited to the above example as long as it is a relay server that relays a remote operation between the operation client terminal and the operated client terminal. For example, the monitoring control relay server may be a relay server to which an operation client terminal is connected.

(B) In the above example, the remote operation information 40 is created by the relay server to which the operated client terminal is connected at the timing when the operation session is established between the operating client terminal and the operated client terminal and the handshake process is performed. An example was given. However, the timing at which the remote operation information 40 is created is not limited to the above example, and may be the timing at which an operation event occurs, for example. Further, the remote operation information 40 may be created not by the relay server to which the operated client terminal is connected, but by the operation client terminal, the operated client terminal, or the relay server to which the operation client terminal is connected.
Further, in the above-described embodiment, an example is given in which the account name of the monitoring client terminal is added to the remote operation information 40 by the monitoring control relay server at the timing when the monitoring session is established. However, the remote operation information 40 is not limited to the timing at which the monitoring session is established.

(C) In the above embodiment, the latest relay group information and relay server information are always shared among all relay servers and client terminals. However, the present invention is not limited to such an embodiment. For example, one of the relay servers may be a center terminal, and the center terminal may manage access rights between the relay servers.
(D) In the above embodiment, an example has been given in which the created and updated remote operation information 40 is shared between the relay server and the client terminal in the relay group.
However, the remote operation information 40 has only the operation client terminal, the operated client terminal, the relay server to which the operation client terminal is connected, and the relay server to which the operated client terminal is connected. 224 may be stored. Then, the monitoring client terminal may determine the remote operation to be monitored by referring to the relay server information 30 and inquiring whether any of the above terminals is being operated remotely.

(E) In the above embodiment, the third relay server C, which is the monitoring control relay server, transmits the image data related to the remote operation between the third client terminal B1 and the fifth client terminal C2 via the monitoring session. An example of transmission to the client terminal A2 has been given.
However, if the monitoring control relay server receives a monitoring request for a plurality of remote operations from the monitoring client terminal in a state where a plurality of remote operations are performed simultaneously via the monitoring control relay server, the monitoring control relay server The image data of each of the plurality of remote operations may be copied and transmitted to the monitoring session.

  For example, in the above embodiment, the third client terminal B1 may perform remote operation not only on the fifth client terminal C2 but also on the fourth client terminal C1 at the same time. At this time, the third relay server C provides not only the operation event and image data related to the remote operation (hereinafter referred to as the first remote operation) between the third client terminal B1 and the fifth client terminal C2, but also the third client terminal B1. And the fourth client terminal C1 are relaying an operation event and image data of a remote operation (hereinafter referred to as a second remote operation).

In this case, if the monitoring request for the second remote operation as well as the first remote operation is received from the second client terminal A2, the third relay server C is similar to the monitoring process of the first remote operation (see FIG. 13). A monitoring process related to the second remote operation between the third client terminal B1 and the fourth client terminal C1 may be performed by the process.
Therefore, the second client terminal A2 can acquire operation contents of a plurality of remote operations, the first remote operation and the second remote operation, using the monitoring session 501.

(F) Each relay server in which an operation session is established may store connection information in association with remote operation information stored in the remote operation information storage unit. The connection information is information related to an operation session established for the relay server itself.
For example, as in the example given in (e) above, it is assumed that the third client terminal B1 is remotely controlling not only the fifth client terminal C2 but also the fourth client terminal C1 at the same time. Here, the operation sessions established between the third client terminal B1 and the second relay server B for the second remote operation are the fourth operation session 601A, the second relay server B, and the third relay server C. An operation session established between the third operation server 601B and the third relay server C and the fourth client terminal C1 is referred to as a sixth operation session 601C. In this case, the third relay server C establishes the second operation session 401B, the third operation session 401C, the fifth operation session 601B, and the sixth operation session 601C with respect to the third relay server C in relation to the remote operation information. Connection information indicating that the connection has been made may be stored.

  Thus, the third relay server C stores connection information established with respect to the third relay server C in relation to the remote operation information. As a result, even if the third relay server C is a monitoring control relay server related to a plurality of remote operations, the third relay server C refers to the remote operation information and the connection information, and operates the remote operation related to the received monitoring request. A session can be better identified and a copy of the image data can be sent to the surveillance session.

  Each client terminal for which an operation session has been established stores the remote operation information stored in the remote operation information storage unit in association with connection information related to the operation session established for the client terminal itself. It may be. For example, the fifth client terminal C2 may store connection information in which the third operation session 401C is established with respect to the fifth client terminal C2 in association with the remote operation information.

(G) In the embodiment, the monitoring client terminal is the second client terminal A1, but the fourth client terminal C1 may monitor the remote operation of the fifth client terminal C2 by the third client terminal B1. In this case, the monitoring control unit 133 of the third relay server C can receive a monitoring request for monitoring the remote operation selected based on the remote operation information from the fourth client terminal C1 when the remote operation is being executed. For example, a monitoring session for monitoring a remote operation is established with the fourth client terminal C1, and image data relayed to the second relay server B is also transmitted to the fourth client terminal C1 via the monitoring session. In this case, when there is a seventh client terminal C3 (not shown) connected to the third relay server C, the third relay server C changes from the second relay server B to the third client terminal B1. When receiving the second remote operation request for the terminal C3, the third relay server performs the following operation. The operation control unit 132 relays the second remote operation request to the seventh client terminal C3, establishes the fourth operation session with the seventh client terminal C3, and performs the fifth operation with the second relay server B. By the third client terminal B1 established through the fourth operation session, the fifth operation session, and the sixth operation session established between the third client terminal B2 and the second relay server B. Relay the operation data of the second remote operation to the seventh client terminal C3. If the monitoring control unit 133 further receives a monitoring request for monitoring the second remote operation from the fourth client terminal C1, the monitoring control unit 133 transmits the operation content of the second remote operation to the fourth client terminal C1 via the monitoring session.

  The present invention can be widely applied to a relay communication system having a plurality of relay servers that can communicate with each other, a plurality of client terminals, and a LAN that connects the client terminals to the relay server, and a relay server used therefor.

1 First LAN
2 Second LAN
3 Third LAN
4 WAN
30 Relay server information 40 Remote operation information 131 Information sharing unit (relay server)
132 Operation control unit (relay server)
133 Monitoring control unit (relay server)
231 Information sharing unit (client terminal)
232 Operation control unit (client terminal)
233 Monitoring control unit (client terminal)
234 Screen setting unit 235 Client communication control unit 224 Display unit 225 Operation input unit A First relay server B Second relay server C Third relay server A1 First client terminal A2 Second client terminal B1 Third client terminal C1 Fourth client Terminal C2 5th client terminal

Claims (10)

A first network;
A second network;
A first relay server connected to the first network;
A second relay server connected to the second network and capable of communicating with the first relay server via a third network;
A first client terminal and a second client terminal connected to the first relay server via the first network;
A third client terminal connected to the second relay server via the second network;
With
The first relay server and the second relay server,
An information sharing unit for sharing the relay server information including the activation information of the relay server itself and the activation information and connection information of the client terminal connected to the relay server itself;
The second relay server is
A request to remotely operate the first client terminal selected with reference to the relay server information from the third client terminal which is an operation client terminal that performs remote operation, the third client terminal and the first client terminal If a first remote operation request including client terminal information is received, the first relay server to which the first client terminal is connected is specified, and the first remote operation request is relayed to the first relay server. The third client is established based on the first remote operation request by establishing a first operation session with the third client terminal, establishing a second operation session with the first relay server, An operation event and image data as operation data of the first remote operation to the first client terminal by the terminal are used as the first operation session. A second operation control unit for relaying between the fine second operation session,
The first relay server is
If the first remote operation request is received from the second relay server, the first remote operation request is relayed to the first client terminal, and a third operation session is established with the first client terminal, The operation data of the first remote operation performed based on the first remote operation request is relayed between the second operation session and the third operation session, and information on the first client terminal and the third client terminal A first operation control unit for creating first remote operation information including:
If the monitoring request for monitoring the first remote operation selected based on the first remote operation information is received from the second client terminal when the first remote operation is being executed, the second client terminal A monitoring session for monitoring the first remote operation with the second client terminal, and image data relayed to the second relay server in the second operation session via the monitoring session. And a monitoring control unit that also transmits to the relay communication system. The operation event relates to the first remote operation received from the user via the operation input unit of the third client terminal , and the first remote operation information includes the first client terminal and the third client. The relay communication system according to claim 1, wherein image information for commonly setting screen information between terminals is created and further included. A fourth client terminal connected to the first relay server via the first network;
If the first relay server receives a second remote operation request from the second relay server to the fourth client terminal by the third client terminal,
The first operation control unit relays the second remote operation request to the fourth client terminal, establishes a fourth operation session with the fourth client terminal, and communicates with the second relay server. establishing a fifth operation session, the fourth operation session, the a fifth operation session, the third said client terminal and via a sixth operation session established between the second relay server is executed Relay the operation data of the second remote operation to the fourth client terminal by the third client terminal;
If the monitoring control unit further receives a monitoring request for monitoring the second remote operation from the second client terminal, the monitoring control unit transmits the operation content of the second remote operation to the second client terminal via the monitoring session. The relay communication system according to claim 1 or 2. A relay server that is communicable with a second relay server connected to the second network, is connected to the first network, and functions as a first relay server connected to the first client terminal via the first network. And
An information sharing unit for sharing the relay server information with the second relay server, including the relay server start information and the start information and connection information of the client terminal connected to the relay server;
A request to remotely operate the first client terminal selected with reference to the relay server information from a third client terminal connected to the second relay server, which is an operation client terminal that performs remote operation, If the first remote operation request including the information on the third client terminal and the first client terminal is received, the first remote operation request is relayed to the first client terminal, and the first remote operation request is transmitted to the second relay server. A second operation session, a third operation session is established with the first client terminal, the second operation session, the third operation session, the third client terminal, and the second relay server, The third client terminal performs the first remote operation request based on the first remote operation request via a first operation session established during The operation event and image data as the operation data of the first remote control to one client terminal and relayed between said second operation session and the third operation session, the first client terminal and information of the third client terminal An operation control unit for creating first remote operation information including:
If a monitoring request for monitoring the first remote operation selected based on the first remote operation information is received from the second client terminal when the first remote operation is being executed, the second client terminal and A monitoring session for monitoring the first remote operation is established, and image data relayed to the second relay server in the second operation session is transmitted to the second client terminal via the monitoring session. And a monitoring control unit that also transmits. The operation event relates to the first remote operation received from the user via the operation input unit of the third client terminal , and the first remote operation information includes the first client terminal and the third client. The relay server according to claim 4, wherein image information for commonly setting screen information between terminals is created and further included. A fourth client terminal is further connected via the first network;
If a second remote operation request for the fourth client terminal by the third client terminal is received from the second relay server,
The operation control unit relays the second remote operation request to the fourth client terminal, establishes a fourth operation session with the fourth client terminal, and sets a fifth operation session with the second relay server. establishing an operating session, and the fourth operation session, and the fifth operation session, wherein the third client terminal and a sixth operation session established between the second relay server, are performed through the first Relay the operation data of the second remote operation to the fourth client terminal by the three client terminals;
If the monitoring control unit further receives a monitoring request for monitoring the second remote operation from the second client terminal, the monitoring control unit transmits the operation content of the second remote operation to the second client terminal via the monitoring session. The relay server according to claim 4 or 5. A first network;
A second network;
A third network;
A first relay server connected to the first network;
A second relay server connected to the second network and capable of communicating with the first relay server via a third network;
A third relay server connected to the third network and connected to the first relay server and the second relay server via a network;
A first client terminal connected to the first relay server via the first network;
A third client terminal connected to the second relay server via the second network;
A fifth client terminal connected to the third relay server via the third network;
With
The first relay server, and the second relay server and the third relay server,
An information sharing unit for sharing the relay server information including the start information of the relay server itself and the start information and connection information of the client terminal connected to the relay server itself;
The second relay server is
A request to remotely operate the first client terminal selected with reference to the relay server information from the third client terminal which is an operation client terminal that performs remote operation, the third client terminal and the first client terminal If a first remote operation request including client terminal information is received, the first relay server to which the first client terminal is connected is specified, and the first remote operation request is relayed to the first relay server. The third client is established based on the first remote operation request by establishing a first operation session with the third client terminal, establishing a second operation session with the first relay server, An operation event and image data as operation data of the first remote operation to the first client terminal by the terminal are used as the first operation session. A second operation control unit for relaying between the fine second operation session,
The first relay server is
If the first remote operation request is received from the second relay server, the first remote operation request is relayed to the first client terminal, and a third operation session is established with the first client terminal, The operation data of the first remote operation performed based on the first remote operation request is relayed between the second operation session and the third operation session, and information on the first client terminal and the third client terminal A first operation control unit for creating first remote operation information including:
If the monitoring request for monitoring the first remote operation selected based on the first remote operation information is received from the fifth client terminal through the third relay server when the first remote operation is being executed. Establishing a monitoring session for monitoring the first remote operation with the fifth client terminal, and transmitting image data relayed to the second relay server in the second operation session via the monitoring session. A monitoring control unit that also transmits to the fifth client terminal. The operation event relates to the first remote operation received from the user via the operation input unit of the third client terminal , and the first remote operation information includes the first client terminal and the third client. The relay communication system according to claim 7, wherein image information for setting screen information between terminals in common is created and further included. A fourth client terminal connected to the first relay server via the first network;
If the first relay server receives a second remote operation request from the second relay server to the fourth client terminal by the third client terminal,
The first operation control unit relays the second remote operation request to the fourth client terminal, establishes a fourth operation session with the fourth client terminal, and communicates with the second relay server. establishing a fifth operation session, the fourth operation session, the a fifth operation session, the third said client terminal and via a sixth operation session established between the second relay server is executed Relay the operation data of the second remote operation to the fourth client terminal by the third client terminal;
If the monitoring control unit further receives a monitoring request for monitoring the second remote operation from the fifth client terminal through the third relay server, the monitoring control unit displays the operation content of the second remote operation via the monitoring session. The relay communication system according to claim 7 or 8, wherein the relay communication system transmits to the fifth client terminal. A first relay server connectable to the second relay server and the third relay server via the WAN, and connectable to the first client terminal and the second client terminal via the LAN,
A first operation session establishing unit for establishing a first operation session with the first client terminal;
A second operation session establishing unit for establishing a second operation session with the second relay server;
An operation relay unit that relays operation data including operation events and image data between the first operation session and the second operation session;
A monitoring session establishing unit for establishing a monitoring session with the second client terminal or the third relay server;
A monitoring controller that relays the image data relayed between the first operation session and the second operation session also to the monitoring session;
A first relay server.

Images