WO2010073869A1

WO2010073869A1 - Client device, screen sharing system, and setting change method

Info

Publication number: WO2010073869A1
Application number: PCT/JP2009/069923
Authority: WO
Inventors: 博史川添; 健作山口; 龍洋澤永; 信哉村井; 真孝後藤; 泰如西林
Original assignee: 株式会社東芝
Priority date: 2008-12-26
Filing date: 2009-11-26
Publication date: 2010-07-01
Also published as: JP2010157005A; JP5305894B2

Abstract

A client device (2) is provided with a screen receiving unit (221) for receiving a displayed screen from a server device (1); a superposition unit (224) for superposing a superposing image onto the displayed screen; a display control unit (222) for displaying the displayed screen superposed with the superposing image on a display device (24); a reception unit (226) for receiving input information which is input on the portion of an area of the displayed screen in which the superposing image has been superposed; and a request transmission unit (223) which transmits, to the server device (1), a change request to change a piece of set information which is predetermined in accordance with the input information, among the pieces of set information to define the operation of the server device (1), into a value which is predetermined in accordance with the input information. The server device (1) is provided with a request receiving unit (124) to receive the change request, and a setting change unit (123) to change the set information on the basis of the received change request.

Description

Client device, screen sharing system, and setting change method

The present invention relates to a client device, a screen sharing system, and a setting change method.

2. Description of the Related Art A screen sharing system that packetizes screen information generated by a server device to a client device from a server device connected via a network is known in real time.

As a technique for realizing a screen sharing system, VNC (Virtual Network Computing) shown in Non-Patent Document 1 is well known. According to VNC, the screen information displayed on the server device can be displayed on the display of the client device. As a result, the user of the client device can perform a remote operation as if the server device is at hand while referring to the same screen information as the screen information displayed on the display of the server device.

In addition, when building a screen sharing system, information on the entire screen of the server device is distributed to the client device at regular intervals, and only the changed portion is displayed in the coordinates each time the screen is updated. There is a method of delivering together. In the latter method of distributing the difference, only the necessary part of the data is generated according to the change in the contents of the screen. Therefore, compared to the former method, the network bandwidth can be used efficiently. Furthermore, there is an advantage that the number of drawing frames per unit time can be increased. Therefore, in the currently provided screen sharing systems, services are often provided by a configuration that distributes only difference data.

In Patent Literature 1, at least a part of screen information is transmitted using a TCP packet or a UDP packet is used depending on any of the data type or display conditions of the screen area, the communication environment, or a user instruction. There has been proposed a screen sharing system that switches between transmission and transmission.

Patent Document 2 proposes a technology related to a system that displays content from a computing device to an alternative display device via wireless communication. Patent Document 2 also proposes an application in which a user of a computing device selects which content of the primary display or secondary display of the computing device is to be transmitted. For example, when the user selects the primary display, the screen information on the computing device is displayed as it is on the alternative display device, and when the secondary display is selected, the extended screen image of the computing device is displayed as an alternative display. Displayed on the device. A state in which the screen information of the primary display is transmitted may be referred to as a mirror mode, and a state in which the screen information of the secondary display is transmitted may be referred to as an extended mode. By applying the above-described application to the screen sharing system, it is possible to refer to and operate not only the primary display of the server apparatus but also the secondary display from the client apparatus.

JP 2005-348262 A JP 2005-129068 A

However, in a screen sharing system that displays the screen information of the server device on the client device in either the mirror mode or the extended mode, the operation (behavior) of the server device related to screen sharing is performed by an operation on the client device side during screen sharing. There was a case where it could not be changed.

This is due to the following reasons. Usually, in order to change the operation of the server device related to screen sharing, a GUI (Graphical User Interface) for changing the operation is displayed on the server device, and the change is instructed by the user operating the GUI. Is used. However, in this case, there may be a problem that the GUI for changing the operation is not necessarily displayed on the display referred to by the client device. For example, this corresponds to a situation where a setting dialog for changing the operation is displayed on the primary display side even though screen sharing is being executed in the extended mode. At this time, since the user of the client device cannot refer to the GUI arranged on the primary display, the operation change process cannot be started. Therefore, the user is forced to change the operation by moving to the installation location of the server device and directly operating the server device. The procedure for changing the operation accompanied by such movement is inefficient and an excessive burden on the user.

The present invention has been made in view of the above, and a client device, a screen sharing system, and a setting change method capable of changing the operation of a server device related to screen sharing by operation of the client device during execution of screen sharing , To provide.

The present invention provides a client device that shares a display screen with a server device connected via a network, a screen receiving unit that receives the display screen from the server device, and a predetermined superimposed image that is displayed on the display screen A superimposing unit that superimposes the image, a display control unit that displays the display screen on which the superimposed image is superimposed, and a region of the displayed display screen that is input on the region where the superimposed image is superimposed. A receiving unit that receives the input information, and a change that changes the setting information that is predetermined according to the input information to the value that is predetermined according to the input information, among the setting information that determines the operation of the server device A request transmission unit configured to transmit a request to the server device.

The present invention also provides a screen sharing system comprising a server device and the client device according to claim 1 connected to the server device via a network and sharing a display screen of the server device. The apparatus includes a request receiving unit that receives the change request, and a setting change unit that changes the setting information based on the received change request.

The present invention is a setting change method executed in a screen sharing system including a server device and a client device connected to the server device via a network and sharing a display screen of the server device, A screen receiving step in which the client device receives the display screen from the server device, a superimposing step in which the client device superimposes a predetermined superimposed image on the display screen, and the client device in the superimposed image A display control step of displaying the display screen on which the superimposed image is superimposed on the display device, and the client device accepts input information input on the region of the displayed display screen on which the superimposed image is superimposed A receiving step, and the client device inputs the input of the setting information that defines the operation of the server device. A request transmission step for transmitting to the server device a change request for changing the setting information predetermined according to the information to a predetermined value according to the input information; and A request receiving step for receiving, and a setting change step for changing the setting information based on the received change request by the server device.

According to the present invention, it is possible to change the operation of the server device related to screen sharing by operating the client device during screen sharing.

1 is a block diagram of a screen sharing system according to a first embodiment. The flowchart which shows the whole flow of the share start process in 1st Embodiment. The figure which shows an example of the screen information showing the list of server apparatuses. The figure which shows an example of the screen information for a screen sharing start. The figure which shows an example of the coordinate expressing position information. The flowchart which shows the whole flow of the screen information transmission process in 1st Embodiment. The figure which shows an example of a mode that the screen information on which the button image was superimposed is displayed on a display apparatus. The flowchart which shows the whole flow of the input information transmission process in 1st Embodiment. The figure which shows an example of a mode that the screen information on which the menu image was superimposed is displayed on a display apparatus. The flowchart which shows the whole flow of the share end process in 1st Embodiment. The state transition figure showing the mode of transition between the states which can be changed by each process. The figure which shows an example of a display screen. The block diagram of the screen sharing system concerning 2nd Embodiment. The flowchart which shows the whole flow of the input information transmission process in 2nd Embodiment. The hardware block diagram of the server apparatus and client apparatus concerning 1st or 2nd Embodiment.

DETAILED DESCRIPTION Exemplary embodiments of a client device, a screen sharing system, and a setting change method according to the present invention will be described below in detail with reference to the accompanying drawings.

(First embodiment)
In the screen sharing system according to the first embodiment, a client device superimposes and displays a predetermined superimposed image on a display screen (screen information) shared with the server device. A change request (change command) for changing the setting information that defines the operation is transmitted.

FIG. 1 is a block diagram showing a configuration of a screen sharing system 100 according to the first embodiment. As shown in FIG. 1, in the screen sharing system 100, an information processing terminal device (server device 1) that functions as a server and an information processing terminal device (client device 2) that functions as a client are connected via a network 3. It has been configured.

The server device 1 is connected to a pointing device 13, a display device 14, and a display device 15. Further, the client device 2 is connected to a pointing device 23 and a display device 24.

The pointing device 13 and the pointing device 23 are devices for instructing and inputting information such as position and state to the connected server device 1 or client device 2 such as a mouse, a trackball, and a stylus pen.

The display device 14, the display device 15, and the display device 24 are still images or moving images output from the connected server device 1 or client device 2 such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display). A device for displaying signals.

The server device 1 has a function of transmitting screen information of application software that operates in its own device to a format suitable for transmission over the network 3 and transmitting the information to the client device 2 in real time. The client device 2 has a function of displaying screen information received from the server device 1. The network 3 is a network conforming to a wired communication protocol such as IEEE802.3, for example. The form of the network 3 is not limited to this. For example, the network 3 may be configured by a network that complies with other protocols such as a wireless communication protocol defined by IEEE 802.11.

Next, an outline of internal components related to the present embodiment in the server device 1 is shown. The server device 1 includes an operating system 11 and a screen sharing server 12 as its components.

The operating system 11 provides basic functions in the server device 1. For example, the operating system 11 allocates resources such as memory and processing time to application software running on the server device 1, creates screen information according to the execution state of the application software and the operation of the pointing device 13, and displays A function of outputting to the device 14 and the display device 15 is provided. The operating system 11 can hold a plurality of screen information respectively corresponding to the display device 14 and the display device 15.

The screen sharing server 12 is application software that provides a server function for screen sharing processing (hereinafter sometimes simply referred to as screen sharing). The screen sharing server 12 includes, as its constituent elements, a screen acquisition unit 121, a screen transmission unit 122, a setting change unit 123, a request reception unit 124, an input generation unit 125, an input reception unit 126, and a session management unit 127. And a communication unit 128.

The screen acquisition unit 121 acquires screen information generated by the operating system 11. The screen transmission unit 122 transmits the screen information acquired from the operating system 11 to the client device 2 via the communication unit 128.

The setting changing unit 123 changes setting information that defines the operation of the server device 1 such as the operation of the operating system 11. The setting information includes, for example, a sharing mode that represents an operation mode related to screen sharing. The shared mode is, for example, one of the above-described mirror mode and extended mode.

The request receiving unit 124 receives a setting change command for requesting a change in the operation of the server device 1 from the client device 2 via the communication unit 128.

The input generation unit 125 converts the input information input from the pointing device 13 or the like into a format that can be interpreted by the operating system 11 and then passes it to the operating system 11 to instruct execution of the input.

The input receiving unit 126 receives input information from the client device 2 via the communication unit 128.

The session management unit 127 starts and ends screen sharing based on a request from the client device 2 via the network 3 or an operation of a user of the server device 1.

The communication unit 128 converts the input information into a format suitable for transmission through the network 3 and transmits the converted information to the client device 2.

Next, an outline of internal components related to the present embodiment in the client device 2 is shown. The client device 2 includes an operating system 21 and a screen sharing client 22 as its constituent elements.

The operating system 21 provides basic functions in the client device 2, similar to the operating system 11 in the server device 1.

The screen sharing client 22 is application software that provides a client function for screen sharing. The screen sharing client 22 includes, as its components, a screen reception unit 221, a display control unit 222, a request transmission unit 223, a superimposition unit 224, an input transmission unit 225, a reception unit 226, a session management unit 227, And a communication unit 228.

The screen receiving unit 221 receives screen information from the server device 1 via the network 3. The screen receiving unit 221 converts the received screen information into a format suitable for display by the display control unit 222.

The display control unit 222 displays the input screen information on the display device 24 connected to the client device 2. The display control unit 222 includes a frame buffer 222a that is a storage area for temporarily storing screen information, and each time screen information is input from the screen reception unit 221 and the superimposition unit 224, the display control unit 222 is based on the contents. To update the information in the frame buffer 222a. The display control unit 222 can also display the information in the frame buffer 222 a on the display device 24 connected to the client device 2 by inputting the information in the frame buffer 222 a to the operating system 21.

The request transmission unit 223 transmits a request (setting change command) for changing the setting of the server device 1 generated by the reception unit 226 described later to the server device 1 via the communication unit 228.

The superimposing unit 224 has a GUI image (GUI image) for superimposing and displaying on the screen information received from the server device 1. The superimposing unit 224 outputs the GUI image so as to be superimposed and displayed on the frame buffer 222a of the display control unit 222.

The input transmission unit 225 transmits the input information input by the reception unit 226 toward the server device 1 via the communication unit 228.

The accepting unit 226 accepts input information generated by the operating system 21. In addition, when the received input information represents an operation in the area of the GUI image that is superimposed and displayed by the superimposing unit 224, the accepting unit 226 is configured to change settings related to the operation of the operating system 11 of the server device 1. Create a change order.

The session management unit 227 manages the state of the screen sharing client 22. For example, the session management unit 227 starts and ends screen sharing based on a request from the server device 1 via the network 3 or a user operation via the reception unit 226.

The communication unit 228 converts the input information into a format suitable for transmission through the network 3 and transmits the converted information to the server device 1.

Next, each process by the screen sharing system 100 according to the first embodiment configured as described above will be described with reference to the drawings as appropriate. In the present embodiment, the operation of the user who operates the client apparatus 2 in a situation where the server apparatus 1 and the client apparatus 2 are connected via the network 3 which is a TCP / IP (Transmission Control Protocol / Internet Protocol) network. The screen sharing with the server device 1 is started based on the above, and the operation of the server device 1 is changed by the operation from the client device 2 side during the screen sharing, and the screen sharing is finally ended. The procedure is shown.

First, sharing start processing representing processing for starting screen sharing in the screen sharing system 100 will be described. FIG. 2 is a flowchart illustrating an overall flow of the sharing start process according to the first embodiment.

When the screen sharing client 22 is activated in the client device 2, the session management unit 227 transitions to the [before start] state of screen sharing. In this [Before Start] state, the session management unit 227 discovers a plurality of server devices that can execute screen sharing from the network 3 according to an arbitrary device discovery procedure. Then, the session management unit 227 notifies the display control unit 222 of the address information of each discovered server device and instructs the screen display of the [Before start] state. Upon receiving this instruction, the display control unit 222 first clears the information in the frame buffer 222a. After that, the display control unit 222 creates screen information representing a list of server devices that can execute screen sharing based on the notified address information, passes it to the operating system 21, and displays it on the display device 24. Is instructed (step S201).

FIG. 3 is a diagram showing an example of screen information representing a list of server devices created at this time. In the example of FIG. 3, messages prompting the user to select a server device for starting screen sharing and icon images 301 to 306 indicating that screen sharing can be executed are listed together with the address information of the server device. ing.

The session management unit 227 requests the reception unit 226 to shift to the [Before start] state in conjunction with the above processing. The receiving unit 226 that has received the request monitors the input notified from the operating system 21. Then, when an input (such as a mouse click) is performed from the pointing device 23 within the rectangular area of the icon image managed by the display control unit 222, the reception unit 226 displays the coordinates of the input position as a session. The management unit 227 is notified. The session management unit 227 identifies the server device 1 that performs screen sharing by converting the coordinates of the input position into the address information of the corresponding server device (step S202). Thereby, the session management unit 227 can start screen sharing with the identified server device 1.

The session management unit 227 does not necessarily need to use the above-described device discovery procedure in order to specify address information of a partner with whom screen sharing is performed. For example, the user may directly enter the address of the server device 1 that starts screen sharing. FIG. 4 is a diagram showing an example of screen information for starting screen sharing created in such a configuration. In the example of FIG. 4, a message prompting the user to input address information of a partner to start screen sharing, an input edit box 401, an input button 402, and a screen sharing start button 403 are displayed. Yes. The user operates the input button 402 to input address information, confirms the input content displayed in the input edit box 401, and then presses the screen sharing start button 403 to instruct start of screen sharing. . The session management unit 227 can specify the address information of the server device 1 that performs screen sharing also by such a procedure.

When the session management unit 227 specifies the address information of the server device 1 that starts screen sharing by the above procedure, the session management unit 227 inputs the address information to the communication unit 228 and requests to establish communication with the server device 1 of this address information. To do. The communication unit 228 performs processing for establishing a TCP connection with the communication unit 128 of the server device 1 so that data communication is possible between the two (step S203). When the TCP connection establishment is completed, the communication unit 128 notifies the session management unit 127 and the communication unit 228 notifies the session management unit 227 that the TCP connection has been established together with the address information of each communication partner.

When the session management unit 127 of the server device 1 receives a notification of completion of establishment of the TCP connection from the communication unit 128, the session management unit 127 starts transmission of screen information (step S204). The session management unit 127 acquires the setting value of the sharing mode for screen sharing stored in the inside or another storage unit (not shown). Then, the session management unit 127 notifies the operating system 11 of the acquired setting value and requests the screen acquisition unit 121 to output the screen information generated by the operating system 11 thereafter.

When the operating system 11 receives the above request, the operating system 11 refers to the notified sharing mode. When the sharing mode is the mirror mode, the operating system 11 detects a new update that occurs in the screen information held by itself (step S205). Then, each time a new update occurs, the operating system 11 outputs screen information relating to the updated portion to the screen acquisition unit 121. At this time, position information indicating the position of the updated portion is added to the screen information. FIG. 5 is a diagram illustrating an example of coordinates representing position information. In the figure, the coordinate value (Left, Top) and the lower right vertex of the updated image rectangle 501 when the upper left of the entire screen is the origin of the xy coordinate plane, the horizontal width of the desktop is Width, and the vertical width is Height. The example of the coordinate which represents positional information with the coordinate value (Right, Bottom) of is shown.

When the sharing mode is the extended mode, the operating system 11 enables the multi-monitor environment. That is, the operating system 11 newly enables an extended screen in addition to the screen information currently held by itself, and enables the presentation of information. For convenience, among the display device 14 and the display device 15, a display device that displays the originally held screen is called a primary display, and a display device that displays an extended screen is called a secondary display. As with the primary display, the operating system 11 holds screen information to be displayed on the secondary display. Thereafter, each time a new update occurs in the screen information of the secondary display, the operating system 11 outputs the screen information related to the updated portion together with the position information.

It has been described so far that every time the content of the screen information held by the operating system 11 is updated, position information is added to the screen information of the updated portion and output. However, the screen information output method is not necessarily this. It is not limited to. For example, a method may be used in which the entire screen information is periodically inspected, a difference from the separately held screen information at the previous inspection is detected, and the screen information related to the difference is output together with the position information. Alternatively, it is possible to follow a method of always outputting the entire screen information periodically held without checking the difference. In this case, adding position information to screen information can be omitted.

The screen information output from the operating system 11 to the screen acquisition unit 121 is transmitted to the client device 2 by the screen transmission unit 122 via the communication unit 128 (step S206).

Next, processing of the session management unit 227 when receiving a notification of completion of TCP connection establishment from the communication unit 228 will be described. Upon receiving the notification, the session management unit 227 starts receiving screen information. Thereafter, when the screen information is transmitted from the server apparatus 1, the screen receiving unit 221 receives the screen information and notifies the display control unit 222 of the received screen information. Then, the display control unit 222 stores the screen information in the frame buffer 222a (Step S207). The superimposing unit 224 generates a predetermined superimposed image corresponding to the current state and superimposes it on a predetermined display position of the screen information in the frame buffer 222a (step S208). For example, the superimposing unit 224 generates a button image that is a superimposed image corresponding to the “screen sharing: button” state, and superimposes the button image on the screen information in the frame buffer 222a.

Also, the session management unit 227 instructs the display control unit 222 to perform screen display in the “screen sharing: button” state. In this state, the display control unit 222 displays, to the operating system 21, the screen information in which the button image corresponding to the “screen sharing: button” state is superimposed on the screen information stored in the frame buffer 222 a by the superimposing unit 224. Output (step S209).

Next, the session management unit 227 instructs the reception unit 226 to shift to the “screen sharing: button” state. Receiving the instruction, the accepting unit 226 acquires information related to the display position of the button image from the superimposing unit 224. Thereafter, the receiving unit 226 monitors the input notified from the operating system 21. Then, when a specific operation is input in an area corresponding to the display position of the button image, the reception unit 226 notifies the session management unit 227 that the button has been operated. The accepting unit 226 outputs the position information to the input transmitting unit 225 when any other input is made.

Next, each process performed after screen sharing is started in the screen sharing system 100 will be described. This processing includes screen information transmission processing and input information transmission processing. The screen information transmission process is a process for transmitting the updated screen information to the client device 2 when the screen information of the server device 1 is updated. The input information transmission process is a process of transmitting input information corresponding to an operation detected by the pointing device 23 on the client device 2 to the server device 1.

First, the screen information transmission process will be described. FIG. 6 is a flowchart illustrating an overall flow of the screen information transmission process according to the first embodiment.

When the operating system 11 of the server apparatus 1 detects the update of the screen information (step S601), the operating system 11 outputs the combination of the screen information and the position information to the screen acquisition unit 121 as described above (step S602). . The screen acquisition unit 121 outputs a combination of screen information and position information to the screen transmission unit 122.

When the screen transmission unit 122 receives the combination of the screen information and the position information, the screen transmission unit 122 performs processing necessary for packetizing the combination by the communication unit 128 (step S603). First, the screen transmission unit 122 performs a conversion process of position information into a network byte order. Next, the screen transmission unit 122 performs compression processing on the screen information by irreversible compression such as JPEG (Joint Photographic Experts Group) or lossless compression method such as gzip (GNU ZIP). Add the indicated value. The processed information is transmitted to the client device 2 via the communication unit 128 (step S604).

Note that the communication unit 128 adds information necessary for transmission over the network 3 to the input information and transmits the information to the client device 2. For example, the communication unit 128 adds a TCP header or an IP header in addition to a value indicating the information type, and transmits it to the client device 2. The information type is information for identifying the type of information to be transmitted. For example, an integer value indicating whether the information to be transmitted is screen information, a setting change command, or input information is set as a value indicating the information type. When the communication unit 228 of the client device 2 determines that the received information is image information based on the value indicating the information type extracted from the received information, the communication unit 228 inputs the received information to the screen reception unit 221.

The screen receiving unit 221 acquires screen information and position information by performing the reverse process of the process performed by the screen transmitting unit 122 on the input information (step S605). That is, the screen receiving unit 221 performs a conversion process of the position information from the network byte order, and performs a screen information expansion process based on the value indicating the compression method. The screen receiving unit 221 outputs the converted position information and screen information to the display control unit 222.

When the position information and the screen information are input, the display control unit 222 overwrites and updates the information indicated by the position information among the information in the frame buffer 222a with the input screen information (step S606). Next, the superimposing unit 224 superimposes the button image corresponding to the current processing state on the screen information of the frame buffer 222a (step S607). The display control unit 222 instructs the operating system 21 to update the display on the display device 24 with the screen information on which the button image is superimposed (step S608).

FIG. 7 is a diagram illustrating an example of how the screen information on which the button image is superimposed is displayed on the display device 24. In the figure, in addition to the

windows

701 and 702, an example in which a button image 711 is superimposed and displayed at the lower right of the screen is shown.

Next, input information transmission processing will be described. FIG. 8 is a flowchart showing an overall flow of the input information transmission processing in the first embodiment.

In the [screen sharing: button] state, the reception unit 226 receives the input information of the pointing device 23 from the operating system 21 (step S801). The accepting unit 226 determines whether or not the input information is an operation on the button image superimposed by the superimposing unit 224 (step S802). For example, the reception unit 226 refers to the position information included in the input information, and determines whether the operation is within the button image area of the superimposition unit 224 notified by the session management unit 227 at the start of screen sharing. It is determined whether or not the state information included in the input information is equal to a click operation (corresponding to a tap operation in the case of a stylus pen). If both are applicable, the accepting unit 226 determines that an operation has been performed on the button image (step S802: YES), and notifies the session management unit 227 to that effect. In other cases (step S802: NO), the reception unit 226 requests the input transmission unit 225 to transmit input information.

First, the processing of the input transmission unit 225 when receiving a request for transmission of input information will be described. Upon receiving a request for transmission of input information, the input transmission unit 225 performs processing necessary for packetizing in the communication unit 228, such as byte order conversion (step S803). The converted input information is passed to the communication unit 228. The communication unit 228 adds information necessary for transmission over the network 3 such as a TCP header and an IP header, in addition to a value indicating the information type, to the input information, and then transmits the information to the server device 1 (step) S804). In this case, the communication unit 228 sets a value indicating that the information is input information as the information type.

The communication unit 128 of the server device 1 determines that the received information is input information based on a value representing the information type extracted from the received information, and inputs the received information to the input reception unit 126. The input reception unit 126 performs processing such as byte order conversion on the input information, and then outputs the obtained input information to the input generation unit 125 (step S805). The input generation unit 125 converts the input information into a format that can be recognized by the operating system 11, and then outputs the converted information to the operating system 11 (step S806). The operating system 11 executes processing that reflects the input information.

Next, the processing of the session management unit 227 when an operation on the button image is performed will be described. When the session management unit 227 receives a notification that an operation has been performed on the button image, the session management unit 227 instructs each of the display control unit 222 and the reception unit 226 to shift to the “screen sharing: menu” state. (Step S807).

In the [screen sharing: menu] state, the display control unit 222 overwrites and updates the frame buffer 222a with the position information and screen information input from the screen reception unit 221 as in the [screen sharing: button] state. At this time, unlike the case of the [screen sharing: button] state, the display control unit 222 superimposes the menu image, which is a superimposed image corresponding to the [screen sharing: menu] state, on the frame buffer 222a. Then, it instructs the operating system 21 to display screen information (step S808).

FIG. 9 is a diagram illustrating an example of how the screen information on which the menu image is superimposed is displayed on the display device 24. In the figure, an example is shown in which a menu image 911 including four choices is superimposed and displayed at the lower right of the screen. Note that this superimposed image (menu image) is an example, and the superimposing unit 224 generates a superimposed image determined in advance according to the current state and superimposes it on a predetermined display position of the screen information in the frame buffer 222a. Good.

The accepting unit 226 acquires information on the display position of the menu image from the superimposing unit 224 in the state of “screen sharing: menu”. Thereafter, the receiving unit 226 monitors the input notified from the operating system 21. Then, when a specific operation such as a click operation is performed in any area of each option of the menu image, the accepting unit 226 executes a process according to the corresponding option (step S809). For example, the reception unit 226 instructs the request transmission unit 223 to transmit a setting change command determined in advance according to the selected option and the executed operation. On the other hand, when an operation is performed at a position not included in any region, the reception unit 226 outputs input information to the input transmission unit 225 as in the [screen sharing: button] state.

In the example of FIG. 9, when “Primary” is selected, the reception unit 226 instructs the request transmission unit 223 to transmit a setting change command to the mirror mode. When “Secondary” is selected, the reception unit 226 instructs the request transmission unit 223 to transmit a setting change command to the extended mode. When “End” is selected, the reception unit 226 instructs the session management unit 227 to end screen sharing. When “Cancel” is selected, the reception unit 226 instructs the session management unit 227 to end the menu display.

When the request transmission unit 223 receives the setting change command for the sharing mode, the request transmission unit 223 transmits the setting change command to the server device 1 via the communication unit 228 (step S810). In this case, the communication unit 228 sets a value indicating that the information is a setting change command in the information type.

The communication unit 128 of the server device 1 determines that the received information is a setting change command based on a value indicating the information type extracted from the received information, and inputs the received setting change command to the request receiving unit 124. . The request reception unit 124 extracts the type of the shared mode after the change from the received information, and transfers this to the setting change unit 123.

The setting changing unit 123 confirms the type of the shared mode after the change, and if it is different from the current sharing mode (obtained by making an inquiry to the session management unit 127), the setting changing unit 123 switches the sharing mode. At this time, the setting change unit 123 instructs the operating system 11 to change the sharing mode (step S811). In response to this, the operating system 11 validates the multi-monitor environment when changing from the mirror mode to the extended mode. Thereafter, each time a new update occurs in the screen information of the secondary display, the operating system 11 outputs the screen information related to the updated portion to the screen acquisition unit 121 together with the position information.

When changing from the extended mode to the mirror mode, the operating system 11 disables the multi-monitor environment. Then, the operating system 11 outputs the screen information related to the updated part of the primary display thereafter to the screen acquisition unit 121 together with the position information.

In either case, the screen information of the entire screen of the corresponding display may be output immediately after the change. After the above processing is completed, the setting change unit 123 requests the session management unit 127 to save the changed sharing mode setting, and the session management unit 127 saves the setting according to the request.

Note that when receiving an instruction to end the menu display, the session management unit 227 instructs the display control unit 222 and the reception unit 226 to shift to the “screen sharing: button” state. The processing of the display control unit 222 and the reception unit 226 when receiving this instruction has already been described. This process is also executed when the accepting unit 226 accepts the selection of “Primary” or “Secondary”.

In addition, when the session management unit 227 receives an instruction to end screen sharing, the session management unit 227 executes sharing end processing representing processing for ending screen sharing. FIG. 10 is a flowchart showing the overall flow of the sharing end process in the first embodiment.

First, the session management unit 227 requests the communication unit 228 to disconnect the currently established TCP connection (step S1001). The communication unit 228 performs a disconnection procedure of the TCP connection currently established with the communication unit 128 of the server device 1 (step S1002). Both the communication unit 128 and the communication unit 228 are notified that the disconnection has been completed.

When the session management unit 127 of the server device 1 receives the notification of the disconnection completion, the session management unit 127 performs a screen information transmission end process (step S1003). Specifically, the session management unit 127 requests the operating system 11 to end the output of the screen information that has been performed so far.

When the session management unit 227 of the client device 2 receives the notification of the disconnection completion, the session management unit 227 instructs the display control unit 222 and the reception unit 226 to shift to the “before start” state (step S1004). The processing of the display control unit 222 and the reception unit 226 when receiving this instruction has already been described.

In the above, each process by the screen sharing system 100 was demonstrated. FIG. 11 is a state transition diagram showing a state of transition between states that can be transitioned by each process. As shown in the figure, in the present embodiment, transition is made between three states: [Before start] state, [Screen sharing: button], and [Screen sharing: menu]. In addition, the state of the same figure is an example, For example, you may comprise so that transition between three or more states is possible during screen sharing. In this case, for example, three or more superimposed images corresponding to each state may be superimposed.

Further, the setting information that can be changed from the client device 2 is not limited to the sharing mode, and any information can be configured as long as it is setting information that defines the operation of the server device 1. For example, the resolution of the display device 14 may be changed. In such a case, the server device 1 may be configured to include only one display device 14.

FIG. 12 is a diagram showing an example of a display screen displayed with a configuration for changing the resolution of the shared display device 14 or display device 15. In the figure, an example is shown in which a menu image 1211 including resolution options is superimposed and displayed at the lower right of the screen.

As described above, in the screen sharing system according to the first embodiment, the GUI for changing the operation of the server device is not always displayed on the display referred to by the client device. It is possible to change the operation of the server device only by the operation from the client device side, without the operator moving to the installation location of the server device and directly operating the server device.

(Second Embodiment)
In the first embodiment, the sharing mode is changed during screen sharing. However, some server devices are restricted so that the sharing mode cannot be changed during screen sharing. Therefore, in the second embodiment, when switching the sharing mode, after the screen sharing is once ended, the sharing mode is changed, and the screen sharing is started again.

FIG. 13 is a block diagram showing a configuration of the screen sharing system 200 according to the second embodiment. As shown in FIG. 13, the screen sharing system 200 has a configuration in which a server device 1-2 and a client device 2 are connected via a network 3.

In the second embodiment, the functions of the setting change unit 123-2 and the session management unit 127-2 of the screen sharing server 12-2 provided in the server device 1-2 are different from those in the first embodiment. Since other configurations and functions are the same as those in FIG. 1 which is a block diagram showing the configuration of the screen sharing system 100 according to the first embodiment, the same reference numerals are given and description thereof is omitted here.

When the setting change unit 123-2 receives the setting change command for the sharing mode, the setting changing unit 123-2 notifies the session management unit 127-2 of the changed sharing mode and instructs the end of screen sharing. This is different from the setting changing unit 123 of the embodiment.

In response to an instruction from the setting change unit 123-2, the session management unit 127-2 stores the changed sharing mode value, ends the screen sharing, and again performs screen sharing with the client device 2. The point which performs a start process is different from the session management part 127 of 1st Embodiment.

Next, input information transmission processing by the screen sharing system 200 according to the second embodiment configured as described above will be described with reference to FIG. Since the sharing start process, the screen information transmission process, and the sharing end process are the same as those in the first embodiment, description thereof is omitted.

FIG. 14 is a flowchart showing an overall flow of input information transmission processing in the second embodiment.

Since the processing from step S1401 to step S1410 is the same as that from step S801 to step S810 in the screen sharing system 100 according to the first embodiment, description thereof is omitted.

The setting change unit 123-2 that has received the setting change command determines whether or not the received setting change command is a sharing mode change command (step S1411). If it is not a change command for the sharing mode (step S1411: NO), the setting change unit 123-2 executes a setting information change process according to the setting change command (step S1412).

If it is a sharing mode change command (step S1411: YES), the setting changing unit 123-2 instructs the session management unit 127-2 together with the changed sharing mode value when switching the sharing mode. Instructs the end of screen sharing. Receiving this, the session management unit 127-2 saves the input shared mode value after the change (step S1413). Next, the session management unit 127-2 instructs the session management unit 227 of the client device 2 to reconnect the screen sharing together with the address information of the server device 1-2 (step S1414).

Upon receiving this instruction, the session management unit 227 of the client device 2 instructs the communication unit 228 to disconnect the TCP connection with the server device 1-2. The communication unit 228 performs a procedure for disconnecting the currently established TCP connection with the communication unit 128 of the server apparatus 1-2 (step S1415). When the disconnection of the connection is completed, the session management unit 227 starts screen sharing based on the address information of the server device 1-2 input together with the reconnection command.

That is, the communication unit 228 performs a TCP connection establishment process with the communication unit 128 of the server device 1-2 (step S1416). When the TCP connection establishment is completed, the communication unit 128 notifies the session management unit 127-2, and the communication unit 228 notifies the session management unit 227 that the TCP connection has been established. When the session management unit 127-2 of the server apparatus 1-2 receives a notification of completion of TCP connection establishment from the communication unit 128, it starts transmitting screen information (step S1417).

As described above, in the screen sharing system according to the second embodiment, even in an environment where the sharing mode cannot be changed while screen sharing is being performed, without additional user operation, You can change the sharing mode.

Next, the hardware configuration of the server device and the client device according to the first or second embodiment will be described with reference to FIG. FIG. 15 is an explanatory diagram illustrating a hardware configuration of the server device and the client device according to the first or second embodiment.

The server device and the client device according to the first or second embodiment include a control device such as a CPU (Central Processing Unit) 51 and a storage device such as a ROM (Read Only Memory) 52 and a RAM (Random Access Memory) 53. A communication I / F 54 that communicates by connecting to a network, an external storage device such as an HDD (Hard Disk Drive) and a CD (Compact Disk) drive device, a display device such as a display device, a keyboard and a mouse, etc. An input device and a bus 61 for connecting each unit are provided, and a hardware configuration using a normal computer is employed.

A program executed by the server device and the client device according to the first or second embodiment is a file in an installable format or an executable format, and is a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD). ), A CD-R (Compact Disk Recordable), a DVD (Digital Versatile Disk), etc., are recorded on a computer-readable recording medium and provided as a computer program product.

Further, the program executed by the server device and the client device according to the first or second embodiment is stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. It may be configured. The program executed by the server device and the client device according to the first or second embodiment may be provided or distributed via a network such as the Internet.

Further, the program according to the first or second embodiment may be provided by being incorporated in advance in a ROM or the like.

The program executed by the server device according to the first or second embodiment includes the above-described units (screen acquisition unit, screen transmission unit, setting change unit, request reception unit, input generation unit, input reception unit, session management). Module configuration including a communication unit). The program executed by the client device according to the first or second embodiment includes the above-described units (screen reception unit, display control unit, request transmission unit, superimposition unit, input transmission unit, reception unit, session management). Module configuration including a communication unit). As actual hardware, the CPU 51 (processor) reads out the program from the storage medium and executes the program, so that each unit is loaded on the main storage device, and each unit described above is generated on the main storage device. Yes.

Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

51 CPU
52 ROM
53 RAM
54 Communication I / F
61

Bus

100, 200 Screen sharing system 1, 1-2 Server device 2 Client device 3 Network 11 Operating system 12, 12-2 Screen sharing server 13

Pointing device

14, 15 Display device 21 Operating system 22 Screen sharing client 23 Pointing device 24 Display device 121 Screen acquisition unit 122 Screen transmission unit 123, 123-2 Setting change unit 124 Request reception unit 125 Input generation unit 126 Input reception unit 127, 127-2 Session management unit 128 Communication unit 221 Screen reception unit 222 Display control unit 222a Frame buffer 223 Request transmission unit 224 Superimposition unit 225 Input transmission unit 226 Reception unit 227 Session management unit 228 Communication unit

Claims

A client device that shares a display screen with a server device connected via a network,
A screen receiving unit for receiving the display screen from the server device;
A superimposing unit that superimposes a predetermined superimposed image on the display screen;
A display control unit for displaying the display screen on which the superimposed image is superimposed on a display device;
A receiving unit that receives input information that is input on a region in which the superimposed image is superimposed in a region of the displayed display screen;
A request for transmitting to the server device a change request for changing the setting information predetermined according to the input information to the predetermined value according to the input information among the setting information defining the operation of the server device A transmission unit;
A client device comprising:
A screen sharing system comprising: a server device; and the client device according to claim 1 connected to the server device via a network and sharing a display screen of the server device.
The server device
A request receiver for receiving the change request;
A setting change unit for changing the setting information based on the received change request;
A screen sharing system comprising:
The request transmission unit transmits the change request for the setting information defining the operation of the second display device connected to the server device to the server device;
The screen sharing system according to claim 2.
The request transmission unit transmits the change request for requesting a change of the second display device sharing the display screen among the plurality of second display devices connected to the server device to the server device. ,
The screen sharing system according to claim 3.
The setting changing unit ends the screen sharing process for sharing the display screen when the request for changing the second display device that shares the display screen is received by the request receiving unit; Changing the setting information based on the change request after the screen sharing process is completed, and starting the screen sharing process after changing the setting information;
The screen sharing system according to claim 4.
The request transmission unit transmits the change request for requesting a change in resolution of the display screen displayed on the second display device to the server device;
The screen sharing system according to claim 5.
The request transmission unit transmits the change request for requesting at least one of a start of a screen sharing process for sharing the display screen and an end of the screen sharing process to the server device;
The screen sharing system according to claim 6.
A setting change method executed in a screen sharing system including a server device and a client device connected to the server device via a network and sharing a display screen of the server device,
A screen receiving step in which the client device receives the display screen from the server device;
A superposition step in which the client device superimposes a predetermined superimposed image on the display screen;
A display control step in which the client device displays the display screen on which the superimposed image is superimposed on a display device;
A receiving step in which the client device receives input information input on a region of the displayed display screen on which the superimposed image is superimposed;
The server issues a change request for changing the setting information predetermined according to the input information from the setting information defining the operation of the server device to a value predetermined according to the input information. A request sending step for sending to the device;
A request receiving step in which the server device receives the change request;
A setting change step in which the server device changes the setting information based on the received change request;
A setting change method characterized by comprising: