JP4611584B2 - Image data display method, image data display server, image data display program, and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image data display method, an image data display server, an image data display program, and a recording medium for displaying a client screen on another client screen in a system including a server and a client.
[0002]
[Prior art]
When using a telephone to support a user on how to use an application on a personal computer, it is difficult to confirm how the user is operating because the user cannot actually see the operation. For example, even if one click operation is performed, the position and shape of the button to be clicked must be explained in detail. Moreover, even if it is explained in this way, it may not be correctly communicated to the user. Therefore, the user's screen is monitored by using software capable of monitoring the screen of another personal computer, and the screen of the support center is monitored by the user.
[0003]
[Problems to be solved by the invention]
However, the software can monitor the display screen of the partner computer by logging in to the partner computer, and the software must be installed on the partner computer. This is difficult when a large number of users like the above support are partners, and there is a security problem because they log in to the partner terminal.
[0004]
In view of such a problem, the present invention makes it possible to monitor the other party's screen and to allow the other party to monitor his / her own screen, and does not need to be installed in advance on a computer that monitors the screen. Another object of the present invention is to provide an image data display method, an image data display server, an image data display program, and a recording medium that can ensure security.
[0005]
[Means for Solving the Problems]
The present invention is an image data display method in which an image data display server causes an image data display server to display an image displayed on the screen of one client, and the image received from the one client. Display data to be displayed on the one or more other clients is generated from the data, the display data is relayed to the one or more other clients, and displayed on the one or more other clients. And
[0006]
In the present invention , for example, an image of a client that provides an image can be relayed to a Web browser with a client that displays an image by using a Web browser and a Web server that are generally spread. Further, since the clients are not directly connected, security is maintained.
[0007]
Further, the present invention is characterized in that the image data is obtained by capturing the screen of the one client at a predetermined interval, and the server receives the image data at a predetermined interval.
[0008]
In the present invention , an image can be displayed as a video by transmitting it at predetermined intervals. In this case, since data flowing on the network increases, the present invention is characterized in that the image data is compressed. In addition to such compression, the present invention is characterized in that the image data is difference data obtained by taking a difference from a previously captured screen.
[0009]
The present invention also provides an image data display method in which an image displayed on the screen of one client is displayed on one or more other clients in a system comprising a plurality of clients and an image data display server to which the clients are connected. The one client generates image data from its own screen, transmits the image data to the image data display server, and the image data display server receives the image data received from the one client. Generating one or more other display data for displaying the image data on the one or more other clients, and the one or more other clients relayed the display data from the image data display server based on the display data The image data is displayed.
[0010]
In the present invention , for example, an image of a client that provides an image can be relayed to a Web browser with a client that displays the image using a Web browser and a Web server that are generally spread. Since the server does not connect directly, it is possible to provide a server capable of ensuring security. Further, if it is assumed that a dedicated program is installed in advance for one client to execute the above process, the clients that execute the above process are limited. From this point of view, the present invention is characterized in that the one client downloads a program relating to generation of the image data and transmission of the image data from the server in advance as needed. A program that performs processing such as transmission of the data-related information and transmission of the data in this way is downloaded and used when connected to the server, for example, and the program disappears when communication with the server is disconnected This eliminates the need for the client to install the program in advance.
[0011]
The present invention is also an image data display server for displaying an image displayed on a screen of one client on one or more other clients, and from the image data received from the one client, the other Display data generating means for generating display data to be displayed on one or more clients, and relay means for relaying the display data to the one or more other clients.
[0012]
In the present invention , for example, a commonly used Web browser and Web server are used, so that an image of a client that provides an image is relayed to a Web browser with a client that displays the image without using dedicated software. It is possible to provide a server capable of Further, since the clients are not directly connected, security can be maintained.
[0013]
The image data display server, the client that provides the data, and the recording medium that records the program in the client that receives the data are optical information such as a CD-ROM, a flexible disk, and a magneto-optical disk (MO). Various types of recording media can be used, such as a recording medium that records information electrically, electrically, or magnetically, or a semiconductor memory that electrically records information, such as a ROM or flash memory.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0015]
In the present embodiment, in the exchange between the support center and the user, the support center and the user monitor each other's screens using a Web browser, and the following description will be given in this form.
[0016]
First, FIG. 1 shows an overall configuration of a data providing system according to an embodiment of the present invention. FIG. 1 shows a Web server (hereinafter referred to as a server) 10, a client 12 that is an HTTP client and a user terminal, and a client 14 that is a terminal of a support center, which are connected via the Internet 16. Has been.
[0017]
As shown in FIG. 2, the computer system of the server 10 and the clients 12 and 14 includes a display device 21, an input device 23, an interface device 25, a drive device 27, a main storage device 31, and an auxiliary storage device. 33 and an arithmetic processing unit 35, which are connected by a bus. Further, the storage medium 29 can be read by the drive device 27.
[0018]
Next, each device will be described. The display device 21 displays various information necessary for operating the computer. The input device 23 includes a keyboard and a mouse that are operated by the user. The interface device 25 is a network interface card (NIC) for connecting a computer and a network, or a modem or router directly connected to the Internet. The drive device 27 can read and write to a detachable recording medium, and is used, for example, when a program is read from the recording medium and installed in the auxiliary storage device 33. The arithmetic processing unit reads and executes the program stored in the auxiliary storage device 33 and uses the main storage device 31 to store the program and data.
[0019]
Returning to the description of FIG. 1, these clients 12 and 14 first log in to the server 10 and connect to each other via the server 10 when monitoring each other's screen. This log-in is first performed by the client 14 as a support center, and the client 12 as a user terminal logs in to a window opened when the client 14 logs in.
[0020]
When the client 14 connects to the server 10, the screen shown in FIG. 3 is displayed on the browser. Note that the window 18 surrounded by a dotted line at the time of connection is not displayed, but is displayed by clicking the new connection start button 20 and inputting information to be described later. The displayed window 18 is provided with three windows, each of which shows a reception window number and a connection status. The reception counter number is a number for identifying a counter. Since there are three counters, numbers from 1 to 3 are assigned. Further, there are two connection states, “waiting for connection” and “being connected”. “Waiting for connection” is a state in which the client 14 as a support center is waiting for a connection from the client 12 as a user. This is an open window from the user's perspective. “Connecting” represents a state in which one or more users are currently connected to the window 3.
[0021]
Next, the process until the client 12 and the client 14 are connected will be described with reference to the sequence diagram of FIG. First, the client 14 notifies the server 10 of the start of the new connection described above. At this time, the support center inputs the window authority password information and the requested window information. The window authority password is a password for establishing a window, and the requested window information indicates window recognition information such as the purpose of use and affiliation.
[0022]
By starting the new connection, the server 10 displays the window 18 (see FIG. 3) on the screen. With the window displayed, the client 14 can be connected from the client 12. Next, the user who operates the client 12 inputs personal information for receiving a connection permission authenticated from the support center in the connection request. This personal information is, for example, a serial number of package software purchased by the user. Then, there is a notification of a port number for connection from the server 10, and connection is made again to that port number.
[0023]
The server 10 displays the personal information received from the client 12 on the client 14 of the support center. The client 14 authenticates the personal information. If the personal information is correct, the client 10 transmits a connection permission to the server 10. When the server 10 receives the connection permission from the client 14, the server 10 notifies the client 14 of the port number, and the client 14 can connect to the client 12 by connecting to the port number. And the state of this connected window is being connected. As described above, by relaying the server 10, it is not possible to know the IP address or port number of the other party, so that security can be ensured.
[0024]
As described above, when the client 14 and the client 12 are connected, the server 10 displays the support screen 28 shown in FIG. 5 on the browser of each of the client 14 and the client 12. The support screen 28 is provided with a self-screen display area 22 on the left side and a partner screen display area 24 on the right side. A self-screen transmission button 26, a transmission stop button 30, a disconnect button 32, and a display area 34 are provided on the screen. The self-screen transmission button 26 is a button for starting transmission of the self-screen to the partner, the transmission stop button 30 is a button for canceling transmission of the self-screen to the partner, and the disconnect button is a connection with the partner. A button to disconnect. The display area 34 is an area indicating whether the screen displayed on the left and right screens is the own screen, the other party's screen, or the like. In this case, the own screen is on the left and the other screen is on the right. Since it is displayed, “self-phase” is displayed.
[0025]
Next, how it is displayed on the support screen 28 shown in FIG. 5 will be described with reference to FIG. FIG. 6 shows a self-terminal screen 36 and a partner terminal screen 38, and the support screen 28 shown in FIG. 5 is displayed on the self-terminal screen 36. In addition, an AP screen 40 displayed by the application is displayed on the own terminal, and an AP screen 42 displayed by the same application is also displayed on the other terminals. These AP screens 40 and 42 are displayed on the support screen 28 in the own screen display area 22 and the partner screen display area 24, respectively. The support screen 28 also displays mouse pointers 44 and 46 that are operated with a mouse as an input device. Although not shown in FIG. 6, the support screen 28 can also be displayed on the partner terminal screen 38. If it is preferable that each other can monitor the support screen 28, such as when supporting, the partner terminal screen 38 is also displayed. Display as necessary, such as displaying.
[0026]
Next, details of the display function will be described. First, in order to display the AP screens 40 and 42 on the support screen 28, a capture function for capturing these screens at regular intervals will be described. The capture function is downloaded as a web browser component when the support screen 28 is downloaded. Then, the capture function performs processing of capturing an image of a screen to be displayed on the support screen 28 and capturing a mouse pointer, and transmitting the captured image to the server. The target to be captured at this time is not limited to a part of the screen but may be the entire screen.
[0027]
The screen image captured by the above processing and the position of the mouse pointer are transmitted as an image obtained by superimposing the screen image and the mouse pointer. The image data may be transmitted as data itself, or the image data may be You may send it after making it into a file. In either case, the data size of the image data is large, so the display size of the image to be captured can be reduced, the data size can be compressed using a known compression method, and the color used in the image can be reduced. The image data size is reduced by reducing the number. In addition, when explaining operations to the user support in the user support, the movement of the mouse is mostly performed and the change in the overall image is not so large. As shown in FIG. It is also possible to divide the cell into cells of the size and transmit only the cell in which the image has changed. More specifically, when the initial client screen is the screen 47 and the next captured screen is the screen 49, only the difference cell 48 is transmitted. By doing so, it is possible to reduce the data size to be transmitted. In this case, in order to ensure the consistency of the display screen, the screen 50 of the entire capture target screen is transmitted at regular intervals. Similarly, when the entire screen is updated, the screen data of the entire screen is transmitted.
[0028]
Next, the flow of image data obtained by the above processing will be described with reference to FIG. FIG. 9 shows the Web server 10, the client 14, and the screen 36 of the client 14. Data flow until the AP screen 40 of the client 14 is displayed on the support screen 28 via the server 10. FIG. FIG. 9 also shows a client screen capture component 52, a display data creation module 54, and a browser 56. Among these, the client screen capture component 52 is a component that has existed since the support screen 28 was downloaded, and is automatically downloaded from the server 10.
[0029]
The client screen capture component 52 captures a target screen to be captured at regular intervals, and converts the obtained image data into image data itself or a file and uploads it to the server 10 at regular intervals. More specifically, for example, a screen is captured at an interval of 20 times per second, and two pieces of image data are transmitted to the server 10 at an interval of 10 times per second. By doing so, it is possible to transmit the target screen to the server 10 as if it were an image captured by a video camera.
[0030]
In addition, the client screen capture component 52 can set an operating environment. The operating environment includes a screen capture interval (for example, the number of frames per second), screen data, or a screen data file. The upload interval to the server 10, the accuracy (resolution) of capturing the screen, the color reduction of the capture screen, the color correction such as the gray scale, and the like. Such a capture function is created using an API (Application Program Interface) for calling each function prepared by an OS (Operating System).
[0031]
Next, the display data creation module 54 will be described. The display data creation module 54 is a module arranged on the server 10 and creates display data that can be displayed on a browser from image data uploaded from a client. At this time, in order not to impair the real-time property of the displayed image data, it is synchronized with the interval at which the client uploads the image data.
[0032]
The browser 56 is provided with a component for rewriting the support screen 28 in real time together with the download of the support screen 28 in order to always display the page rewritten by the display data creation module 54.
[0033]
Next, the data flow will be described in the order of (1) to (7) in the figure. (1) shows the capture process of the AP screen to be image-captured. At this time, the data size is compressed. (2) shows the conversion of the screen data captured in (1) into a file. In this case, as described above, the screen data may be transmitted to the server 10 as it is without being made into a file. (3) indicates uploading of an image data file or image data to the server 10. (4) indicates that display data to be displayed on the browser is generated from the screen data received from the client 14, and the display data creation module reads the screen data and generates display data. The interval for generating the display data is made equal to the interval for capturing the AP screen 40. (5) shows the incorporation of the generated display data into HTML. (6) shows the update of the display data incorporated in the HTML. The browser of the client 14 recognizes that the currently displayed page has been updated by the server 10, and reads the updated display data. (7) indicates that the display data is displayed on the support screen 28.
[0034]
As described above, according to the present embodiment, a module for capturing a screen can be downloaded to the browser as a component by using a commonly used Web browser or Web server. The other party's screen can be displayed on his / her own screen without using the software.
[0035]
In addition, if a client has a function that does not work with a standard web browser, the client can access the counter if it has a module that performs a series of operations such as downloading the function from the Internet and installing the function. At that time, a dedicated application can be installed, and applications with improved operability and functionality can be used. In this case, the client to be provided is executed in normal HTML until connected to the window, and the client screen capture application downloaded by the previous module is started when the capture condition and the connection condition are finally determined. This application connects to the relay application of the Web server, and captures and transmits the screen.
[0036]
In the Web server, connection management with the client is performed by a program operating on the virtual machine of the Web server. In order to execute a plurality of connection managements simultaneously, a relay process is performed with a thread that is the minimum unit of processing when the OS divides a process of one application into a plurality of processes. Thereby, even when a plurality of connections are made, the dispatcher can perform processing in parallel. In addition, a client that displays an image displays an image with an application installed by downloading.
[0037]
In this embodiment, an image is provided from a providing client to a client to be displayed. However, since the two clients can both start a Web browser, it goes without saying that images can be provided and received from each other. . In addition, there may be a plurality of clients that provide image data and clients that receive the provision. It is also possible to send and receive audio while monitoring each other's images by using the VoIP technology or the like.
[0038]
In the embodiment described above and FIG. 9, the display data creation module 54 corresponds to the display data generation means and the display data generation procedure, and (6) corresponds to the relay means and the relay procedure.
[0039]
【The invention's effect】
As described above, according to the present invention, it is possible to monitor the other party's screen and to allow the other party to monitor his / her own screen, and it is not necessary to install in advance on the computer that monitors the screen. An image data display method, an image data display server, an image data display program, and a recording medium that can ensure security are obtained.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of a screen display system according to an embodiment of the present invention.
FIG. 2 is a hardware block diagram of a Web server and a client.
FIG. 3 is a diagram showing a connection screen.
FIG. 4 is a client connection sequence diagram.
FIG. 5 is a diagram showing a support screen.
FIG. 6 is a display image diagram of a client.
FIG. 7 is a diagram illustrating transmission of difference data.
FIG. 8 is a diagram illustrating transmission of difference data and entire image data.
FIG. 9 is a diagram showing a functional configuration in the present embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Web server 12, 14 ... Client 16 ... Internet 18 ... Window 20 ... New connection start button 22 ... Self-screen display area 24 ... Other party screen display area 26 ... Self-screen transmission button 28 ... Support screen 30 ... Transmission stop button 32 ... Disconnect button 34 ... display area 36 ... own terminal screen 38 ... partner terminal screen 40, 42 ... AP screen 44, 46 ... mouse pointer 47, 49, 50 ... screen 48 ... difference 52 ... client screen capture component 54 ... display data creation module 56 ... Browser

Claims (8)

An image data display method in which an image data display server displays an image displayed on the screen of one client on one or more other clients,
From images data received from said one client generates display data to be displayed on said other one or more clients, and transmitting the display data to the other one or more clients, the other 1 Display to more than one client ,
2. The image data display method according to claim 1, wherein the image data is obtained by capturing the screen of the one client at a predetermined interval, and the server receives the image data at a predetermined interval . The image data, the image data display method according to claim 1, characterized in that it is compressed. The image data display method according to claim 1 , wherein the image data is difference data obtained by taking a difference from a previously captured screen. An image data display method in which an image displayed on a screen of one client is displayed on one or more other clients in a system including a plurality of clients and an image data display server to which the clients are connected.
The one client generates image data from its own screen,
Sending the image data to the image data display server;
The image data display server generates display data for displaying the image data on the one or more other clients from the image data received from the one client;
One or more other clients to which the display data is transmitted from the image data display server display the image data based on the display data ,
The image data is obtained by capturing the screen of the one client at a predetermined interval, and the image data display server receives the image data at a predetermined interval. Method. The image data display method according to claim 4 , wherein the one client downloads a program related to generation of the image data and transmission of the image data in advance from the server as needed. An image data display server that displays an image displayed on the screen of one client on one or more other clients,
From images data received from said one client, and display data generating means for generating display data to be displayed on said other one or more clients,
Have a transmitting means for transmitting the display data to the other one or more clients,
The image data display server , wherein the image data is obtained by capturing the screen of the one client at a predetermined interval, and receives the image data at a predetermined interval . The computer, from the image picture data received from a single client, a display data generation step for generating display data to be displayed on the one or more other clients, the display data to said one or more other clients an image data display program for executing a transmission step of transmitting,
The image data display program , wherein the one client is obtained by capturing the screen of the client at a predetermined interval, and causes the image data to be received at a predetermined interval . A computer-readable recording medium on which the image data display program according to claim 7 is recorded.

Images