JP2003108504A - Image data display method, image data display server, image data display program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image data display method, an image data display server, an image data display program, and a recording medium for monitoring a mating image screen, allowing a mate to monitor one's own image screen, requiring no preinstallation in a computer for monitoring the image screen, and enabling the provision of security. SOLUTION: This image data display method allows the image data display server 10 to display an image displayed on the image screen 36 of one client on one or more other clients, forms display data to be displayed on one or more other clients from image data 36 received from the one client, relays the display data to one or more other clients, and displays the data on one or more clients.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data display method, an image data display server, an image data display program and a record for displaying the screen of a client on the screen of another client in a system comprising a server and a client. Regarding the medium.

[0002]

2. Description of the Related Art It is difficult to confirm how a user operates because he / she cannot actually see the user's operation when he / she supports the user by using a method of using an application of a personal computer. Is.
For example, even if only one click operation is performed, the position and shape of the click button must be explained in detail. Moreover, even if the explanation is made in this way, it may not be properly transmitted to the user. Therefore, using software that can monitor the screen of other personal computers,
In addition to monitoring the user's screen, the user was required to monitor the screen of the support center.

[0003]

However, the software described above can monitor the display screen of the partner's personal computer by logging in to the partner's personal computer, and if the software is not installed in the partner's personal computer. I won't. This is difficult for a large number of users as in the case of the above support, and also has a security problem because the user logs in to the terminal of the other party.

In view of the above problems, the present invention allows the screen of the other party to be monitored and allows the other party to monitor its own screen, and the computer for monitoring the screen is installed in advance. Image data display method that does not require security and can ensure security,
An object is to provide an image data display server, an image data display program, and a recording medium.

[0005]

According to a first aspect of the present invention, an image data display server causes an image data display server to display an image displayed on the screen of one client to one or more other clients. A method comprising:
Display data to be displayed on the other one or more clients is generated from the image data received from one client, the display data is relayed to the one or more other clients, and the display data is relayed to the one or more other clients. The feature is that it is displayed on the client.

According to the first aspect of the present invention, for example, a commonly used Web browser and Web server are used,
The image of the client that provides the image can be relayed to the Web browser with the client that displays the image.
Further, the clients are not directly connected to each other, so that the security is maintained.

According to a second aspect of the present invention, the image data is obtained by the one client capturing its own screen at predetermined intervals, and the server receives the image data at predetermined intervals. It is characterized by doing.

According to the second aspect, the image can be displayed as a video by transmitting the image at a predetermined interval. In this case, since the amount of data flowing on the network increases, the invention according to claim 3 is characterized in that the image data is compressed. In addition to such compression, in the invention according to claim 4, the image data is
It is characterized in that it is difference data that is the difference from the screen captured previously.

According to a fifth aspect of the present invention, in a system including a plurality of clients and an image data display server to which the clients are connected, an image displayed on the screen of one client is displayed on one or more other clients. In the image data display method, the one client generates image data from its own screen, transmits the image data to the image data display server, and the image data display server causes the one client to display the image data. One or more other clients that generate display data for causing the other one or more clients to display the image data from the image data received from the client, and have the display data relayed from the image data display server. However, the image data is displayed based on the display data.

In a fifth aspect of the present invention, for example, a commonly used Web browser and Web server are used,
The image of the client that provides the image can be relayed to the Web browser with the client that displays the image, and the clients are not directly connected to each other. Therefore, it is necessary to provide a server capable of ensuring security. You can Further, if one client preliminarily installs a dedicated program in order to execute the above processing, the clients that execute the above processing are limited. From this point of view, claim 6
The invention described in (1) is characterized in that the one client downloads a program relating to the generation of the image data and the transmission of the image data from the server in advance as needed. In this way, a program that performs processing such as transmission of the data-related information and transmission of the data is downloaded and used when connecting to the server, and the program disappears when communication with the server is disconnected. Therefore, the client does not need to install the above program in advance.

According to a seventh aspect of the present invention, there is provided an image data display server which causes an image displayed on the screen of one client to be displayed on one or more other clients.
Display data generation means for generating display data to be displayed on the one or more other clients from the image data received from the one client, and relay for relaying the display data to the one or more other clients. And means.

According to the present invention, for example, since a commonly used Web browser and Web server are used, the image of the client that provides the image and the client that displays the image can be used without using dedicated software. We
It is possible to provide a server capable of relaying to a browser. Further, since the clients are not directly connected to each other, security can be maintained.

The image data display server, the client that provides the data, and the recording medium that records the program in the client that receives the data are a CD.
-ROM, flexible disk, magneto-optical disk (M
Various types of recording media such as a recording medium for optically, electrically or magnetically recording information like O) or a semiconductor memory for electrically recording information like a ROM or a flash memory are used. be able to.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

In the present embodiment, in the exchange between the support center and the user, the support center and the user
This is a mode in which the screens of both are monitored using a Web browser, and the following description will be given in this mode.

First, FIG. 1 shows the overall configuration of a data providing system according to an embodiment of the present invention. In FIG. 1, W
An eb server (hereinafter referred to as a server) 10 and a client 1 that is an HTTP client and a user terminal
2 and a client 14 that is a terminal of the support center are shown, and they are connected by the Internet 16.

These server 10 and clients 12, 1
The computer system of No. 4, as shown in FIG.
It has a display device 21, an input device 23, an interface device 25, a drive device 27, a main storage device 31, an auxiliary storage device 33, and an arithmetic processing device 35, which are connected by a bus. Further, the storage medium 29 can be read by the drive device 27.

Next, each device will be described. Display device 21
Displays various information necessary for operating the computer. The input device 23 includes a keyboard and a mouse operated by the user. Interface device 25
Is an NI for connecting a computer to a network
C (Network Interface Card)
Alternatively, it is a modem or router directly connected to the Internet. The drive device 27 can read and write a removable recording medium, and is used, for example, when a program is read from the recording medium and installed in the auxiliary storage device 33. The arithmetic processing unit reads and executes the program stored in the auxiliary storage device 33, and uses the main storage device 31 to store the program, data, and the like.

Returning to the description of FIG. 1, these clients 12 and 14 first log in to the server 10 and monitor each other's screens, and connect to each other via the server 10. This log-in is first performed by the client 14 which is the support center, and the client 12 which is the user's terminal logs in to the window opened when the client 14 logs in.

When the client 14 connects to the server 10, the screen shown in FIG. 3 is displayed on the browser. The window 18 surrounded by a dotted line is not displayed at the time of connection, but is displayed by clicking the new connection start button 20 and inputting information described below. The displayed counter 18 has three counters in it.
The reception counter number and connection status are shown respectively. This reception window number is a number that identifies the window, and since there are three windows, numbers 1 to 3 are given. Further, there are two connection states, "waiting for connection" and "connecting", and "waiting for connection" is a state in which the client 14 which is a support center is waiting for a connection from the client 12 which is a user. It is an open window from the user's point of view. “Connecting” represents a state in which one or more users are currently connected to the window 3.

Next, the client 12 and the client 1
4 will be described with reference to the sequence diagram of FIG. First, the client 14 notifies the server 10 of the above-described new connection start. At this time, the support center inputs the contact authority password information and the requested contact information. The window authority password is a password for opening a window, and the window information requested is the window recognition information such as the purpose of use and affiliation.

With the start of this new connection, the server 10
The window 18 (see FIG. 3) is displayed on the screen. With this window displayed, the client 14
Will be able to be connected from. Next, the user operating the client 12 inputs the personal information for being authenticated by the support center and receiving the connection permission in the connection request. This personal information is, for example, the serial number of the packaged software purchased by the user. Then, the server 10 notifies the server of the port number for connection, and connects again to the port number.

The server 10 displays the personal information received from the client 12 on the client 14 of the support center, the client 14 authenticates the personal information, and if the personal information is correct, the connection permission is transmitted to the server 10. . Upon receiving the connection permission from the client 14, the server 10 notifies the client 14 of the port number,
The client 14 can connect to the client 12 by connecting to the port number. Then, the state of the connected window becomes the connected state. By relaying the server 10 in this way, the IP address and port number of the other party are not known, so that security can be ensured.

When the client 14 and the client 12 are connected as described above, the server 10 displays the support screen 28 shown in FIG. 5 on the browser of each of the client 14 and the client 12. Support screen 2
8, the self-screen display area 22 is provided on the left side and the partner screen display area 24 is provided on the right side. A self-screen transmission button 26, a transmission stop button 30, a disconnection button 32, and a display area 34 are provided on the screen. The self-view transmission button 26 is a button for starting to send the self-view to the other party, the transmission stop button 30 is a button for canceling the transmission of the self-view to the other party, and the disconnection button is for connection with the other party. It is a button to disconnect. In the display area 34, whether the screens displayed on the left and right screens are their own screens,
In the area that indicates whether it is the screen of the other party, in this case, the self screen is displayed on the left and the partner screen is displayed on the right.
Phase "is displayed.

Next, how the support screen 28 shown in FIG. 5 is displayed will be described with reference to FIG. FIG. 6 shows the own terminal screen 36 and the partner terminal screen 38, and the own terminal screen 36 includes the support screen 2 shown in FIG.
8 is displayed. Further, the AP screen 40 displayed by the application is displayed on the own terminal, and the AP screen 42 displayed by the same application is displayed on the other terminal. These AP screens 40 and 42 are displayed on the support screen 28 in the own screen display area 22 and the partner screen display area 24, respectively. Also, this support screen 28
The mouse pointers 44 and 46 operated by a mouse as an input device are also displayed on the screen. Although not shown in FIG. 6, the support screen 28 can also be displayed on the partner terminal screen 38, and can be displayed on the partner terminal screen 38 when it is preferable to monitor each other, such as when supporting. Display as needed, such as by displaying.

Next, the details of the display function will be described. First of all, the AP screens 40 and 42 are supported by the support screen 2
A description will be given of a capture function for capturing those screens at regular intervals in order to display them on the screen of FIG. The capture function is downloaded as a component of the WEB browser when the support screen 28 is downloaded. Then, the capture function performs a process of capturing an image of a screen to be displayed on the support screen 28, capturing a mouse pointer, and transmitting the image to the server. The target to be captured at this time is not limited to a part of the screen, and may be the entire screen.

The screen image and the position of the mouse pointer captured by the above process are transmitted as an image in which the screen image and the mouse pointer are superimposed, but the image data may be the data itself. The image data may be converted into a file and then transmitted. In either case, since the data size of the image data is large, the size on the display of the image to be captured is reduced, the data size is compressed using a well-known compression method, and the color used in the image is reduced. The image data size is reduced by reducing the number. Further, in the user support, when explaining the operation to the user, the movement of the mouse is often large, and the change in the overall image is not so large. Therefore, as shown in FIG. It is also possible to divide the cells into cells of a size and transmit only the cells in which the image has changed. Specifically, with reference to FIG. 8, when the initial client screen is the screen 47 and the next captured screen is the screen 49, only the difference cell 48 is transmitted. By doing so, the size of data to be transmitted can be reduced. In this case, in order to guarantee the consistency of the display screen, the screen 50 of the entire capture target screen is transmitted at regular intervals. Also, when the entire screen is updated, the screen data of the entire screen is similarly transmitted.

Next, the flow of image data obtained by the above processing will be described with reference to FIG. FIG. 9 shows the Web
Server 10, client 14, client 14
Screen 36 of the client 14 is shown.
FIG. 6 is a diagram showing a data flow until the screen 40 is displayed on the support screen 28 via the server 10. Also,
FIG. 9 shows a client screen capture component 52, a display data creation module 54, and a browser 5.
6 is also shown. Of these, the client screen capture component 52 is a component that has existed since the support screen 28 was downloaded, and is automatically downloaded from the server 10.

The client screen capture component 52 captures the target screen to be captured at regular intervals, converts the obtained image data into image data itself or a file, and uploads it to the server 10 at regular intervals. More specifically, for example, 1
The screen is captured at intervals of 20 times per second, and two pieces of image data are transmitted to the server 10 at intervals of 10 times per second. By doing so, the target screen can be transmitted to the server 10 as if it were an image taken by a video camera.

The client screen capture component 52 can set an operating environment, and the operating environment includes the screen capturing interval (for example, the number of screen frames per second), screen data or The interval of uploading the screen data file to the server 10, the accuracy (resolution) of capturing the screen, the color reduction of the capture screen, and the color correction such as gray scale are included. Such a capture function is provided by the OS (Operat
API for calling each function prepared by
Created using (Application Program Interface).

Next, the display data creation module 54 will be described. The display data creation module 54 is the server 1
In the module arranged on 0, the display data that can be displayed on the browser is created from the image data uploaded from the client. At this time, in order not to impair the real-time property of the displayed image data, it is synchronized with the interval at which the client uploads the image data.

In the browser 56, a component for rewriting the support screen 28 in real time is provided together with the download of the support screen 28 in order to constantly display the page rewritten by the display data creation module 54.

Next, the data flow will be described in the order of (1) to (7) in the figure. (1) shows the capture process of the AP screen of the image capture target,
At this time, the data size is compressed. (2) is
The conversion of the screen data captured in (1) into a file is shown. In this case, as described above, the screen data may be directly transmitted to the server 10 without being made into a file.
(3) shows uploading of an image data file or image data to the server 10. (4) indicates that the display data to be displayed on the browser is generated from the screen data received from the client 14, and the display data creation module reads the screen data and generates the display data. The interval for generating this display data is A
It is equal to the interval at which the P screen 40 is captured. (5)
Shows the incorporation of the generated display data into HTML. (6) shows the update of the display data incorporated in HTML. The browser of the client 14 recognizes that the currently displayed page has been updated by the server 10, and reads the updated display data.
Then, (7) displays the display data on the support screen 28.
Is displayed.

As described above, according to this embodiment, a module for capturing a screen can be downloaded as a component into a browser by using a generally popular Web browser or Web server. Therefore, it is possible to display the screen of the other party on the own screen without using dedicated software.

Further, when the client has a function that does not operate with a standard Web browser, it is equipped with a module for performing a series of operations of downloading the function from the Internet and installing the function. When you access the window, you can use an application that has a special application built in and improved operability and functionality. In this case, the client to be provided executes the normal HTML until connecting to the window, and when the capture condition and the connection condition are finally determined, the client screen capture application downloaded by the previous module is activated. Then, this application connects to the relay application of the Web server and captures and transmits the screen.

Further, in the Web server, connection management with the client is performed by a program operating on the virtual machine of the Web server. Then, in order to execute a plurality of connection managements at the same time, the relay process is performed by the thread which is the minimum unit of the process when the OS divides the process of one application into a plurality of processes. As a result, even when a plurality of connections are made, the dispatcher can perform processing in parallel. Also, the client that displays the image is
Display the image with the application that is embedded in the download.

In the present embodiment, the image is provided from the providing client to the displaying client.
It is needless to say that the two clients can both start up the web browser and thus provide and receive the images with each other. Further, there may be a plurality of clients providing image data and a plurality of clients receiving the image data. Further, by using the VoIP technology or the like, it is possible to transmit and receive voice while monitoring mutual images between the clients.

In the embodiment described above and in FIG.
The display data creation module 54 corresponds to the display data generation means and the display data generation procedure, and (6) corresponds to the relay means and the relay procedure.

[0039]

As described above, according to the present invention,
An image data display method that enables monitoring of the screen of the other party and allows the other party to monitor its own screen, and does not need to be installed in advance on the computer that monitors the screen, and can also ensure security, An image data display server, an image data display program and a recording medium can be obtained.

[Brief description of drawings]

FIG. 1 is an overall configuration diagram of a screen display system according to an embodiment of the present invention.

FIG. 2 is a hardware block diagram of a Web server and a client.

FIG. 3 is a diagram showing a connection screen.

FIG. 4 is a client connection sequence diagram.

FIG. 5 is a diagram showing a support screen.

FIG. 6 is a display image diagram of a client.

FIG. 7 is a diagram showing transmission of difference data.

FIG. 8 is a diagram showing transmission of difference data and entire image data.

FIG. 9 is a diagram showing a functional configuration according to the present embodiment.

[Explanation of symbols]

10 ... Web server 12, 14 ... Client 16 ... Internet 18 ... window 20 ... New connection start button 22 ... Self-view display area 24 ... Display area of partner screen 26 ... Self-portrait send button 28 ... Support screen 30 ... Transmission stop button 32 ... Disconnect button 34 ... Display area 36 ... Own terminal screen 38 ... partner terminal screen 40, 42 ... AP screen 44, 46 ... Mouse pointer 47, 49, 50 ... screen 48 ... difference 52 ... Client screen capture component 54 ... Display data creation module 56 ... Browser

Claims (9)

[Claims] 1. An image data display method for causing an image data display server to display an image displayed on the screen of one client to one or more other clients, wherein the image data display server receives the image from the one client. Generating display data to be displayed on the one or more other clients from image data, relaying the display data to the one or more other clients, and displaying the display data on the one or more other clients. Characteristic image data display method. 2. The image data is obtained by the one client capturing its own screen at a predetermined interval, and the server receives the image data at a predetermined interval. Item 1. The image data display method according to item 1. 3. The image data display method according to claim 1, wherein the image data is compressed. 4. The image data display method according to claim 2, wherein the image data is difference data obtained by taking a difference from a previously captured screen. 5. An image data display method for displaying an image displayed on the screen of one client on one or more other clients in a system including a plurality of clients and an image data display server to which the clients are connected. Where the one client generates image data from its own screen, transmits the image data to the image data display server, and the image data display server receives the image data from the one client. Generate display data for causing the other one or more clients to display the image data, and the other one or more clients relayed the display data from the image data display server, based on the display data. A method for displaying image data, characterized in that the image data is displayed. 6. The image data display according to claim 5, wherein the one client downloads a program relating to the generation of the image data and the transmission of the image data from the server in advance as needed. Method. 7. An image data display server for displaying an image displayed on the screen of one client to one or more other clients, the image data display server including the image data received from the one client, An image data display server, comprising: a display data generation unit that generates display data to be displayed on one or more clients of 1; and a relay unit that relays the display data to the one or more other clients. 8. A computer, from the image data received from one client,
An image data display program for executing a display data generation procedure for generating display data to be displayed on the one or more other clients, and a relay procedure for relaying the display data to the one or more other clients. 9. A computer, from the image data received from one client,
An image data display program for executing a display data generation procedure for generating display data to be displayed on the other one or more clients, and a relay procedure for relaying the display data to the one or more other clients. The recorded computer-readable recording medium.