JP4390925B2 - Remote screen sharing system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a remote screen sharing system in which two terminals connected by a communication network share a screen with each other to exchange information.
[0002]
[Prior art]
In a video conference system, a screen called a whiteboard is shared between a transmitting terminal and a receiving terminal, and various information is written on the screen to facilitate communication. Such a screen sharing system is called a remote screen sharing system in the present invention. In this type of system, for example, when text or graphic information is written on a shared screen at a transmitting terminal, the contents of the screen are transferred and displayed on the receiving side. When new character or graphic information is written on the receiving side, this is transferred to the transmitting side and the same screen is displayed on the transmitting side.
[0003]
[Problems to be solved by the invention]
By the way, the conventional techniques as described above have the following problems to be solved.
Remote screen sharing systems as well as video conference systems can be widely used in various fields. For example, an operator in charge of data input operates one terminal, and an assistant operates the other terminal. When the operator in charge of data input various information on the shared screen, the assistant monitors the contents and gives advice if necessary. By using such a system, it is possible for an assistant to give detailed advice on the operation of a terminal in a remote place.
[0004]
However, in this type of system, when the data input operator inputs data using the kana-kanji conversion function, when the assistant finds an error in the selected kanji, the assistant prompts the operator to input again. Or, it is necessary to show the correct kanji on the assistant side and communicate this to the operator in charge of data input.
[0005]
However, when a kanji selection error is communicated to the other party by voice, the meaning may not be transmitted accurately, and misunderstandings may occur. If the assistant shows the correct kanji on the shared screen, the data entry operator can correctly recognize the kanji, but the data entry operator refers to the kanji entered by the assistant and displays the kanji. The complicated work of inputting to the terminal was necessary.
[0006]
[Means for Solving the Problems]
The present invention adopts the following configuration in order to solve the above points.
<Configuration 1>
The first terminal and the second terminal are connected to each other via a communication network, and the first terminal converts an input character input by the first operator to convert an unconfirmed conversion character. A character conversion confirming means for generating the converted character as a confirmed character by confirming input, a window control unit for displaying the converted character and the confirmed character on a display, and the confirmed character as image data or a character code. Remote screen sharing including another window control unit that displays the determined character in common on the display based on the received image data or character code. In the system, the screen sharing control unit transmits the unconfirmed converted character as image data or a character code to the second terminal. The other window control unit of the second terminal is a remote screen sharing system for displaying the converted character on the display to confirm the correctness based on the received image data or character code. The conversion confirmation unit generates candidate characters to be converted from the input characters, the window control unit displays a list table including the candidate characters on the display, and the screen sharing control unit captures the list table as image data. The other window control unit of the second terminal transmits the list table based on the received image data and a virtual keyboard on which candidate characters can be selected to be displayed on the display, and transmits the second terminal to the second terminal. terminals, on a selection signal indicating the candidate characters selected by the second operator the virtual keyboard A selection control unit for transmitting to the first terminal is further provided, and the character conversion confirming means of the first terminal sets the corresponding candidate character as a confirmed character when receiving the selection signal. Remote screen sharing system.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described using specific examples.
<Specific example 1>
FIG. 1 is a block diagram showing a specific example of a remote screen sharing system according to the present invention.
The first terminal 1 and the second terminal 2 are connected to each other via a communication network 3. Each of the first terminal 1 and the second terminal 2 includes a general personal computer, a workstation, or the like. A window 6 is displayed on the display 4 of the first terminal 1. A window 7 is displayed on the display 5 of the second terminal 2. The screen sharing control unit 8 and the window control unit 9 provided in the first terminal 1 control to match the contents of the window 6 displayed on the first terminal 1 and the window 7 displayed on the second terminal 2. To implement a so-called remote screen sharing system.
[0010]
Since the hardware configuration of each terminal is the same as that of a well-known personal computer or the like, the description thereof is omitted, but the screen sharing control unit 8 is configured by an application program that operates on the first terminal 1. The window control unit 9 is also configured by a program for controlling window display in cooperation with the operating system of the first terminal 1 or the like.
For the communication network 3, various types such as a LAN (local area network), a public line, a dedicated line, and the Internet can be used.
[0011]
Next, the operation of the screen sharing control unit 8 will be described more specifically. A program for transmitting a window displayed on the first terminal 1 to the second terminal 2 as it is and displaying it is widely known. This program copies the image of the window 6 displayed on the first terminal 1 into a data file, compresses it, and transmits it to the second terminal 2 through the communication network 3. In the second terminal 2, the received image of the window 6 is displayed on the display 5 as a window 7. Various timings for transmitting the image of the window 6 displayed on the first terminal 1 to the second terminal 2 can be considered. Here, for example, the following method is adopted.
[0012]
First, when operating the first terminal 1, when the keyboard is pressed and an OnKeyUp event is detected, the character code input in the field for storing the characters to be displayed on the window 6 at that timing is used as the second terminal. 2 to send. That is, in this case, the screen is shared by transferring the character code every time a character is input. Also, there is a method in which a timer for starting a character transfer program is set and all character codes input in the field are transferred to the partner terminal at a fixed time. In the latter case, the second terminal performs processing such as replacing all character codes in the field.
[0013]
In addition, there is a method for transferring image data of the entire window. This method is the simplest for matching the displayed contents of the window. This is because the display content of the window may change in addition to the input of characters. In this case, for example, an OnKeyUp event is detected, image data of the window is copied at that timing, and this is transferred to the other party. In this way, the contents of the windows displayed on the first terminal and the second terminal are matched with each key input. Of course, this image data transfer method can also be a method of automatically transferring the screen image data to the partner terminal at regular time intervals.
[0014]
FIG. 2 is an operation explanatory diagram of the screen sharing control unit in the present invention.
This example shows a case where, for example, the image of the window is transferred from the first terminal to the second terminal. First, in step S1, it is assumed that the window 6 is displayed on the first terminal while nothing is displayed on the second terminal. This window 6 is copied and transferred to the second terminal (step S1). In the next step S2, the image is displayed as it is in the window 7 of the second terminal.
[0015]
Here, it is assumed that one character “ko” is input at the first terminal in step S2. In this case, the window 6 of the first terminal is copied as it is and transferred to the second terminal. Thereby, in step S3, a window including the character “ko” is displayed. As can be seen from this figure, in the first terminal, unconfirmed characters for kana-kanji conversion are input in the name column of the window 6. A broken underline is a display indicating an unconfirmed character. Since the content of the image transferred to the second terminal is fixed, the broken underline is not displayed.
[0016]
Through the processes in steps S2, S3, and S4 in the figure, the operator operating the first terminal inputs the characters “konno”, and then converts these characters into kanji in step S5. Conventionally, the screen displayed on the first terminal 1 in step S1 is transferred to the second terminal, and then the operator operating the first terminal ends the kana-kanji conversion, and the window 6 shown in step S5 is displayed. After being displayed, the contents are transferred to the second terminal.
[0017]
However, in the present invention, unconfirmed characters are sequentially transmitted to the second terminal as they are. Even in the unconfirmed state, the image of the window 6 of the first terminal is copied exactly and transmitted to the second terminal, and the content as it is is displayed in the window 7 of the second terminal.
[0018]
In this way, in the second terminal, the process of executing the kana-kanji conversion process in the first terminal can be clearly read between step S2 and step S6. Therefore, for example, when an operator operating the first terminal makes a mistake in the kana-kanji conversion operation, the operator using the second terminal at the step S6 asks the operator who uses the first terminal Notify that the kanji is different. The operator who operates the first terminal can select another kanji and display it again because the converted kanji is not yet determined in step S6. Therefore, the operator who operates the first terminal can select the correct Chinese character again, and can quickly and efficiently select and confirm the Chinese character.
[0019]
That is, since the operator operating the second terminal can find a kanji selection mistake before the kana-kanji conversion is confirmed, the operator operating the first terminal only has to re-convert the kanji, The trouble of repeating the processes in steps S2 to S5 in FIG. 2 is eliminated.
[0020]
FIG. 3 is a flowchart of the character conversion operation.
The above operations will be specifically described in order with reference to FIG. First, in step S1, character conversion input in the shared window is started on the first terminal. That is, the operator who operates the first terminal starts the kana-kanji conversion program. Next, in step S2, “konno” is input. Here, in step S3, the screen sharing control unit 8 shown in FIG. 1 acquires the image data of the window, and transmits it to the second terminal in step S4. On the second terminal 2, the image is displayed. In this example, image data is transferred after the input of three characters, not one character at a time. However, like the description using FIG. 2, these are still undefined characters.
[0021]
Next, in step S5, character conversion is performed at the first terminal. Here, first, for example, “konno” is first converted to “konno”. Here, the screen sharing control unit 8 of the first terminal 1 acquires the image data of the window and transmits it to the second terminal 2 (steps S6 and S7). As a result, the second terminal 2 displays “Konno” after conversion, which is first selected. Here, the operator who operates the second terminal 2 notifies the operator who operates the first terminal 1 by voice or the like whether or not the converted kanji is correct. If it is wrong, the process proceeds from step S8 to step S5 to perform reconversion. That is, this time, another kanji is selected and displayed on the window 6 of the first terminal 1 again.
[0022]
In steps S6 and S7, the contents are transferred to the second terminal 2 and displayed as before. Here, if the operator of the second terminal approves the contents of the kanji, the process proceeds from step S8 to step S9, and the operator of the first terminal 1 determines the converted character.
[0023]
<Effect of specific example 1>
As described above, in this specific example, in step S5 of FIG. 2, the character “Ikono” displayed in the window 6 of the first terminal is unconfirmed and is displayed in the window 7 of the second terminal. Since it is displayed, when the selected Chinese character is incorrect, the operator of the first terminal can select another character as it is, and there is no need to re-input from the hiragana. Since the operator of the first terminal and the operator of the second terminal can visually monitor the process of character conversion input and determine the converted character, both operations are unnecessary, Efficient work can be performed.
[0024]
In the example shown in FIG. 2, all the processes for inputting the kana are also displayed on the second terminal, and in the example of FIG. 3, the hiragana before conversion is displayed on the second terminal. The character may be displayed on the second terminal.
In the above example, the example in which information such as an image to be shared unilaterally is transmitted from the first terminal 1 to the second terminal 2 has been described, but the same applies to the second terminal 2 side. A function is provided, and the screen may be shared by transmitting data in both directions.
[0025]
<Specific example 2>
In the above example, when an undetermined character for character conversion input is displayed on the first terminal, it is displayed on the second terminal. In the following specific example 2, when the first terminal performs character conversion input, a drop-down list indicating conversion candidates is also displayed on the second terminal. This is because the second terminal can quickly designate the Chinese character to be selected.
[0026]
FIG. 4 shows an operation explanatory diagram of the second specific example.
In the upper right of FIG. 4, a window 6 and a drop-down list 11 displayed on the first terminal are shown. A window 7 having the same contents is displayed on the second terminal (step S1). The contents of the window 6 of the first terminal are the same as those in the specific example 1, and the drop-down list 11 is This indicates a conversion candidate when a character “konno” is input. The window control unit 9 shown in FIG. 1 displays such a drop-down list 11 on the display 4 of the first terminal 1. At this time, the screen sharing control unit 8 acquires a window handle (step S2). Thereby, the screen sharing control unit 8 copies the image of the drop-down list 11 and transfers the image data to the second terminal (step S3).
[0027]
As a result, in step S4, the same drop-down list 11 as that displayed on the first terminal is displayed together with the window 7 on the second terminal. Thereby, the operator who operates the second terminal can recognize the candidates listed during the kana-kanji conversion from the drop-down list 11. The operator who operates the second terminal transmits the correct characters in the drop-down list 11 by voice, for example, to the operator who operates the first terminal by designating the number. Thus, the operator who operates the first terminal can quickly select the correct Chinese character and confirm the content.
[0028]
<Effect of specific example 2>
That is, in the method of the second specific example, when there are a large number of character candidates to be converted, the operator operating the second terminal can promptly and accurately notify the operator operating the first terminal. It becomes possible.
[0029]
<Specific example 3>
In the second specific example, the drop-down list is transferred from the first terminal 1 to the second terminal 2 and displayed, and the operator who operates the second terminal specifies a candidate to be selected. I was told to the operator who operates 1 terminal. At this time, if the operator who operates the second terminal can directly select the corresponding candidate, it is more efficient. However, it is not easy to directly operate a character conversion input program activated in the first terminal in the second terminal.
[0030]
Therefore, in this example, a virtual keyboard is displayed on the second terminal so that the keyboard functions in the same way as the keyboard of the first terminal, and a candidate in the drop-down list is selected using the virtual keyboard. I made it.
[0031]
FIG. 5 is an operation explanatory diagram of the third specific example.
As shown in the figure, first, in step S1, a screen having the same contents as the window 6 of the first terminal is displayed in the window 7 of the second terminal. In step S2, the drop-down list 11 is displayed on the first terminal as described in the specific example 2. At this time, in this specific example, the window control unit 9 shown in FIG. 1 detects the display of the drop-down list 11 in the first terminal (step S3). Then, the detection result is notified to the screen sharing control unit 8.
[0032]
When the drop-down list 11 is displayed, the screen sharing control unit 8 transfers the image to the second terminal and controls the display of the virtual keyboard 12 for the second terminal in step S4. As a result, the virtual keyboard 12 is displayed on the second terminal in step S5. The virtual keyboard 12 is displayed on the display of the second terminal. When a number is designated using a pointing device such as a mouse and the OK button is clicked, the result is notified to the first terminal 1.
[0033]
That is, the operator who operates the second terminal looks at the drop-down list 11 and selects an appropriate conversion candidate for character conversion input. Then, a number corresponding to the character is input using the virtual keyboard 12. When this number is notified to the first terminal 1, it is recognized as a key event through the screen sharing control unit 8 of the first terminal 1. That is, in step S6, exactly the same signal as when the keyboard of the first terminal is operated is generated and recognized by the operating system of the first terminal.
[0034]
As a result, the same processing is executed as when the appropriate character in the drop-down list is selected by operating the keyboard of the first terminal (step S7). That is, the same processing is realized as if the operator who operates the second terminal operates the keyboard of the first terminal and selects the conversion candidate.
[0035]
The display control of the virtual keyboard may be performed by a screen sharing control unit provided in the first terminal, or an application program for virtual keyboard control installed in the second terminal is started by the screen sharing control unit. Then, a virtual keyboard may be displayed.
[0036]
FIG. 6 shows an operation flowchart of Example 3 as described above.
First, in step S1, a drop-down list for selecting a conversion candidate is displayed on the first terminal. The processing so far is the same as in the second specific example. Next, in step S2, the drop-down list display is detected and the image data is transferred. In step S3, the second terminal performs control to display a virtual keyboard together with the drop-down list.
[0037]
In step S4, a virtual keyboard is displayed on the second terminal, and the operator of the second terminal clicks the corresponding key using a pointing device such as a mouse in step S5. Based on this click event, the second terminal transmits information including the number of the clicked key to the first terminal. When this is received, the first terminal generates a key event that is exactly the same as when the keyboard of the first terminal is operated, thereby selecting the corresponding conversion candidate in the drop-down list.
[0038]
Thereafter, in step S8, the selected character is displayed. That is, even if the operator of the first terminal does not perform the kanji conversion process, the selection of the necessary kanji is completed by the operation of the operator of the second terminal. Therefore, selection of conversion candidates and determination of characters can be performed more smoothly than in the case of the specific example 2.
[0039]
The system as described above can be used in the following cases, for example. First, an operator who operates the first terminal starts an operation of inputting data including his / her name and address according to a certain format. At this time, the operator who operates the second terminal monitors the operation content of the operator who operates the first terminal while viewing the shared screen. Then, for example, when an operator operating the first terminal is at a loss in selecting personal name kanji, the operator operating the second terminal assists this.
[0040]
In another case, the operator who operates the first terminal hears necessary information while speaking to the operator who operates the second terminal by voice or the like, and performs data input work. An operator who operates the first terminal mainly inputs data, and inquires an operator who operates the second terminal about an unknown personality kanji. Then, the operator operating the second terminal is made to select kanji as necessary. In this way, the operator who operates the first terminal completes necessary data while having a conversation with the operator who operates the second terminal.
[0041]
In addition, the above system can be used for distance education. For example, a teacher who operates the first terminal inputs a hiragana character, and shows a drop-down list for students who operate the second terminal in a state where the kanji is unconfirmed. Students see the drop-down list and select the kanji they think is correct. If the teacher selects the wrong student, the teacher operating the first terminal informs the student of the kanji that has been correctly selected by informing the user of that fact.
[0042]
Further, for example, when automating a financial counter service, a customer needs to operate a terminal and input his / her name, address, and the like. However, there are many customers who are not necessarily familiar with terminal operation. In such a case, the customer cannot operate the terminal successfully, leading to wasted time and a failure.
[0043]
In order to avoid this, the screens are shared between the terminal operated by the customer and the terminal of the operator who monitors the operation of the customer at a remote place through the network. For example, the customer communicates his / her name to the operator using a microphone, and the operator inputs the customer's name using a keyboard. Then, the conversion result is displayed to the customer. At this time, the conversion result remains as an undetermined character.
[0044]
When the customer requests another character, a drop-down list is displayed to the customer to select which Kanji character. At this time, it is more accurately transmitted to the operator that the selected character is transmitted by number than to the customer who transmits the selected character to the operator by voice. The operator selects the appropriate character according to the customer's request. Since it is in an indeterminate state, it is possible to switch the selected kanji many times. That is, no re-input is required.
[0045]
Further, according to the third specific example, the virtual keyboard as described above is displayed on the customer's terminal, and the customer selects the Chinese character. Thereby, the corresponding Chinese character is selected more accurately. Further, even if the operation is simple, the customer can operate the terminal so that the customer is interested in the operation of the terminal, gets used to the operation, and then can operate the terminal without the assistance of the operator. There is also an effect.
[0046]
<Effect of specific example 3>
As described above, by using the system of the present invention, operators operating two terminals connected by a communication network under various conditions can cooperate with each other to execute character conversion processing such as kana-kanji conversion. Is possible. In the above example, the kana-kanji conversion process in Japanese has been described. However, any specific example can be similarly used for the conversion process of, for example, Chinese or other languages.
[Brief description of the drawings]
FIG. 1 is a block diagram of a remote screen sharing system according to the present invention.
FIG. 2 is an operation explanatory diagram of a screen sharing control unit.
FIG. 3 is a flowchart of a character conversion operation.
FIG. 4 is an operation explanatory diagram of a specific example 2;
FIG. 5 is an operation explanatory diagram of a specific example 3;
6 is an operation flowchart of specific example 3. FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 1st terminal 2 2nd terminal 3 Communication network 4, 5 Display 6, 7 Window 8 Screen sharing control part 9 Window control part

Claims (1)

A first terminal and a second terminal connected to each other via a communication network, wherein the first terminal converts an input character input by the first operator to convert an unconfirmed conversion character; A character conversion confirming means for generating and generating the converted character as a confirmed character by a confirmed input; a window control unit for displaying the converted character and the confirmed character on a display; and the confirmed character as image data or a character code. Remote screen sharing including another window control unit that displays the fixed character in common on a display based on received image data or character code. In the system,
The screen sharing control unit transmits the unconfirmed converted character as image data or a character code to the second terminal,
The other window control unit of the second terminal is a remote screen sharing system that displays the converted character on the display based on the received image data or character code so as to confirm the correctness of the converted character.
The character conversion determination unit generates candidate characters to be converted from the input characters, the window control unit displays a list table including the candidate characters on a display, and the screen sharing control unit uses the list table as image data. Capture and send to the second terminal,
The other window controller of the second terminal displays the list table based on the received image data and a selectable virtual keyboard of candidate characters on the display,
The second terminal further includes a selection control unit that transmits a selection signal indicating a candidate character selected by the second operator with the virtual keyboard to the first terminal,
The character conversion confirming means of the first terminal sets a corresponding candidate character as a confirmed character when receiving the selection signal.
A remote screen sharing system.

Images