JP2006227933A - Remote control system and remote control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a user to receive a remote control more at ease. <P>SOLUTION: An control source terminal 2 is provided with an emulator starting means starting an emulator, a status reproduction means reproducing a state of a terminal to be controlled on the emulator, a control information storage means storing respective pieces of control information, a control video transmission means transmitting a control video to the terminal to be controlled. and a control information transmission means transmitting control information to the terminal to be controlled. The terminal to be controlled 2 is provided with a status information transmission means transmitting status information of the terminal to be controlled to the control source terminal, a reproducing means reproducing the control video, an indication reception means transmitting a remote control execution request to the control source terminal, and a control execution means executing control information received from the control source terminal. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a remote operation technology via a network.

  In recent years, the functions of terminals such as PCs (Personal Computers) have rapidly become sophisticated, and one terminal can perform various processes in a general manner. However, the operability deteriorates due to the high functionality of the terminal, and the operation of the terminal is difficult and difficult to handle especially for users who lack literacy such as the elderly.

  In such a case, there are a method for confirming the operation method in a manual (instructions, online manual, etc.), a method for inquiring by telephone to the support center, and a method for directly instructing a person who is familiar with the operation. However, it is difficult to intuitively understand the operation in the manual, it is difficult to convey a complicated operation in the case of a telephone, and it is not possible to directly tell if there is no person familiar with the operation.

Therefore, there is a method of remotely operating (controlling) a terminal located in a remote place via a network. For example, Patent Literature 1 describes a remote cooperative training method in which the contents of operation of a teacher terminal can be viewed on a student terminal connected via a network.
JP 2000-29861 A

  In the conventional remote operation, the remote operator who is requested to operate directly operates the client's terminal. For this reason, the client may not be able to predict the state of his / her terminal as a result of the remote operation, and may feel uneasy about the remote operation. In other words, in the case of a remote operator who is not acquainted such as a support center or a remote operator who is not very trustworthy, the client is worried that operations other than the operation intended by the client may be performed. is there.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to enable a user to receive a remote operation more safely.

  In order to solve the above problems, the present invention is a remote operation system including, for example, an operation source terminal that performs remote operation and an operated terminal that is remotely operated by the operation source terminal. The operating source terminal receives emulator starting means for starting an emulator that realizes the environment of the operated terminal, and status information indicating the status of the operated terminal, and displays the status of the operated terminal on the emulator based on the status information. Status reproduction means for reproducing the operation information, operation information storage means for storing each of the operation information received on the emulator, operation video transmission means for transmitting the operation video output by executing the operation information to the operated terminal, Operation information transmission means for receiving a remote operation execution request from the operated terminal, reading operation information stored in the operation information storage means, and transmitting the operation information to the operated terminal. Then, the operated terminal collects status information of the operated terminal, transmits status information transmitting means to the operating source terminal, receiving means for receiving and playing back the operation video from the operating source terminal, and execution of remote operation An instruction receiving unit that receives an instruction and transmits a remote operation execution request to the operation source terminal, and an operation execution unit that executes operation information received from the operation source terminal.

  According to the present invention, even a remote operator who is not acquainted can receive a remote operation with peace of mind.

  Hereinafter, embodiments of the present invention will be described.

  FIG. 1 is an overall configuration diagram of a remote control system to which an embodiment of the present invention is applied.

  The illustrated remote operation system includes an operated terminal 1 that is a target of remote operation, and an operation source terminal 2 that performs the remote operation. The operated terminal 1 and the operation source terminal 2 are connected by a network 3 that performs communication of information necessary for remote operation.

  As illustrated, the operated terminal 1 includes a processing unit 11, a terminal type transmission unit 12, a status information transmission unit 13, an operation information reproduction unit 14, an instruction reception unit 15, and an operation reproduction unit 16. Have.

  The processing unit 11 controls (or links) the other units 12 to 16 and performs processing necessary for remote operation and processing other than remote operation. The processes other than the remote operation are processes for realizing the function of the operated terminal 1 itself, for example, a connection process with the network 3 and a screen display process. The terminal type transmission unit 12 transmits the terminal type of the operated terminal 1 to the operation source terminal 2 according to the instruction of the processing unit 11.

  The status information transmission unit 13 collects status information indicating the current state of the operated terminal 1 in accordance with an instruction from the processing unit 11 and transmits the collected status information to the operation source terminal 2. The status information is, for example, a transition state of a screen displayed on an output device (not shown), information on a running application (application identification information, processing status, display position on the screen, etc.).

  The operation information reproducing unit 14 reproduces the video information transmitted from the operation source terminal 2 in real time and outputs it to the output device. The instruction receiving unit 15 receives various instructions (operation execution, operation rejection, operation end) input by the user, and sends the received instructions to the processing unit 11. The operation reproduction unit 16 operates the operation information received from the operation source terminal 2 in accordance with the instruction from the processing unit 11, and executes the same operation as that on the operation source terminal 2 on the operated terminal 1.

  As shown in the figure, the operation source terminal 2 includes a processing unit 21, an emulator activation unit 22, an emulator selection unit 23, a status reproduction unit 24, an operation video transmission unit 25, an operation information reception unit 26, and operation information. A sending unit 27 and a storage unit 28 are included.

  The processing unit 21 controls (or links) the other units 22 to 27 to perform processing necessary for remote operation and processing other than remote operation. The processes other than the remote operation are processes for realizing the functions of the operation source terminal 2 itself, such as a connection process with the network 3 and a screen display process.

  The emulator activation unit 22 holds a plurality of different emulators for the operated terminal 1 and activates the emulator selected by the emulator selection unit 23. Then, the emulator activation unit 22 cooperates with the status reproduction unit 24 and the operation video transmission unit 25 to reproduce the operation status of the operated terminal 1 on the emulator and execute an operation on the operated terminal 1. The emulator is software or hardware for reproducing (implementing) functions of another OS (Operating System) or CPU on the operation source terminal 2 and operating an application for the OS (or CPU). . In other words, the emulator operates an application for the OS (or CPU) by reproducing the function (API) of the target OS or the instruction of the CPU.

  The emulator selection unit 23 specifies an emulator corresponding to the terminal type received from the operated terminal 1 with reference to an emulator table 281 described later. Then, the emulator selection unit 23 instructs the emulator activation unit 22 to activate the identified emulator. Based on the status information received from the operated terminal 1, the status reproduction unit 24 instructs the emulator activation unit 22 to reproduce the operation status (processing status) of the operated terminal 1.

  The operation video transmission unit 25 transmits the operation video of the emulator activated by the emulator activation unit 22 to the operated terminal 1. The operation information accepting unit 26 accepts an operation on the emulator activated by the emulator activation unit 22 and sends the accepted operation to the emulator activation unit 22 as operation information. In addition, the operation information receiving unit 26 stores the received operation information in the operation information accumulation DB 282 of the storage unit 28. The operation information includes, for example, commands (commands) input by the user using a keyboard or the like, and mouse operation information (movement of the cursor, click of a mouse button, etc.) by operating the mouse.

  The operation information sending unit 27 extracts predetermined operation information from the operation information accumulation DB 282 of the storage unit 28 according to the instruction of the processing unit 21 and transmits it to the operated terminal 1. The storage unit 28 includes an emulator table 281 in which terminal types and emulators are associated with each other, and an operation information accumulation DB 282 in which operation information is accumulated.

  FIG. 2 is a diagram illustrating an example of the emulator table 281.

  The illustrated emulator table 281 has a terminal type 31 and an emulator ID 32 for identifying each emulator. For example, when the terminal type 31 is “terminal A”, the emulator ID 32 of the corresponding emulator is “emulator A”.

  Each of the operated terminal 1 and the operation source terminal 2 described above includes, for example, a CPU 901 as shown in FIG. 3, a memory 902, an external storage device 903 such as an HDD, and an input device 904 such as a keyboard and a mouse. A general-purpose computer system including an output device 905 such as a display or a printer and a communication control device 906 for connecting to a network can be used. In this computer system, the CPU 901 executes a predetermined program loaded on the memory 902, thereby realizing each function of each device.

  For example, each function of the operated terminal 1 and the operation source terminal 2 is performed by the CPU 901 of the operated terminal 1 in the case of the program for the operated terminal 1, and the operation source terminal 2 in the case of the program for the operation source terminal 2. Each of the CPUs 901 is implemented by executing each of them. Note that the memory 902 or the external storage device 903 of the operation source terminal 2 is used for the storage unit 28 of the operation source terminal 2. Note that the operated terminal 1 in the present embodiment may be an appliance terminal specialized for a specific function. That is, the operated terminal 1 may be, for example, a consumer game machine, a Web browsing dedicated terminal, a mail transmission / reception dedicated terminal, or the like.

  Next, remote operation processing of this system will be described. In the present embodiment, a case where the user of the operated terminal 1 (hereinafter, “operated user”) requests remote operation from the user of the operating source terminal 2 (hereinafter, “operating source user”) is described below as an example. explain.

  FIG. 4 is a processing flowchart of remote operation processing. First, the processing unit 11 of the operated terminal 1 receives an instruction from the operated user, and establishes a remote operation communication session with the processing unit 21 of the operation source terminal 2. Then, the processing unit 11 of the operated terminal 1 transmits a remote operation request message to the operation source terminal 2 (S1).

  Then, the processing unit 21 of the operation source terminal 2 receives the remote operation request message and outputs a message or screen indicating that the remote operation request has been made to the output device (S2). In addition, the processing unit 21 outputs a remote operation request message and the like, and also outputs an instruction input area for accepting an input of an instruction for the operation source user to accept or reject the remote operation request to the output device. The instruction input area may be displayed on a part of a screen indicating that a remote operation request has been made, or may be displayed on an operation screen such as a dialog box.

  The operation source user refers to a remote operation request message or the like displayed on the output device, and uses an input device to input an acceptance instruction for accepting the remote operation request or a rejection instruction for rejecting the remote operation request. To enter. Then, the processing unit 21 of the operation source terminal 2 determines whether or not the instruction input in the instruction input area is an acceptance instruction (S3). When the acceptance instruction is accepted (S3: YES), the processing unit 21 transmits a remote operation acceptance message to the operated terminal 1.

  Then, the processing unit 11 of the operated terminal 1 receives the remote operation acceptance message and outputs a start message (or screen) indicating the start of the remote operation to the output device. In addition, the processing unit 11 controls the instruction receiving unit 15 to output an operation button or an icon to the output device in order to receive an instruction of the operated user. The operation buttons and the like are for accepting an input of “operation execution”, “operation rejection”, or “operation end” instructions.

  Then, the processing unit 11 sends a remote operation start message, the terminal type sending unit 12 gives the terminal type of the operated terminal 1, and the status information sending unit 13 gives the current status information of the operated terminal 1, respectively. (S4). The terminal type transmission unit 12 acquires the terminal type stored in the memory or the external storage device, and transmits it to the operation source terminal 2. Further, the status information transmission unit 13 collects status information indicating the current state of the operated terminal 1 stored in a memory or the like and transmits the collected status information to the operation source terminal 2.

  Then, the processing unit 21 of the operation source terminal 2 receives the remote operation start message, the terminal type, and the status information. Then, the processing unit 21 sends the terminal type to the emulator selection unit 23. Then, the emulator selection unit 23 specifies (selects) an emulator ID corresponding to the terminal type of the operated terminal 1 with reference to the emulator table 281 (see FIG. 2). Then, the emulator selection unit 23 sends the specified emulator ID to the emulator activation unit 22 and instructs activation of the emulator of the emulator ID.

  In the case of the emulator table 281 shown in FIG. 2, when the terminal type of the operated terminal 1 is “terminal A”, the emulator selection unit 23 specifies “emulator A”. Then, the emulator selection unit 23 instructs the emulator activation unit 22 to activate the “emulator A” emulator.

  Then, the processing unit 21 sends the received status information to the status reproduction unit 24. Then, the status reproduction unit 24 reproduces the current operation status of the operated terminal 1 on the emulator activated by the emulator activation unit 22 based on the status information (S5).

  Then, the operation source user of the operation source terminal 2 performs various operations using the input device on the activated emulator. That is, the operation source user performs, for example, command input and operation using a mouse (movement of a cursor, click of a mouse button, etc.). The operation information receiving unit 26 receives a user operation, sends the received operation as operation information to the emulator activation unit 22, and stores it in the operation information accumulation DB 282. Then, the emulator activation unit 22 executes processing (operation) according to the operation information, and outputs the processing result to the output device.

  The operation video transmission unit 25 then sequentially (in real time) transmits image information (operation video) of the output screen output by the operation of the operation source user to the operated terminal 1 (S6).

  Then, the operation information reproducing unit 14 of the operated terminal 1 outputs the image information sequentially transmitted from the operation source terminal 2 to the output device in real time (S7). That is, on the output device of the operated terminal 1, an operation result image on the emulator of the operation source terminal 2 is displayed in real time. Note that an operation button (or icon) is always displayed on the output device of the operated terminal 1 at a predetermined position of the output device after S4 in order to receive the operated user's instruction output by the instruction receiving unit 15 in S4. It is possible to input the operated user's instruction at any time.

  Then, the operated user of the operated terminal 1 refers to the image information reproduced (displayed) on the output device, and inputs any instruction by clicking an operation button or the like as necessary. The instruction receiving unit 15 receives an instruction from the operated user, and determines whether or not the instruction is an “execute operation” instruction (S8). For example, the operated user needs to perform an operation performed on the emulator of the operation source terminal 2 also on the operated terminal 1 and also performs the operation on the operated terminal 1. If it is determined that there is no problem, an “execute operation” instruction is input.

  When the “operation execution” instruction is received (S8: YES), the instruction reception unit 15 sends an “operation execution” instruction to the processing unit 11. Then, the processing unit 11 transmits an operation execution message to the operation source terminal 2 (S9). The operation execution message indicates the operation executed by the emulator of the operation source terminal 2 on the operated terminal 1 from when the image information is output in S7 to when the instruction receiving unit 15 receives the “operation execution” instruction. A message requesting execution.

  The processing unit 21 of the operation source terminal 2 receives the operation execution message. Then, the processing unit 21 controls the operation information sending unit 27 to sequentially transmit the operation information stored in the operation information storage DB 282 to the operated terminal 1 in the order stored in the operation information storage DB 282 (S10). ). The operation information sending unit 27 reproduces the operation information input by the operation source user in S6 until the operation execution message is received after reproducing the state of the operated terminal 1 in S5 (or S13). And transmitted to the operated terminal 1.

  The operation reproduction unit 16 of the operated terminal 1 receives each operation information and sends it to the processing unit 11 in the order of reception. And the process part 11 implement | achieves (executes) the same operation as the operation performed on the emulator of the operation origin terminal 2 on the operated terminal 1 by performing operation of each operation information in order (S11). .

  After executing all the operation information received from the operation source terminal 2, the processing unit 11 instructs the status information transmission unit 13 to collect the current status information. Then, the processing unit 11 transmits a state reproduction request message, and the status information transmission unit 13 transmits the collected status information to the operation source terminal 2 (S12). The state reproduction request message is a message for requesting the operation source terminal 2 to reproduce the current state of the operated terminal 1 based on the collected status information.

  Then, the processing unit 21 of the operation source terminal 2 receives the state reproduction request message and the status information from the operated terminal 1. Then, the processing unit 21 causes the status reproduction unit 24 to reproduce the current situation of the operated terminal 1 based on the received status information on the active emulator (S13). Then, the processing unit 21 returns to S6 and accepts the operation of the operation source user in the state of the operated terminal 1 reproduced in S13.

  On the other hand, when the instruction received in S7 is other than the “operation execution” instruction (S8: NO), the instruction receiving unit 15 determines whether or not the instruction is an “operation rejection” instruction (S14). For example, when the operated user determines that the operation performed on the emulator of the operation source terminal 2 does not need to be performed on the operated terminal 1, or the operation is performed on the operated terminal 1. When it is determined that there is a problem with the operation, an “operation refusal” instruction is input.

  When an “operation refusal” instruction is received (S14: YES), the instruction reception unit 15 sends an “operation refusal” instruction to the processing unit 11. Then, the processing unit 11 proceeds to S12 described above, and transmits a state reproduction request message and status information to the operation source terminal 2. Note that when an “operation refusal” instruction is received, image information operated by the emulator of the operation source terminal 2 is displayed on the output device, but the operation is not executed on the operated terminal 1. Accordingly, the status information sent by the status information transmission unit 13 at this time transmits the previous operation execution message (S11), and transmits the status at the time when the operation is performed (S13) or the remote operation start message of S4. This is the situation at the time. That is, the operation executed by the emulator of the operation source terminal 2 after the state of the operated terminal 1 is reproduced in S14 or S5 until the operation rejection instruction is accepted is canceled.

  If the instruction received in S7 is not an “operation rejection” instruction, that is, an “operation end” instruction (S14: NO), the instruction receiving unit 15 sends an “operation end” instruction to the processing unit 11. Then, the processing unit 11 transmits a communication end message to the operation source terminal 2 and performs a remote operation communication session end process. In addition, the processing unit 11 outputs a message or screen indicating the end of the remote operation to the output device (S15).

  Then, the processing unit 21 of the operation source terminal 2 receives the communication end message and outputs a message or a screen indicating that the remote operation is ended to the output device. Then, the processing unit 21 stops the emulator activated by the emulator activation unit 22 (S17).

  When a remote operation rejection instruction is received in S3 (S3: NO), the processing unit 21 of the operation source terminal 2 transmits a remote operation rejection message to the operated terminal 1 (S16). Then, the processing unit 11 of the operated terminal 1 receives the remote operation rejection message, and performs the process of S15 described above. Then, the processing unit of the operation source terminal 2 receives the communication end message and outputs a message or a screen indicating that the remote operation is ended to the output device (S17).

  The embodiment of the present invention has been described above.

  In the present embodiment, the operated terminal 1 is not directly operated remotely, but is operated once by the emulator of the operation source terminal 2, and the operation content on the emulator is transmitted to the operated terminal 1. Then, the operated user of the operated terminal 1 confirms the operation content and the operation result by watching this video, and selects whether or not to perform the operation on the operated terminal 1. Then, the operation source terminal 2 transmits operation information to the operated terminal 1 according to an operation execution instruction of the operated user, and performs remote operation. As a result, the operated user can confirm the operation result by the remote operation in advance, so even if the user is not familiar with the operation source user or the operation source user cannot trust, the remote operation can be performed safely. Can receive. In addition, the operated user can prevent a risk that an operation not intended by the user is performed.

  In the present embodiment, the operation source terminal 2 receives a terminal type from the operated terminal 1 and activates an emulator corresponding to the terminal type. As a result, even when the operation source terminal 2 supports the operation of a plurality of different types of terminals, the operation can be easily (smoothly) performed by an emulator according to the terminal type of the operated terminal without prior consultation. can do. For example, in a support center that collectively supports operation of multiple types of appliance terminals, by using the operation source terminal 2 of the present embodiment, it is possible to save trouble of inquiring the user type of the terminal from the operated user and smoothly and efficiently. Remote operation service can be provided.

  In addition, this invention is not limited to said embodiment, Many deformation | transformation are possible within the range of the summary.

1 is an overall configuration diagram of a remote control system to which an embodiment of the present invention is applied. It is a figure which shows an example of an emulator table. It is a figure which shows the hardware structural example of each apparatus. It is a processing flowchart of remote operation processing.

Explanation of symbols

  1: operated terminal, 11: processing unit, 12: terminal type transmission unit, 13: status information transmission unit, 14: operation information reproduction unit, 15: instruction reception unit, 16: operation reproduction unit, 2: operation source terminal, 21: Processing unit, 22: Emulator activation unit, 23: Emulator selection unit, 24: Status reproduction unit, 25: Operation video transmission unit, 26: Operation information reception unit, 27: Operation information transmission unit, 28: Storage unit, 281 : Emulator table, 282: Operation information storage DB

Claims (4)

A remote operation system having an operation source terminal that performs remote operation and an operated terminal that is remotely operated by the operation source terminal,
The operation source terminal is
Emulator starting means for starting an emulator for realizing the environment of the operated terminal;
Status reproduction means for receiving status information indicating the status of the operated terminal and reproducing the status of the operated terminal on the emulator based on the status information;
Operation information storage means for storing each operation information received on the emulator;
Operation video transmission means for transmitting an operation video output by executing the operation information to the operated terminal;
An operation information transmitting unit that receives a remote operation execution request from the operated terminal, reads the operation information stored in the operation information storage unit, and transmits the operation information to the operated terminal;
The operated terminal is:
Status information transmission means for collecting the status information of the operated terminal and transmitting it to the operation source terminal;
Reproducing means for receiving and reproducing the operation video from the operation source terminal;
An instruction receiving means for receiving a remote operation execution instruction and transmitting the remote operation execution request to the operation source terminal;
An operation execution means for executing the operation information received from the operation source terminal. The remote control system according to claim 1,
The operated terminal further includes terminal type transmitting means for transmitting the terminal type of the operated terminal to the operation source terminal,
The remote operation system, wherein the emulator activation means of the operation source terminal activates an emulator corresponding to a terminal type received from the operated terminal. A remote operation method in a remote operation system having an operation source terminal performing remote operation and an operated terminal remotely operated by the operation source terminal,
The operation source terminal includes an operation information storage unit that stores operation information, and a first processing unit,
The first processing unit includes:
An emulator starting step for starting an emulator for realizing the environment of the operated terminal;
Receiving status information indicating the status of the operated terminal, and reproducing the status of the operated terminal on the emulator based on the status information; and
An operation information storage step for receiving the operation information and executing it on the emulator, and storing the operation information in the operation information storage unit;
An operation video transmission step of transmitting an operation video output by executing the operation information to the operated terminal;
An operation information transmission step of receiving a remote operation execution request from the operated terminal, reading the operation information stored in the operation information storage unit, and transmitting the operation information to the operated terminal, and
The second processing unit of the operated terminal is
Collecting the status information of the operated terminal and transmitting the status information to the operation source terminal;
A playback step of receiving and playing back the operation video from the operation source terminal;
An instruction receiving step of receiving a remote operation execution instruction and transmitting the remote operation execution request to the operation source terminal;
An operation execution step of executing the operation information received from the operation source terminal. The remote operation method according to claim 3,
The second processing unit of the operated terminal is
A terminal type transmission step of transmitting the terminal type of the operated terminal to the operation source terminal is further performed,
The first processing unit of the operation source terminal is
In the emulator starting step, an emulator corresponding to a terminal type received from the operated terminal is started.

Images