JP2013246539A - Device with screen to be operated - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve usability and reliability by processing a remote operation signal in accordance with a transmission source of an operation signal or the state of a device with a screen to be operated.SOLUTION: An operation event processing unit 40 acquires an operation event signal from an event conversion unit 50, and executes operation event processing in accordance with the operation event signal. The event conversion unit 50 holds the association of a local operation signal and a local operation event signal. Also, the event conversion unit 50 acquires the local operation signal from a local operation reception unit or a remote operation reception unit 10, and converts the local operation signal into the local operation event signal on the basis of the association. A state notification unit 60 notifies an operation control unit 70 of the state of a device 1 with a screen to be operated. The operation control unit 70 determines whether or not preliminarily designated conditions are satisfied, and when determining that those conditions are satisfied, changes the association of the local operation signal and the operation event signal held by the event conversion unit 50.

Description

  Embodiments described herein relate generally to a screen operated device in which a display screen is remotely operated by a screen operating device.

  In recent years, remote operation technology for operating a display screen of a computer by another computer via a network has been widespread. Remote operation technology has been proposed under the concept of remote desktop, and typically, VNC (Virtual Network Computing) as shown in Patent Document 1 has been proposed.

  The environment for realizing the remote operation technology is established by connecting a screen operated device whose display screen is remotely operated and a screen operation device for remotely operating the display screen via a network. Each of the screen operation device and the screen operated device is a computer that operates according to an application, and includes a display screen that is an output device, a mouse and a keyboard that are input devices, and the like.

  As the server device, the screen operated device transmits image information of the display screen to the screen operating device via the network and transfers the screen. On the other hand, the screen operation device displays the received image information on the display screen as a client device. The screen operating device generates an operation signal by operating an input device such as a mouse or a keyboard, and transmits the operation signal to the screen operated device via a network.

  The screen-operated device passes the received operation signal to the application, and converts the operation signal into an operation event signal. At this time, the operation signal and the operation event signal are associated in advance. The screen operated device executes various event processes in accordance with the operation event signal, and outputs the executed process results to the display screen. Specific event processing includes a mouse press event and a pointer movement event.

  Incidentally, the screen operated device not only receives an operation signal from the screen operated device side but also receives an operation signal from an input device attached to the screen operated device itself. An operation signal from the screen operating device side is called a remote operation signal, and an operation signal from the screen operated device side is called a local operation signal.

  The screen operated device determines whether the received operation signal is a remote operation signal or a local operation signal. By discriminating such an operation signal, the operation signal can be selectively permitted or discarded depending on whether the operation signal is transmitted from the screen operation device or the screen operated device.

JP 2008-71312 A

  In the conventional screen-operated device, it is determined whether it is a remote operation signal or a local operation signal, but the remote operation signal itself is not determined for each screen operation device. Therefore, when a plurality of screen operation devices are simultaneously connected to one screen operated device, the remote operation signal cannot be permitted or discarded only for a specific screen operation device. Therefore, there is a demand for a remote operation technique that permits a remote operation signal from one screen operation device and discards the remote operation signal from another screen operation device.

  Further, in the conventional screen operated device, when the operation signal is permitted or discarded, the screen operated device does not consider the state of the device. The state of the screen operated device is information derived from the OS, X server, various devices, setting files, and the like, and specifically appears as the following states.

  That is, how to set the input device of the screen-operated device, who is using the user, what application is being used, what type of window is being used, and what is the CPU usage rate and memory usage rate It takes a value, what time is the internal clock, where is the position of the text cursor and mouse cursor, etc. Development of a remote operation technique capable of appropriately responding to the situation by allowing or discarding the operation signal in consideration of the state of these devices is awaited.

  In recent years, OSs have also been diversified. In recent years, there are not a few environments in which screen operated devices and screen operating devices use different OSs. For this reason, if the association between the operation signal and the operation event signal is fixed, an operation signal peculiar to a certain OS cannot be handled by a screen operated device that does not use the OS.

  Furthermore, the setting of the input device may be different between the screen operated device side and the screen operating device side. Also in this case, if the association between the operation signal and the operation event signal is fixed, there is a possibility that the screen operated device will handle the remote operation signal from the screen operating device. Therefore, it is required not only to perform two kinds of processing of permitting or discarding the operation signal, but also to change the association between the operation signal and the operation event signal and flexibly cope with the operation signal. .

  The screen-operated device of the present embodiment has been proposed in order to solve the above problems. An object of the present embodiment is to provide a screen-operated device that can process an operation signal in accordance with a source of the operation signal and a state of the screen-operated device, and improve usability and reliability. .

In order to achieve the above object, the screen-operated device according to this embodiment includes the following components (a) to (g).
(A) A local operation receiving unit that receives a local operation signal obtained by an input operation of a local user.
(B) A remote operation receiving unit that receives a remote operation signal generated by the screen operation device via a network and converts the received remote operation signal into a local operation signal.
(C) holding an association between the local operation signal and the local operation event signal, obtaining the local operation signal from the local operation receiving unit or the remote operation receiving unit, and performing a local operation on the local operation signal based on the association An event converter that converts event signals.
(D) An operation event processing unit that acquires the operation event signal from the event conversion unit and executes an operation event process according to the operation event signal.
(E) A screen configuration unit that configures the processing result of the operation event processing unit as screen information.
(F) A screen information transmission unit that transmits the screen information configured by the screen configuration unit to the screen operation device via a network.
(G) An operation control unit that determines whether a predetermined condition is satisfied and determines that the condition is satisfied, and changes an association between the local operation signal held by the event conversion unit and the operation event signal.

The block diagram containing 1st Embodiment. The block diagram of 1st Embodiment. The processing flow figure of a 1st embodiment. The processing flow figure of a 1st embodiment. The flowchart of 1st Embodiment. The flowchart of 1st Embodiment. The flowchart of 1st Embodiment. The schematic diagram in the case of the state which discards the remote operation signal from a specific screen operation apparatus in 1st Embodiment. FIG. 6 is a schematic diagram when a remote operation signal for a specific application window is discarded in the first embodiment. The schematic diagram in the case of changing the correlation of the operation signal and the operation event signal in the first embodiment.

[First Embodiment]
A screen operated apparatus according to the first embodiment will be described with reference to FIGS.
(Configuration overview)
FIG. 1 is a block diagram showing a remote operation environment of the screen operated device 1. As shown in FIG. 1, the screen operated device 1 and the screen operating device 100 are connected by a network N. The network N is a network that complies with a wired communication protocol such as IEEE802.3, a wireless communication protocol defined by IEEE802.11, or other protocols, and is a communication line such as a LAN, the Internet network, or a dedicated line.

  The screen operated device 1 and the screen operating device 100 are both computers. The former is a VNC server device that operates a VNC server application on the OS, and the latter is a VNC client device that operates a VNC client application on the OS. The screen operated device 1 and the screen operating device 100 are respectively arithmetic control devices 1a and 100a composed of CPUs, main storage devices 1b and 100b that are RAMs, external storage devices 1c and 100c such as HDDs that store OS and applications, Communication interfaces 1d and 100d, screens 21 and 121 as display screens, keyboards 30 and 130 as input devices, and mice 31 and 131 are provided.

(Detailed configuration)
The configuration of the screen operated device 1 and the screen operating device 100 will be described in detail with reference to the block diagram of FIG.
(Screen operation device)
The configuration of the screen operation device 100 will be described. As shown in FIG. 2, the screen operation device 100 is provided with a remote operation transmission unit 110 and a screen information reception unit 120. The functions of these components are realized by starting the OS and the VNC client application.

  The remote operation transmission unit 110 and the screen information reception unit 120 communicate with other terminals using a protocol such as TCP / IP through the communication interface 100d. These may use a common communication unit. The remote operation transmission unit 110 adds a TCP or IP header to the remote operation signal received from the mouse 131 or the keyboard 130 and transmits the remote operation signal to the screen operated device 1 via the network N.

  The screen information receiving unit 120 removes the TCP and IP headers from the data received from the screen information transmitting unit 20 of the screen operated device 1 via the network N, performs decoding processing to obtain screen difference information, and stores it in the frame buffer. Overwrite on the other hand. The frame buffer is a virtual space allocated in the main storage device 1b. The screen 121 visualizes the contents of the frame buffer and provides screen data to the user on the screen operation device 100 side.

(Screen operated device)
Next, the configuration of the screen operated device 1 will be described. As shown in FIG. 2, the screen operated device 1 includes a remote operation receiving unit 10, a screen information transmitting unit 20, a screen configuration unit 22, a local operation receiving unit 32, an operation event processing unit 40, an event converting unit 50, and a status notification. A unit 60 and an operation control unit 70 are provided. The functions of these components are realized by starting the OS and the VNC server application.

  The screen operated device 1 outputs the contents of the frame buffer to the screen 21 and transfers the screen data of the screen 21 to the screen operating device 100 via the network N by the communication interface 1d. The screen 21 is a display screen that provides screen data visualizing the contents of the frame buffer to the user on the screen operated device 1 side.

(Remote operation receiver)
The remote operation receiving unit 10 is a part of the VNC server, and communicates with other terminals using a protocol such as TCP / IP through the communication interface 1d. The remote operation reception unit 10 receives a remote operation signal from the remote operation transmission unit 110 of the screen operation device 100 via the network N, removes a TCP or IP header from the remote operation signal, and converts it to a local operation signal.

(Screen information transmitter)
The screen information transmission unit 20 is also a part of the VNC server, and communicates with other terminals using a protocol such as TCP / IP through the communication interface 1d. The remote operation receiving unit 10 and the screen information transmitting unit 20 may use a common communication unit. The screen information transmission unit 20 adds a TCP or IP header to the screen data received from the screen configuration unit 22 and transmits it to the screen operation device 100 via the network N.

  When transmitting screen data, the screen information transmission unit 20 does not transmit all the screen data, but extracts a difference from the screen information received from the screen configuration unit 22 immediately before by performing bit comparison or the like, and performs an arbitrary compression process. (For example, JPEG or gzip) is applied and transmitted to the screen operating device 100.

(Screen component)
The screen configuration unit 22 is a part that receives a drawing request from the operation event processing unit 40, converts it into screen data, and stores it in a storage area such as a frame buffer. The screen configuration unit 22 is realized by a drawing function of an X server employed in, for example, the UNIX system (registered trademark). The screen configuration unit 22 stores the drawing information received from the operation event processing unit 40 in the frame buffer as visual information. The drawing information is drawing objects such as windows, buttons, character strings, and images.

(Local operation receiver)
The local operation receiving unit 32 receives an input operation signal as a local operation signal from a local user via an input device such as a mouse 31 or a keyboard 30.

(Operation event processing part)
The operation event processing unit 40 acquires a local operation event signal from the event conversion unit 50 and executes an operation event process according to the local operation event signal. For example, the operation event processing unit 40 processes an operation event signal in various applications in accordance with an input from an input device such as the mouse 31, and sends a drawing request to the screen configuration unit 22 as a processing result. Drawing requests include drawing requests such as a mouse cursor as well as characters and drawings.

(Event converter)
The event conversion unit 50 acquires a local operation signal from the local operation reception unit 32 or the remote operation reception unit 10 and converts the local operation signal into a local operation event signal. The local operation signal is a signal including a hardware button identification number and a pressed state of the button, and the local operation event signal is a signal including an identification number indicating the processing content of the operation.

  The event conversion unit 50 holds the association between the local operation signal and the local operation event signal. When the local operation signal is received from the operation event processing unit 40, the event conversion unit 50 converts the operation signal into a corresponding operation event signal based on the association. It returns to the event processing unit 40. The operation event signal includes a pointer movement event signal of a mouse pointer, a button pressing event signal of a keyboard or a mouse button, and the like.

  The event conversion unit 50 is implemented as a key map of the X server employed in the UNIX system (registered trademark) or the like. The key map is a correspondence table for converting a key code (also called a scan code) that is a physical button identifier into a key symbol that is a software button identifier. For example, the key code of the space key of the Japanese 106 keyboard is an integer 61, and the key symbol corresponding to the “space” of X11, which is one implementation of the X server, is the identifier XK_SPACE. Yes.

(Status notification part)
The state notification unit 60 is a part that notifies the operation control unit 70 of the state of the screen-operated device 1, and is one of the characteristic parts of the present embodiment. The screen operated device 1 according to the first embodiment is characterized in that the status notification unit 60 recognizes the state of the screen operated device 1 and then permits or discards the remote operation signal.

  The state of the screen-operated device 1 includes the setting state of the input device of the screen-operated device 1, the user in use, the type of application in use and the window in use, the usage rate of the CPU 1a, and the memory usage rate in the main storage device 1b. There are the time and time zone of the internal clock incorporated in the screen-operated device 1, the state of the application, and the like. The state of the application includes the type of task being executed by the application and the position of the object such as how the objects such as the window, button, and mouse cursor of the application are arranged on the screen.

(Operation control unit)
The operation control unit 70 is a part that changes the association between the local operation signal and the operation event signal held by the event conversion unit 50, and this component is also one of the characteristic parts of this embodiment. The operation control unit 70 receives the state of the screen operated device 1 notified by the state notification unit 60 and receives local operation signals from the local operation receiving unit 32 and the remote operation receiving unit 10.

  The operation control unit 70 determines whether or not a predetermined condition is satisfied. If the condition is determined, the association between the local operation signal and the operation event signal is changed, and the conversion operation of the event conversion unit 50 is changed. . The process for changing the conversion process of the event conversion unit 50 in the operation control unit 70 includes an empty setting in which the operation event processing unit 40 does not execute any process. The conversion operation of the event conversion unit 50 can be changed by directly editing the correspondence relationship between the local operation signal and the local operation event signal held by the event conversion unit 50, or another correspondence relationship prepared in advance. And replace it with

  The condition for changing the conversion operation of the event conversion unit 50 in the operation control unit 70 is that the remote operation signal is transmitted from a predetermined screen operation device 100, the application to be used is a predetermined one, and Is set with the state of the screen-operated device 1 described above.

(Process flow)
Next, the flow of processing in the screen operated device 1 according to the present embodiment will be described.
(Screen transfer process)
FIG. 3 shows screen transfer processing from the screen operated device 1 to the screen operating device 100. First, the screen configuration unit 22 updates the image data in the frame buffer according to the drawing request (step S01), and draws the contents of the frame buffer on the screen 21 (step S02).

  When the screen configuration unit 22 updates the frame buffer, the screen information transmission unit 20 transmits screen data indicating the updated contents to the screen operation device 100 (step S03). Specifically, when the screen information transmission unit 20 monitors the frame buffer and the screen configuration unit 22 updates the frame buffer, the screen information transmission unit 20 reads the image data in the frame buffer.

  When reading the image data in the frame buffer, the screen information transmission unit 20 compares the previous image data held in advance with the read image data to detect a difference portion. Then, the screen information transmitting unit 20 compresses the image data at the difference portion, adds a TCP or IP header to the compressed image data at the difference portion, and transmits the image data to the network N as screen data. The TCP or IP header includes an IP address of the screen operation device and a port number indicating the screen information receiving unit 20.

  On the screen operating device 100 side, when the screen information receiving unit 120 receives screen data from the screen information transmitting unit 20 via the network N (step S04), the contents of the screen data are drawn on the screen 121 (step S05). Specifically, when receiving the screen data, the screen information receiving unit 120 expands the screen data after removing the TCP or IP header, and writes the obtained image data in the difference portion of the frame buffer.

(Button pressing process on screen operation device)
FIG. 4 shows a mouse button or keyboard pressing process of the screen operation device 100. First, when the user of the screen operation device 100 presses a mouse button using the mouse 131 or performs an operation of pressing a key using the keyboard 130, a button press event occurs (step S11).

  When a button press event occurs, the remote operation transmission unit 110 generates a remote operation signal. A TCP or IP header is added to the remote operation signal. TCP and IP include the IP address of the screen operated device 1 and the port number indicating the remote operation receiving unit 10. The remote operation transmission unit 110 transmits a remote operation signal to the screen operated device 1 via the network N (step S12).

  On the screen operated device 1 side, the remote operation receiving unit 10 receives the remote operation signal, removes the TCP or IP header from the remote operation signal, converts it to a local operation signal, and transmits this to the operation control unit 70. (Step S13). An ID of the screen operation device 100 such as an IP address is added to the local operation signal.

  If the setting is not to discard the operation of the screen operation device 100 (No in step S14), the operation control unit 70 changes the conversion process of the event conversion unit 50 to the default setting (step S15-2). In the default setting, the event conversion unit 50 is set to perform conversion processing so that all operation signals received by the event conversion unit 50 are converted into local operation event signals according to the standard settings of the OS and the X server. Yes.

  On the other hand, if the setting is to discard the operation of the screen operation device 100 (Yes in step S14), the operation control unit 70 changes the conversion process of the event conversion unit 50 to an empty setting (step S15-1). In the empty setting, the conversion processing of the event conversion unit 50 is changed so that all operation signals received by the event conversion unit 50 are converted into local operation event signals of a type that the operation event processing unit 40 does not perform any processing. To do.

  If the conversion processing of the event conversion unit 50 is set to be empty, the operation event processing unit 40 does not perform any processing, so that the remote operation signal transmitted from the screen operation device 100 is discarded. Further, whether or not to discard the operation of the screen operation device 100 may be managed by a table or the like that associates the ID of the screen operation device 100 and whether or not to discard the operation.

  The operation control unit 70 transmits a local operation signal to the operation event processing unit 40 (step S16). The operation event processing unit 40 sends a local operation signal to the event conversion unit 50, and the event conversion unit 50 converts the local operation signal into a local operation event signal (step S17). At this time, the event conversion unit 50 executes signal conversion processing according to the empty setting set in step S15-1 or the default setting set in step S15-2.

  The operation event processing unit 40 receives the local operation event signal from the event conversion unit 50, and performs each process according to this signal (step S18). Here, when the conversion processing of the event conversion unit 50 is set to be empty, the local operation event signal obtained by the operation event processing unit 40 is a type of local operation event signal that does not perform any processing. Therefore, the operation event processing unit 40 does not perform any processing, and the remote operation signal transmitted from the screen operation device 100 is discarded.

(Button pressing process on screen operated device)
FIG. 5 is a flowchart showing a mouse button or keyboard pressing process of the screen operated device 1. When the user first presses a mouse button using the mouse 31 or performs an operation of pressing a key using the keyboard 30, a local operation event signal is generated (step S21). The local operation receiving unit 32 transmits a local operation event signal to the operation control unit 70.

  Here, if the setting is not to discard the operation of the screen operated device 1, (No in step S22), the operation control unit 70 performs the local operation according to the standard setting of the OS or the X server for the conversion processing of the event conversion unit 50. Default settings for conversion to event signals are set (step S23-2). On the other hand, if the setting is to discard the operation of the screen operated device 1 (Yes in step S22), the operation control unit 70 does not perform the conversion process of the event conversion unit 50, and the operation event processing unit 40 does not perform any process. To empty setting (step S23-1).

  Further, the operation control unit 70 transmits a local operation signal to the operation event processing unit 40 (step S24). The operation event processing unit 40 sends a local operation signal to the event conversion unit 50, and the event conversion unit 50 converts the local operation signal into a local operation event signal (step S25). At this time, the event conversion unit 50 executes signal conversion processing according to the empty setting set in step S23-1 or the default setting set in step S23-2.

  The operation event processing unit 40 receives a local operation event signal from the event conversion unit 50, and performs each event process according to this signal (step S26). Here, if the conversion processing of the event conversion unit 50 is set to be empty, the event processing unit 40 does not perform any processing as described above.

(Processing for changing the association between the remote operation signal and the operation event signal 1)
FIG. 6 is a flowchart showing processing for accepting a remote operation signal after changing the association between the remote operation signal and the operation event signal. In the processing of FIG. 6, whether or not the association between the remote operation signal and the operation event signal can be changed is determined based on whether or not the operation control unit 70 satisfies a predetermined condition.

  When the remote operation receiving unit 10 of the screen operated device 1 receives a remote operation signal from the screen operating device 100 or when the local user performs an input operation on the screen operated device 1, the local operation receiving unit 32 causes the local operation event signal to be received. Is received (step S31). If the setting of the association between the local operation signal and the operation event signal is not changed in the operation control unit 70 (No in step S32), the operation control unit 70 performs the conversion process of the event conversion unit 50 by the OS or the X server. A default setting for conversion to a local operation event is set according to the standard setting (step S33-2).

  On the other hand, if the operation control unit 70 is set to change the association between the local operation signal and the operation event signal (Yes in step S32), the operation control unit 70 changes the rule for converting the local operation signal into the local operation event signal. Then, the conversion process of the event conversion unit 50 is changed (step S33-1).

  Further, the operation control unit 70 transmits a local operation signal to the operation event processing unit 40 (step S34). The operation event processing unit 40 sends a local operation signal to the event conversion unit 50, and the event conversion unit 50 converts the local operation signal into a local operation event signal (step S35). At this time, the event conversion unit 50 performs conversion processing in accordance with the setting in step S33-1 or step S33-2. The operation event processing unit 40 performs each process according to the local operation event signal (step S36).

(Process 2 for changing the association between the local operation signal and the operation event signal)
FIG. 7 is a flowchart showing processing for changing the association between the local operation signal and the operation event signal based on the content of the remote operation signal and the state of the screen operated device 1. First, the state notification unit 60 monitors the state of the screen operated device 1 and notifies the operation control unit 70 of the obtained state information (step S41).

  If the operation control unit 70 determines that the information indicating the state of the screen operated device 1 does not satisfy the condition specified in advance (No in step S42), the operation control unit 70 performs the conversion process of the event conversion unit 50 on the OS Or the default setting for conversion to a local operation event according to the standard setting of the X server (step S43-2). Further, when the operation control unit 70 determines that the information indicating the state of the screen operated device 1 satisfies a predetermined condition (Yes in step S42), the operation control unit 70 changes the conversion process of the event conversion unit 50 (step S43-1). ). Specifically, the rule that associates the local operation signal with the local operation event is changed.

(Function and effect)
The operation and effect of the screen operated device 1 having such a configuration and operation will be described with reference to FIGS. 8 and 10, the screen operation device 100 is shown as screen operation devices A and B.

(1) A remote operation signal transmitted from a specific screen operation device B can be discarded.
As shown in FIG. 8, the screen operated device 1 transfers and draws the screen data of the screen 21 of the screen operated device 1 on the screen of the screen operated device A and on the screen of the screen operated device B.

  In the screen operated device 1, the remote operation signal from the screen operating device A to the screen operated device 1 is permitted, and the remote operation signal from the screen operating device B to the screen operated device 1 is discarded in advance. In other words, as a condition for changing the conversion operation of the event conversion unit 50 to the empty setting in the operation control unit 70, the remote control signal is transmitted from the screen operation device B. When the condition is satisfied, the operation control unit 70 changes the conversion process of the event conversion unit 50 to an empty setting in which the operation event processing unit 40 performs no processing.

  The button operation generated on the screen of the screen operating device A is transmitted to the screen operated device 1. In this case, since the source of the received remote operation signal is the screen operation device A, the operation control unit 70 of the screen operated device 1 does not satisfy the condition specified in advance. Therefore, the event conversion unit 50 and the operation event processing unit 40 on the screen operated device 1 side perform the conversion and processing into button operation signals as usual, and the processing results are displayed on the screen operating device A and the screen operated device. 1 reflected on each screen.

  On the other hand, the remote operation signal of the button operation generated on the screen of the screen operating device B is transmitted to the screen operated device 1. In this case, if the source of the received remote operation signal is the screen operation device B, the operation control unit 70 of the screen operated device 1 satisfies the predesignated condition, and the event conversion unit 50 performs the conversion process. The operation event processing unit 40 is changed to an empty setting so that no processing is performed. As a result, the operation event processing unit 40 discards the button operation without performing the button operation process. Thus, the button operation is not reflected on the screen 21 of the screen operated device 1.

  According to the first embodiment as described above, it is possible to discard only the button operation from the screen operation device B designated in advance. Therefore, when a plurality of screen operation devices are connected to one screen operated device, it is possible to discard only a remote operation signal from a specific screen operation device.

(2) A remote operation signal to a specific application window can be discarded.
As shown in FIG. 9, the screen operated device 1 transfers and draws screen information on the screen of the screen operated device 1 on the screen of the screen operated device 100. FIG. 9A is a schematic diagram showing a state in which the application A of the screen operated device 1 is operated from the screen operating device 100, and FIG. 9B shows the application B of the screen operated device 1 from the screen operating device 100. It is a schematic diagram showing the state which is operating.

  The operation control unit 70 of the screen operated device 1 is set in advance to permit the remote operation signal from the screen operated device 100 to the application A of the screen operated device 1 but discard the remote operation signal to the application B. That is, when the remote operation receiving unit 10 of the screen operated device 1 receives a remote operation signal from the screen operating device 100, if the application being used by the screen operated device is the application B, the operation control unit 70 converts the event. The conversion processing of the unit 50 is changed to an empty setting so that the operation event processing unit 40 does not perform any processing.

  As shown in FIG. 9A, when a button operation generated on the screen of the screen operating device 100 is transmitted to the screen operated device 1, the application A is being used in the screen operated device 1. Therefore, the event conversion unit 50 and the operation event processing unit 40 on the screen operated device 1 side perform conversion and processing into button operation signals as usual. The processing result of the operation event processing unit 40 is reflected on each screen of the screen operation device 100 and the screen operated device 1.

  On the other hand, as shown in FIG. 9B, when a remote operation signal of button operation generated on the screen of the screen operating device 100 is transmitted to the screen operated device 1, the application being used in the screen operated device 1 If B, the operation control unit 70 changes the conversion process of the event conversion unit 50 to an empty setting so that the operation event processing unit 40 does not perform any processing. As a result, the operation event processing unit 40 discards the button operation processing in the application B.

  Thus, the button operation is not reflected on the screen 21 of the screen operated device 1. In this way, it is possible to discard only the operation from the screen operation device 100 to the specific application B of the screen operated device 1. According to the first embodiment as described above, it is possible to discard input operation information in units of applications.

(3) The association between the local operation signal and the operation event signal can be changed.
As shown in FIG. 10, the screen operated device 1 transfers and draws screen information of the screen operated device 1 on the screen of the screen operated device A and on the screen of the screen operated device B. The operation control unit 70 of the screen operated device 1 is set so as to change the association between the input operation signal from the screen operating device B and the operation event signal in the screen operated device 1 when a predetermined condition is satisfied. To do.

  When the button operation generated on the screen of the screen operating device A is transmitted to the screen operated device 1 and the predesignated condition is not satisfied, the event converting unit 50 on the screen operated device 1 side and the operation event processing The unit 40 performs conversion into button operation signals and processing as usual, and reflects the processing results on the screens of the screen operating device A and the screen operated device 1.

  On the other hand, when the button operation generated on the screen of the screen operation device B is transmitted to the screen operated device 1 and a predesignated condition is satisfied, the operation control unit 70 changes the conversion process of the event conversion unit 50. The association between the local operation signal and the operation event signal is changed. That is, the operation control unit 70 changes the conversion process of the event conversion unit 50 by changing the rule for converting the local operation signal into the local operation event signal, and accepts the local operation signal.

(3-1) Example 1 in which association change between local operation signal and operation event signal is effective 1
For example, the screen operated device 1 and the screen operating device A use an OS “S”, and the screen operating device B uses an OS “L” different from “S”. Further, it is assumed that a “Copy key” exists as a local operation event signal specific to the OS “S”.

  At this time, the association between the local operation signal and the operation event signal is set as a change condition that “the application in use of the screen operated device 1 is an application using the local operation event“ Copy key ””. It is assumed that a specific local operation signal is converted as a local operation event signal “F1 key” as the operation event signal before the change.

  When the operation control unit 70 determines that the above change condition is satisfied, the local operation signal converted as the local operation event signal “F1 key” before the change is converted into the local operation event signal “Copy key”. To do. Conventionally, it is not possible to perform a pressing operation of the “Copy key” unique to the OS “S” being used in the screen operated device 1 from the screen operating device B whose OS is “L”.

  However, according to the first embodiment, when the “F1 key” key is pressed on the screen operating device B, the screen operated device 1 changes the corresponding operation event signal to the “Copy key”. . As a result, even by a remote operation signal from the screen operation device B, the “Copy key” can be pressed in the screen operated device 1, and usability is further improved.

(3-2) Example 2 in which association change between local operation signal and operation event signal is effective Part 2
Further, as another example in which the change in association between the operation signal and the operation event signal is effective, there is a case where the input settings of the mouse on the screen operated device 1 side and the screen operating device B are different. That is, “the screen-operated device 1 side is the standard setting for the press key of the left and right buttons of the mouse, and the screen operating device B side is set by switching left and right”, and the association between the local operation signal and the operation event signal. Is a condition to change. When this change condition is satisfied, the operation control unit 70 changes the operation event signal corresponding to the pressed key of the left and right buttons of the mouse so that the conversion processing of the event conversion unit 50 is reversed left and right.

  When the user of the screen operating device 1 and the screen operating device B is right-handed and the user of the screen operating device B is left-handed, when the screen operating device B is remotely operated from the screen operating device B, the screen operating device B The user needs to perform the mouse operation with the right hand that is not the dominant hand, or the mouse operation equivalent to the right hand with the left hand, so that the user's feeling of using the mouse of the screen operation device B is lowered.

  In such a case, when the mouse pressing key is on the left or right, the left / right switching setting is possible. In other words, whether the user is right-handed or left-handed, the operation content of the mouse pressing key can be switched between the left and right, and the right button input operation and the left button input operation having a difference in use frequency can be switched. Since there is a difference in use frequency in the input operation of the left and right buttons, it becomes possible to press the button on the side where the use frequency is high with a dominant hand, and the usability is improved.

  Here, it is assumed that the screen operated device 1 is set as a standard setting, and the screen operating device B is set to change the above-described change condition by switching the left and right keys of the mouse. Therefore, the mouse pressing key is set upside down between the screen operated device 1 and the screen operating device B. With the settings as described above, when the screen operated device 1 receives a remote operation signal indicating that the right button is pressed from the screen operating device B, the screen operating device B in which the right button is switched to the left button is not the right button but the left button. This means that the button was pressed.

  In the conventional screen operated device 1, the association between the local operation signal and the operation event signal is not changed. Therefore, the local operation signal of the right button pressing operation is processed as it is as the right button pressing operation event. As a result, the conventional screen operated device 1 cannot execute an accurate remote operation.

  Therefore, in the first embodiment, the change condition is that “the screen-operated device 1 is the standard setting regarding the pressing key of the left and right buttons of the mouse, and the screen operating device B is set to switch left and right”. Then, the operation control unit 70 changes the association between the local operation signal and the operation event signal when the change condition is satisfied, and the operation event signal corresponding to the pressed key of the left and right buttons of the mouse for the conversion process of the event conversion unit 50. So that left and right are reversed.

  Therefore, when the remote operation receiving unit 10 of the screen operated device 1 receives the remote operation signal when the left button of the mouse is pressed on the screen operating device B, the operation control unit 70 receives the local operation signal and the operation event signal. Change association with. Therefore, the event conversion unit 50 changes the local operation signal generated from the left button operation of the mouse into an operation event signal that guides the right button pressing operation.

  Further, when the remote operation receiving unit 10 of the screen operated device 1 receives a remote operation signal when the right button of the mouse is pressed on the screen operating device B, the operation control unit 70 also detects the local operation signal and the operation event. Change the association with the signal. Therefore, the event conversion unit 50 changes the local operation signal generated from the right button operation of the mouse into the operation event signal that guides the left button pressing operation.

  According to the first embodiment as described above, even if the user of the screen operation device B performs the setting for switching the left and right mouse buttons to operate the mouse with the left hand, the screen operated device 1 receives the remote operation signal. The display screen is remotely operated after the contents are accurately grasped. Thereby, in 1st Embodiment, it can contribute to the improvement of the reliability and ease of use of a remote control technique.

[Other Embodiments]
As mentioned above, although embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  For example, in the above embodiment, the remote operation signal is discarded for the screen operated device 1 that is using a specific screen operating device or a specific application, but the specific screen operating device or the specific application is used. Only the remote operation signal for the screen operated device may be set to be permitted. Further, the remote operation signal may be permitted or discarded not only in the application unit but also in the user unit or the component part of the application window.

  Further, as a change condition for changing the association between the remote operation signal and the operation event signal, there are the following examples. For example, the in-use user is not allowed to use the in-use application, the CPU usage rate or the memory usage rate is greater than a certain value, the value of the internal clock is a specific time or time zone, and in use The position of the text cursor or mouse cursor during user input may be on a text box or button that is not permitted to be input by the user in use. According to such an embodiment, it is possible to carry out a remote operation of the display screen by limiting the use state of a specific user or screen operated device, and further a specific time. Thereby, reliability and security can be improved.

1 screen operated device 1a arithmetic control device 1b main storage device 1c external storage device 1d communication interface 10 remote operation reception unit 20 screen information transmission unit 21, 121 screen 22 screen configuration unit 30, 130 keyboard 31, 131 mouse 32 local operation reception Unit 40 operation event processing unit 50 event conversion unit 60 status notification unit 70 operation control unit 100 screen operation device 110 remote operation transmission unit 120 screen information reception unit N network

Claims (5)

A local operation receiving unit for receiving a local operation signal obtained by an input operation of a local user;
A remote operation receiving unit that receives a remote operation signal generated by the screen operation device via a network and converts the received remote operation signal into a local operation signal;
An association between the local operation signal and the local operation event signal is held, the local operation signal is acquired from the local operation reception unit or the remote operation reception unit, and the local operation signal is converted into a local operation event signal based on the association. An event converter to convert,
An operation event processing unit that acquires the operation event signal from the event conversion unit and executes an operation event process according to the operation event signal;
A screen configuration unit configured to configure the processing result of the operation event processing unit as screen information;
A screen information transmission unit configured to transmit the screen information configured by the screen configuration unit to the screen operation device via a network;
An operation control unit that determines whether or not a condition specified in advance is satisfied and determines that the condition is satisfied, and changes an association between the local operation signal held by the event conversion unit and the operation event signal; A screen-operated device characterized by that. A state notification unit for detecting the state of the screen operated device and notifying the operation control unit;
The operation control unit receives the state of the screen operated device from the state notification unit, and holds the event conversion unit when the content of the local operation signal and the state of the screen operated device satisfy the specified criteria 2. The screen operated device according to claim 1, wherein the association between the local operation signal to be performed and the operation event signal is changed.   The status notification unit includes the status of the screen operated device as the status of the input device of the screen operated device, the user in use, the application in use, the type of window in use, the usage rate of the CPU or memory, and the time of the internal clock. The screen operated device according to claim 2, wherein one of a time zone, a type of task being executed in the application, and an object position is notified.   The operation control unit performs empty setting processing in which the operation event processing unit does not perform any processing as processing for changing the conversion processing of the event conversion unit. The screen operated device according to the item.   The operation control unit includes the setting state of the input device of the screen-operated device, the user in use, the application in use, the type of window in use, the usage rate of the CPU or memory, the time and time zone of the internal clock, and the application executing task 5. The screen operation according to claim 1, wherein the association between the local operation signal and the operation event signal is changed on the condition of either the type of the object or the position of the object. apparatus.

Images