JP4973358B2 - Remote operation system - Google Patents

Description

  The present invention relates to a system and method for efficiently transferring screen data in a remote operation system including a plurality of networks and computers connected to the networks.

  The remote operation system operates an operated terminal in a remote location connected by a network from the operating terminal. Specifically, on the display of the operating terminal, the remote operating system displays on the operated terminal display. The same content is displayed, and the operation performed by the user on the operation side terminal is actually performed on the operated side terminal. As an implementation method, the display content (drawing data) is transferred from the operated terminal to the operating terminal so that the content on the display of the operated terminal is displayed on the remote operating terminal. Since the amount of data is transfer of screen drawing data, there is a general tendency that the amount of data is larger than that of normal character-based data. For example, Japanese Patent No. 2753277.

  In recent years, the proportion of mobile terminals used in corporate activities has been increasing year by year, such as via a wireless LAN (Local Area Network) or mobile phone, via a slower line than a wired LAN. Network access is increasing. In such a low-speed line environment between terminals of the remote operation system, as described above, when there is a large amount of transfer data, a delay occurs in data transfer to the operation side terminal, and as a result, a display for user operation of the operation side terminal The response of the screen on the side is reduced, and the user operability is significantly reduced. Therefore, it is desirable to reduce the transfer data between terminals.

Patent 2753277

  In general, reducing the transfer data causes degradation of image quality. It is an object of the present invention to maintain good display image quality and reduce transfer data between terminals in a remote operation system.

  An object of the present invention is to solve the above-described problems by changing the data processing method according to the number of colors in the bitmap and the communication speed. Specifically, the image is divided into 32 × 24 pixels on the display of the operated terminal. This divided bitmap is called a divided rectangle. Here, the pixels are 32 × 24 pixels, but can be divided into arbitrary sizes. The number of colors used is calculated from the divided rectangular bitmap, and the image is compressed using an appropriate compression method according to the number of colors. In addition, the image compression method is optimized so as to suppress the degradation of the apparent image quality of the user according to the communication speed between the terminals of the remote operation system, so that the transfer size of the screen data is reduced and the operability and visual recognition of the user are reduced. Do not impair the sex.

  According to the present invention, in the remote operation system, even when the line between the operation terminal and the operated terminal is a medium / low speed line, it is possible to suppress deterioration of the image quality without impairing user operability. In addition, since the transfer data flowing to the network between the terminals is reduced according to the present invention, traffic between the terminals of the remote operation system is reduced.

  The best mode for carrying out the present invention will be described below in detail with reference to the drawings.

  FIG. 1 shows an external appearance of a remote operation system which is an embodiment of the present invention. A display 101A, a keyboard 103A, and a mouse 104A are connected to the operation terminal 102A. Similarly, a display 101B, a keyboard 103B, and a mouse 104B are connected to the operated terminal 102B. The network 107 can use a line such as a LAN if it is on the premises, and can also use a wireless communication line such as a wireless LAN, a cellular phone, or a PHS (Personal Handyphone System) if it is outside the premises. The same screen as that of the operated terminal 102B is displayed on the display 101A of the operating terminal 102A, and the flow of display data is represented by an arrow 105. The keyboard and mouse input input at the operation terminal 102A is reproduced at the operated terminal, and the flow of input data is represented by an arrow 106. A general-purpose personal computer may be used as the operation terminal and the operated terminal, or a dedicated terminal can be provided. Further, when the display 101B, the keyboard 103B, and the mouse 104B are not connected to the operated terminal 102B, it can be used similarly.

  In the following description, the operated terminal is referred to as an agent terminal, and the operating terminal is referred to as a controller terminal.

  FIG. 2 shows a control sequence between the controller terminal and the agent terminal. When the controller terminal connects to the agent, it performs connection start processing 201 and selects an optimal image transfer option according to the communication line speed between the controller terminal and the agent terminal. The image transfer option will be described later with reference to FIGS. The controller terminal adds the setting option information selected in the connection start process 201 and transmits a connection request 202 to the agent terminal. In response to the connection request, the agent terminal acquires screen information of the agent terminal (203). The full screen information acquired after the start of connection is subjected to drawing data optimization 204. Acquisition of screen information and optimization of a drawing command will be described in detail with reference to FIGS. 4 and 7 to 13 described later. The optimized drawing command is transmitted to the controller terminal (205). The controller terminal restores the received optimized drawing command in the same manner as the method optimized at the agent terminal (206). By executing the restored drawing command on the operating system, the drawing command is displayed on the screen of the controller terminal (207). The controller terminal transmits a completion notice 208 to the agent terminal after the screen is displayed.

  The agent terminal that has received the completion notification 208 acquires screen information of the changed portion from the sent screen (209). The acquired screen information is optimized for a drawing command and transmitted to the controller terminal (210). The controller terminal restores the drawing command (211) and displays it on the screen (212), and transmits a completion notice 214 to the agent terminal. These screen information transfer procedures are repeated. Thereafter, when an end notification 215 is sent from the controller terminal to the agent terminal, the transfer of screen information is completed at the agent terminal, and communication between the controller terminal and the agent is disconnected.

  FIG. 3 shows the module configuration and screen data flow of the controller terminal and agent terminal of this embodiment. The agent terminal 301 includes a communication control unit 304 that controls communication with the controller terminal 311, a display driver hook control unit 305 that supplements a drawing command output from the operating system 303 to the display driver 306, and overall control of each control unit. Then, software configured by a basic control unit 308 that obtains a drawing command from the display driver hook control unit and performs processing operates.

  The controller terminal 311 includes a communication control unit 312 that controls communication with the agent terminal 301, a display control unit 314 that restores a drawing command from the agent terminal 301 and executes a drawing command to the operating system, and an operating system 317. Software composed of a display driver 315 that displays actual drawing data according to a drawing command, a display 316 that is a display medium, and a basic control unit 313 that performs overall control of each control unit operates.

On the controller terminal, a keyboard hook control unit 321 that captures keyboard data input from the keyboard driver 323 to the operating system 317 and a mouse hook control unit 322 that captures mouse data input from the mouse driver 324 to the operating system 317. Configured software runs.
The operation data supplemented by the mouse hook control unit and the keyboard hook control unit is transmitted to the agent terminal through the communication control unit 312 and input and executed in the operating system 303 on the agent terminal. The operating system 303 sends a drawing command to the display driver 307 in response to the operation data, and displays a screen on the display.

  When the drawing command 302 is transmitted from the operating system 303 on the agent terminal, the drawing command for the screen data portion changed by the display driver hook control unit is captured (310). The captured drawing command is optimized by the basic control unit 308 and transmitted to the controller terminal through the communication control unit 304. The drawing command transmitted by the basic control unit 314 on the controller terminal is restored to the drawing command captured by the display driver hook control unit. The restored drawing command is displayed on the display 316 through the operating system 317 and the display driver 315.

  FIG. 4 is a diagram showing an outline of the drawing data optimization process performed at the agent terminal. Among the drawing commands supplemented by the display driver hook control unit 305, coordinate information is stored in the queue of the display driver hook control unit (401). The coordinate information is represented by coordinate information (xn, yn) representing the upper left corner of the rectangle and coordinate information (xn ', yn') representing the lower right corner of the rectangle. The coordinate basic control unit 308 acquires the coordinate information stored in the queue at regular intervals, and when there are two or more coordinate information, the coordinate information of the coordinate information is deleted so that there is no overlapping part from the coordinate information of the changed drawing command. Optimization is performed (402). Note that optimization is not performed when the coordinate information stored in the queue is one. The basic control unit 308 divides the optimized coordinate information into a rectangle of 32 × 24 pixels (403). This divided rectangle is represented by a combination of coordinate information (xn, yn) representing the upper left vertex of the rectangle and (w, h) representing the width and height of the divided rectangle. Next, a bitmap in the rectangle is acquired from the divided rectangle information (404). Next, in order to reduce the amount of bitmap data, the number of colors used in the bitmap is calculated, and appropriate compression is performed according to the number of colors to optimize drawing information (405). The processes of (403), (404), and (405) are repeatedly executed for all parts of the changed drawing data.

  In this embodiment, it is divided into 32 × 24 pixels, but it may be divided into other sizes. This size information is stored in storage devices not shown in FIG. 3 of the controller terminal and the agent terminal. The data is divided by the predetermined size read from the storage device. The predetermined size is unified between the controller terminal and the agent terminal, and when the size to be divided is changed, the division size information is notified when the divided drawing information is transmitted.

  FIG. 5 is a flowchart showing a flow of connection start processing in the controller terminal. The controller terminal calculates the communication speed by measuring the line speed between the agent terminals at the start of connection (501). The communication speed setting can be calculated from the communication speed set in the controller terminal. Next, option settings for image transfer are read based on the calculated communication speed (502). This option setting is set in advance so as to be optimized for user operability and visibility according to the communication speed. Depending on the communication speed, this image transfer option setting is a low speed option (503) if it is less than 128 Kbps, a medium speed option (504) if it is less than 1 Mbps, and a high speed option (505) if it is 1 Mbps or more. Selected. Here, the value of the communication speed can be set to an arbitrary value.

  FIG. 6 is a table showing option settings according to the communication speed in the controller terminal, and is stored in a storage device not shown in FIG. In this embodiment, in order to achieve the purpose of reducing the transfer size and not impairing the operability and visibility of the user, the number of screen color reductions 601, application of JPEG compression of bitmap (valid / invalid) 602, bitmap The settings such as application of cache (valid / invalid) 603 and application of transmission data flow control (valid / invalid) 604 are operated based on values optimized in advance according to the communication speed. The screen color reduction 601 option reduces the amount of data by reducing the number of colors actually set on the screen of the agent terminal from 4 bits to 16 bits. The bitmap JPEG compression 602 option reduces the amount of data by performing compression using JPEG, which is generally used as a bitmap data compression method. The bitmap cache 603 will be described later with reference to FIG. The transmission data flow control 604 is effective when drawing information transmitted from the agent terminal to the controller terminal is transmitted after a completion notification from the controller terminal is transmitted. If transmission is performed without waiting for a completion notification from the controller terminal, the setting is invalidated. The contents of the option setting can be changed to any setting.

  FIG. 7 is a flowchart showing a flow of drawing data optimization processing in the agent terminal. The basic control unit 308 acquires coordinate information from one or more drawing data stored in the queue of the display driver hook control unit 305 (701). The acquired one or more pieces of coordinate information are optimized and divided into rectangles of 32 × 24 pixels (702). Drawing data (bitmap file) is acquired from the acquired coordinate information of 32 × 24 pixels (703). The number of colors used is calculated from the acquired bitmap, and a bitmap file conversion method is selected according to the number of colors (704). If the total number of colors in the bitmap is two or less, 1 bitp (bit per pixel) conversion of the bitmap file is performed (705). Here, by 1 bpp conversion, one pixel is replaced with 2-bit information, and color information attached to the drawing data becomes two-color data (color palette), so that the amount of transfer data can be reduced. Similarly, if the total number of colors in the bitmap is 4 or less, the bitmap file is subjected to 2bpp conversion (706), if it is within 8 colors, 3bpp conversion (707), and if it is within 16 colors, 4bpp is converted. Conversion (708) is performed. In this way, by converting to a necessary data format according to the number of colors in the bitmap, it is possible to reduce the amount of data while suppressing image degradation. Next, processing when the total number of colors in the bitmap is 16 or more will be described later with reference to FIG.

  Next, when the total number of colors in the bitmap is 4 or less, 1 bpp conversion (S705) and 2 bpp conversion (706) are performed, respectively, but MMR (Modified Modified Read) compression is performed on the converted bitmap ( 710). Note that MMR compression can be expected to reduce the amount of data according to the complexity and number of colors of the bitmap, but since this conversion depends on the bitmap data, the amount of data in the compressed file is less than that of the uncompressed file. It can be larger than the amount of data. Therefore, the file sizes of the file after MMR compression and the file before MMR compression are compared (711). When the file after MMR compression is smaller than the file before MMR compression, the file after MMR compression is selected (712). Conversely, when the file is smaller than the file before MMR compression, it is determined that MMR compression is inappropriate and before MMR compression. A file is selected (713). If the total number of colors in the bitmap is 8 or more, it is considered that the complexity of the bitmap is high and the effect of the MMR compression is low, so the file after bpp conversion is selected as it is (714). After converting and compressing the bitmap according to the number of colors in this way, the coordinate information and the drawing data are stored in a queue for transmission to the controller terminal (715). Next, it is determined whether the conversion and compression of all the divided bitmaps have been completed (716). If the conversion has not been completed, the total number of colors is obtained from the bitmap acquisition (703) for each divided bitmap. The acquisition, conversion, and compression processes are repeated.

FIG. 8 is a flowchart showing the flow of drawing data cache processing in the agent terminal. The basic control unit 308 acquires the drawing data stored in 715 of FIG. 7 (801), and creates a digest from the acquired drawing data (802). Regarding the digest creation method, here, the 16-byte data sampled from the drawing data is used as the digest, but other methods and other sizes are also possible. Next, a search is performed to determine whether drawing data existing in the queue has already been transmitted and is present in the cache (803).
The basic control unit 308 stores the drawing data transmitted to the controller terminal in a cache, and does not transmit the drawing data when transmitting the same drawing data next time, but transmits only the index information of the cache. Here, the same cache is held in each of the agent terminal and the controller terminal, and at the same time, the digest is listed as an index of drawing data. In the cache search, the digests listed are compared with each other to check for the presence of the same digest (804).
If the digest exists in the list, the drawing data existing in the cache is acquired from the digest, and the drawing data in the queue is compared with the drawing data in the cache (806). If the drawing data match, the drawing data in the queue is replaced with the cache index (807). If the comparison does not match the drawing data, the same digest in the cache is searched (808), and the presence or absence of the digest is determined (809).
If the same digest exists, the drawing data in the cache and the drawing data in the queue are compared again (806). If the same digest does not exist in 809, it is determined that the drawing data is not cached, the digest is registered in the list (810), and the drawing data is stored in the cache (811). If it is determined in 804 that the same digest does not exist at all, the digest is registered (810) and the drawing data is stored in the cache (811). The cache processing from 801 to 812 is repeated for each drawing data in the queue.

  FIG. 9 is a flowchart showing a flow of data compression after cache processing in the agent terminal. In step 812 of FIG. 8, the values of w and h in the coordinate information are omitted for the drawing command in the queue for which the cache processing has been completed (901). Next, when drawing data in the queue are continuously present, the same data is omitted and optimization is performed (902). The optimization of continuous identical data will be described later with reference to FIG. Finally, zip compression is performed on one or more drawing commands in the queue for which optimization has been performed to compress the data into single data (903).

  FIG. 10 is a flowchart showing the flow of processing when the number of colors in the drawing data bitmap in the agent terminal is greater than 16. It is determined whether the JPEG option is enabled in the option setting (1001). If the JPEG option is enabled (medium speed option setting), the 32 × 24 pixel bitmap is zip compressed (1002). Next, the same 32 × 24 pixel bitmap is JPEG compressed (1003). The sizes of the Zip compressed file and the JPEG compressed file are compared (1004). If the Zip-compressed file is larger, the JPEG-compressed file is selected (1005).

  When the JPEG option is disabled (low speed, high speed option setting), and when the Zip compressed file is smaller, the color reduction option setting is acquired (1006), and the 32 × 24 pixel bitmap is reduced by the setting value. Process (1007), and select a bitmap after color reduction (1008). In these cases, the Zip-compressed file used for comparison is discarded. Since the optimized drawing data is finally compressed together by Zip compression (step 903 in FIG. 9), it is not necessary to save the file after Zip compression.

  FIG. 11 shows an example of a drawing command that is queued at the time when the bitmap is divided into 32 × 24 pixels in the agent terminal and subjected to color reduction / compression processing. At this point, the drawing command includes coordinate information, width and height (x, y, w, h), rectangular information (1101), and drawing data (1102) subjected to color reduction / compression processing.

  The drawing data includes a combination of compression information including a color palette and compressed data, or a bitmap after color reduction. In the case of a compressed bitmap, a flag indicating a bitmap, a flag indicating a conversion method such as 1 bpp conversion, a flag indicating that MMR compression has been performed, and a color palette (in the case of 1 bpp conversion, a two-color palette, 4 bpp conversion) In this case, it is composed of 16 color palettes) and data after conversion and compression. The bit map after color reduction includes a flag indicating that it is a bit map and data after color reduction.

  In the case of a JPEG-compressed bitmap, it is composed of a flag indicating that compression is performed using JPEG and compressed data.

  FIG. 12 shows a cache table of drawing data. There is a digest table consisting of a drawing data digest (1201), a data pointer (1202), and the same digest pointer (1203), and an index table consisting of an index (1204) and drawing data (1205). These tables can be created in accordance with a common algorithm on the agent and controller sides.

  In the case of the present invention, in order to reduce transfer data, such table information is not transferred, but a table created by either one can be transferred to the other as a table held in common by the agent and the controller. is there.

  When the drawing data is the same as the drawing data registered in this table, the agent replaces the drawing data with an index and stores it in the transmission queue. The controller performs the reverse process of the agent to restore the actual drawing data from the index.

  FIG. 13 shows an example of a drawing data processing method when the agent terminal further optimizes drawing data after color reduction / compression / cache in order to reduce transfer data.

  The drawing data after color reduction / compression / cache is either a file after compression / color reduction or an index. (1301) to (1304) are information displayed on the same line because the X coordinate is the same. In this embodiment, since W = 32 pixels and H = 24 pixels are fixed, the Y coordinate, width, and height are values obtained by calculation.

Therefore, the width and height are first omitted from (1301) and processed to (1310). Next, (1302) to (1304) have the same X coordinate and index information.
Therefore, a repeat counter (repeat) is provided, and processing is performed so as to omit duplicate data as in (1311).

  Similarly, optimization is performed for all drawing data.

  The optimized drawing command is transmitted from the agent terminal to the controller terminal and restored by the basic control unit on the controller terminal. In the restoration, the data type of the drawing data is determined by the flab set at the head of the drawing data, and the restoration is performed accordingly. If it is determined that the file is a bitmap file based on the flag set at the top of the drawing data, a flag indicating that MMR compression has been performed is determined. If the flag is on, MMR compression is decompressed. Drawing data (bitmap file) that has not been subjected to MMR compression or has been decompressed by MMR can be processed by the operating system as it is, so there is no need to perform restoration processing. Finally, W and H indicating the width and height of the omitted drawing command are set, and a drawing command is issued to the operating system. If the flag set at the beginning of the drawing data determines that the file is a JPEG file, the JPEG file is restored to a bitmap file, W and H indicating the width and height of the omitted drawing command are set, A drawing command is issued to the operating system.

  When there is an index and repeat counter as in (1311), the corresponding drawing data is read from the cache table of the agent terminal from the index, restored in the same manner as described above, and a drawing command is issued to the operating system. . Thereafter, the same processing is executed as many times as the number of repeat counters. Alternatively, W and H can be recalculated from the repeat counter value and executed as one drawing command.

  In the embodiment, the optimized drawing data are collectively compressed in Zip and transmitted after being compressed into a single data. However, the compression method is not limited to Zip, and other compression methods can be used. You can use it. In this case, the compression method in step 1002 in FIG. 10 is compressed by the other compression method.

  In this embodiment, the optimized coordinate information is divided into rectangles of 32 × 24 pixels (FIG. 4, 403), but three points are selected from the coordinates instead of the rectangles, and this is repeated to form a triangular shape. You can also divide by.

  In this embodiment, when the number of colors is 16 or more, the compression method is changed according to the number of equal colors for JPEG compression (FIG. 7). For example, when the number of colors is 32 or more, the compression rate is 80. %, The compression rate of compression can be changed according to the number of colors, for example, the compression rate is 60% when the color is 64 colors or more.

It is a figure which shows the external appearance of a remote operation system. It is a figure which shows the control sequence between a controller terminal and an agent terminal. It is a figure which shows the module structure of a controller terminal and an agent terminal, and the flow of screen data. It is a figure which shows the outline | summary of the optimization process of the drawing data performed in an agent terminal. It is a figure which shows the flowchart which shows the flow of the connection start process in a controller terminal. FIG. 6 is a diagram illustrating an image transfer option setting table. It is the flowchart which showed the flow of the optimization process of the drawing data in an agent terminal. It is the flowchart which showed the flow of the cache process of the drawing data in an agent terminal. It is the flowchart which showed the flow of the data compression after the cache process in an agent terminal. It is a figure which shows the flowchart which shows the flow of a process when there are more colors of a bitmap than 16 colors. It is a figure which shows an example of the queued drawing command. It is a figure which shows the cache table of drawing data. It is a figure which shows an example of the optimization process for reducing transfer data.

Explanation of symbols

101A Operation terminal display 101B Operation terminal display 102A Operation terminal 102B Operation terminal 103A Operation terminal keyboard 103B Operation terminal keyboard 104A Operation terminal mouse 104B Operation terminal mouse 105 Display data flow 106 Input data flow 107 Network 201-213 Controller Sequence diagram 301 showing control of terminal and agent terminal 301 Agent terminal 302 Modified drawing data 303 Operating system 304 Communication control unit 305 on agent terminal side Display driver hook control unit 306 on agent terminal side Display driver 307 on agent terminal side Agent terminal Side display 308 Agent terminal side basic control unit 309 Changed drawing command flow 31 Captured drawing command flow 311 Controller terminal 312 Communication control unit 313 on the controller terminal side Basic control unit 314 on the controller terminal side Display control unit 315 on the controller terminal side Display driver 316 on the controller terminal side Display 317 on the controller terminal side Controller terminal Drawing command flow 321 on the controller terminal 321 Keyboard hook control unit 322 Mouse hook control unit 323 Keyboard driver 324 Mouse driver 401 Coordinate information 402 of the modified drawing command Coordinate information 403 of the optimized drawing command Coordinate information 404 divided into 32 × 24 pixels 404 Obtained 1-rectangular drawing information 405 Optimized 1-rectangular drawing information 501 to 505 Flow showing connection start processing of the controller terminal Charts 601 to 604 Image transfer option setting tables 701 to 716 Flowcharts 801 to 812 showing a flow of drawing data optimization processing Flowcharts 901 to 903 showing a flow of cache processing Flowcharts 1001 to 901 showing a flow of data compression 1008 Flowchart showing processing flow when there are more than 16 colors 1101 Drawing data rectangular information 1102 Drawing data 1201-1205 Drawing data cache tables 1301-1309 Drawing commands before optimization 1310-1314 Drawing commands after optimization

Claims (12)

An information transmission method in a remote operation system in which an operated terminal is operated by an operating terminal via a network,
The operated terminal is:
Obtain the image data of the changed part from the drawing data displayed on the display before the predetermined time has elapsed,
Generate divided image data obtained by dividing the image data of the changed part into a predetermined size,
Select a compression method for each of the divided image data according to the number of colors of each of the divided image data,
Generating first compressed image data obtained by compressing the divided image data by the selected compression method;
Generating second compressed image data obtained by compressing the first compressed image data by a compression method different from the selected compression method;
Compare the first compressed image data and the second compressed image data, select compressed image data with a small amount of data,
The selected compressed image data is stored in a transmission queue, a digest is created from the stored compressed image data, the digest is listed as index information of the compressed image data, and is stored in the first cache of the operated terminal. Store and
Check if the same digest exists from the list stored in the first cache;
If the same digest exists, the compressed image data stored in the first cache corresponding to the same digest and the previously stored compressed image data are further compared with the compressed image data stored in the transmission queue. ,
If the compressed image data is the same, the compressed image data stored in the transmission queue is replaced with index information indicating the previously stored compressed image data ,
Transmitting compressed image data or data that is index information stored in the transmission queue to the operation terminal;
The operation terminal is
When the data transmitted from the operated terminal is compressed image data, index information is created and stored in the second cache of the operating terminal,
When the data transmitted from the operated terminal is the index information, the previously stored compressed image data corresponding to the index information is retrieved from the second cache,
Restoring the divided image data from the compressed image data or previously stored compressed image data ;
Drawing data is generated from the restored divided image data,
Displaying the generated drawing data on the display of the operation terminal ;
A method for transmitting information in a remote operation system. An information transmission method in the remote operation system according to claim 1,
The operated terminal is:
Along with the selected compressed image data, information indicating the selected compression method is transmitted to the operation terminal,
The operation terminal is
Remotely recovering the divided image data from the selected compressed image data transmitted by a decompression method corresponding to the compression method specified based on the transmitted information indicating the selected compression method Information transmission method in operation system. It is the information transmission method in the remote operation system of Claim 1 or 2, Comprising:
The operated terminal is:
Coordinate information when the divided image data is displayed on the display together with the selected compressed image data is transmitted to the operation terminal,
The operation terminal is
An information transmission method in a remote operation system, wherein drawing data is generated based on the restored divided image data and the coordinate information. It is the information transmission method in the remote operation system of any one of Claims 1-3,
The selected compressed image data has a color palette,
The operation terminal is
An information transmission method in a remote operation system, wherein the divided image data is restored from the selected compressed image data transmitted based on color information of the color palette. A remote operation system for operating an operated terminal by an operation terminal via a network,
The operated terminal is:
A display image information acquisition unit for acquiring image data of a changed portion from drawing data displayed on a display before a predetermined time has elapsed;
An image data dividing unit for generating divided image data obtained by dividing the image data of the changed part into a predetermined size;
First image compression that selects a compression method for each of the divided image data according to the number of colors of each of the divided image data, and generates first compressed image data in which the divided image data is compressed by the selected compression method And
A second image compression unit for generating second compressed image data obtained by compressing the first compressed image data by a compression method different from the selected compression method;
A selection unit that compares the first compressed image data with the second compressed image data and selects compressed image data with a small amount of data;
The selected compressed image data is stored in a transmission queue, a digest is created from the stored compressed image data, the digest is listed as index information of the compressed image data, and a first cache of the operated terminal is stored. And confirms whether the same digest exists from the list stored in the first cache. If the same digest exists, the same digest stored in the first cache is stored. The compressed image data previously stored corresponding to the digest is further compared with the compressed image data stored in the transmission queue, and when the compressed image data is the same compressed image data, the compressed image data stored in the transmission queue is stored. and replaced with the index information indicating the compressed image data stored in the previously, stored in the transmission queue The data is compressed image data or the index information has a communication control unit to be transmitted to the operation terminal,
The operation terminal is
When the data transmitted from the operated terminal is compressed image data, index information is created and stored in the second cache of the operating terminal, and the data transmitted from the operated terminal is the index information. if it was in, an image restoration unit for restoring the compressed image data stored previously corresponding to the index information taken out from the second cache, the divided image data from the compressed data stored in the previous SL compressed image data or previously When,
A display image generation unit for generating drawing data from the restored divided image data;
A display control unit for displaying the generated drawing data on the display of the operation terminal;
Having
A remote operation system characterized by that. The remote operation system according to claim 5,
The communication control unit of the operated terminal transmits information indicating the selected compression method to the operating terminal together with the selected compressed image data,
The image restoration unit of the operation terminal is a restoration method corresponding to the compression method specified based on the transmitted information indicating the selected compression method, and the divided image is transmitted from the transmitted compressed image data. A remote operation system characterized by restoring data. The remote operation system according to claim 5 or 6,
The communication control unit of the operated terminal transmits the coordinate information when the divided image data is displayed on the display together with the selected compressed image data to the operating terminal,
The remote operation system, wherein the display image generation unit of the operation terminal generates drawing data based on the restored divided image data and the coordinate information. The remote operation system according to any one of claims 5 to 7,
The selected compressed image data has a color palette,
The remote operation system, wherein the image restoration unit of the operation terminal restores the divided image data from the selected compressed image data transmitted based on color information of the color palette. An information transmission method in a remote operation system in which an operated terminal is operated by an operating terminal via a network,
The operated terminal is:
Obtain the image data of the changed part from the drawing data displayed on the display before the predetermined time has elapsed,
Generate divided image data obtained by dividing the image data of the changed part into a predetermined size,
Measure the communication speed between the operation terminal and the operated terminal, select a compression method according to the measured communication speed,
Generating compressed image data obtained by compressing the divided image data with the selected compression method;
The compressed image data is stored in a transmission queue, a digest is created from the stored compressed image data, the digest is listed as index information of the compressed image data, and stored in a first cache of the operated terminal. ,
Check whether the same digest exists from the list stored in the first cache. If the same digest exists, the list corresponds to the same digest stored in the first cache. Further compare the previously stored compressed image data with the compressed image data stored in the transmission queue,
If the compressed image data is the same, the compressed image data stored in the transmission queue is replaced with index information indicating the previously stored compressed image data ,
Transmitting compressed image data or data that is index information stored in the transmission queue to the operation terminal;
The operation terminal is
When the data transmitted from the operated terminal is compressed image data, index information is created and stored in the second cache of the operating terminal,
When the data transmitted from the operated terminal is the index information, the previously stored compressed image data corresponding to the index information is retrieved from the second cache,
Restoring the divided image data from the compressed image data or previously stored compressed image data ;
Drawing data is generated from the restored divided image data,
Displaying the generated drawing data on the display of the operation terminal ;
A method for transmitting information in a remote operation system. An information transmission method in the remote operation system according to claim 9,
The operated terminal is:
Along with the compressed image data, information indicating the selected compression method is transmitted to the operation terminal,
The operation terminal is
In the remote operation system, wherein the divided image data is restored from the transmitted compressed image data by a restoration method corresponding to the compression method specified based on the transmitted information indicating the selected compression method Information transmission method. An information transmission method in the remote operation system according to claim 9 or 10,
The operated terminal is:
Along with the compressed image data, the coordinate information when the divided image data is displayed on the display is transmitted to the operation terminal,
The operation terminal is
An information transmission method in a remote operation system, wherein drawing data is generated based on the restored divided image data and the coordinate information. It is the information transmission method in the remote operation system of any one of Claims 9-11,
The compressed image data has a color palette,
The operation terminal is
An information transmission method in a remote operation system, wherein the divided image data is restored from the transmitted compressed image data based on color information of the color palette.

Images