JP2010186369A - Image transfer device, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve efficient image transfer by reducing the calculation load of update data without damaging ease of use for a user in an image transfer technology for transferring the image of a display screen of a CPU. <P>SOLUTION: An image transfer device 100 is connected to an image display terminal 200, and configured to transfer the image of the GUI(graphical interface) of an operation system to the image display terminal 200 as necessary, wherein an active window in the GUI is detected, and image data in the active window are compressed, and transferred to the image display terminal 200. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image transfer apparatus, method, and program, and more particularly, to a technique for efficiently transferring a GUI (graphical interface) screen.

  As a screen transmission method, there is a method using a moving image compression method such as MPEG. In this method, the entire latest frame (screen) is compared with the previous frame (that is, an image obtained by encoding and decoding the previous frame), and the difference is compressed and transmitted. That's it. In this method, when the data amount of the difference between frames is larger than the transmission band, the image quality is lowered by increasing the quantization step at the time of compression, and when the difference is small and the band becomes free, the image quality is increased. This is done by reducing the quantization step. According to this method, the update frequency of the image data to be transmitted does not greatly decrease compared to the actual screen update frequency due to the limitation of the bandwidth that can be used for screen transfer. Image transmission becomes possible.

  As technologies related to the present invention, there are technologies described in Patent Documents 1 and 2. In Patent Document 1, the screen is divided into blocks, priorities are set for each block according to the number of times the screen has changed, and the screen of the block with the lower number of screen changes is preferentially transmitted. The method is described.

  Further, in Patent Document 2, the transmission image data determination unit acquires information about the generation result of the update image data at a fixed time period, and based on the acquired information and the transmission result information, the update image data or the screen Which of the latest screen image data held in the holding unit is to be transmitted, which image data area is to be transmitted when transmitting the latest screen image data, and the image data to be transmitted There is a description of a screen transfer device that determines image quality (see paragraphs 0028, 0035-0041, etc.).

JP 2004-086550 A JP 2006-246153 A

  In the above-described moving image compression method such as MPEG, in order to obtain the difference between the transmitted frame and the current frame, a process for decoding the encoded data is generated, and the burden is large. Further, even when the control as in Patent Document 1 is performed, there is a possibility that waste is generated because the update rectangle is calculated in units of a predetermined rectangular area.

  In the conventional screen transfer method, after full screen capture is performed, update data is generated by comparison with the previous screen data. In this method, it is necessary to secure a memory area for holding previous screen data extending over the entire drawing area. Furthermore, a load for calculating the update area is applied. In addition, the display screen is captured at a fixed time period as described above, update data is generated by comparing the captured image data with the previous image data, and the moving image data is compressed based on the update data. Hereinafter, the method of transferring to another terminal may be referred to as “differential transfer method”.

  In the technical field of image transfer devices that transfer and display GUI screens to image display terminals connected via a network, the available network bandwidth is limited, and the amount of data transferred is compressed. It is required to do. Therefore, the transfer data is compressed using the above-described differential transfer method and data compression technology using predictive coding.

  However, when the available hardware resources are insufficient, there is a problem that it takes time to capture the GUI screen, and delays image transfer. On the other hand, if importance is placed on avoiding the image transfer delay and the time period for capturing the display screen is made large and the image quality is lowered, the ease of use for the user is impaired.

  Therefore, in view of the above circumstances, the present invention realizes efficient image transfer by reducing the calculation burden of update data without losing the ease of use for the user in the image transfer technology for transferring the image of the GUI display screen. An object of the present invention is to provide an image transfer apparatus, method and program.

  In order to achieve the above object, an image transfer apparatus, method, and program having the following configurations are provided.

  An image transfer apparatus according to the present invention includes a transfer means for transferring a GUI (graphical interface) image of an operating system to an image display terminal, an active window detection means for detecting an active window of the GUI, and the detected active A capture unit that captures a drawing area of a window, and a change in the image of the active window image captured by the capture unit in a certain period of time is extracted and transferred using the extracted change in the image Compression means for compressing an image.

  An image transfer method according to the present invention includes an active window detection step for detecting an active window in a GUI (graphical interface) of an operating system, and a capture step for capturing a drawing area of the detected active window at a constant time period. And extracting a change in the captured image of the active window, compressing the image using the extracted change in the image, and transferring the compressed image to the image display terminal It is characterized by including these.

  An image transfer program according to the present invention captures, in a computer, an active window detection process for detecting an active window in a GUI (graphical interface) of an operating system, and a drawing area of the detected active window at a constant time period. Capture processing, extracting changes in the captured image of the active window, using the extracted changes in the image, compressing the image, and transferring the compressed image to the image display terminal The transfer processing is executed.

  According to the present invention, in an image transfer technique for transferring an image on a GUI display screen, it is possible to realize efficient image transfer by reducing the calculation burden of update data without impairing ease of use for the user. Become.

1 is a block diagram illustrating a configuration of an image transfer apparatus according to a first embodiment of the present invention. It is a block diagram which shows the structure of the image transfer apparatus which concerns on 2nd Embodiment by this invention. It is a figure which shows an example of the GUI screen for demonstrating operation | movement of the image transfer apparatus which concerns on 2nd Embodiment by this invention. It is a flowchart which shows operation | movement of the image transfer apparatus which concerns on 2nd Embodiment by this invention.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

<First Embodiment>
Referring to FIG. 1, an image transfer apparatus 100 according to the present embodiment is shown together with an image display terminal 200. The image transfer apparatus 100 illustrated in FIG. 1 is realized by causing a general computer including a central processing unit, a storage device, an input device, and the like to execute the program according to the present embodiment.

  The image transfer apparatus 100 includes an operating system having a graphical interface (Graphical User Interface: hereinafter referred to as “GUI”), and has a function of displaying the GUI on a display device (not shown). Hereinafter, the information displayed on the display device is referred to as a “GUI screen”.

  The image transfer apparatus 100 has a function of compressing and encoding image data related to the GUI screen and transferring the image data to the image display terminal 200 connected via a network by wired or wireless communication. The image display device 200 decodes the transferred image data, generates a GUI screen, and provides it to the user.

  As illustrated in FIG. 1, the image transfer apparatus 100 includes an active window detection unit 101, a capture unit 102, a previous image data storage unit 103, an update data generation unit 104, a compression unit 105, and a communication unit 106. Each part generally has the following functions.

  The active window detection unit 101 has a function of detecting an active window in the GUI. Here, the “active window” refers to a window having an input focus on the GUI screen. As a specific method of detection, a method of obtaining information on input focus from the operating system can be used.

  The capture unit 102 has a function of capturing a part or all of the GUI screen as image data. The capture operation by the capture unit 102 is continuously performed at a constant time period (for example, every several milliseconds).

  In this embodiment, the capture unit 102 can capture the entire GUI screen, that is, the entire GUI drawing area. However, when an active window is detected by the active window detection unit 101, only the drawing area of the active window is captured. To capture.

  The previous image data storage unit 103 functions as a storage area in which the image data captured by the capture unit 102 is temporarily stored for subsequent compression processing.

  The update data generation unit 104 has a function of generating update data by comparing the image data captured by the capture unit 102 with the previous image data temporarily stored in the previous image data storage unit 103.

  The compression unit 105 has a function of compressing moving image data based on the still image based on the image data captured by the capture unit 102 and the update data generated by the update data generation unit 104. The capture unit 102 to the compression unit 105 function as means for realizing efficient moving image compression by predictive encoding.

  The communication unit 106 has a function of transmitting and transferring the compressed and encoded moving image data to the image display terminal 200. Various types of communication interfaces can be used regardless of whether they are wired or wireless. The image display terminal 200 has a function of decompressing / decoding compressed moving image data and displaying it.

  Hereinafter, the effect of this embodiment will be described.

  In the conventional screen transfer method, after full screen capture is performed, update data is generated by comparison with the previous screen data. In this method, it is necessary to secure a memory area for holding previous screen data extending over the entire drawing area. Furthermore, a load for calculating the update area is applied. On the other hand, since the above embodiment first detects an active window and captures only that portion, the following advantages can be obtained.

  First, since only the active window is captured from the previous screen capture process, the capture time can be reduced. Next, a memory area for holding the previous screen data can be saved. Furthermore, the greatest merit is that it is not necessary to perform processing for generating update data. That is, according to the above-described embodiment, it is possible to greatly reduce the calculation burden for detecting an update area by detecting an active window and updating only that area.

  Note that, according to the above embodiment, drawing is not updated for the drawing area excluding the active window, but the drawing of the active window that is likely to be mainly operated by the user at present is performed for a certain period of time. Since it is updated periodically, the ease of use for the user is not significantly impaired.

  That is, according to the above-described embodiment, in the image transfer technology for transferring the image of the GUI display screen, it is possible to efficiently transfer the image by reducing the calculation burden of the update data without impairing the ease of use for the user. .

<Second Embodiment>
In addition to the configuration of the first embodiment, the present embodiment includes the following configuration and operation for drawing and displaying a pointer (mouse pointer or the like) in a GUI, and is easier for the user than the first embodiment. This is a further improvement.

  The problem in the first embodiment described above is that updating only the active window, even if the user holds the mouse pointer in a part that is not the active window, that part is not recognized as the active window. The mouse pointer is not displayed because it is not updated.

  Therefore, in the present embodiment, when the mouse pointer moves to a portion that is not an active window, the previous screen is transmitted once. When the mouse pointer moves over the active window after that, only the active window is updated.

  Referring to FIG. 2, an image transfer apparatus 100 according to the present embodiment is shown together with an image display terminal 200. The image transfer apparatus 100 illustrated in FIG. 2 includes a pointer position determination unit 107 in addition to the configuration illustrated in FIG. In addition, description is abbreviate | omitted about the site | part provided with the function equivalent to what was once demonstrated by description of the said 1st Embodiment.

  The pointer position determination unit 107 has a function of detecting the position of the pointer in the GUI and determining whether the pointer is on the active window. As a specific method of detecting the pointer position, a method of obtaining the coordinates of the pointer on the GUI display screen from the operating system can be used.

  Referring to FIG. 3, an example of a GUI display screen for explaining the operation of the image transfer apparatus 100 according to the present embodiment is shown. As illustrated, in the image display terminal 200 according to the present embodiment, the transferred GUI display screen is displayed on the entire image display unit 201, and an active window 202 and an inactive window 203 are arbitrarily arranged.

  In this embodiment, when the pointer is on the active window (for example, the position of the pointer 204 in FIG. 3), partial capture is performed only within the active window to generate update data, and compression / transfer processing is performed. On the other hand, when it is outside the active window (for example, the position of the pointer 205 in FIG. 3), the entire drawing area (full screen) of the GUI is captured and update data is generated. This will be described in more detail with reference to the flowchart of FIG.

  FIG. 4 shows the operation of the image transfer apparatus 100 according to the present embodiment. The operation shown in FIG. 4 is a single image transfer (remote screen) operation. In FIG. 4, when the image transfer of this embodiment is started, the drawing of all the drawing areas is first updated (step S101), and the pointer position determination unit 107 determines whether or not the pointer is on the active window. Judgment is made (step S102).

  When the pointer is on the active window, the capture unit 102 performs partial capture of only the drawing area of the active window (step S103), and generation of update data, compression processing, and transfer processing are performed based on the result (step S105). ).

  On the other hand, if the pointer is not on the active window, the capture unit 102 captures the entire drawing area of the GUI (step S104), and update data generation, compression processing, and transfer processing are performed based on the result (step S105). ).

  The above steps S102 to S105 are continued until the image transfer is completed (step S106 / Yes).

  Hereinafter, the effect of this embodiment will be described.

  According to the above-described embodiment, in the image transfer technology for transferring the image of the GUI display screen, it is possible to reduce the calculation load of the update data and to efficiently transfer the image without impairing the ease of use for the user. In addition, problems that may occur when the pointer goes out of the active window (such as a missing pointer) can be solved. That is, ease of use for the user is improved.

<Other embodiments>
In each embodiment described above, an active window is detected, a change in the drawing area of the detected active window is extracted as update data, and moving image data to be transferred is compressed based on the extracted change. No particular mention is made of the mode of data compression outside the drawing area of the active window.

  However, even when an active window is detected, transferring an image outside the active window drawing area is advantageous in that it improves the ease of use for the user. Therefore, in the second embodiment, even when the active window detection unit 101 detects an active window, the capture unit 102 captures the entire drawing area of the GUI at a constant time period, and the captured image data The moving image data is compressed based on the change. Here, it is assumed that the time period for capturing the entire drawing area of the GUI is longer than the time period for capturing the drawing area of the active window.

  According to this configuration, since the entire drawing area of the GUI is captured and transferred, the usability for the user is improved, while the time period for capturing the entire drawing area of the GUI captures the drawing area of the active window. Therefore, the effects of reducing the load and reducing the amount of transferred data in image transfer are not suppressed.

  In addition, with regard to load reduction and transfer amount reduction in image transfer, the compression rate may be changed for the active window and the others, and only the active window may be transmitted with high image quality. This configuration is a configuration in which an image related to the drawing area of the active window is compressed and transferred at a lower compression than an image related to the other drawing area.

  Even in this configuration, since the entire drawing area of the GUI is captured and transferred, the usability for the user is improved. On the other hand, the image in the area excluding the active window is transferred with high compression. The effect of reducing the load in transfer and reducing the amount of transferred data is not suppressed.

  The preferred embodiment of the present invention has been described above, but the present invention is not limited to this, and various modifications can be made without departing from the scope of the invention. The configurations disclosed in the above embodiments can be implemented in combination with each other.

DESCRIPTION OF SYMBOLS 100 Image transfer apparatus 101 Active window detection part 102 Capture part 103 Previous image data storage part 104 Update data generation part 105 Compression part 106 Communication part 107 Pointer position judgment part 200 Image display terminal 201 Image display part 202 Active window 203 Inactive window 204 , 205 pointer

Claims (7)

Connected to an image display terminal,
An image transfer device for transferring a GUI (graphical interface) image of an operating system to the image display terminal as needed,
Detecting an active window in the GUI;
Compress the image data in the active window,
An image transfer apparatus for transferring to the image display terminal. Transfer means for transferring an image of a GUI (graphical interface) of the operating system to an image display terminal;
Active window detection means for detecting an active window of the GUI;
Capture means for capturing a drawing area of the detected active window;
For the image of the active window captured by the capture means at a constant time period, a change means for extracting a change in the image and using the change in the extracted image to compress the image to be transferred;
An image transfer apparatus comprising: Pointer position determining means for determining whether or not the GUI pointer is on the active window;
The capture means captures the drawing area of the active window when the pointer position determining means determines that the pointer is on the active window, and if it is determined that the pointer is not on the active window, The image transfer apparatus according to claim 2, wherein an area is captured. The capture means captures the entire drawing area of the GUI with a period longer than the fixed time period for capturing the drawing area of the active window,
The compression unit extracts a change in an image with respect to the entire drawing area of the captured GUI, and compresses an image to be transferred using the change in the extracted image. 3. The image transfer device according to 3.   5. The image transfer apparatus according to claim 4, wherein the compression means compresses the image of the active window with low compression compared to the image of the area excluding the active window. An active window detection step of detecting an active window in an operating system GUI (graphical interface);
A capture step of capturing a drawing area of the detected active window at a constant time period;
For the captured image of the active window, a change step of extracting a change in the image and using the change in the extracted image to compress the image;
A transfer step of transferring the compressed image to the image display terminal;
An image transfer method comprising: On the computer,
An active window detection process for detecting an active window in an operating system GUI (graphical interface);
Capture processing for capturing a drawing area of the detected active window at a constant time period;
For the captured image of the active window, a change in the image is extracted, and a compression process for compressing the image using the change in the extracted image;
An image transfer program for executing a transfer process for transferring a compressed image to an image display terminal.

Images