JP3797372B2 - Image transfer using drawing command hook - Google Patents

Description

  The present invention relates to an image display system including an image supply device and an image display device connected via a network.

  As a kind of projector for projecting and displaying an image, a projector capable of projecting and displaying an image supplied from a personal computer via a network (hereinafter also referred to as “network projector”) has been proposed (for example, Patent Document 1). . In addition, a technique called VNC (Virtual Network Computing) is used to supply such a network projector with an image from a personal computer via a network.

International Publication No. 01/093583 Pamphlet

  However, when using VNC, an image is temporarily transferred from a VRAM (frame memory) in the computer to a RAM (system memory), and then the image is acquired from the RAM and transferred to the projector via the network. . Since transfer of image data from the VRAM to the RAM takes a considerable time, there is a case where a sufficient transfer speed cannot be obtained as a whole. Such a problem is not limited to a network projector system, and is generally a problem common to systems that supply and display images from an image supply apparatus to an image display apparatus via a network.

  An object of the present invention is to provide a technique for improving the transfer speed when transferring an image to an image display device via a network.

In order to achieve at least a part of the above object, a first image supply device according to the present invention is an image supply device for supplying an image to an image display device via a network for display.
An application program capable of issuing image drawing commands;
A drawing module for processing a drawing command issued by the application program;
A hook processing module that hooks a specific drawing command issued by the application program in advance and draws an image in a specific transfer image storage area in the general-purpose memory according to the acquired drawing command;
An image transfer processing module that acquires an image from the transfer image storage area and transfers the acquired image to the image display device via the network;
With
The hook processing module stores, in the general-purpose memory, first change area information indicating a first change area that is an area in which an image is drawn in response to the specific drawing command in the transfer image storage area. A function of writing, and a function of supplying the drawing command to the drawing module after processing the specific drawing command;
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) An image part corresponding to the first change area and an image part corresponding to an area not included in the first change area and included in the history area among the second change areas Obtain from the transfer image storage area without obtaining from the frame memory,
(Iv) The acquired image portion is transferred to the image display device via the network together with screen update area information indicating a screen update area which is a sum of the first and second change areas.

  According to this image supply device, when a specific drawing command is issued, the hook processing module hooks and acquires the drawing command instead of the drawing module, and draws the image in the general-purpose memory. As described above, the entire transfer processing speed can be improved as compared with the case where the entire image to be transferred is transferred from VRAM to RAM (general purpose memory). Further, among the second change areas, areas that are not included in either the first change area or the history area are areas that are not drawn by the hook processing module, but this image is acquired from the frame memory. It is possible to transfer images without missing. Further, an image portion corresponding to the first change region and an image portion of the second change region that is not included in the first change region and that corresponds to the region included in the history region are not acquired from the frame memory. Since it is acquired from the transfer image storage area, the time required to acquire these image portions can be reduced.

A second image supply device according to the present invention is an image supply device for supplying and displaying an image on an image display device via a network,
An application program capable of issuing image drawing commands;
A drawing module for processing a drawing command issued by the application program;
A hook processing module that hooks a specific drawing command issued by the application program in advance and draws an image in a specific transfer image storage area in the general-purpose memory according to the acquired drawing command;
An image transfer processing module that acquires an image from the transfer image storage area and transfers the acquired image to the image display device via the network;
With
The hook processing module stores, in the general-purpose memory, first change area information indicating a first change area that is an area in which an image is drawn in response to the specific drawing command in the transfer image storage area. A function of writing, and a function of supplying the drawing command to the drawing module after processing the specific drawing command;
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) Of the second change area, an image portion corresponding to an area included in at least one of the first change area and the history area is acquired from the transfer image storage area without obtaining from the frame memory. Acquired,
(Iv) The acquired image portion is transferred to the image display device via the network together with screen update area information indicating a screen update area that is the same as the second change area.

  Even in the second image supply device, when a specific drawing command is issued, the hook processing module hooks and acquires the drawing command instead of the drawing module, and draws the image in the general-purpose memory. Thus, compared to the conventional case where the entire image to be transferred is transferred from the VRAM to the RAM (general purpose memory), the speed of the entire transfer process can be improved. Further, among the second change areas, areas that are not included in either the first change area or the history area are areas that are not drawn by the hook processing module, but this image is acquired from the frame memory. It is possible to transfer images without missing. Further, since the image portion corresponding to the area included in at least one of the first change area and the history area in the second change area is acquired from the transfer image storage area without acquiring from the frame memory, The time required to acquire the image portion can be shortened.

  The image transfer processing module acquires, from the frame memory, an image portion corresponding to an area that is not included in any of the first change area and the history area in the second change area and transfers the image. After writing in the image storage area, an image portion corresponding to the screen update area may be acquired from the transfer image storage area.

  According to this configuration, when the image is finally transferred via the network, the image can be acquired from one transfer image storage area, so that the processing is simplified.

The image transfer processing module is
The hook processing module may be loaded into the application program when the image transfer processing module is activated, and the hook processing module may be unloaded from the application program when the execution of the image transfer processing module is completed.

  According to this configuration, since the hook processing module works only when the image transfer process is performed, it is possible to prevent the hook processing module from desirably executing the hook process when the hook process is unnecessary.

  The present invention can be realized in various forms, for example, an image display system, an image supply device and an image display device that constitute the image display system, a hook processing module and an image transfer used in the image supply device. The present invention can be realized in various modes such as a processing module, an image supply method, a computer program for realizing the method or apparatus, and a recording medium on which the computer program is recorded.

Hereinafter, embodiments of the present invention will be described in the following order.
A. First embodiment:
B. Second embodiment:
C. Variations:

A. First embodiment:
FIG. 1 is an explanatory diagram showing the configuration of an image display system as an embodiment of the present invention. The image display system 10 of the present embodiment includes a personal computer 100 as an image supply device, a projector 200 as an image display device, and a network 300 that connects the computer 100 and the projector 200. The computer 100 has a function of supplying an image to the projector 200 via the network 300 and causing the projector 200 to project the image and display it on the projection display screen 70.

  FIG. 2 is a block diagram showing the internal configuration of the computer 100 and the projector 200. The computer 100 includes a CPU 102, a ROM 104, a RAM 106 as a general-purpose memory (also referred to as “system memory”), a hard disk drive 108, an input unit 110 including a keyboard and a pointing device, a network interface unit 112, A VRAM 114 as a frame memory, a graphic controller 116, a display device 118 such as a liquid crystal display, and a bus 120 for connecting these elements are provided.

  The RAM 106 stores various computer programs including an application program 122, a graphical device interface (GDI) 124, a display driver 126, a projector driver 128, and a VNC server 130 as an image transfer processing module. Has been. The GDI 124, the display driver 126, and the projector driver 128 function as a part of the operating system. In this embodiment, Windows (registered trademark) provided by Microsoft Corporation is assumed as the operating system. Such various computer programs are provided in a form recorded on a computer-readable recording medium such as a flexible disk or a CD-ROM.

  The projector 200 includes a CPU 202, a ROM 204, a RAM 206, an input unit 210 including various operation buttons, a network interface unit 212, an image processing unit 214, a light source lamp, a liquid crystal panel, and a projection unit 216 including a projection optical system. And a bus 218 for connecting these elements. The ROM 204 stores various computer programs including the image processing driver 228 and the VNC client 230.

  Hereinafter, before describing the function of the computer 100 in the embodiment, the function of a conventional computer using a VNC server will be described as a comparative example.

  FIG. 3 is an explanatory diagram showing a hierarchical structure of computer software and hardware in a comparative example. The application program 122 and the VNC server 130 belong to an application program layer (also referred to as “user application program layer”). The GDI 124, the display driver 126, and the projector driver 128 belong to the kernel layer. Further, the RAM 106 as a general-purpose memory, the network interface unit 112, the VRAM 114, the graphic controller 116, and the display device 118 belong to the hardware layer.

  In the first embodiment, a case where a presentation program (for example, Microsoft PowerPoint) is used as the application program 122 will be described.

  The application program 122 issues various drawing requests to the GDI 124. The GDI 124 is a computer program that uniformly manages drawing on a display device (the display device 118 or the projector 200 of the computer 100) or a printing device (not shown). As is well known, the GDI 124 provides an application program interface (API: Application Program Interface) relating to drawing called “GDI function” to various application programs. Note that an API generally refers to a collection of procedures for an application program to use various functions of an operating system.

  For example, the application program 122 issues a drawing request for a presentation sheet image included in the presentation file to the GDI 124. Usually, the drawing request also includes information regarding the output destination of the image (that is, information specifying whether to output the image to the display device or the printing device).

  Note that other general-purpose drawing APIs may be used instead of the GDI 124. The “general-purpose drawing API” means an API that is commonly used for many application programs.

  The GDI 124, the display driver 126, and the projector driver 128 all function as a drawing module that executes a drawing process in accordance with a drawing command. Specifically, the GDI 124 receives the drawing request issued from the application program 122, checks the output destination of the image based on the drawing request, and if the output destination is a display device, the display driver 126 and the projector driver A drawing request is passed to each of 128.

  The display driver 126 draws image data in the VRAM 114 in accordance with the received drawing request. As is well known, the drawing request is not limited to drawing the entire screen, but frequently drawing a part of the screen. When receiving a drawing command for drawing a part of the screen, the display driver 126 draws only an image portion of an area to be drawn (also referred to as “change area”) in the VRAM 114. The graphic controller 116 displays an image on the display device 118 based on image data (that is, bitmap data) drawn in the VRAM 114. On the other hand, the projector driver 128 writes the change area information 106 a indicating the position and size of the drawing target area (that is, the change area) in the VRAM 114 in the RAM 106 in accordance with the drawing request received from the GDI 124.

  4A and 4B are explanatory diagrams showing examples of the change area information 106a and the image data storage area 106b in the RAM 106. FIG. Here, it is assumed that the image to be projected and displayed by the projector changes from FIG. 4 (A) to FIG. 4 (B). As will be described later, the image data storage area 106 b stores image data transferred from the VRAM 114 by the VNC server 130. The image data storage area 106b preferably has the same data capacity as the VRAM 114 (frame memory).

  The change area Ra is an area of an image portion drawn by a drawing command when changing the image from FIG. 4 (A) to FIG. 4 (B). The change area information 106a regarding the change area Ra includes the x coordinate Xa and the y coordinate Ya of the reference point (usually the upper left point) of the change area Ra, and the width Wa and the height Ha of the change area Ra. The coordinates (Xa, Ya) of the reference point are data indicating the position of the change area Ra, and the width Wa and the height Ha are data indicating the size of the change area Ra.

  The VNC server 130 refers to the change area information 106 a in the RAM 106 and transfers the image data in the change area Ra from the VRAM 114 to the image data storage area 106 b in the RAM 106. The image data storage area 106b is a memory area having a size equal to the display resolution of the computer 100 or the projector 200, for example. The VNC server 130 further reads out the image data of the change area Ra from the image data storage area 106b, and creates a screen update message including the change area information 106a of the change area Ra and the image data.

  FIG. 5 is an explanatory diagram showing the structure of the screen update message. This screen update message conforms to the structure of the frame buffer update “FramebufferUpdate” defined in the VNC protocol. The screen update message has a message header portion indicating a message type and a parameter portion indicating a message parameter.

  The message header part includes a message type (1 byte) indicating the type of message and a change area number (2 bytes) indicating the number of change areas. The data value of the message type indicating the screen update message is “0”. The data value of the number of change areas is normally set to “1”. However, when the change area information 106a regarding a plurality of change areas remains unprocessed in the RAM 106, the screen update message includes information about the change areas (change area information and image data). It is also possible to construct an update message. The change area information included in the screen update message is also referred to as “screen update area information”. The message type data and the change area number data are divided through 1-byte padding.

  In the parameter section, the x coordinate (2 bytes) and y coordinate (2 bytes) of the reference point of the change area, the width (2 bytes) and height (2 bytes) of the change area, and the transfer of the image data of the change area Time encoding format (4 bytes) and change area image data (n bytes) are included. A predetermined format such as a JPEG format (data value “7”) is used as the encoding format.

  The VNC server 130 transfers the screen update message thus created to the projector 200 via the network interface unit 112.

  As described above, in the comparative example shown in FIG. 3, the image data once stored in the VRAM 114 is transferred to the RAM 106 which is a general-purpose memory, and then the image data in the RAM 106 is transferred to the projector 200. For this reason, it takes a considerable time to transfer data from the VRAM 114 to the RAM 106, and a sufficient transfer speed may not be obtained.

  FIG. 6 is an explanatory diagram showing a hierarchical structure of computer software and hardware in the first embodiment. 3 is different from the comparative example shown in FIG. 3 in that a hook processing module 400 is added between the application program 122 and the GDI 124, and two types of change area information 106a1 and 106a2 (hereinafter, “ Change area information A1, A2 ") is stored, and transfer history information 106c (hereinafter referred to as" transfer history information HA1 ") is stored in the RAM 106. It is the same as FIG. The hook processing module 400 hooks a specific drawing command issued by the application program 122 in advance and executes processing described later. Other drawing commands are processed by the GDI 124 as usual. Note that the hook processing module 400 is preferably configured exclusively for each application program so as to hook only a specific drawing command appropriate for the application program 122.

  The hook processing module 400 can also be considered as a kind of drawing API, but is distinguished from a general-purpose drawing API such as the GDI 124. That is, the hook processing module 400 is typically configured to process only specific drawing commands, whereas the GDI 124 is configured to process all drawing commands. .

  The hook processing module 400 is loaded (installed) into the application program 122 as a DLL (dynamic link library), for example. In general, an application program operates in units of “processes”. During execution of an application program, multiple modules (GDI32.DLL, Kernel32.DLL, etc.) are loaded in the process, and the application program performs various operations by using the functions in those modules. Is going. The “loading” of the hook processing module 400 means that the module 400 in which the hook procedure is mounted is incorporated in the process of the application program 122.

  When the hook processing module 400 is loaded, the function address of a specific drawing command issued from the application program 122 is the address of another function for executing a unique function implemented in the hook processing module 400. Is replaced. In the function of this unique function, processing for writing change information and image data into the RAM (described later) is executed, and then a normal GDI function is executed. When the hook processing module 400 is unloaded, the hook processing module 400 loaded in the process is unloaded after the function address replaced at the time of loading is restored.

  The hook processing module 400 can be loaded into the application program 122 and unloaded at any time as necessary. However, when the VNC server 130 is activated, the hook processing module 400 is loaded into a specific application program 122 (presentation program in this embodiment) that is operating, and when the VNC server 130 is terminated, the hook processing module 400 is loaded. Preferably, the VNC server 130 is configured to unload. In this way, since the hook processing module 400 can be operated only during the period when the VNC server 130 is being executed, a drawing command is hooked when the VNC server 130 is not executed (not activated) on the computer 100. And undesirable results can be avoided. In the present specification, the phrase “VNC server 130 is running on computer 100” means that the process of VNC server 130 is activated.

  FIG. 7 is an explanatory diagram showing an example of two change area information A1, A2 and transfer history information HA1. FIGS. 7A-1 to 7A-3 show states at time t0, and FIGS. 7B-1 to 7B-3 show states at time t1. It is assumed that the VNC server 130 repeatedly executes the transfer process at a predetermined time interval, and after the transfer is executed at time t0, the next transfer is executed at time t1.

  The first change area information A1 shown in FIG. 7A-1 is an area RA1 (referred to as “change area RA1”) drawn directly in the image data storage area 106b in the RAM 106 by the hook processing module 400 (FIG. 6). ). The image in the change area RA1 is referred to as “image G1”. The first change area information A1 is written into the RAM 106 by the hook processing module 400. Second change area information A2 shown in FIG. 7A-2 represents an area RA2 (referred to as “change area RA2”) drawn in the VRAM 114 by the display driver 126 (FIG. 6). The image in the change area RA2 is referred to as “image G2.” The second change area information A2 is written in the RAM 106 by the projector driver 128. The transfer history information HA1 shown in FIG. 7A-3 is the past first change area RA1 drawn directly in the image data storage area 106b by the hook processing module 400 and transferred by the VNC server 130 in the preceding predetermined period. (Referred to as “history area”). In this embodiment, only the first change area that is the transfer target in the immediately preceding transfer process is set as the history area. However, a change area in a plurality of preceding transfer processes can be a history area. In the transfer process immediately before time t0, it is assumed that the first change area RA1 does not exist, and therefore transfer history information HA1 is not registered at time t0. The transfer history information HA1 is written into the RAM 106 by the VNC server 130.

  In the example of FIG. 7, a state in which the respective areas are arranged in the same screen area SCA is shown. Here, the “screen area SCA” means the entire area of the image stored in the VRAM 114, which is equal to the entire area of the image data storage area 106b. Strictly speaking, the first change area RA1 is an area defined by an address in the image data storage area 106b, and the second change area RA2 is an area defined by an address in the VRAM 114. It can be expressed by the same screen area SCA.

  As shown in FIGS. 7B-1 to (B-3), at time t1, the first change area information A1 does not exist, and the second change area information A2 is the area of the image G1 (that is, time This represents the same region as the first change region RA1 at t0). In this state, the drawing command for the image G1 is processed by being hooked by the hook processing module 400 immediately before time t0, and then processed by the drawing modules 124, 126, and 128 between times t0 and t1. Sometimes it can happen. As shown in FIG. 7B-3, at time t1, the first change area information A1 at time t0 is registered as transfer history information HA1. In FIG. 7 (B-3), the history area RHA1 is drawn with a one-dot chain line for convenience of illustration. In the transfer process (process by the VNC server 130) described below, the transfer process at these times t0 and t1 will be described.

  FIG. 8 is a flowchart illustrating operations of the hook processing module 400 and the VNC server 130 in the embodiment. Steps S10 to S16 are executed by the hook processing module 400, and steps S20 to S26 are periodically executed by the VNC server 130.

  When a specific drawing command is issued from the application program 122, the hook processing module 400 hooks the drawing command in step S10 and acquires the drawing command instead of the GDI 124. Note that it is preferable that the drawing command hooked by the hook processing module 400 is not limited to all drawing commands, but only limited specific drawing commands. This is because there are many drawing commands issued by the application program 122. If all of them are hooked, there is a possibility that the processing speed may be lowered or the desired drawing may not be performed. Because.

  In step S12, a process of registering the change area information is executed according to the acquired drawing command. Specifically, the hook processing module 400 acquires the position and size of the area (the change area RA1 in FIG. 7A-1) where drawing is performed according to the drawing command, and changes area information A1 representing the change area is obtained. A process of directly writing to the RAM 106 is executed.

  In step S14, the image data writing process of the change area RA1 is executed. Specifically, the hook processing module 400 executes a process of expanding the image data in the change area RA1 in accordance with the drawing command and directly writing the image data in the image data storage area 106b in the RAM 106.

  In step S16, the hook processing module 400 calls a normal GDI function according to the drawing command acquired in step S10, and executes the drawing process. Note that the display driver 126 and the projector driver 128 (FIG. 6) execute respective processes in accordance with the drawing command received from the GDI 124, as in the comparative example shown in FIG. That is, the display driver 126 draws an image in the VRAM 114, and the projector driver 128 writes the change area information A2 indicating the drawing area in the RAM.

  On the other hand, in step S20, the VNC server 130 determines an area of image data to be transferred to the projector 200 (referred to as a “screen update area”). FIG. 9 is a flowchart showing the detailed procedure of step S20. In steps S50 and S52, first and second change area information A1 and A2 are acquired. As described above, the first change area information A1 is information representing an area drawn in the image data storage area 106b in the RAM 106 by the hook processing module 400, and is written in the RAM 106 by the hook processing module 400. It is. On the other hand, the second change area information A2 is information representing an area drawn in the VRAM 114 by the display driver 126, and is written in the RAM 106 by the projector driver 128.

  FIG. 10A shows two change areas RA1, RA2 and a history area RHA1 at time t0. However, the history area RHA1 does not exist at the time t0.

  In the example of FIG. 10A, the two change areas RA1 and RA2 partially overlap, but the two change areas RA1 and RA2 may have various other relationships. For example, there are cases where the two change regions RA1 and RA2 do not overlap at all, and there are also cases where the two change regions RA1 and RA2 completely overlap. For example, when the hook processing module 400 performs drawing in accordance with a specific drawing command, and then the normal drawing modules 124, 126, and 128 perform drawing in the same region in accordance with the same drawing command, the two change regions RA1, RA2 Are the same. However, even when the same drawing command is executed, there may be a slight difference between the two change areas RA1 and RA2 depending on the setting of each module.

  FIG. 10B shows an area TG1 where the VNC server 130 directly acquires an image from the RAM 106 at the time t0 when the image is transferred, and FIG. 10C shows an area TG2 where the image is acquired from the VRAM 114. These areas TG1 and TG2 are also referred to as “transfer areas”. As can be understood from this example, when the transfer history information HA1 does not exist, the image of the first change area RA1 (= TG1) is directly acquired from the RAM 106, and the first change area RA2 includes the first change area RA1. The image of the area TG2 that is not included in the change area RA1 is acquired from the VRAM 114. The reason for this distinction is to reduce the number of images acquired from the VRAM 114 as much as possible and reduce the time required for the acquisition. For example, when the two change areas RA1 and RA2 are completely overlapped, it is not necessary to acquire an image from the VRAM 114, so that the transfer process can be performed at high speed. FIG. 10D shows the entire area (screen update area TTG) transferred by the VNC server 130 at time t0. This screen update area TTG corresponds to the sum area of the first and second change areas RA1 and RA2.

  The first change area information A1 may include a plurality of information corresponding to a plurality of drawing commands. In this case, the sum area of the drawing areas of the plurality of drawing commands becomes the first change area RA1. The same applies to the second change region RA2.

  In step S54 of FIG. 9, the difference between the change areas RA1 and RA2 represented by the two types of change area information A1 and A2 is calculated. Specifically, a region TG2 (FIG. 10C) that is not included in the first change region RA1 in the second change region RA2 is calculated.

  In step S56, transfer history information HA1 is acquired, and in step S58, the difference between the history area RHA1 and the second change area RA2 is calculated. The process in step S58 is a process for adjusting the second transfer area TG2 (an area in which an image is acquired from the VRAM 114 at the time of transfer). Since the transfer history information HA1 does not exist at the time t0, the processing content will be described later in the context of the processing at the time t1. In step S60, the transfer history information HA1 is discarded, and in step S62, the first change area information A1 is registered as transfer history information HA1. As a result, the first change area information A1 at time t0 can be used as transfer history information HA1 at the next time t1.

  In step S64, the optimization process of the screen update area TTG (FIG. 10D) is executed. FIG. 11 is an explanatory diagram showing the contents of the optimization process of the screen update area. FIG. 11A shows the same screen update area TTG as in FIG. In the optimization process, as shown in FIG. 11B, the screen update area TTG is divided into adjacent rectangular areas R11 to R13 that do not overlap each other. And these rectangular area | regions R11-R13 are employ | adopted as a screen update area | region after optimization. Since each of the screen update areas R11 to R13 is rectangular, it is possible to reduce the amount of image data to be transferred compared to the case of transferring a rectangular image including the screen update area TTG before optimization. is there.

  In step S22 of FIG. 8, image data of an image portion corresponding to the screen update area is acquired. FIG. 12 is a flowchart showing the detailed procedure of step S22. In step S70, it is checked whether or not the second transfer area TG2 (FIG. 10C) is empty. The “second transfer area TG2” is an area in which image data is acquired from the VRAM 114 in the screen update area TTG as described above. When the second change area information A2 does not exist, and when the second change area A2 is completely included in the sum area of the first change area A1 and the history area, the second transfer area TG2 is empty. If the second transfer area TG2 is empty, the process proceeds to step S74 described later. On the other hand, if the second transfer area TG2 is not empty, step S72 is executed. In step S72, the VNC server 130 acquires the image data in the second transfer area TG2 from the VRAM 114 and writes it in the image data storage area 106b in the RAM 106. As a result, all the images in the screen update area TTG shown in FIG. 10D are stored in the image data storage area 106b. In step S74, the VNC server 130 acquires the image data of the screen update area TTG from the image data storage area 106b of the RAM 106. Note that what is actually acquired is the image data of the individual screen update areas R11 to R13 (FIG. 11B) after the optimization processing.

  In step S24 of FIG. 8, the image data is converted in accordance with a preset encoding format. In step S <b> 26, a screen update message (FIG. 5) including the image data thus converted and the change area information is created and transferred to the projector 200 via the network interface unit 112.

  When the network interface unit 212 (FIG. 2) of the projector 200 receives the communication data supplied via the network 300, the network interface unit 212 (FIG. 2) extracts a screen update message included in the communication data and passes it to the VNC client 230. The VNC client 230 passes the image data included in the screen update message to the image processing driver 228. The image processing driver 228 controls the image processing unit 214 according to the image data, and causes the image processing unit 214 to develop the image data in a display memory (not shown) in the image processing unit 214. As a result, the projector 200 can display an image on the projection display screen 70 as shown in FIG.

  Also at time t1, the processes of steps S20 to S26 in FIG. 8 are executed in the same manner. However, at time t1, as shown in FIGS. 7B-1 to 7B-3, the first change area information A1 does not exist, and the second change area information A2 represents the area of the image G1. Information, and transfer history information HA1 is also information representing the area of the image G1.

  FIG. 13 is an explanatory diagram showing the relationship among the change area, history area, and screen update area at time t1. As shown in FIG. 13A, the first change area RA1 does not exist, and the second change area RA2 is the same as the history area RHA1. At this time, in the process of step S54 in FIG. 9, a portion of the second change region RA2 that is not included in the first change region RA1 is created as the second transfer region TG2 (FIG. 13B). . In the process of step S58 in FIG. 9, the history area RHA1 is deleted from the second transfer area TG2. FIG. 13D shows the second transfer area TG2 'after the correction. In this example, since the second change area RA2 and the history area RHA1 are the same, the second transfer area TG2 'after correction is empty. On the other hand, the area deleted from the second transfer area TG2 is added to the first transfer area TG1 (area in which an image is directly acquired from the RAM 106) (FIG. 13C). At time t1, since the second transfer area TG2 ′ is empty, the entire image in the screen update area TTG (FIG. 13E) is acquired directly from the image data storage area 106b without acquiring an image from the VRAM 114. Forwarded.

  As can be understood from the examples of FIGS. 10 and 13, the screen update area TTG that is the target of image transfer is independent of the presence or absence of the history area RHA1, and is the sum area of the first and second change areas RA1 and RA2. As determined. The history area RHA1 is used to determine a first transfer area TG1 for acquiring image data directly from the image data storage area 106b and a second transfer area TG2 (or TG2 ') acquired from the VRAM 114. Specifically, the sum area of the first change area RA1 and the area included in the history area RHA1 in the second change area RA2 is the first transfer area TG1. In addition, an area that is not included in any of the first change area RA1 and the history area RHA1 in the second change area RA2 is the second transfer area TG2 (or TG2 '). The first transfer region TG1 is a sum region of the first change region RA1 and the second change region RA2 that is not included in the first change region RA1 and included in the history region RHA1. We can think that there is. By dividing the first and second transfer areas TG1 and TG2 in this way, the amount of image data acquired from the VRAM 114 can be reduced.

  Assuming that the transfer history information HA1 is not used, the image of the second change area RA2 should be acquired from the VRAM 114 at time t1. On the other hand, in this embodiment, the image of the change area RA2 can be directly acquired from the image data storage area 106b in the RAM 106 without accessing the VRAM 114. This is because the change area RA2 is drawn directly in the image data storage area 106b by the hook processing module 400 at the previous time, so that a correct image can be obtained from the image data storage area 106b without obtaining it from the VRAM 114. This is because it can be acquired. As can be understood from this description, the present embodiment reduces the possibility that a large amount of image data must be acquired from the VRAM 114 due to the difference between the execution timing of the drawing command and the transfer timing of the image. be able to.

  FIG. 14 is an explanatory diagram showing another example of the two change area information A1 and A2 and the transfer history information HA1. FIGS. 14A-1 to 14A-3 show states at a certain transfer timing at time t10, and FIGS. 14B-1 to 14B-3 show the next transfer timing. The state at time t11 is shown. At time t10, the two change area information A1a and A1b indicating the change areas RA1a and RA1b of the two images G1 and G2 are registered as the first change area information A1 to be processed. Only information indicating the change area RA2 of one of the two images G1 and G2 is registered as the second change area information A2. It is assumed that the transfer history information HA1 does not exist at time t10. On the other hand, at time t11, the first change area information A1 does not exist, and information indicating the change area of the image G2 is registered as the second change area information A2. Further, as the transfer history information HA1, two change area information A1a and A1b at time t0 are registered.

  In the transfer process at time t10, as shown in FIG. 15A, the sum area of the two types of change areas RA1 and RA2 is employed as the screen update area TTG to be transferred. This screen update area TTG is equal to the sum area of the first change areas RA1a and RA1b shown in FIG. Accordingly, the images in the screen update area TTG do not need to be acquired from the VRAM 114, and are all acquired directly from the image data storage area 106b.

  In the transfer process at time t11, since the first change area RA1 does not exist, the second change area RA2 is directly adopted as the screen update area TTG as shown in FIG. However, the second change area RA2 is completely included in the history areas RHA1a and RHA1b (FIG. 14B-3). Therefore, the image in the screen update area TTG at time t11 does not need to be acquired from the VRAM 114, and is all acquired directly from the image data storage area 106b.

  As described above, in the first embodiment, the hook processing module 400 hooks a specific drawing command to prefetch and executes a process of directly writing the image data of the change area to the RAM 106. As a result, the time required to transfer the image data from the VRAM 114 to the RAM 106 in the comparative example shown in FIG. 3 can be saved. As a result, there is an advantage that the transfer speed when image data is transferred to the projector 200 can be sufficiently increased.

  In a general personal computer, data transfer from the RAM 106 to the VRAM 114 can be executed at a very high speed, whereas data transfer from the VRAM 114 to the RAM 106 can be executed only several times slower than that. There are many. In the first embodiment, since the data transfer from the low-speed VRAM to the RAM can be reduced, the time required to transfer the image data to the projector 200 can be greatly reduced.

  In the first embodiment, the image portion of the region TG2 that is not included in any of the first change region RA1 and the history region RHA1 in the second conversion region RA2 is acquired from the VRAM 114 and projector. 200. As a result, the images drawn by the normal drawing modules 124, 126, and 128 can be transferred to the projector 200 without omission.

B. Second embodiment:
FIG. 16 is an explanatory diagram showing the relationship between the change area and the screen update area in the second embodiment, and corresponds to FIG. 10 in the first embodiment. In the second embodiment, the overall apparatus configuration and processing flow are the same as those in the first embodiment, but the area to be transferred is different from that in the first embodiment. Specifically, in the second embodiment, the first transfer region TG1 (FIG. 16B) in which an image is directly acquired from the RAM 106 is a portion where the first and second change regions RA1 and RA2 overlap. Set to In other words, the first transfer area TG1 is set to an area included in the first change area RA1 in the second change area RA2. On the other hand, the second transfer region TG2 (FIG. 16C) from which an image is acquired from the VRAM 114 is included in the first change region RA1 in the second change region RA2, as in the first embodiment. It is set to the part that is not. Accordingly, the screen update area TTG (FIG. 16D), which is the entire area to be transferred, is the same as the second change area RA2.

  As described with reference to FIG. 7, since transfer history information HA1 does not exist at time t0, all image portions in the second transfer area TG2 (FIG. 16C) are acquired from the VRAM 114. However, if the transfer history information HA1 exists, the area overlapping the history area in the second transfer area TG2 is deleted from the second transfer area TG2 and the first transfer area TG2, as in the first embodiment. It is added to the transfer area TG1 (see FIG. 13). In other words, the image stick portion of the area included in either the first change area RA1 or the history area RHA1 in the second change area RA2 is directly acquired from the RAM 106. In addition, the bar portion of the second change area RA2 that is not included in any of the first change area RA1 and the history area RHA1 is acquired from the VRAM 114. Therefore, the processing at time t1 shown in FIG. 13 is the same as that in the first embodiment in the second embodiment.

  Thus, in the second embodiment, the image in the second change area RA2, which is an area drawn in the VRAM 114 by the GDI 124 and the display driver 126, is transferred by the VNC server 130. The reason for performing the processing in this way is as follows. That is, since the drawing command hooked by the hook processing module 400 is again executed by the GDI 124 and the display driver 126, basically all drawing commands are executed by the GDI 124 and the display driver 126. Therefore, if the image of the second change area RA2 is transferred, the projector 200 can display a correct image. However, if all the image data is acquired from the VRAM 114 at this time, there arises a problem that a considerable time is required for data transfer. Therefore, in the second embodiment, by directly acquiring image data from the RAM 106 for the portion included in either the first change area RA1 or the history area RHA1 in the second change area RA2, The time required for data acquisition is shortened. Further, the second embodiment is preferable to the first embodiment in that the amount of image data transferred is much smaller than that of the first embodiment.

  In many cases, the area drawn by the hook processing module 400 (first change area RA1) and the area drawn by the GDI 124 and the display driver 126 (second change area RA2) coincide with each other. Yes. Therefore, as in the second embodiment, even if the image transfer of the portion not included in the second change area RA2 in the first change area RA1 is omitted, the image displayed on the projector 400 is uncomfortable. The possibility is negligible. On the other hand, in the first embodiment described above, the image is transferred also in the portion of the first change area RA1 that does not overlap with the second change area RA2. Therefore, the first and second change areas RA1 are assumed. , RA2 has the advantage of being able to transfer a more complete image even if it is significantly different.

  FIG. 17 is an explanatory diagram showing another example of the two change area information A1 and A2 and the transfer history information HA1 in the second embodiment. At time t20 shown in FIGS. 17A-1 to 17A-3, information A1a and A1b representing the regions RA1a and RA1b of the two images G10 and G20 are registered as the first change region information A1. As the second change area information A2, information A2 indicating the area RA2 (= RA1a) of the first image G10 is registered. Here, for convenience of explanation, it is assumed that the first image G10 is an image that occupies the entire screen area SCA, and that the second image G11 is an image that has been slid to the right. Such a state occurs, for example, when slide out is set as an animation effect of an image object. At time t20, the transfer history information HA1 does not exist.

  At time t21 shown in FIGS. 17B-1 to 17B-3, the first change area information A1 does not exist, and the second change area information A2 includes the area RA2 (= There is information A2 representing RA1b). Further, as the transfer history information HA1, two pieces of first change area information A1a and A1b existing in the RAM 106 at the immediately preceding time t20 are registered. As described above, also in the example of FIG. 17, the registration timing of the first change area information A1 by the hook processing module 400 and the registration timing of the second change area information A2 by the projector driver 128 are slightly shifted. Assume the case.

  FIG. 18 is an explanatory diagram showing screen update areas at time t20 and time t21 in the second embodiment. As described above, in the second embodiment, the screen update area TTG is always the same as the second change area RA2. Of the second change area RA2, the drawing portion of the area included in either the first change area RA1 or the history area RHA1 is directly acquired from the RAM 106, and any of the first change area RA1 and the history area RHA1 is acquired. The image portion of the area not included in the VRAM 114 is acquired from the VRAM 114. As shown in FIG. 17, at time t20, the second change region RA2 is included in the first change regions RA1a and RA1b. At time t21, the second change area RA2 is included in the history areas RHA1a and RHA1b. Accordingly, all images are acquired directly from the RAM 106 at times t20 and t21.

  When the image moves at a high speed due to the animation effect as in the example of FIG. 17, there is a high possibility that the registration timing of the first and second change area information A1 and A2 will be shifted. At this time, if the transfer history information HA1 is not used, there arises a problem that it becomes necessary to acquire the entire image in the second change area RA2 from the VRAM 114 at time t21. On the other hand, in the first and second embodiments, the image portion in the transfer history area RHA1 is directly acquired from the RAM 106, so that the time required for acquiring the image can be shortened.

  The VNC server 130 includes a plurality of transfer modes including the transfer process of the first embodiment (also referred to as “first transfer mode”) and the transfer process of the second embodiment (also referred to as “second transfer mode”). It may be configured such that one can be selected and executed. The transfer mode may be selected by the user or automatically by the VNC server 130 in accordance with the processing capability (CPU processing speed, bus transfer speed, etc.) of the computer 100 (generally an image supply device). Alternatively, the transfer mode may be selected. In this way, it is possible to select a transfer mode in which a preferable image can be transferred at an appropriate transfer speed.

C. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

C1. Modification 1:
As the screen update message supplied from the computer 100 to the projector 200, various messages other than those used in the above embodiment can be used. For example, a screen update message that includes at least image data can be used. Specifically, for example, when transferring image data of the entire screen, there is no need to transfer change area information.

  The screen update message preferably includes at least information indicating the position and size of the change area (screen update area) and image data in the change area. This is because the amount of data to be transferred is reduced if only the data relating to the change area (change area information and its image data) is transferred.

C2. Modification 2:
In the above-described embodiment, the image portion of the second transfer area TG2 is acquired from the VRAM 114 and once written in the image data storage area 106b in the RAM 106, and then the image of the screen update area TTG is acquired from the RAM 106 to connect the network. However, the process of writing to the image data storage area 106b can be omitted. However, according to the procedure of the above-described embodiment, when an image is finally transferred through the network, the image can be acquired from one image data storage area 106b.

C3. Modification 3:
In the above embodiment, the drawing module is divided into three modules, the GDI 124, the display driver 126, and the projector driver 128. However, the modules can be arbitrarily classified, and these functions can be combined into one module. It is also possible to combine the functions of the display driver 126 and the projector driver 128 into one module.

C4. Modification 4:
In the above embodiment, a personal computer is used as the image supply device, but other types of computers (mobile computer, handheld computer, workstation, etc.) may be used instead. In addition to these computers, devices that can be connected to a network and have the same functions as the computer may be used. Such devices include, for example, information mobile terminals, mobile phones, mail terminals, game machines, set top boxes, and the like. As the image display device, various display devices other than the projector can be used.

C5. Modification 5:
As the network, in addition to a local area network (LAN), various networks such as a wide area network (WAN), the Internet, and an intranet can be applied. The network may be configured by wire or may be configured by radio.

C6. Modification 6:
In the above embodiment, a part of the functions realized by software may be realized by hardware, or a part of the functions realized by hardware may be realized by software.

It is explanatory drawing which shows the structure of an image display system as one Example of this invention. It is a block diagram which shows the internal structure of a computer and a projector. It is explanatory drawing which shows the hierarchical structure of the software and hardware of a computer in a comparative example. It is explanatory drawing which shows the example of the change area information and image data storage area in 1st Example. It is explanatory drawing which shows the structure of a screen update message. It is explanatory drawing which shows the hierarchical structure of the software and hardware of a computer in an Example. It is explanatory drawing which shows the example of two change area information A1, A2 and transfer history information HA1 in 1st Example. It is a flowchart which shows operation | movement of the hook process module and VNC server in 1st Example. It is a flowchart which shows the detailed procedure of step S20. It is explanatory drawing which shows the relationship between the change area | region and screen update area | region in 1st Example. It is explanatory drawing which shows the content of the optimization process of a screen update area | region. It is a flowchart which shows the detailed procedure of step S22. It is explanatory drawing which shows the relationship between the change area | region at the time t1 of 1st Example, a history area | region, and a screen update area | region. It is explanatory drawing which shows the other example of two change area information A1, A2 and transfer history information HA1 in 1st Example. It is explanatory drawing which shows the screen update area | region in the time t10 and the time t11 of 1st Example. It is explanatory drawing which shows the relationship between the change area | region and screen update area | region in 2nd Example. It is explanatory drawing which shows the other example of two change area information A1, A2 and transfer history information HA1 in 2nd Example. It is explanatory drawing which shows the screen update area | region in the time t20 and the time t21 of 2nd Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image display system 60 ... Display screen 70 ... Projection display screen 100 ... Computer 102 ... CPU
104 ... ROM
106 ... RAM
106a ... Change area information 106b ... Image data storage area 106c ... Transfer history information 108 ... Hard disk drive 110 ... Input section 112 ... Network interface section 114 ... VRAM
116: Graphic controller 118 ... Display device 120 ... Bus 122 ... Application program 124 ... GDI (graphics device interface)
126 ... Display driver 128 ... Projector driver 130 ... VNC server 200 ... Projector 202 ... CPU
204 ... ROM
206 ... RAM
210: Input unit 212 ... Network interface unit 214 ... Image processing unit 216 ... Projection unit 218 ... Bus 228 ... Image processing driver 230 ... VNC client 300 ... Network 400 ... Hook processing module

Claims (12)

An image supply device for supplying and displaying an image on an image display device via a network,
An application program capable of issuing image drawing commands;
A drawing module for processing a drawing command issued by the application program;
A hook processing module that hooks a specific drawing command issued by the application program in advance and draws an image in a specific transfer image storage area in the general-purpose memory according to the acquired drawing command;
An image transfer processing module that acquires an image from the transfer image storage area and transfers the acquired image to the image display device via the network;
With
The hook processing module stores, in the general-purpose memory, first change area information indicating a first change area that is an area in which an image is drawn in response to the specific drawing command in the transfer image storage area. A function of writing, and a function of supplying the drawing command to the drawing module after processing the specific drawing command;
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) An image part corresponding to the first change area and an image part corresponding to an area not included in the first change area and included in the history area among the second change areas Obtain from the transfer image storage area without obtaining from the frame memory,
(Iv) The acquired image portion is transferred to the image display device via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas. Feeding device. An image supply device for supplying and displaying an image on an image display device via a network,
An application program capable of issuing image drawing commands;
A drawing module for processing a drawing command issued by the application program;
A hook processing module that hooks a specific drawing command issued by the application program in advance and draws an image in a specific transfer image storage area in the general-purpose memory according to the acquired drawing command;
An image transfer processing module that acquires an image from the transfer image storage area and transfers the acquired image to the image display device via the network;
With
The hook processing module stores, in the general-purpose memory, first change area information indicating a first change area that is an area in which an image is drawn in response to the specific drawing command in the transfer image storage area. A function of writing, and a function of supplying the drawing command to the drawing module after processing the specific drawing command;
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) Of the second change area, an image portion corresponding to an area included in at least one of the first change area and the history area is acquired from the transfer image storage area without obtaining from the frame memory. Acquired,
(Iv) The image supply device, wherein the acquired image portion is transferred to the image display device via the network together with screen update region information indicating a screen update region that is the same as the second change region. The image supply device according to claim 1, wherein:
The image transfer processing module acquires, from the frame memory, an image portion corresponding to an area that is not included in either the first change area or the history area in the second change area. An image supply device that acquires an image portion corresponding to the screen update area from the transfer image storage area after writing to the image storage area. The image supply device according to any one of claims 1 to 3,
The image transfer processing module
An image supply configured to load the hook processing module into the application program when the image transfer processing module is activated and unload the hook processing module from the application program when the execution of the image transfer processing module is completed. apparatus. An image transfer processing module program used in an image supply device for supplying and displaying an image on an image display device via a network,
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing module program is configured to cause a computer to realize a function of acquiring an image from the transfer image storage area and transferring the acquired image to the image display device via the network.
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module program is
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) An image part corresponding to the first change area and an image part corresponding to an area not included in the first change area and included in the history area among the second change areas Obtain from the transfer image storage area without obtaining from the frame memory,
(Iv) causing a computer to realize a function of transferring the acquired image portion to the image display device via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas. An image transfer processing module program. An image transfer processing device used in an image supply device for supplying and displaying an image on an image display device via a network,
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing device has a function of acquiring an image from the transfer image storage area and transferring the acquired image to the image display device via the network,
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing device includes:
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) An image part corresponding to the first change area and an image part corresponding to an area not included in the first change area and included in the history area among the second change areas Obtain from the transfer image storage area without obtaining from the frame memory,
(Iv) The acquired image portion is transferred to the image display device via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas. Transfer processing device. An image transfer processing module program used in an image supply device for supplying and displaying an image on an image display device via a network,
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing module program is configured to cause a computer to realize a function of acquiring an image from the transfer image storage area and transferring the acquired image to the image display device via the network.
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module program is
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) The transfer image without obtaining an image portion corresponding to an area included in at least one of the first change area and the history area among the second change areas from the frame memory. Obtained from the storage area,
(Iv) causing the computer to realize a function of transferring the acquired image portion to the image display device via the network together with screen update area information indicating a screen update area that is the same as the second change area. An image transfer processing module. An image transfer processing device used in an image supply device for supplying and displaying an image on an image display device via a network,
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing device has a function of acquiring an image from the transfer image storage area and transferring the acquired image to the image display device via the network,
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing device includes:
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) The transfer image without obtaining an image portion corresponding to an area included in at least one of the first change area and the history area among the second change areas from the frame memory. Obtained from the storage area,
(Iv) The image transfer processing device, wherein the acquired image portion is transferred to the image display device via the network together with screen update region information indicating a screen update region that is the same as the second change region. . A drawing module for processing a drawing command issued by an application program and a specific drawing command issued by the application program are hooked in advance, and a specific transfer image is stored in the general-purpose memory according to the acquired drawing command. A method for supplying an image to an image display device via a network from an image supply device having a hook processing module for drawing an image in a region and displaying the image (a) The hook processing module is issued by the application program Drawing an image in a specific image storage area for transfer in the general-purpose memory in response to a specific drawing command;
(B) The hook processing module uses the first change area information indicating the first change area, which is an area in which an image is drawn in accordance with the specific drawing command, in the transfer image storage area. Writing into the memory;
(C) the hook processing module supplying the drawing command to the drawing module after processing the specific drawing command;
(D) the drawing module drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module;
(E) the drawing module writing, into the general-purpose memory, second change area information indicating a second change area, which is an area in which an image is drawn in the frame memory in response to the drawing command; ,
(F) The image supply device sets the first change area existing in the transfer process in a predetermined preceding period as a history area and writes transfer history information representing the history area in the general-purpose memory;
(G) The image supply device refers to the first and second change area information and the transfer history information stored in the general-purpose memory, and includes the first change area in the second change area. Obtaining an image portion corresponding to an area not included in any of the change area and the history area from the frame memory;
(H) The image supply device corresponds to an image portion corresponding to the first change area and an area not included in the first change area and included in the history area among the second change areas. Acquiring from the transfer image storage area without acquiring the image portion from the frame memory;
(I) The image supply apparatus transfers the acquired image portion to the image display apparatus via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas. Process,
An image supply method comprising: A drawing module for processing a drawing command issued by an application program and a specific drawing command issued by the application program are hooked in advance, and a specific transfer image is stored in the general-purpose memory according to the acquired drawing command. A method of supplying and displaying an image on an image display device via a network from an image supply device including a hook processing module for drawing an image in an area,
(A) the hook processing module rendering an image in a specific transfer image storage area in the general-purpose memory in response to a specific rendering command issued by the application program;
(B) The hook processing module uses the first change area information indicating the first change area, which is an area in which an image is drawn in accordance with the specific drawing command, in the transfer image storage area. Writing into the memory;
(C) the hook processing module supplying the drawing command to the drawing module after processing the specific drawing command;
(D) the drawing module drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module;
(E) the drawing module writing, into the general-purpose memory, second change area information indicating a second change area, which is an area in which an image is drawn in the frame memory in response to the drawing command; ,
(F) The image supply device sets the first change area existing in the transfer process in a predetermined preceding period as a history area and writes transfer history information representing the history area in the general-purpose memory;
(G) The image supply device refers to the first and second change area information and the transfer history information stored in the general-purpose memory, and includes the first change area in the second change area. Obtaining an image portion corresponding to an area not included in any of the change area and the history area from the frame memory;
(H) The image supply device does not acquire, from the frame memory, an image portion corresponding to an area included in at least one of the first change area and the history area among the second change areas. Obtaining from the transfer image storage area;
(I) The image supply device transfers the acquired image portion to the image display device via the network together with screen update region information indicating a screen update region that is the same as the second change region. ,
An image supply method comprising: A computer-readable recording medium recording an image transfer processing module used in an image supply device for supplying an image to an image display device via a network for display.
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing module obtains an image from the transfer image storage area, and causes a computer to execute a process of transferring the acquired image to the image display device via the network.
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) An image part corresponding to the first change area and an image part corresponding to an area not included in the first change area and included in the history area among the second change areas Obtain from the transfer image storage area without obtaining from the frame memory,
(Iv) A recording medium having a function of transferring the acquired image portion to the image display device via the network together with screen update area information indicating a screen update area that is a sum of the first and second change areas. . A computer-readable recording medium recording an image transfer processing module used in an image supply device for supplying an image to an image display device via a network for display.
The image supply device hooks an application program capable of issuing an image drawing command, a drawing module for processing a drawing command issued by the application program, and a specific drawing command issued by the application program. And a hook processing module for drawing an image in a specific transfer image storage area in the general-purpose memory in accordance with the acquired drawing command.
The image transfer processing module obtains an image from the transfer image storage area, and causes a computer to execute a process of transferring the acquired image to the image display device via the network.
The hook processing module has a function of drawing an image in the transfer image storage area in accordance with a drawing command acquired by the hook, and an image is drawn in the transfer image storage area in accordance with the specific drawing command. A function of writing first change area information indicating a first change area, which is an area, into the general-purpose memory; and a function of supplying the drawing command to the drawing module after processing the specific drawing command ,
The drawing module has a function of drawing an image in a frame memory in accordance with a drawing command received from the application program or the hook processing module, and an area in the frame memory in which an image is drawn according to the drawing command. Having a function of writing second change area information indicating a second change area in the general-purpose memory;
The image transfer processing module
(I) a function of setting the first change area existing in the transfer process in a predetermined preceding period as a history area and writing transfer history information representing the history area in the general-purpose memory;
(Ii) Referring to the first and second change area information and the transfer history information stored in the general-purpose memory, the first change area and the history among the second change areas An image portion corresponding to an area not included in any of the areas is acquired from the frame memory,
(Iii) Of the second change area, an image portion corresponding to an area included in at least one of the first change area and the history area is acquired from the transfer image storage area without obtaining from the frame memory. Acquired,
(Iv) Recording having a function of transferring the acquired image portion to the image display device via the network together with screen update area information indicating a screen update area that is the same as the second change area Medium.

Images