JP5402415B2 - Image transmitting apparatus and program - Google Patents

Description

The present invention relates to an image transmission apparatus for transmitting image data.

  In a conventional server-client system, for example, there is a server-based computing system (SBC) in which a desired application program is activated and executed on the server device side by remote operation from the client device. In this SBC system, display drawing data that is processed and updated on the server device side in response to an operation input of the client device is transmitted and transferred from the server device to the client device every time the drawing is updated. As a result, the client device only needs to have a minimum function such as display and input, and resources such as applications are centrally managed by the server device.

  As described above, since the application program is executed on the server device, the SBC system can realize a highly functional computing environment even when the performance of the client device is low.

  Conventionally, the SBC system is designed on the premise of a high-speed network having no delay such as an in-house LAN (Local Area Network). However, when the amount of screen update data is large or the network communication speed (bandwidth) is low (narrow), the screen drawing data generated by the server device in response to the operation input of the client device is It takes time until the data is transmitted and transferred to the client device and is completed and displayed as one screen, which causes a problem of poor responsiveness.

  Therefore, in order to improve the user's apparent reaction in the client device, first, screen rendering data with a low data amount and a small amount of data is transmitted, and then high-quality screen rendering data is transmitted at intervals. A screen transfer device that transfers screens is considered (see, for example, Patent Document 1).

JP 2006-246153 A

  In the conventional screen transfer device, although the apparent response of the screen display according to the user operation in the client device is improved, if all screen transfers are performed in two stages, the total amount of screen data transmitted increases conversely. Will do.

  For this reason, for example, in an environment where transmission delay is likely to occur due to a wireless WAN (Wide Area Network), for example, the load on the entire system becomes larger, its performance deteriorates, and it does not contribute to improvement of the original operation response There's a problem.

  In the SBC system, particularly, a situation in which the operation response is likely to deteriorate is a case where long data such as a web page is displayed while continuously scrolling.

  That is, in the case of this continuous scrolling, every time scrolling by one unit, a process of generating screen drawing data having a width corresponding to the one scroll and transferring it to the client device occurs. In an environment where this occurs, the operation response cannot be displayed even more.

The present invention aims to provide when displaying while continuously scrolling a long data such as Web pages, while reducing the load on the system as a whole, a device it is possible to display operation with good response And

Claim 1 is an image transmission device for transmitting image data, and when an arbitrary screen update operation by the user for the image data displayed on the display screen is detected, the screen update is continuously scrolled continuously . A first determination unit that determines whether or not the screen is updated; and a second determination unit that determines whether or not the communication state is good when the first determination unit determines that the screen is continuously updated. , in the case where it is judged to be good second discriminating means, the image data of the display area is updated in accordance with the display scroll immediately transmitted in real time as a low-quality image image, whereas, in the second determining means is judged if it is determined not to be good, and the first transmission control means for transmitting said collectively poor quality picture image at predetermined intervals, wherein a screen update not continuous scrolling by the first discriminating means And time is characterized by comprising a second transmission control means for transmitting image data in the display area corresponding to the screen update as a high-quality picture images, a.

According to the present invention, when displaying while continuously scrolling a long data such as Web pages, while reducing the load to the entire system, to be capable of displaying operation response may ing.

1 is a block diagram showing a configuration of an SBC (Server Based Computing) system according to an embodiment of the present invention. The block diagram which shows the circuit structure of the server apparatus 10 in the said SBC system. The relationship between the drawing data G of the Web page HP generated on the client frame buffer RAM 14a of the server device 10 and the drawing data G (Gk + Gnew) transmitted to the client device 20 when the screen is scrolled (every scroll). FIG. The block diagram which shows the circuit structure of the client apparatus 20 in the said SBC system. FIG. 3 is a diagram showing a configuration of software modules in the server device 10 and the client device 20. The figure which shows the main data memories prepared in RAM14 of the said server apparatus 10. FIG. The description will be made by comparing the screen transmission schedules in the “Eager Update mode” and “Lazy Update mode” set according to the state of the scrolling flag of the server device 10 and the transmission delay time t acquired by the RTT measurement module 13a1. To do. The flowchart which shows the outline | summary of the drawing update process which the server apparatus 10 of the said SBC system performs. The flowchart which shows the determination process (SA) of the continuous screen scroll (during scroll) accompanying the drawing update process which the said server apparatus 10 performs. The flowchart which shows the compression process (SB) of the update drawing data G according to the judgment process (SA) of whether the continuous screen scroll is accompanying the drawing update process which the said server apparatus 10 performs. The flowchart which shows the screen transfer scheduling process (SC) according to the determination process (SA) of whether the continuous screen scroll is accompanying the drawing update process which the said server apparatus 10 performs. The flowchart which shows the screen transmission process (SD) according to the determination process (SA) of whether the continuous screen scroll is accompanying the drawing update process which the said server apparatus 10 performs.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an SBC (Server Based Computing) system according to an embodiment of the present invention.

  This SBC system includes a server device 10 and a plurality of client devices (thin clients) 20 connected to a network N composed of a wired or wireless LAN (Local Area Network) or WAN (Wide Area Network).

  The server device 10 has a plurality of application programs such as a text creation processing program, a Web browser program, a spreadsheet processing program, and a mail processing program, and an operation input from the client devices 20 connected to the server device 10 ( It is activated in response to the (input event) signal and executes its processing.

  In the server device 10, the display output drawing data G generated on the client frame buffer RAM 14a (see FIG. 2) in accordance with the execution of the application program according to the operation input signal from the client device 20,. , Image transmission and command transmission are combined and transferred to the access source client device 20.

  Then, in the client devices 20,..., The drawing data G transferred from the server device 10 is expanded in the frame buffer RAM 25 (see FIG. 4) and displayed on the display screen of the display device 26.

  That is, each client device (Thin client) 20 in the SBC system has an input function corresponding to a user operation such as a keyboard and a mouse and an output function to an LCD display unit and a printer as main functions. The server device 10 does not have various application functions and data file management functions.

  And, in principle, a data file generated with various processes activated and executed in the server device 10 in response to an operation input (input event) signal from the client device 20,. It is stored and stored for each user account or as a shared file in a storage device such as a magnetic disk managed and managed by the server device 10.

  FIG. 2 is a block diagram showing a circuit configuration of the server device 10 in the SBC system.

  The server device 10 includes a CPU 11 as a computer, and a ROM 13, a RAM 14, a frame buffer RAM 15, and a display device 16 are connected to the CPU 11 via a bus 12.

  Further, a communication I / F (interface) 19 with an input device 17 such as a keyboard and a mouse, an external storage device 18, a client device 20,.

  The CPU 11 controls the operation of each part of the circuit by using the RAM 14 as a working memory in accordance with a system program or various application programs stored in advance in the program memory 13a of the ROM 13, and controls key input signals from the input device 17 and communication I / O. The various programs are started and executed in response to a processing command (input event) signal corresponding to a user operation from the client device 20 received via F19.

  In this server device 10, various data generated in accordance with an application program activated / executed in response to an input event signal from the client device 20 is stored in the external storage device 18 in association with the user ID, for example. . The drawing data G for client display is generated using the client frame buffer RAM 14 a in the RAM 14, transferred from the communication I / F 19 to the client device 20, and displayed.

  For example, when an input event from the client device 20 is an event for requesting display of the Web page HP, the client device 20 uses the entire data of the Web page HP acquired by the Web browser program of the server device 10 in response to this event. The drawing data G matching the display screen (26) is generated and stored in the client frame buffer RAM 14a (see FIG. 3A).

  The drawing data G generated in the client frame buffer RAM 14a is compressed by an appropriate compression method, transferred to the client device 20 at an appropriate timing, and stored in the frame buffer RAM 25 (see FIG. 4). And displayed.

  The drawing data G generated in the client frame buffer RAM 14a is extracted as an image only in a rectangular area including a portion having a difference between the drawing data G-1 generated in response to the previous input event and the image, It is transmitted together with the coordinates indicating the display position on the display screen (26).

  At this time, among the drawing data G generated in the client frame buffer RAM 14a, for the rectangular area having no difference between the drawing data G-1 generated in response to the previous input event and the image, a copy of the rectangular area is made. Coordinates indicating the display position on the display screen (26) are transmitted together with the command.

  FIG. 3 shows drawing data G of the Web page HP generated on the client frame buffer RAM 14a of the server device 10, and drawing data G (Gk + Gnew) transmitted to the client device 20 when the screen is scrolled (every scroll). It is a figure which shows the relationship.

  With respect to drawing data G for client display generated in response to an input event from the client device 20, when a screen scroll occurs in the state of FIG. 3A, the image changes as shown in FIG. For a rectangular area Gk that does not exist, coordinates (a, b) indicating the display position on the display screen (26) corresponding to the width Ds of one scroll are transmitted together with the copy command of the rectangular area Gk.

  At the same time, for the rectangular area Gnew in which the image is changed with the width Ds of one scroll, the coordinates (c, d) indicating the display position on the display screen (26) are transmitted together with the image data.

  On the other hand, drawing data to be displayed on the display device 16 of the server device 10 itself is generated on the frame buffer RAM 15.

  FIG. 4 is a block diagram showing a circuit configuration of the client device 20 in the SBC system.

  The client device 20 includes a CPU 21 as a computer, and a ROM 23, a RAM 24, and a frame buffer RAM 25 are connected to the CPU 21 via a bus 22. Then, the drawing data G (see FIG. 3) written in the frame buffer RAM 25 is displayed on the display screen of the display device 26.

  The CPU 21 is connected to an input device 27, an external storage device 28, and a communication I / F (interface) 29 with the server device 10 via the bus 22.

  The CPU 21 controls the operation of each part of the circuit by using the RAM 24 as a working memory in accordance with a system program stored in the program memory 23a of the ROM 23. The CPU 21 receives the key input signal from the input device 27 via the communication I / F 29. The system program is activated and executed in response to an application response signal, transfer drawing data G, and the like from the server apparatus 10 to be executed.

  In the client device 20, various data generated by executing the application program in the server device 10 is appropriately read and stored in the external storage device 28, and the generated drawing data G for transfer is In accordance with the image and command, it is expanded in the frame buffer RAM 25 and displayed on the display device 26.

  FIG. 5 is a diagram showing the configuration of software modules in the server device 10 and the client device 20.

  The server control program stored in the program memory 13a of the server device 10 includes an RTT (Round Trip Time) measurement module 13a1, a screen control module 13a2, and a communication module 13a3.

  The RTT measurement module 13a1 has a function of measuring the degree of transmission delay with the client device 20 in the network N and notifying the measurement result to the screen control module 13a2.

  The screen control module 13a2 activates an application program corresponding to an input event from the client device 20 to generate display drawing data G on the client frame buffer RAM 14a, and the transmission delay notified from the RTT measurement module 13a1. According to the measurement result and the screen scroll state, the transmission schedule of the drawing data G to the client device 20 is controlled.

  The communication module 13a3 has a function of passing the input event data and RTT measurement data received from the client device 20 to the screen control module 13a2 and the RTT measurement module 13a1, and the RTT measurement data received from the RTT measurement module 13a1. And the drawing data G received from the screen control module 13a2 has a function of transmitting to the client device 20.

  On the other hand, the client control program stored in the program memory 23a of the client device 20 includes a communication module 23a1, an input module 23a2, and a screen display module 23a3.

  The communication module 23a1 transmits / receives data for RTT measurement to / from the server device 10 and the input event data received from the input module 23a2 to the server device 10, and is received from the server device 10. It has a function of delivering display drawing data G to the screen display module 23a3.

  The input module 23a2 has a function of delivering input event data corresponding to a user operation to the communication module 23a1.

  The screen display module 23a3 has a function of developing the display drawing data G received from the communication module 23a1 in the frame buffer RAM 25 in accordance with the image or command and displaying it on the display device 26.

  FIG. 6 is a diagram showing main data memories prepared in the RAM 14 of the server device 10.

  In addition to the client frame buffer RAM 14a, the RAM 14 includes a copy command time stamp memory 14b, a copy command continuous count memory 14c, a scrolling flag memory 14d, a drawing update area coordinate memory 14e, a screen transmission FIFO memory 14f, a transmission scheduling timer. A memory 14g, an RTT memory 14h, a drawing update mode memory 14i, a screen transfer reservation number memory 14j, and the like are prepared.

  In the copy instruction time stamp memory 14b, when it is determined that a drawing command issued in accordance with an application program corresponding to an input event from the client device 20 is a copy command of the drawing data Gk, a time is displayed every time the command is issued. Stamp data is stored.

  When the difference between the previous time stamp stored in the copy command time stamp memory 14b and the time stamp associated with the current copy command is less than a preset threshold, the copy command continuous count memory 14c stores the copy command The issuance count is added and stored.

  The scrolling flag memory 14d updates the drawing data G by continuous screen scrolling when the number of issued copy commands stored in the copy command continuous number of times memory 14c exceeds a preset threshold value. A flag indicating that this is being performed is stored.

  In the drawing update area coordinate memory 14e, when the drawing data G for client display is updated, the coordinates of the rectangular area in the entire image including the image change portion with the previous drawing data G-1 are stored. The

  In the screen transmission FIFO memory 14f, the drawing data G generated in the client frame buffer RAM 14a is buffered as data to be transferred to the client device 20. At this time, the previous drawing data G-1 and Only the data in the rectangular area updated by the comparison is compressed and stored, for example, in units of the screen scroll width Ds (see FIG. 3) set by the application that generated the drawing data G.

  The transmission scheduling timer memory 14g stores timer data for repeatedly executing the screen transmission schedule setting at regular intervals.

  The RTT memory 14h stores a transmission delay time t between the client device 20 measured by the RTT measurement function of the RTT measurement module 13a1.

  Sequences (1) to (5) of the RTT measurement process by the RTT measurement module 13a1 of the server device 10 are as follows.

  (1) The server device 10 transmits an RTT measurement request to the client device 20 at regular intervals.

  (2) When the client device 20 receives the RTT measurement request transmitted from the server device 10, the client device 20 transmits an upstream RTT measurement packet to the server device 10.

  (3) When the server device 10 receives the upstream RTT measurement packet transmitted from the client device 20, the server device 10 transmits the downstream RTT measurement packet to the client device 20.

  (4) When the client device 20 receives the downlink RTT measurement packet transmitted from the server device 10, the time stamp when the uplink RTT measurement packet is transmitted and the time when the downlink RTT measurement packet is received The difference is set as a transmission delay time t and notified to the server device 10.

  (5) The transmission delay time t notified to the server device 10 is transferred to the RTT measurement module 13a1 and stored in the RTT memory 14h, and is also transferred to the screen control module 13a2.

  The screen control module 13a2 sets the flag set (on) / reset (off) state stored in the scrolling flag memory 14d and the transmission delay between the client device 20 stored in the RTT memory 14h. Based on the time t, the transmission schedule of the drawing data G for updating the client screen is set to “Eager Update mode” or “Lazy Update mode” to be described later and controlled.

  FIG. 7 shows screen transmission schedules according to the “Eager Update Mode” and “Lazy Update Mode” set according to the state of the scrolling flag of the server device 10 and the transmission delay time t acquired by the RTT measurement module 13a1. It is a figure demonstrated in contrast.

  There are mainly the following two scheduling methods by which the server apparatus 10 transmits a screen.

(1) A method in which drawing data G is transmitted in real time according to the timing at which the application issues a drawing command (Eager Update mode: see FIG. 7A).

(2) A method in which intervals of a certain time are set and the drawing data G during that interval are transmitted together as the fixed time expires (Lazy Update mode: see FIG. 7B).

  As for the transmission schedule of the drawing data G for updating the client screen, ideally an eager update mode in which drawing commands are transmitted in real time in the order of generation as shown in FIG.

  However, if transmission of drawing data G with a large amount of data (high-quality transmission) is repeated many times in an environment with a large transmission delay, transmission overhead occurs many times and further increases the transmission delay.

  For this reason, when continuous screen scrolling that requires repeated transmission of the updated drawing data G occurs many times, drawing data G with high compression (low image quality) and a small amount of data is transmitted with a small transmission delay. Occasionally, by transmitting in the eager update mode, the update screen is displayed with good response on the client device 20 side. Then, after the continuous screen scrolling is completed, the drawing data G that has been updated with low compression (high image quality) is retransmitted in Lazy Update mode with respect to the already updated drawing data G that has been transmitted with high compression (low image quality). By doing so, the image quality can be quickly recovered after the screen is crawled.

  In the drawing update mode memory 14i in the RAM 14, the transmission delay time t stored in advance is stored in the RTT memory 14h when the flag in the scrolling flag memory 14d is set (on) and stored in the RTT memory 14h in advance. When the threshold value TH is less than or equal to the set threshold TH, data indicating the eager update mode is stored, and the scrolling flag is in a reset (off) state, or the transmission delay time t exceeds the threshold value TH, Data indicating the Lazy Update mode is stored.

  In the screen transfer reservation number memory 14j, the number of reservations for waiting for transfer of the drawing data G buffered in the screen transmission FIFO memory 14f to the client device 20 (in the case of screen scrolling, Ds unit) is stored.

  Next, the operation of the server device 10 of the SBC system configured as described above will be described.

  FIG. 8 is a flowchart showing an outline of the drawing update process executed by the server device 10 of the SBC system.

  An application program (for example, a Web browser program) executed in response to an input event from the client device 20 acquires a Web page HP from a Web site on the Internet N, and the drawing data G for client display is newly or updated. When it is generated on the frame buffer RAM 14a, it is determined whether or not the current generation process of the drawing data G is a drawing update process accompanying continuous screen scrolling (scrolling) according to the contents of the drawing command ( Step SA).

  Here, when it is determined that the drawing update process is associated with continuous screen scrolling (during scrolling) (step Sj (Yes)), the drawing data G specialized for the continuous screen scrolling (for the scroll width Ds). The image compression processing of the drawing data Gnew) of the rectangular area is executed (step SBs). On the other hand, when it is determined that the drawing update process accompanying the continuous screen scrolling is not in progress (step Sj (No)), the normal image compression process is executed (step SBn).

  Similarly, when it is determined that the drawing update process accompanying the continuous screen scroll is being performed (step Sj (Yes)), the screen transfer scheduling process specialized for the continuous screen scroll is executed (step Sj). SCs). On the other hand, when it is determined that the drawing update process accompanying the continuous screen scrolling is not being performed (step Sj (No)), the normal screen transfer scheduling process is executed (step SCn).

  Then, the update drawing data G that has been subjected to image compression processing and scheduled according to whether or not the drawing update processing accompanying the continuous screen scrolling is being performed is transmitted according to the transmission mode in the eager update mode or the lazy update mode. (Step SD).

  FIG. 9 is a flowchart showing a determination process (SA) of continuous screen scrolling (scrolling) accompanying the drawing update process executed by the server device 10.

  When a drawing command issued from an application program (for example, a Web browser program) is confirmed as the drawing data G is generated for the client frame buffer RAM 14a (step A1), whether or not a copy command is included in the drawing command. Judgment is made (step A2).

  If it is determined that the copy command is included in the drawing command (step A2 (Yes)), the time stamp associated with the issue of the copy command is stored in the copy command time stamp memory 14b (step A3).

  Then, a difference (time interval) between the previous time stamp and the current time stamp stored in the copy instruction time stamp memory 14b is obtained (step A4), and the time stamp difference between the previous time and the current time is calculated in advance. It is determined whether or not the value is equal to or less than a set threshold value, that is, whether or not a copy command for updating drawing data G has been issued continuously at a short time interval (step A5).

  If it is determined that a copy command for updating the drawing data G has been issued continuously at the preset time interval (step A5 (Yes)), the number of times the continuous copy command is issued is copied. It is counted and stored in the instruction continuous count memory 14c (step A6).

  Then, whether or not the number of continuous copy commands issued stored in the continuous copy command memory 14c is greater than or equal to a preset threshold value, that is, at a certain short time interval and continuously for a certain number of times. It is determined whether or not a copy command for updating has been issued (step A7).

  Here, if it is determined that a copy command having a short time interval has been issued continuously for the preset number of times (step A7 (Yes)), it is in a state of drawing update processing accompanying continuous screen scrolling. As a result, a flag indicating that the scroll is being performed is set (on) in the scroll flag memory 14d (step A8).

  On the other hand, if it is determined in step A2 that the copy command is not included in the drawing command (step A2 (No)), the drawing update process associated with screen scrolling is not in the state (for example, drawing data for one screen). G is newly generated), and the scrolling flag (14d) is not set.

  Further, in step A5, the time stamp difference associated with the copy command between the previous time and the current time is not less than or equal to a preset threshold value and is not a continuous copy command issued (or disappeared) at short time intervals. Is determined (step A5 (No)), the number of continuous copy commands issued stored in the continuous copy command memory 14c is reset, and the scroll set in the scroll flag memory 14d is reset. The middle flag is reset (step A9).

  In Step A7, when it is determined that the number of continuous copy commands issued stored in the copy command continuous number of times memory 14c does not reach a preset threshold value (Step A7 (No)), It is determined that the state is not in the drawing update process accompanying the screen scrolling, and the scrolling flag (14d) is not set.

  Thus, when it is determined that the drawing update process is in a state of continuous screen scrolling, a flag is set (on) in the scrolling flag memory 14d.

  FIG. 10 is a flowchart showing the compression process (SB) of the updated drawing data G according to the determination process (SA) as to whether or not the continuous screen scrolling is involved in the drawing update process executed by the server device 10.

  It is determined whether or not a scrolling flag is set (on) in the scrolling flag memory 14d (step B1, B2), and if it is determined that the scrolling flag is set (step B2 ( Yes)), the coordinates in the entire image of the rectangular area (Gnew) (see FIG. 3) for the scroll width Ds updated by the current screen scroll are extracted and stored in the drawing update area coordinate memory 14e ( Step B3).

  Then, the drawing data Gnew in the rectangular area corresponding to the scroll width Ds is compressed to a small data amount in the high compression mode (step B4), and is registered in the screen transmission FIFO memory 14f together with the coordinate information indicating the drawing position on the display screen. (Step B5).

  On the other hand, if it is determined in step B2 that the scroll flag is not set in the scroll flag memory 14d (off) (step B2 (No)), the drawing data G that has been updated this time is normally compressed. It is compressed in the mode (step B6) and registered in the screen transmission FIFO memory 14f (step B5).

  In this way, the drawing data Gnew that has been updated as the screen scrolls continuously is subjected to high compression processing, is registered in the screen transmission FIFO memory 14f, and is transmitted to the client device 20 in accordance with the subsequent screen transmission processing (SD). On the other hand, for example, the drawing data G whose drawing has been updated on the entire screen, or the drawing data Gnew which has been updated with a non-continuous one-time screen scrolling, is normally compressed and registered in the screen transmission FIFO memory 14f. It is transmitted to the client device 20 according to the screen transmission process (SD).

  FIG. 11 is a flowchart showing a screen transfer scheduling process (SC) corresponding to a determination process (SA) of whether or not continuous screen scrolling is being performed accompanying the drawing update process executed by the server device 10.

  When the timer handler is started according to the timer data stored in the transmission scheduling timer memory 14g (step C1), the drawing update area coordinate memory 14e is already subjected to high compression processing and sent to the screen transmission FIFO memory 14f for transmission. It is determined whether or not the coordinates of the rectangular area (Gnew) (see FIG. 3) corresponding to the scroll width Ds are stored (step C2).

  If it is determined that the coordinates of the rectangular area (Gnew) for the scroll width Ds are stored in the drawing update area coordinate memory 14e (step C2 (Yes)), the contents of the in-scroll flag memory 14d are It is read (step C3), and it is judged whether or not the scrolling flag is reset (off) (step C4).

  When it is determined that the scrolling flag has been reset (off), that is, it is after continuous screen scrolling (step C4 (Yes)), the contents of the RTT memory 14h are read (step S4). C5), it is determined whether or not the transmission delay time t is equal to or less than a preset threshold value TH (step C6).

  When it is determined that the transmission delay time t is equal to or less than a preset threshold value TH, that is, the transmission delay between the server and the client is small and is in a good communication state (step C6 (Yes)), The contents of the screen transfer reservation number memory 14j are read (step C7), and the transfer waiting reservation number of the drawing data G that has been buffered in the screen transmission FIFO memory 14f and has not been transmitted is less than a preset threshold value. (Step C8).

  Here, the number of transfer waiting reservations for the drawing data G buffered in the screen transmission FIFO memory 14f is equal to or less than a preset threshold value, that is, the load associated with the screen transfer of the server device 10 is light. If it is determined (step C8 (Yes)), the drawing data Gnew corresponding to the coordinates of the rectangular area corresponding to the scroll width Ds stored in the drawing update area coordinate memory 14e (compressed and transmitted in the high compression mode) ) Is compressed in the normal compression (high quality) mode (step C9).

  If it is confirmed that there is no input event from the client device 20 at present (step C10 (Yes)), the drawing data Gnew compressed in the normal compression (high quality) mode becomes the drawing position on the display screen. Is registered in the screen transmission FIFO memory 14f together with the coordinate information instructing the image data, and is transmitted to the client device 20 in accordance with the subsequent screen transmission processing (SD) (step C11).

  Then, the coordinates of the rectangular area corresponding to the scroll width Ds that has been subjected to the normal compression process and registered in the screen transmission FIFO memory 14f are deleted from the drawing update area coordinate memory 14e (step C12), and further, the transmission scheduling timer memory The timer data stored in 14g is initially set (step C13).

  On the other hand, in step C2, is it determined that the coordinates of the rectangular area (Gnew) corresponding to the scroll width Ds that has been subjected to the high compression processing and the transmission processing have not been stored in the drawing update area coordinate memory 14e (step C2)? (No), or even if it is determined that the coordinates of the rectangular area (Gnew) that has been transmitted with the same high compression are stored (step C2 (Yes)), the scrolling flag is reset (off) in step C4. It is not determined, that is, whether it is currently determined that continuous screen scrolling is in progress (step C4 (No)), or even if it is determined that the same scrolling flag is reset (off) (step C4). (Yes)) In step C6, is it determined that the transmission delay time t between the server and the client is not less than or equal to a preset threshold value TH? In Step C6 (No)) or in Step C8, if it is determined that the number of transfer waiting reservations of the drawing data G buffered in the screen transmission FIFO memory 14f is not less than a preset threshold value (Step C8) (No)) The compression / transmission processing (steps C9 to C12) in the normal compression mode (high quality) of the drawing data Gnew corresponding to the rectangular area (Gnew) is not executed, and is stored in the transmission scheduling timer memory 14g. The timer data thus set is initially set (step C13).

  In addition, the following conditions are cleared: the scrolling flag (off) (step C4 (Yes)), the RTT threshold value TH or less (step C6 (Yes)), and the screen transfer reservation number threshold value or less (step C8 (Yes)). Even if the drawing data Gnew (highly compressed and transmitted) corresponding to the coordinates of the rectangular area corresponding to the scroll width Ds is compressed in the normal compression (high quality) mode in step C9, the input event from the client device 20 is currently present. If it is determined that there is (step C10 (No)), the transmission processing (steps C11 and C12) of the drawing data Gnew subjected to the normal compression processing is not executed, and the timer data stored in the transmission scheduling timer memory 14g is not stored. Initially set (step C13).

  Thus, the drawing data Gnew of the rectangular area corresponding to the scroll width Ds compressed and transmitted in the high compression mode in the image compression process (SB) in accordance with the continuous screen scrolling is the screen transfer scheduling process (SC). Are compressed in the normal compression mode (high quality) on condition that continuous screen scrolling is not performed, transmission delay is small, the number of screen transfer reservations is small, and there is no input event from the client device 20. Retransmitted.

  FIG. 12 is a flowchart showing a screen transmission process (SD) corresponding to a determination process (SA) as to whether or not a continuous screen scroll is being performed accompanying the drawing update process executed by the server device 10.

  It is determined whether or not the flag is set (on) in the in-scrolling flag memory 14d (steps D1 and D2), and if it is determined that the in-scrolling flag is set (step D2 (Yes) )), It is determined whether or not the transmission delay time t stored in the RTT memory 14h is equal to or less than a preset threshold value TH (step D3).

  When it is determined that the transmission delay time t is equal to or less than a preset threshold TH, that is, the transmission delay between the server and the client is currently small and in a good communication state (step D3 (Yes)), The transmission schedule of the drawing data G (Gnew) for updating the client screen stored in the screen transmission FIFO memory 14f is set to “Eager Update mode” and stored in the drawing update mode memory 14i (step D4).

  On the other hand, even if it is determined in step D2 that the scrolling flag is reset (off) (step D2 (No)), or even if it is determined that the scrolling flag is set (on). (Step D2 (Yes)), when it is determined that the transmission delay time t is not less than or equal to the preset threshold value TH and the transmission delay between the server and the client is currently large (Step D3 ( No)), the transmission schedule of drawing data G (Gnew) for updating the client screen stored in the screen transmission FIFO memory 14f is set to "Lazy Update mode" and stored in the drawing update mode memory 14i ( Step S4).

  Therefore, according to the server device 10 of the SBC system having the above-described configuration, during continuous screen scrolling, the drawing data Gnew in the rectangular area corresponding to the scroll width Ds is displayed in the “high compression mode” only when the transmission delay is small. ”Is compressed to a small amount of data (low quality) and transmitted to the client device 20 in real time in“ Eager Update mode ”to update the display. Then, after the continuous screen scrolling is finished, it is determined that the transmission delay is small, the number of screen transmission reservations is small, and there is no input event from the client device 20, and the high-compression is transmitted. The drawing data Gnew is compressed in the “normal compression mode” (high quality), and is retransmitted collectively to the client device 20 in the “Lazy Update mode” to update the display with high image quality.

  For this reason, the drawing data Gnew that is sequentially updated is displayed in real time with high compression (low quality) during the screen scrolling only when the drawing update process is performed by continuous screen scrolling, which tends to deteriorate the operation response. Transmit to ensure responsiveness, and after scrolling the screen, transmit the drawing data Gnew that has already been transmitted in a low-compression (high quality) manner to ensure readability. This can be done by checking when the delay is small. Therefore, even in a network environment where transmission delay is unavoidable, such as a wireless WAN, it is possible to realize a comfortable screen scroll response while suppressing the load on the entire system.

  Note that each processing method by the server device 10 of the SBC system described in the embodiment, that is, the drawing update process shown in the flowchart of FIG. 8 and the continuous screen scrolling accompanying the drawing update process shown in the flowchart of FIG. Scrolling) determination processing (SA), and updated drawing data G compression processing (SB) in accordance with the determination processing (SA) of whether or not continuous screen scrolling accompanying the drawing update processing shown in the flowchart of FIG. The screen transfer scheduling process (SC) according to the determination process (SA) as to whether or not the continuous screen scrolling is accompanying the drawing update process shown in the flowchart of FIG. 11, and the drawing update process shown in the flowchart of FIG. Each method such as screen transmission processing (SD) according to the determination processing (SA) of whether or not continuous screen scrolling is involved, As programs that can be executed by a computer, external recording devices such as memory cards (ROM cards, RAM cards, etc.), magnetic disks (floppy disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), semiconductor memories, etc. It can be stored and distributed on 18 media. Then, the computer (CPU 11) of the server device 10 reads the program stored in the medium of the external storage device 18 into the storage device (flash ROM 13 or RAM 14), and the operation is controlled by the read program, thereby The two-stage transmission function of the drawing data Gnew specialized for the drawing update process associated with the continuous screen scrolling described in the embodiment can be realized, and the same process by the above-described method can be executed.

  Further, program data for realizing each of the above methods can be transmitted on the communication network (N) in the form of a program code, and from a computer device (program server) connected to the communication network (N). The program data is fetched into the server device 10 and stored in the storage device (flash ROM 13 or RAM 14), thereby realizing the two-stage transmission function of the drawing data Gnew specialized for the drawing update processing accompanying the continuous screen scroll described above. You can also.

  Note that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention at the stage of implementation. Further, each of the embodiments includes inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent elements are deleted from all the constituent elements shown in each embodiment or some constituent elements are combined in different forms, the problems described in the column of the problem to be solved by the invention If the effects described in the column “Effects of the Invention” can be obtained, a configuration in which these constituent requirements are deleted or combined can be extracted as an invention.

DESCRIPTION OF SYMBOLS 10 ... Server apparatus 20 ... Client apparatus 11, 21 ... CPU
12, 22 ... Bus 13, 23 ... ROM
13a, 23a ... Program memory 13a1 ... RTT measurement module 13a2 ... Screen control module 13a3, 23a1 ... Communication module 23a2 ... Input module 23a3 ... Screen display module 14, 24 ... RAM
14a: Client frame buffer RAM
14b ... Copy command time stamp memory 14c ... Copy command continuous count memory 14d ... Scrolling flag memory 14e ... Drawing update area coordinate memory 14f ... Screen transmission FIFO memory 14g ... Transmission scheduling timer memory 14h ... RTT memory 14i ... Drawing update mode memory 14j ... screen transfer reservation memory 15, 25 ... frame buffer RAM
16, 26 ... Display device 17, 27 ... Input device 18, 28 ... External storage device 19, 29 ... Communication I / F
N ... Communication network HP ... Web page G ... Drawing data for client display Ds ... Scroll width (movement amount)
Gnew: Scroll width update drawing data

Claims (6)

An image transmission device that transmits image data,
First discriminating means for discriminating whether or not the screen update is continuous continuous scrolling when an arbitrary screen update operation by the user for the image data displayed on the display screen is detected;
A second discriminating unit that discriminates whether or not a communication state is good when the first discriminating unit determines that the continuous scrolling is performed ;
Wherein in the case where it is judged to be good second discriminating means, the image data of the display area is updated in accordance with the display scroll immediately transmitted in real time as a low-quality image image while satisfactorily in the second discriminating means If it is determined that not a first transmission control means for transmitting the low-quality picture images together at predetermined intervals,
Wherein when the it is judged that the screen update not continuous scrolling in the first determination means, second transmission control means for transmitting image data in the display area corresponding to the screen update as a high-quality picture images,
An image transmitting apparatus comprising: Wherein the first determination means, the screen update is determined whether or not the screen update of the continuous scroll on the basis of whether screen updating operation for and scrolling successively at short time intervals is performed,
The image transmitting apparatus according to claim 1. Wherein the first determination means, the difference between the previous image data and current image data is the screen update is determined whether or not the screen update of the continuous scroll on the basis of whether it is above a predetermined threshold value,
The image transmission apparatus according to claim 1, wherein the image transmission apparatus is an image transmission apparatus. The second determination means determines whether the communication state is good based on whether the transmission delay time of the image data is equal to or less than a specific threshold value,
The image transmission apparatus according to claim 1, wherein The image transmission device is a device that generates image data in response to an input event from a client device via a network, and transmits the image data to the client device.
Detection of the screen update operation is detected by detecting a scroll request by a user operation on the client device.
The image transmission apparatus according to claim 1, wherein the image transmission apparatus is an image transmission apparatus. A program for controlling a computer of an image transmission apparatus for transmitting image data,
The computer,
First discriminating means for discriminating whether or not the screen update is a continuous scroll screen update when an arbitrary screen update operation by the user for the image data displayed on the display screen is detected;
A second discriminating unit that discriminates whether or not a communication state is good when the first discriminating unit determines that the continuous scroll screen update is performed ;
Wherein in the case where it is judged to be good second discriminating means, the image data of the display area is updated in accordance with the display scroll immediately transmitted in real time as a low-quality image image while satisfactorily in the second discriminating means If it is determined that not the first transmission control means for transmitting the low-quality picture images together at predetermined intervals,
Wherein when the it is judged that the screen update not continuous scrolling in the first determination means, second transmission control means for transmitting image data in the display area corresponding to the screen update as a high-quality picture images,
A computer-readable program designed to function as a computer.

Images