JP2008217585A - Client device for server base computing system, and client control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform, in a client device of a server base computing system, recording of screen data generated and transferred from a server device and reproduction thereof based on an event operation by a user. <P>SOLUTION: Receiving data 43 such as drawing data generated and transferred from the server device according to an input event associated with a user operation is stored in a recording file 24D1 while associating a kind 42d of the input event in the order of a frame number 42a every generation and update of the data, and successively outputted for screen display. The receiving (recording) data 43 stored in the recording file 24D1 is successively read for each frame number 42a, and recording data 43 associated with the same input event kind 42d is continuously reproduced. When recording data 43 associated with a different input event kind 42d is read, reproduction output of the receiving (recording) data 43 concerned is restarted according to input event determination accompanying a new user operation. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention is a server / client system connected via a network such as a LAN (Local Area Network), and operates all applications input / output / displayed on the client device on the server device. The present invention relates to a client device of a server-based computing system and a control program thereof.

  Conventionally, in each PC terminal connected to a network such as an in-house LAN, the individual PC terminal independently starts and executes an application such as document creation or table creation, and various files generated in association with this start up and execution. The data is stored in its own storage device in the PC terminal, or stored in an external storage medium such as a magnetic disk, an optical disk, or a small semiconductor memory, or transferred to and stored in a storage device managed by a server on the network.

  In such a conventional server / client system, the generated files are managed in each PC terminal itself. Therefore, if the PC terminal is lost or illegally taken out and put into the hands of a third party, the stored file is saved. May be read out and important information and confidential information may be leaked.

  In addition, if each PC terminal starts and executes an application independently, it is necessary to update, change, and add the application for each PC terminal, which is cumbersome and expensive. .

  For this reason, in recent server-client systems, all applications that are input / output / displayed on each PC terminal (client device) are started and executed on the server, and all generated files associated therewith are also managed on the server side. Such an SBC (Server Based Computing) system has been introduced (for example, see Patent Document 1).

In such a server-based computing system, drawing data of application software executed on the server side is transferred to the client device, and the drawing data is simply displayed on the client device.
JP 2004-171063 A

  As described above, in the server-based computing system, all applications that are input / output / displayed on the client device are started and executed on the server, and all generated files associated with this are managed on the server side. Therefore, even if the client device is lost or illegally taken out and passed into the hands of a third party, no saved file is read out, and there is no possibility of leakage of important information or confidential information .

  However, in an environment outside the network where the server cannot be connected, the client device cannot be used at all. For example, even if a presentation material image is created using the server, it must be taken out by the client device. There is a problem that cannot be displayed and explained on the customer side.

  In view of the above problems, an object of the present invention is to provide a client device of a server-based computing system that can record data generated and transferred by a server device and reproduce the data based on an event operation by a user. It is to provide a client control program.

  The client device of the server-based computing system according to the present invention includes an event transmission unit that transmits event information corresponding to a user operation to the server device, and the server device according to the event information transmitted by the event transmission unit. A data receiving means for receiving the data generated by the data receiving means, a received data recording means for recording the data received by the data receiving means in association with the frame number in the receiving order and the event information, and the received data recording And data reproducing means for reproducing the received data recorded by the means based on the frame number and event information.

  In addition, the client device of the server-based computing system according to the present invention further includes recording mode setting means for setting a recording mode in accordance with a user operation, and the received data recording means includes the recording mode setting means. In a state in which the recording mode is set, the data received by the data receiving means is recorded in association with the frame number of the reception order and the event information.

In the client device of the server-based computing system according to the present invention, the received data recording means corresponds to the event information corresponding to the data received last time by the data receiving means and the data received this time. Recording event information determining means for determining whether or not the current event information matches, and the event information corresponding to the data received this time is received last time by the recording event information determining means. If it is determined that the event information is the same as the event information corresponding to the data, the data received this time is recorded in association with the frame number common to the previous received data,
It is characterized by that.

  In the client device of the server-based computing system according to the present invention, the data reproducing means sends the received data recorded by the received data recording means in a direction of sending a frame number associated with the received data. It is determined whether or not the number direction reproduction means to be reproduced and the event information of the frame number associated with the reception data reproduced last time are the same as the event information of the frame number associated with the reception data to be reproduced this time. Event information determining means at the time of reproduction, and the event information associated with the reception data to be reproduced this time by the event information determining means at the time of reproduction is different from the event information associated with the reception data reproduced last time If it is determined that the received data to be reproduced this time is It is characterized by playing according to the input event in response.

  Also, in the client device of the server-based computing system according to the present invention, the number direction reproducing means has a sending degree varying means for making variable the degree of sending the frame number when reproducing the received data. It is characterized by.

  In the client device of the server-based computing system according to the present invention, the data reproducing means returns and sets a frame number of received data to be reproduced in accordance with a return input event according to a user operation. The number direction reproduction means further includes a number setting means, and the number direction reproduction means receives the reception data recorded by the reception data recording means when the frame number of the reception data to be reproduced is set by the return frame number setting means. It is characterized in that data is reproduced in the direction of sending the frame number from the set frame number.

  Further, in the client device of the server-based computing system according to the present invention, the return frame number setting means determines the degree of return of the frame number when setting the frame number of the received data to be reproduced. It is characterized by having a return degree variable means for making it variable.

  In the client device of the server-based computing system according to the present invention, the data reproduction means further includes a time measurement means for measuring a preset time, and the reproduction event information determination means reproduces this time. When it is determined that the event information associated with the received data to be received is different from the event information associated with the previously reproduced received data, the time preset by the time measuring means is counted. The received data to be reproduced this time is reproduced at the time of the event or according to the input event according to the user operation.

  In the client device of the server-based computing system according to the present invention, the time measuring means measures a time set to infinity in advance.

  According to the present invention, it is possible to provide a client device and a client control program of a server-based computing system that can record data generated and transferred by a server device and reproduce it based on an event operation by a user. .

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a server-based computing system including a server device 10 and its client devices 20 according to an embodiment of the present invention.

  This server-based computing system includes a server device 10 and a plurality of client devices 20,... Connected on a network N composed of a LAN (Local Area Network) or a WAN (Wide Area Network).

  The server device 10 has a plurality of application programs such as a document creation processing program, a spreadsheet processing program, a presentation material creation program, a mail processing program, an Internet connection processing program, and a Web display program, and is connected to the server device 10. It is activated in response to an operation input (input event) signal from the client device 20,.

  In this server device 10, the display output drawing data generated in accordance with the execution of the application program according to the operation input signal from the client device 20,... Is transferred by the built-in accelerator circuit 10A (see FIG. 2). And is compressed / encrypted (hereinafter referred to as encoding) and transmitted (transferred) to the client device 20 of the access source.

  In the client devices 20,..., The drawing data transferred from the server device 10 is decompressed and decoded (hereinafter referred to as decoding) in the accelerator circuit 20A (see FIG. 3) in each client device, and the display is performed. Displayed in the section.

  That is, each of the client devices 20,... In this server-based computing system has an input function corresponding to a user operation such as a keyboard and a mouse and an output function to an LCD display unit and a printer as main functions. However, it does not have at least various application functions and data file management functions that the server apparatus 10 has.

  And the data file produced | generated with the various processes started and performed in the server apparatus 10 according to the operation input (input event) signal from the client apparatus 20 ... is fundamentally in the said server apparatus 10. Alternatively, it is stored and stored for each user account or as a shared file in a storage device such as a magnetic disk managed and managed by the server device 10.

  On the other hand, the client device 20 displays the drawing data generated and transferred by the server device 10 in response to an operation input (input event) at the client device 20 on the display unit. In addition, it has a function of freely reproducing and displaying in accordance with any user operation such as operation of a mouse button or operation of a predetermined key.

  FIG. 2 is a block diagram showing a circuit configuration of the server device 10 in the server-based computing system.

  The server device 10 includes a CPU 11 serving as a computer. The CPU 11 is connected to a ROM 13, a RAM 14, and a display device 15 via a bus 12, and is connected to the accelerator circuit 10 </ b> A. A VRAM 16 is mounted on the accelerator circuit 10A.

  Further, an input unit 17 such as a keyboard and a mouse, an external storage HDD (Hard Disk Drive) 18, and a transmission / reception control unit 19 with the client device 20 are connected to the CPU 11 via the bus 12.

  The CPU 11 controls the operation of each part of the circuit by using the RAM 14 as a working memory in accordance with a system program or various application programs stored in advance in the ROM 13. The various programs are activated and executed in response to a processing command signal corresponding to a user operation (input event) from the client device 20 received.

  In this server device 10, various data generated according to an application program that is activated and executed in response to an input event signal from the client device 20 is stored in the external storage HDD 18 in association with the user ID, for example. The drawing data for display is converted into drawing data for transfer by using the VRAM 16 in the accelerator circuit 10A, transferred from the transmission / reception control unit 19 to the client device 20, and displayed.

  FIG. 3 is a block diagram showing a circuit configuration of the client device 20 in the server-based computing system.

  The client device 20 includes a CPU 21 as a computer, to which a ROM 23 and a RAM 24 are connected via a bus 22 and an accelerator circuit 20A is connected. The accelerator circuit 20A is equipped with a VRAM 25, and drawing data written in the VRAM 25 is output to the display device 26 and displayed.

  The CPU 21 also has an input unit 27 such as a keyboard, mouse, microphone, optical sensor, external storage HDD (Hard Disk Drive) 28, authentication device, barcode reader, CCD camera, printer, wireless device, etc. A USB / parallel / serial interface 30 for connecting the various external devices and a transmission / reception control unit (wired / wireless) 31 with the server device 10 are connected.

  The CPU 21 uses the RAM 24 as a working memory in accordance with a system program stored in advance in the ROM 23 to control the operation of each part of the circuit. The server 21 receives a key input signal from the input unit 27 and the transmission / reception control unit 31. The system program is activated and executed in accordance with an application response signal from 10, transfer drawing data, and the like.

  In the client device 20, various data generated by executing the application program in the server device 10 is appropriately read and stored in the external storage HDD 28, and the generated drawing data for display is stored in an accelerator circuit. The data is decoded at 20A, written in the VRAM 25, and displayed on the display device 26.

  At this time, the client device 20 displays each drawing data generated and transferred by the server device 10 in response to an operation input (input event) at the client device 20 on the display unit. And can be freely reproduced and displayed in accordance with an arbitrary user operation such as an operation of a mouse button or an operation of a predetermined key.

  Here, the outline of the transfer drawing data generation function in the accelerator circuit 10A of the server device 10 will be described (see FIGS. 5B and 5C).

  In the server device 10, when the drawing data (before change) G that has been generated and transferred in accordance with the running application program changes to drawing data (after change) G ′, first, the drawing data in the changed drawing data G ′ is changed. A region Q ′ corresponding to the image change portion (difference) P is cut out, and the drawing data of the region Q ′ having this change is converted into transfer drawing data H by the accelerator circuit 10A.

  The transfer drawing data H is obtained by comparing the area Q ′ cut out from the changed drawing data G ′ according to the image change portion (difference) P and the same area Q in the drawing data G before the change. Of these, the background matching region R is generated by being converted into transparent drawing data (color image data that does not require rewriting) S.

  As a result, the client device 20 writes the transfer drawing data H, excluding the transparent image data S, at the coordinate position corresponding to the image change area Q ′ on the pre-change drawing data G. Drawing data G ′ can be displayed.

  The drawing data for transfer H generated in the accelerator circuit 10A of the server device 10 is compressed and encrypted (encoded) and transferred to the client device 20.

  That is, in this server-based computing system, the display drawing data generated by the server device 10 in response to an event such as a key input from the client device 20 is generated and updated every time the drawing data is generated and updated. The image change area Q ′ after the drawing update is generated as transfer drawing data H and transferred to the client device 20. Then, only the image change region Q ′ is sequentially rewritten with respect to the drawing data already displayed on the client device 20 and the display is updated. Thereby, the amount of data transferred from the server device 10 to the client device 20 is greatly reduced.

  In the client device 20 of the present embodiment, the display device 26 displays the drawing data generated and transferred from the server device 10 in response to a user operation input (input event) on the client device 20. The RAM 24 has a function of recording drawing data for each drawing update corresponding to a user operation (input event). After that, it has a function of freely reproducing and displaying in accordance with any user operation such as operation of a mouse button or operation of a predetermined key (button).

  FIG. 4 is a diagram showing a display state of the playback operation buttons accompanying the playback processing of the recorded drawing data in the client device 20 of the server-based computing system.

  When the client device 20 is set to a reproduction mode for reproducing the drawing data recorded in the RAM 24, as shown in FIG. 4, the “feed” button F and the “return” button along the lower end of the display screen in the display device 26. "Button R is displayed.

  The “feed” button F and the “return” button R are displayed as transparent buttons on the display screen (26). For example, the button selection by the cursor key and the instruction by the enter key, or the button selection by the mouse pointer and the left click. An operation is input by an instruction.

  When the “feed” button F and the “return” button R are not transmissive buttons, the display areas of the buttons F and R are distinguished from the reproduction display area of drawing data. Further, the “feed” button F and the “return” button R may not be displayed on the display screen (26), but may be provided as physical buttons that are pressed by the keyboard in the input unit 27.

  As described above, every time drawing data generated and transferred from the server apparatus 10 in response to an operation input (input event) in the client apparatus 20 is updated and displayed in the client apparatus 20, it is stored in the RAM 24 of the client apparatus 20. The recorded drawing data is reproduced and displayed for each drawing update in accordance with the input of the “Send” button F, and the reproduced drawing data in response to the input of the “Return” button R. The return playback display is performed.

  FIG. 5 is a diagram showing a feed playback display screen associated with the input of the “feed” button F and a return playback display screen associated with the input of the “return” button R when the client device 20 is set to the playback mode. .

  That is, when the “feed” button F is operated and input as shown in FIG. 5A in the state where the reproduction mode is set in the client device 20, first, as shown in FIG. The drawing data X for one screen recorded at the initial display in the recording mode is displayed.

  Here, when the “feed” button F is further operated and input, as shown in FIG. 5C, only the drawing data b of the changed portion that has been updated is rewritten and reproduced and displayed. When button "F" is operated and input, as shown in FIG. 5D, only the drawing data c of the changed portion that has been subsequently updated is rewritten and reproduced and displayed.

  Thereafter, when the “return” button R is operated and input as shown in FIG. 5 (E), the drawing data c of the changed portion most recently drawn and updated as shown in FIG. 5 (E) → (F). When the replay display corresponding to the drawing data a and b of each changed part immediately before rewriting is performed and the screen state is returned to one, and the “return” button R is further operated, FIG. 5 ( As shown in G), corresponding to the drawing data a of the changed portion (in this case, drawing data X for one screen) until immediately before being rewritten to the drawing data b of the changed portion that was updated most recently as described above. Replay display is performed and the screen state is returned to one.

  As a result, each drawing data for each drawing update that is sequentially generated in the server device 10 in response to an input event in the client device 20, transferred to the client device 20, and displayed and recorded is input by the client device 20 alone. The drawing data corresponding to the event can be sequentially sent and played back and displayed back and back.

  Next, the operation of the server-based computing system configured as described above will be described.

  FIG. 6 is a flowchart showing server processing by the server device 10 of the server-based computing system.

  When a server control program stored in the ROM 13 in advance is started by the CPU 11 by starting up the power supply of the server device 10, various initialization setting processes such as flag memory reset and counter memory reset in the RAM 14 are executed. (Step S1), a server connection request signal reception waiting state from the client device 20 is entered (Step S2).

  When the server connection request signal from the client device 20 is received via the transmission / reception control unit 19 in the server connection request signal reception standby state (step S2 (YES)), the connection request source client device 20 is in communication with the server connection request signal. An authentication process based on the client ID and user password is executed (step S3).

  If it is determined that the authentication of the connection request source client device 20 is “OK” by this authentication processing (step S4 (YES)), the display screen size / display of the client device 20 determined to be the authentication OK is possible. It is determined whether terminal information (instance information) unique to the client device 20 such as the number of colors is already stored and managed in the RAM 14 (step S5).

  If it is determined that the terminal information (instance information) is not stored and managed for the client device 20 determined to be authenticated, the client device 20 is informed (step S5 (NO)). The terminal information request signal is transmitted, and the terminal information (instance information) is generated in response to the terminal information request signal (step S6). Then, rendering data for default display corresponding to the generated terminal information (instance information) of the client device 20 is generated (step S7), and transferred to a frame buffer in the RAM 14 reserved for the client device 20. At the same time, the data is transmitted / transferred to the client device 20 via the transmission / reception control unit 19 (step S9).

  On the other hand, if it is determined in step S4 that the terminal information (instance information) is stored and managed in the RAM 14 for the client device 20 determined to be authenticated (step S5 (YES)). The terminal information (instance information) of the client device 20 is acquired and default display drawing data is generated (step S8), transferred to the frame buffer in the RAM 14 reserved for the client device 20, and transmitted / received. The data is transmitted / transferred to the client device 20 via the control unit 19 (step S9).

  Thus, when the input event signal corresponding to the user operation is received from the client device 20 after the connection with the server device 10 and the initial screen display are performed (step S2 (NO) → S10 (YES) → S11 ( YES)), the received input event signal is transferred to the window manager for generating the display screen (drawing data) of the client device 20, and is also transferred to the application program corresponding to the input event signal. The program is started (step S12).

  When a display screen (drawing data) corresponding to the application program activated in response to an input event from the client device 20 is generated (step S13), the accelerator circuit 10A responds to the image change portion. The generated drawing data for transfer is converted and generated (step S14), transferred to the client device 20 and displayed (step S15).

  Thereafter, in a state where a process end signal is not received from the client device 20 connected to the server device 10 (step S16 (NO)), application processing and display drawing according to the input event signal from the client device 20 are performed. Data generation / transfer processing is repeatedly executed (steps S10 to S15).

  FIG. 7 is a flowchart showing client processing by the client device 20 of the server-based computing system.

  When the client control program stored in advance in the ROM 23 is started by the CPU 21 by starting up the power supply of the client device 20, it is determined whether or not the display drawing data recording mode is set according to the user operation. (Reproduction mode) is determined (step A1), and if it is determined that the recording mode is selected (step A1 (YES)), the file authentication for allowing the user to input the authentication code of the recording file to be generated A key input screen is displayed on the display device 26, and the authentication code of the recording file is input and stored (step A2).

  Then, a connection information input confirmation screen for allowing the user to input connection information of the client device 20 for connection / authentication to the server device 10 is displayed on the display device 26 (step A3). When an ID and a user password are input as connection information of the client device 20, connection to the server device 10 is made via the transmission / reception control unit 31 together with connection information such as the display screen size of the display device 26 and the number of displayable colors. A request signal is transmitted (step A4).

  In response to the transmission of the connection request signal to the server device 10, when an “authentication OK” determination signal is received from the server device 10 (step A5 (YES)), an input event signal corresponding to the user operation is sent. Based on input event thread generation processing (step AB) for generating and transmitting to the server device 10 (see FIG. 8), and display drawing data received from the server device 10 in response to transmission of the input event signal A reception data thread generation process (step AC) (see FIG. 9 / FIG. 18) for generating and recording display screen data is started and executed.

  Thereafter, when “connection disconnection” with the server apparatus 10 is determined according to the user operation (step A6 (YES)), a user selection screen of “log off” or “log off (session information retention)” is displayed ( In step A7), if the latter “logoff (session information hold)” user selection is determined (step A7 (NO)), a logoff request signal for holding the session (logon period) information and logging off is sent to the server device 10. (Step A8).

  Then, the input event thread generation process (step AB) and the reception data thread generation process (step AC) are ended (step A9), and the recording file generation process is also ended (step A10).

  As a result, display drawing data generated and transferred by the server device 10 is sequentially received and displayed in response to an input event accompanying a user operation from the client device 20, and is recorded in a recording file by setting the recording mode. Become so.

  On the other hand, if it is determined in step A1 that the playback mode is set, the process proceeds to playback processing (step AE) (see FIG. 13).

  FIG. 8 is a flowchart showing input event thread generation processing (AB) accompanying client processing by the client device 20 of the server-based computing system.

  When this input event thread generation process is activated, it enters a standby state for receiving an input event signal from the input unit 27 or an input event signal from the USB / parallel / serial interface 30 according to a user operation (step B1).

  If the CPU 21 determines that an input event signal has been received (step B1 (YES)), the event signal (information) is acquired (step B2), and whether the event content is keyboard data associated with a key input operation (step B1). Step B3), left / right click and coordinate designation data associated with mouse click operation (step B5), or data from an external device received via USB / parallel / serial interface 30 (step B7) Whether or not the clipboard data for other pre- and post-processing is determined (step B9).

  Here, when the key code is received and acquired by the CPU 21 by operating the keyboard in the input unit 27 (step B3 → B4), the corresponding input event status (key input content) + number (the same key event) Input event type data consisting of the number of inputs) is generated and temporarily stored in the RAM 24 (step B10). Then, after a message number corresponding to the keyboard data is given (step B11), transmission data to which a protocol header corresponding to the key event is added is generated (step B12), and packet communication processing is performed to the server device 10. It is transmitted (step B13).

  Further, when the designated coordinate on the display screen is detected and acquired by the CPU 21 by operating the mouse in the input unit 27 or the click signal of the left and right wheels is received and acquired (step B5 → B6), the corresponding input event status. Input event type data consisting of (pointer coordinates and left and right click contents) + number (number of times the same mouse event has been input) is generated and temporarily stored in the RAM 24 (step B10). Then, after a message number corresponding to the mouse pointer data is given (step B11), transmission data to which a protocol header corresponding to the mouse event is added is generated (step B12), packet communication processing is performed, and the server device 10 is processed. (Step B13).

  When various external devices are connected via the USB / parallel / serial interface 30 and data from the external device is received by the CPU 21 (step B7 (YES)), the received data is encoded. (Step B8), input event type data consisting of the corresponding input event status (contents of external device data) + number (number of times of input of the same external device input event) is generated and temporarily stored in the RAM 24 (Step B10). ). Then, after a message number corresponding to the external device data is given (step B11), transmission data to which a protocol header corresponding to the external device input event is added is generated (step B12), packet communication processing is performed, and the server It is transmitted to the apparatus 10 (step B13).

  Each time the input event signal is received, the input event thread generation process (AB) is repeated without being ended (step B14 (NO) → B1).

  FIG. 9 is a flowchart showing a first received data thread generation process (AC1) accompanying the client process by the client device 20 of the server-based computing system.

  When this reception data thread generation process is started, first, the reception data waiting state from the server device 10 is entered (step C11). The drawing data for display (or sound data for audio output or video data generated and transferred by the server device 10 in accordance with the contents of the input event in the client device 20 generated and transmitted by the input event thread generation processing. Or other serial data) is received via the transmission / reception controller 31 (step C11 (YES)), the process proceeds to the first recording header data generation process in FIG. 10 (step CD1).

  FIG. 10 is a flowchart showing a recording header data generation process (CD1) accompanying the first reception data thread generation process (AC1) of the client device 20.

  FIG. 11 is a diagram showing a recording format 24D1 of the reception data (drawing data) generated by the recording header data generation process (CD1) accompanying the first reception data thread generation process (AC1) of the client device 20. is there.

  FIG. 12 is a diagram showing a format of a recording file generated and recorded by the recording header data generation process (CD1) accompanying the first reception data thread generation process (AC1) of the client device 20.

  When any data generated in response to the input event is received from the server apparatus 10 and the recording header data generation process (CD1) in FIG. 10 is started, the recording mode of the client apparatus 20 is changed. In order to record the received data (drawing data) in the recording file with the authentication code set according to the setting judgment, the initial data for display (display area size, number of displayable colors, compression / decompression / decryption method) Subsequently to the type, the authentication code of the recording file, and the total frame size) 41, a memory area having a capacity composed of the size of the predetermined header 42 (42a to 42d) and the size of the received data 43 is secured in the RAM 24 (step D11). .

  Then, a frame number (reception order number) 42a to be described in the header 42 of the recording file of the received data received this time is generated and written in the header 42 of the recording file secured in the RAM 24 (step D12). Further, current time data (time stamp) 42b is acquired and written in the header 42 (step D13).

  Further, in the input event thread generation process (AB) in FIG. 8, input event type data 42d consisting of input event status + number generated and held for each input event is read and written in the header 42 of the same recording file. (Step D14) The frame size of the received data is calculated and written in the header 42 (Step D15), and the received data (main body) 43 is written subsequently to the header 42 of the recording file (Step D16). ).

  In this way, the received data transferred for each generation / update of various data from the server device 10 is written and stored in the recording file in the RAM 24 in the client device 20 according to the recording format 24D1 as shown in FIG. Then (step C12), the header information 42 is acquired by the CPU 21 (step C13), whether the received data is drawing data (step C14), sound data (step C17), or video data. Or other serial data (step C110 → C113).

  If it is determined that the data is drawing data (step C14 (YES)), the drawing data is decoded by the accelerator circuit 20A, transferred to the frame buffer in the VRAM 25, and output to the display device 26 for display. (Steps C15 and C16).

  If it is determined that the data is sound data (step C17 (YES)), the sound data is decoded into an audio signal, transferred to the audio output device, and output as audio (steps C18 and C19).

  If it is determined that the data is video data (step C110 (YES)), the video data is decoded as video data by the accelerator circuit 20A, transferred to the frame buffer in the VRAM 25, and output to the display device 26. Then, a moving image is displayed (steps C111 and C112).

  At this time, as described with reference to FIG. 5, the received (drawing) data from the server apparatus 10 other than the drawing data corresponding to the initial screen is equivalent to one screen obtained with the previous drawing update. It is received as transfer drawing data H corresponding to only the image change portion (difference) P in the drawing data X. Therefore, in the accelerator circuit 20A, the received (drawing) data corresponding to the current image change portion (difference) P is overwritten in the same change area with respect to the image data for one screen generated and displayed in the VRAM 25 last time. In this way, image data for one new screen is generated and output and displayed.

  Then, every time any data generated in response to the input event is received from the server device 10, the first received data thread generation process (AC1) is repeated without being ended (step C114 (NO) → C11).

  That is, in the first reception data thread generation process (AC1) in the client device 20 configured as described above, reception data transferred from the server device 10 for each generation / update of various data is output as display, audio, or video. Accordingly, the data is sequentially written and recorded in the recording file in the RAM 24 in accordance with the recording format 24D1 as shown in FIG. As a result, the recording file in the RAM 24 specifically includes, for example, a frame number 42a starting with initial data 41 as shown in FIG. 12, a time stamp 42b, an input event type 42d, a frame size 42c, and actual data. The recording file 24D1 consisting of 43 is recorded in the format.

  Next, in the initial step A1 accompanying the client process in the client device 20, it is determined that the playback mode is not the “recording mode” described so far, and the playback process (step AE) (see FIG. 13). The operation when moving to is described.

  FIG. 13 is a flowchart showing playback processing (AE) accompanying client processing by the client device 20 of the server-based computing system.

  FIG. 14 is a flowchart showing a first REC file read header process (EF1) accompanying the reproduction process (AE) by the client device 20 of the server-based computing system.

  When the reproduction process is started in the client device 20, first, an input of an authentication code for specifying a recording file input in advance by the user in the initial step A2 in the recording mode is awaited (step E1).

  Here, if it is determined that reproduction is possible by inputting the authentication code of the recording file to be reproduced (step E1 (YES)), the process proceeds to the first REC file read header process (EF1) in FIG. Then, the data recorded in the authenticated recording file 24D1 (see FIG. 12) is read in frame number units (step EF).

  Then, whether the data of the frame number unit read by the REC file read header process (EF1) is drawing data (step E2), sound data (step E5), or video data. It is determined whether the serial data is other than (step E8 → E11).

  If it is determined that the data is drawing data (step E2 (YES)), the drawing data is decoded as image data by the accelerator circuit 20A, transferred to the frame buffer in the VRAM 25, and output to the display device 26. And displayed (steps E3 and E4).

  If it is determined that the data is sound data (step E5 (YES)), the sound data is decoded into an audio signal, transferred to the audio output device, and output as audio (steps E6 and E7).

  If it is determined that the data is video data (step E8 (YES)), the video data is decoded as video data by the accelerator circuit 20A, transferred to the frame buffer in the VRAM 25, and output to the display device 26. Then, a moving image is displayed (steps E9 and E10).

  Also in this case, the reception (drawing) data read from the recording file 24D1 is as described with reference to FIG. 5 in the same manner as the display process accompanied by the recording of the reception (drawing) data in the “recording mode”. Other than the drawing data corresponding to the initial screen, the drawing data H for transfer corresponding to only the image change portion (difference) P in the drawing data X for one screen obtained with the previous drawing update is received. Yes. Therefore, in the accelerator circuit 20A, the received (drawing) data corresponding to the current image change portion (difference) P is overwritten in the same change area with respect to the image data for one screen generated and displayed in the VRAM 25 last time. In this way, image data for one new screen is generated and output and displayed.

  When it is determined that there is data having the next frame number in the recording file 24D1 (see FIG. 12), the next recording data is designated (step E12 → E13), and the REC file reading header is specified. The process is repeatedly transferred to the process (EF1), and the recording data having the next frame number is read (step EF).

  Thus, after the data in the recording file 24D1 (see FIG. 12) is read and reproduced in units of frame numbers, it is determined that the data to be read in the recording file is complete (step E12 (YES) )), The recording file 24D1 is closed, and the series of reproduction processes is completed (step E14 → end).

  Here, the first REC file read header process (EF1) in FIG. 14 will be described in detail.

  In step E1 immediately after the transition to the reproduction process, it is confirmed as authentication OK that the authentication code of the recording file (24D1) to be reproduced has been determined to be reproducible, and the first REC file is read. When header processing (EF1) is started (step F11), header structures (1), (2) for temporarily storing data read from the recording file 24D1 in the RAM 24 for each frame number (FIG. 11). The memory area similar to the recording format shown) and each memory area of the counter are initialized and cleared (step F12).

  Then, the recording file 24D1 (see FIG. 12) stored and saved in the RAM 24 is opened (step F13), and the data is based on the display area size, the number of display colors, etc. recorded as the initial data 41 at the head thereof. A display screen is generated and displayed on the VRAM 25 (step F14).

  Then, the header 42 of the recording file 24D1 (see FIG. 12) and the pointer pt for specifying the head position in the storage area of the received data 43 are set, and the header 42 of the head frame “1” designated by the pointer pt. (Frame number 42a / time stamp 42b / input event type (+ number) 42d / frame size 42c) and received (recorded) data 43 are sequentially stored in the header structure (1) (step F15).

  Then, the frame number 42a (in this case “1”) stored in the header structure (1) and the frame number 42a stored in the previous read process stored in the header structure (2) are stored. It is determined whether or not the number incremented by “+1” (in this case, “none” for the first time processing) matches (step F16).

  In the case of this initial reading process, since it is determined that there is a mismatch (step F16 (NO)), the header 42 and the received (recorded) data corresponding to the frame number “1” stored in the header structure (1). 43 is directly copied to the header structure (2) (step F17).

  The pointer pt for designating the reading position of the recording file 24D1 (see FIG. 12) is stored in the header structure (1) and the header 42 of the frame number “1” that has been read this time and received. The frame size corresponding to the data 43 is updated and the start position of the next frame “2” is designated (step F18).

  Then, the reception (recording) data 43 of the frame number “1” read-processed this time and copied to the header structure (2) is processed by steps E2 to E11 in the reproduction process (see FIG. 13). As the protocol analysis is executed, reproduction output such as display of drawing data, sound output of sound data, or display of video data is started.

  Thereafter, since it is determined that there is data having the next frame number in the recording file 24D1 (step E12 (NO)), reading of the next recording data is designated (step E13), and the reading of the REC file is performed. The process returns to the process from Step F15 in the header process (EF1) (Step E13 → F15).

  In this case, the header 42 and received data 43 of the frame “2” currently designated by the pointer pt are overwritten and stored in the header structure (1) (step F15).

  Then, the frame number 42a (in this case “2”) stored in the header structure (1) and the frame number 42a (in this case “1”) stored in the header structure (2) are “ It is determined that the number incremented by +1 (in this case, “2”) matches (step F16 (YES)), and it is confirmed that the data is continuous frames. Next, the header structure (1) The time stamp 42b [06] of the frame “1” copied to the header structure (2) with respect to the time stamp 42b [06/11/27 19: 00: 00: 200] of the frame “2” stored in the header structure (2). 11/27 19: 00: 00: 100], it is determined whether or not a value obtained by sequentially adding the timer count value (+1 msec) has been reached (steps F19 → F110 → F19).

  That is, in the process of step F19 + F110, the reception end time of the reception (recording) data of the frame “1” read and reproduced in the previous reading process and the reception of the frame “2” read in the current reading process (Recording) Synchronization with the data reproduction start time is achieved.

  Here, step F19 (YES), that is, the reproduction end time of the reception (recording) data of the frame “1” stored in the header structure (2) and the header structure (1) are stored. If it is determined that the reproduction start time of the received (recorded) data of frame “2” matches, whether or not the input event type (+ number) 42d of the received (recorded) data is the same. Determination is made (step F111). That is, it is determined whether or not the recording data of the frame “1” read and reproduced last time and the recording data of the frame “2” to be read and reproduced this time are data generated in response to the same user input event. The

  In step F111 (YES), that is, the record data of the frame “1” read and reproduced last time and the record data of the frame “2” read and reproduced this time are both the same user input event [mouse left press event 000. ], The header 42 and the received (recorded) data 43 corresponding to the frame number “2” stored in the header structure (1) are directly used in the header structure. It is copied to the body (2) (step F17).

  Then, the pointer pt for designating the reading position of the recording file 24D1 (see FIG. 12) is stored in the header structure (1) and the header 42 of the frame number “2” that has been read this time and received. The frame size corresponding to the data 43 is updated, and the head position of the next frame “3” is designated (step F18).

  Then, following the reading / reproducing output of the reception (recording) data 43 of the frame number “1”, the reception (recording) of the frame number “2” read-processed this time and copied to the header structure (2). With respect to the data 43, protocol analysis is performed by the processing in steps E2 to E11 in the same reproduction processing (see FIG. 13), drawing data display, sound data audio output, video data display, and the like. The reproduction output is started.

  Thereafter, the processing in steps E12, E13, F15, F16, F19, F110, F111, F17, F18, E2 to E11 is repeated in the same manner, so that the same user input event [mouse left press event 000 ] Corresponding to the next frame number “4” (user input event [mouse coordinate event 001]) after the reception (recording) data 43 of the next frame number “3” generated according to the above is reproduced and output. When the header 42 and the received (recorded) data 43 are newly read from the record file 24D1 and stored in the header structure (1) (step F15), steps F16 (YES), F19 (YES), F111 (NO) In other words, the record data of the frame “3” read and reproduced last time and the record data of the frame “4” read and reproduced this time are: It is determined that the data is not generated in response to the same user input event.

  Then, for example, as shown in FIG. 4, a user operation input of the “feed” button F displayed along the lower end of the display screen of the display device 26 is detected, such as a key event, a mouse event, other voice input events, Whether it is determined that there has been some input event associated with the user operation such as a sensor detection event (step F112 (YES)), or it is determined that a certain time has passed without any input event associated with the user operation. (Step F113 (YES)), the reproduction output of the reception (recording) data 43 of the frame number “4” read this time into the header structure (1) is started (steps F17, F18, E2). E11).

  Then, through the same reading process as described above, the reproduction (output) of the reception (recording) data 43 from frame numbers “5” to “8” generated by the same user input event [mouse coordinate event 001] is performed. Each time the received (recorded) data generated by the different user input event is newly stored in the header structure (1) after being sequentially executed, it waits for some input event accompanying the user operation (step F112), or Waiting for a certain period of time (step F113) to enter a determination waiting state, and further reading and reproducing the recording data after the next frame number according to the determination of whether there has been an input event associated with the new user operation or whether a certain period of time has elapsed. The process is repeated.

  Incidentally, by setting the fixed time for determining the setting in step F113 to infinity, it is assumed that an input event associated with a new user operation is an absolute condition, and the process shifts to reading and playback processing of recording data after the next frame number. Can be performed.

  Therefore, according to the client device 20 of the server-based computing system configured as described above, received data such as drawing data, sound data, and video data generated and transferred from the server device 10 in response to an input event accompanying a user operation. Are written and stored in the recording file 24D1 in the RAM 24 in association with the types of the input events in the order of the frame numbers for each data generation update, and are sequentially displayed on the screen or output as audio. According to the first REC file read header function (EF1) accompanying the reproduction process, the received (recorded) data stored in the recording file 24D1 in the RAM 24 is sequentially read for each frame number, and the same input event The received (recorded) data associated with the type is continuously reproduced and output, and when the received (recorded) data associated with a different input event type is read, an input event associated with a new user operation The reproduction / output of the received (recorded) data is resumed according to the determination or the determination of the passage of a certain time. As a result, the data generated and transferred from the server apparatus 10 in response to a user operation input event from the client apparatus 20 and reproduced and output by the client apparatus 20 can be reproduced and output by the client apparatus 20 alone. The received (recorded) data corresponding to each user operation input event can be reproduced and output in the same manner as the data generation output for each user operation input event.

  In the first REC file read header function (EF1) shown in FIG. 14, received (recorded) data corresponding to input events of various user operations stored in the recording file 24D1 (see FIG. 12) in the RAM 24. However, the second REC file reading header function (EF2) shown in FIG. 15 and FIG. 16 is used. As described above, a configuration may be adopted in which the same different input events are divided and can be reproduced and output by changing in either the “feed” or “return” direction by a user operation.

  FIG. 15 is a flowchart showing a second REC file read header process (EF2) accompanying the reproduction process (AE) by the client device 20 of the server-based computing system.

  FIG. 16 is a flowchart showing a “return mode” setting routine associated with the second REC file read header process (EF2).

  In the second REC file read header process (EF2), the processes in steps F21 to F211 are the same as the processes in steps F11 to F111 in the first REC file read header process (EF1) shown in FIG. Since they are the same, a detailed description of each processing step is omitted.

  That is, when the authentication of the recording file 24D1 is confirmed (step E1) and the second REC file read header process (EF2) is started (step F21), the initialization process is performed (step F22). First, immediately after the recording file 24D1 (see FIG. 12) is opened (step F23) and a data display screen is generated and displayed on the VRAM 25 based on the initial data 41 at the head (step F24), "Feed mode" is set (step F213).

  Immediately after the header 42 and the received (recorded) data 43 corresponding to the frame n in the recording file 24D1 designated by the pointer pt are read and stored in the header structure (1) (step F25), It is determined whether or not “mode” is set (“return mode”) (step F214). If it is determined that “feed mode” is set, processing for determining the order of frame numbers of read data (Step F26), synchronization determination processing (Steps F29, F210) between the data reproduction time read last time and the data reproduction time read this time, and the input event type (+ number) corresponding to the data read last time And a determination process for matching the input event type (+ number) corresponding to the data read this time (step F211) It will be migrated.

  Then, for example, when the recording data with the frame number “1” is read into the header structure (1) for the first time and it is determined that the frame numbers of the read data are not in order (step F26 (NO)), or When it is determined that the input event type (+ number) corresponding to the data read last time matches the input event type (+ number) corresponding to the data read this time (step F211 (YES)) The frame number count (+1) process (step F215) is performed, the currently read data is copied from the header structure (1) to (2) (step F27), and the data start designation position of the next frame after the pointer pt Through the updating process up to (step F28), reproduction output of the current read data copied to the structure (2) is started. That (step E2~E11).

  In step F211, if it is determined that the input event type (+ number) corresponding to the previously read data is different from the input event type (+ number) corresponding to the current read data, As shown in FIG. 4, a pressing determination process of the “feed” button F or the “return” button R displayed along the lower end of the display screen of the display device 26 is performed (steps F212F / F212R).

  If it is determined that the “feed” button F has been pressed by a user operation (step F212F (YES)), the count value of the frame number count at that time is cleared (F216), and the Returning to the process from step F215, after newly performing the frame number count (+1) process, the reproduction output process of the reception (recording) data 43 corresponding to the frame number of the input event type that has been read this time is performed. (Steps F27, F28 → E2 to E11).

  That is, in the setting state of the “feed mode” in the second REC file read header process (EF2), each frame number is recorded from the recording file 24D1 in the same manner as in the first REC file read header process (EF1). The received (recorded) data 43 read in the frame is sequentially reproduced and output within the range of the same input event type, and the received (recorded) data 43 of the frame number read this time is associated with different input event types. In the case of the received data, the reproduction output from the data 43 of the frame number associated with the different input event type is resumed by the user operation of the “Send” button F.

  Thus, by repeating the reading / reproducing process of the reception (recording) data 43 for each frame number in the “feed mode”, for example, the frame numbers “4” to “4” stored in the recording file 24D1 shown in FIG. The data 43 for 5 frames corresponding to the input event type [mouse coordinate event 001] up to “8” is sequentially reproduced and output, and the frame number “1” stored in the header structure (1) by the current read processing. When the input event type [mouse left press event 001] 42d corresponding to the data 43 of 9 ″ is determined to be different (NO in step F211), the “return” button R is operated by the user. If it is determined that it has been (step F212R (YES)), the routine proceeds to the “return mode” setting routine shown in FIG.

  That is, when the “return” button R is operated, the designated position of the pointer pt is moved back to the position of the first frame number “1” in the recording file 24D1 (see FIG. 12) (step G1). The header 42 and the received (recorded) data 43 corresponding to the previously reproduced frame number “8” stored in the header structure (2) are cleared (step G2).

  Then, from the frame number “9” of the data stored in the header structure (1), the frame count number “5” corresponding to the input event type [mouse coordinate event 001] counted in the step F215. "Is subtracted to set the return frame number" 4 "(step G3), and" return mode "is set (step G4).

  Then, the process proceeds to step F25, where the header 42 and the received (recorded) data 43 corresponding to the first frame number “1” newly designated by the pointer pt are stored in the header structure (1). In step F214, since it is determined that the current state is the “return mode” setting state and not the “feed mode”, the frame number “1” of the data stored in the header structure (1) is determined in step G3. It is determined whether or not it matches the set return frame number “4” (step F217).

  In this case, since it is determined that they do not match (step F217 (NO)), the new reproduction output of the data is temporarily stopped (step F218), and the frame “corresponding to the data stored in the header structure (1)” It is determined whether or not the input event type 1 ”[mouse left press event 000] matches the input event type corresponding to the data stored in the header structure (2) (step F219).

  In this case, since the stored data of the header structure (2) is cleared in the step G2, the input event type of the frame “1” corresponding to the stored data of the header structure (1) [mouse left press It is determined that the input event type corresponding to the data stored in the header structure (2) does not match the event 000] (step F219 (NO)), and the frame count of the frame count is set to “1”. (Step F220).

  Then, the processes of steps F27 and F28 → E2 to E11 are executed while the reproduction output is paused, and the data 42 and 43 with the frame number “1” stored in the header structure (1) are headers. The data is copied to the structure (2), and the pointer pt is moved to the head designated position of the data 42 and 43 corresponding to the next frame number “2”.

  Then, the data 42 and 43 corresponding to the frame number “2” newly designated by the pointer pt are overwritten and stored in the header structure (1) (step F25). If it is determined that there is no “return mode” (step F214 (NO)), the frame number “2” of the data stored in the header structure (1) is still the return frame number set in step G3. It is determined that it does not match “4” (step F217 (NO)).

  Therefore, the new reproduction output of the data continues to be paused (step F218), and the input event type [mouse left press event 000] of the frame “2” corresponding to the data stored in the header structure (1). ] And the input event type [mouse left press event 000] of the frame “1” corresponding to the storage data copied to the header structure (2) (step F219 (YES)), the frame count number “1” is incremented (+1) and set to “2” (step F221).

  Then, in the same manner as described above, the processes of steps F27 and F28 → E2 to E11 are executed while the reproduction output is paused, and the data 42 and 43 of the frame number “2” stored in the header structure (1). Is copied to the header structure (2), and the pointer pt is moved to the head designated position of the data 42 and 43 corresponding to the next frame number “3”.

  After that, similarly to the above, the processing of steps F25, F214 → F217 to F219 → F221 → F27, F28 → E2 to E11 is repeated in the pause state of the reproduction output, so that the frame newly designated by the pointer pt The data 42 and 43 with the number “4” are stored in the header structure (1) (step F25), and if it is determined that the “return mode” continues (step F214 (NO)), the header structure ( 1) When it is determined that the frame number “4” corresponding to the stored data matches the return frame number “4” set in step G3 (step F217 (YES)), the setting is switched to “feed mode”. (Step F222), the temporary output stop of the data is canceled and the output is set to ON (Step F223). .

  Then, a determination is made to wait for a user operation for the “feed” button F or the “return” button R (step F212F / F212R). If it is determined that the “feed” button F is operated by the user (step F212F (YES) )), Received (recorded) data for 5 frames corresponding to the input event type [mouse coordinate event 001] that has already been reproduced and output by the process of steps F216, F215, F27, F28 → E2 to E11. Playback starts from the point 43 returns to the top frame number “4”.

  On the other hand, the frame number “4” corresponding to the data stored in the header structure (1) matches the return frame number “4” (step F217 (YES)), and is switched to the “feed mode” (step S21). F222), the temporary output stop of the data is canceled and the output is set to ON (step F223), and in the state waiting for the user operation for the “feed” button F or the “return” button R (steps F212F / F212R) When it is further determined that the “return” button R has been operated by the user (step F212R (YES)), the routine again proceeds to the “return mode” setting routine shown in FIG.

  Then, the return frame number is set back to the first frame number “1” of the data 42 and 43 for three frames corresponding to the input event type [mouse left press event 000] reproduced and output before that. At the same time, the "return mode" is reset (steps G1 to G4). Then, in the same manner as described above, after the repetitive processing in the data output pause state of steps F25, F214 → F217 to F219 → F221 → F27, F28 → E2 to E11 and the feed mode restart processing in the steps F217 → F222, F223, According to the user operation determination of the “Send” button F (Step F212F (YES)), the reproduction output of the reception (recording) data 43 from the point where the return frame number “1” is returned is started (Step F216). , F215, F27, F28 → E2-E11).

  Therefore, according to the second REC file read header function (EF2) accompanying the reproduction processing in the client device 20 of the server-based computing system having the above-described configuration, the reception (recording) stored in the recording file 24D1 in the RAM 24 is performed. ) Data is sequentially read for each frame number, and received (recorded) data associated with the same input event type is continuously reproduced and output, and received (recorded) data associated with different input event types. Is read, the reproduction output of the received (recorded) data is resumed according to the operation determination of the “Send” button F. On the other hand, when the operation of the “return” button R is determined, the first frame number of the received (recorded) data associated with the previously reproduced input event type is set as the return destination frame number and all received Only the reading of the data for each frame number and the generation processing of the output data are repeated with the reproduction output of the data from the reception (recording) data corresponding to the first frame number of the (recording) data being paused. When the received (recorded) data corresponding to the return destination frame number is read, the playback output is temporarily stopped and the return is made after the operation determination of the “Send” button F is made. Resume playback output from pre-received (recorded) data.

  As a result, the data generated and transferred from the server device 10 in response to a user operation input event from the initial client device 20 and output by the client device 20 can be obtained for each input event of the initial user operation by the client device 20 alone. In the same manner as in the data output, the received (recorded) data corresponding to each input event corresponding to the operation input of the “Send” button F can be reproduced and output as well as the operation of the “Return” button R. By returning to the data start position for each input event and re-operating the “Send” button F, the data from the return position can be reproduced and output.

  In this case, the reproduction output of the reception (recording) data from the return position is repeated only for the output data generation / update process from the beginning of all reception (recording) data with the reproduction output of the data being paused. Since the reproduction output is temporarily stopped from the received (recorded) data at the return position, the output data is generated, updated, and reproduced and output, as described with reference to FIG. Is transferred to the client device 20 of the server-based computing system in which each screen data generated in step (b) is transferred as transfer drawing data H that is cut out only in the region Q ′ corresponding to the image change portion (difference) P. However, the received (recorded) data from the return position can be drawn and displayed as correct screen data.

  Note that the second REC file read header function (EF2) shown in FIGS. 15 and 16 receives in response to input events of various user operations stored in the recording file 24D1 (see FIG. 12) in the RAM 24 (see FIG. 12). Record) data is divided into different input events, and when the “Send” button F is operated, it is read and updated in the direction of sending sequentially, and is reproduced and output, while one each time the “Return” button R is operated. The received (recorded) data corresponding to the previous input event is set as the return destination of the reproduction start, and the reproduction output can be performed again in response to the operation of the “feed” button F. When the “slow feed” button is added and the button is operated, the timer count speed in the step F210 is increased by N times or reduced by 1 / N. By the variable or may be configured to fast forward or slow feeding of data reproduction output.

  For example, when the “fast return” button is added and the button is operated, the frame number (return frame number) of the return destination data set in step G3 corresponds to the input event read this time. By returning to the first frame number of the received (recorded) data corresponding to the input event two or three times before the first frame number of the received (recorded) data, it is returned with one return operation. A configuration is also possible in which the return width (speed) of received (recorded) data to be reproduced can be varied (fast).

  Further, when the “fast forward” button is added and the button is operated in the same manner as when the “fast return” button is added, the frame number (feed frame number) of the destination data to be reproduced next is By setting the first frame number of the received (recorded) data corresponding to the input event read this time to the first frame number of the received (recorded) data corresponding to the second or third input event, A configuration may be employed in which the feed width (speed) of received (recorded) data to be sent and reproduced in response to a single feed operation can be varied (fast).

  In the first received data thread generation processing (AC1) by the client device 20 described with reference to FIGS. 9 to 12, the recording mode is set by the user selection from the start of the processing, and the server device 10 All the received data generated and transferred are sequentially written in the memory by the recording header data generation process (CD1) and stored as the recording file 24D1 in the RAM 24. However, the configuration shown in FIGS. As described in the second received data thread generation process (AC2), the received data generated and transferred by the server device 10 can be written in the memory within an arbitrary range of the user and stored in the recording file 24D2 in the RAM 24. Good.

  FIG. 17 is a flowchart showing second received data thread generation processing (AC2) accompanying client processing by the client device 20 of the server-based computing system.

  FIG. 18 is a flowchart showing a recording header data generation process (CD2) accompanying the second reception data thread generation process (AC2) of the client device 20.

  FIG. 19 is a diagram showing a recording format 24D2 of the reception data (drawing data) generated by the recording header data generation process (CD2) accompanying the second reception data thread generation process (AC2) of the client device 20. is there.

  FIG. 20 is a diagram showing a format of a recording file generated and recorded by the recording header data generation process (CD2) accompanying the second reception data thread generation process (AC2) of the client device 20.

  When the second received data thread generation process is started, first, the “replay mode” is set, and the input event status (2) in which the contents of the input event corresponding to the previous received data is saved is cleared. Then, it is initialized (step C21) and enters a standby state for data received from the server device 10 (step C22). Then, display drawing data (or sound for sound output) generated and transferred by the server device 10 according to the contents of the input event in the client device 20 generated and transmitted by the input event thread generation processing (AB). When data, video data, or other serial data) is received via the transmission / reception control unit 31 (step C22 (YES)), it is determined whether or not the “recording mode” is set (step S22). C23).

  Here, if it is determined that the current client device 20 is the “reproduction mode” initially set in step C21 and not the “recording mode” (step C23 (NO)), for example, provided in the input unit 27. It is determined whether or not the “record start” button is pressed (step C24).

  If it is determined that the “recording start” button has not been pressed (step C24 (NO)), the “reproduction mode” setting state is continued, and the server device 10 generates and updates various data. The header information 42 of the received data transmitted and transferred every time is acquired by the CPU 21 (step C25), whether the received data is drawing data (step C26), sound data (step C29), or video. It is determined whether it is data or other serial data (steps C212 → C215).

  If it is determined that the data is drawing data (step C26 (YES)), the drawing data is decoded as image data by the accelerator circuit 20A, transferred to the frame buffer in the VRAM 25, and output to the display device 26. And displayed (steps C27 and C28).

  If it is determined that the data is sound data (step C29 (YES)), the sound data is decoded into an audio signal, transferred to the audio output device, and output as audio (steps C210 and C211).

  If it is determined that the data is video data (step C212 (YES)), the video data is decoded as video data by the accelerator circuit 20A, transferred to the frame buffer in the VRAM 25, and output to the display device 26. The moving image is displayed (steps C213 and C214).

  That is, in this “reproduction mode” setting state, the received data generated and transferred from the server device 20 is sequentially decoded and reproduced and outputted (steps C224 (NO) → C22 to C215).

  Here, for example, if it is determined that the “recording start” button has been pressed at the timing when user-received received data is reproduced (step C24 (YES)), the “recording mode” is set (step C216). A new recording file 24D2 is secured in the RAM 24 (step C217).

  Then, initial data 41 'including the display area size in the client device 20, the number of displayable colors, the type of received data encoding method, and the authentication code of the recording file 24D2 is generated (step C218), and the current display device 26 The image data of one screen reproduced and displayed at is captured from the frame buffer (FB) in the VRAM 25 or captured from the server device 10 (step C219).

  Then, the process proceeds to the recording header data generation process (CD2) in FIG. 18 (step CD2).

  When the recording header data generation process (CD2) is started, the input event status (contents of the input event) generated and held for each input event in the input event thread generation process (AB) in FIG. 8 is acquired. (Step D21), the input event type corresponding to the current received data acquired from the input event status is different from the input event type corresponding to the previous received data held in the input event status (2). Or the same (step D22).

  At the initial stage when the recording header data generation process (CD2) is started, no input event type is held in the input event status (2), so the content of the input event status (2) is It is determined that the input event type corresponds to the current received data acquired from the input event status (step D22 (YES)).

  Then, in order to record the received data (drawing data) in the recording file 24D2 newly secured in the RAM 24 in the step C217, the initial data for display (display area size, number of displayable colors, received data encoding) Type of method, recording file authentication code, overall frame size, FB capture data (recording start screen data)) 41 ', followed by a capacity consisting of a predetermined header 42 (42a to 42d) size and received data 43 size Is reserved in the RAM 24 (step D25).

  Then, a frame number (reception order number) 42a to be described in the header 42 of the recording file 24D2 of the received data received this time is generated and written to the header 42 of the recording file 42D2 secured in the RAM 24 (step D26), current time data (time stamp) 42b is acquired and written in the header 42 (step D27).

  Further, the input event type data 42d consisting of the input event status + number is read and written in the header 42 of the recording file 24D2 (step D28), and the frame size of the received data is calculated and stored in the header 42. It is written (step D29).

  Then, the data of the input event status corresponding to the current received data is rewritten and stored in the input event status (2) (step D210).

  The current reception data (main body) 43a is subsequently written in the header 42 of the recording file 24D2 (step D24).

  Thus, after the “recording mode” is set, the received data transmitted / transferred every time various types of data are generated / updated in the server device 10 are stored in the recording file 24D2 in the RAM 24 in the client device 20 in FIG. When written and stored in accordance with the recording format as shown (step C220), the received data 43a stored this time is also subjected to protocol analysis according to the processing of steps C25 to C215, decoded and reproduced and output.

  Subsequently, drawing data for display (or sound data for audio output, video data, or other serial data) generated and transferred by the server device 10 is received via the transmission / reception control unit 31. At this time (step C22 (YES)), it is determined that the “recording mode” is set (step C23 (YES)), and for example, the “recording stop” button provided in the input unit 27 is pressed. If not judged (step C221 (NO)), the process is repeatedly transferred to the recording header data generation process (step CD2) in FIG.

  In this recording header data generation process (step CD2), the input event type corresponding to the current reception data acquired from the input event status and the previous reception data held in the input event status (2) are supported. When it is determined that the input event type is the same (steps D21 and D22 (NO)), as shown in FIGS. 19 and 20, the record data 24D2 in which the previous received data 43a is written is stored. Following the previous reception data 43a, a memory area corresponding to the current reception data 43b is secured and written (steps D23 and D24).

  That is, in the recording header data generation process (CD2) associated with the second reception data thread process (AC2), a plurality of reception data 43a, 43b,. The header 42 is used as a common one and is continuously written in the recording file 42D2.

  Thereafter, when it is determined that the “recording stop” button has been pressed (step C221 (YES)), the “reproduction mode” is set again (step C222), and the recording file 24D2 is closed and stored. (Step C223). Then, the next received data generated / transferred from the server apparatus 10 is sequentially reproduced and output as it is without being stored in the recording file 24D2.

  Thereby, the received data from the server device 10 is stored in the recording file in the RAM 24 in a period (range) in which the “recording mode” is set by the user operation, for example, as shown in FIG. Are recorded in the format of a recording file 24D2 composed of a frame number 42a starting with initial data 41 ', a time stamp 42b, an input event type 42d, a frame size 42c, actual data 43a, 43b,.

  Therefore, according to the second received data thread generation function (AC2) accompanying the client process in the client device 20 of the server-based computing system having the above-described configuration, the “record start” button and the “record stop” button by the user By this operation, it becomes possible to selectively write / save only received data to be stored in the recording file 24D2, and to suppress the necessary minimum storage capacity. In addition, regarding the reception data generated in response to the same input event, even if the reception data includes a plurality of reception data, it is associated with one common header 42 managed by one frame number 42a. Since the REC file read header process (EF) accompanying the reproduction process (AE) shown in FIG. 13 is written and stored continuously, the reproduction start / stop of the received data is simply separated for each frame number. What is necessary is just to control, and the process step can be simplified greatly.

  Note that each processing method by the client device 20 of the server-based computing system described in the embodiment, that is, client processing by the client device 20 shown in the flowchart of FIG. 7, and client processing shown in the flowchart of FIG. The input event thread generation process (AB) involved, the first reception data thread generation process (AC1) accompanying the client process shown in the flowchart of FIG. 9, and the first reception data thread generation process (AC1) shown in the flowchart of FIG. ) Recording header data generation process (CD1) accompanying the client process shown in the flowchart of FIG. 13, the first REC file accompanying the playback process (AE) shown in the flowchart of FIG. Read header processing ( F1), second REC file read header processing (EF2) accompanying the reproduction processing (AE) shown in the flowcharts of FIGS. 15 and 16, and second received data thread generation accompanying the client processing shown in the flowchart of FIG. Each method such as the processing (AC2) and the recording header data generation processing (CD2) accompanying the second reception data thread generation processing (AC2) shown in the flowchart of FIG. 18 can be executed by the computer. The program can be stored and distributed in an external recording medium such as a memory card (ROM card, RAM card, etc.), magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, etc. . The computer (CPU 21) of the client device 20 reads the program stored in the external recording medium into the storage device (23), and the operation is controlled by the read program. The client device 20 can realize the recording function and the arbitrary reproduction function of the received data from the server device 10, and can execute the same processing as described above.

  In addition, program data for realizing each of the above methods can be transmitted on a communication network (LAN) N in the form of a program code, and a computer device (program server) connected to the communication network (LAN) N The program data is taken in via the transmission / reception control unit (31) and stored in the storage device (23), and the recording function and the arbitrary reproduction function of the received data from the server device 10 in the client device 20 described above are realized. You can also.

  Note that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention at the stage of implementation. Further, each of the embodiments includes inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in each embodiment or some constituent features are combined, the problems described in the column of the problem to be solved by the invention can be solved. When the effects described in the column of the effect of the invention can be obtained, a configuration in which these constituent elements are deleted or combined can be extracted as an invention.

1 is a block diagram showing a configuration of a server-based computing system including a server device 10 and client devices 20 according to the embodiment of the present invention. The block diagram which shows the circuit structure of the server apparatus 10 in the said server base computing system. The block diagram which shows the circuit structure of the client apparatus 20 in the said server-based computing system. The figure which shows the display state of the reproduction | regeneration operation button accompanying the reproduction | regeneration processing of the recording drawing data in the client apparatus 20 of the said server base computing system. FIG. 5 is a diagram showing a feed playback display screen accompanying input of a “feed” button F and a return playback display screen accompanying input of a “return” button R in a state where the client device 20 is set to a playback mode. The flowchart which shows the server process by the server apparatus 10 of the said server base computing system. The flowchart which shows the client process by the client apparatus 20 of the said server-based computing system. The flowchart which shows the input event thread | sled generation process (AB) accompanying the client process by the client apparatus 20 of the said server-based computing system. The flowchart which shows the 1st reception data thread production | generation process (AC1) accompanying the client process by the client apparatus 20 of the said server-based computing system. The flowchart which shows the recording header data generation process (CD1) accompanying the 1st reception data thread generation process (AC1) of the said client apparatus 20. FIG. The figure which shows the recording format 24D1 of the reception data (drawing data) produced | generated by the recording header data production | generation process (CD1) accompanying the 1st reception data thread production | generation process (AC1) of the said client apparatus 20. FIG. The figure which shows the format of the recording file produced | generated and recorded by the recording header data production | generation process (CD1) accompanying the 1st reception data thread production | generation process (AC1) of the said client apparatus 20. FIG. The flowchart which shows the reproduction | regeneration processing (AE) accompanying the client process by the client apparatus 20 of the said server base computing system. The flowchart which shows the 1st REC file reading header process (EF1) accompanying the reproduction | regeneration processing (AE) by the client apparatus 20 of the said server-based computing system. The flowchart which shows the 2nd REC file reading header process (EF2) accompanying the reproduction | regeneration processing (AE) by the client apparatus 20 of the said server-based computing system. The flowchart which shows the setting routine of the "return mode" accompanying the said 2nd REC file reading header process (EF2). The flowchart which shows the 2nd reception data thread production | generation process (AC2) accompanying the client process by the client apparatus 20 of the said server-based computing system. The flowchart which shows the recording header data generation process (CD2) accompanying the 2nd reception data thread generation process (AC2) of the said client apparatus 20. FIG. The figure which shows the recording format 24D2 of the reception data (drawing data) produced | generated by the recording header data production | generation process (CD2) accompanying the 2nd reception data thread production | generation process (AC2) of the said client apparatus 20. FIG. The figure which shows the format of the recording file produced | generated and recorded by the recording header data production | generation process (CD2) accompanying the 2nd reception data thread production | generation process (AC2) of the said client apparatus 20. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Server apparatus 20 ... Client apparatus 10A, 20A ... Accelerator circuit 11, 21 ... CPU
12, 22 ... Bus 13, 23 ... ROM
14, 24 ... RAM
15, 26 ... display device 16, 25 ... VRAM
17, 27 ... Input unit 18, 28 ... External storage HDD
19, 31 ... Transmission / reception control unit 30 ... USB / parallel / serial interface N ... Communication network F ... "Send" button R ... "Return" button G ... Drawing data before change G '... Drawing data after change X ... One screen Drawing data of minutes a, b, c ... Drawing data of changed part P ... Changed part Q ... Area before change corresponding to image changed part (difference) P Q '... Area corresponding to image changed part (difference) P R ... background matching area S ... transparent drawing data (color image data that does not require rewriting) S
H: Drawing data for transfer 24D1, 24D2 ... Recording format (recording file)
41, 41 '... initial data 42 ... header 42a ... frame number 42b ... time stamp 42c ... frame size 42d ... input event type (+ number)
43 ... received (recorded) data

Claims (10)

A client device of a server-based computing system,
Event transmission means for transmitting event information corresponding to a user operation to the server device;
Data receiving means for receiving data generated by the server device in accordance with the event information transmitted by the event transmitting means;
Received data recording means for recording the data received by the data receiving means in association with the frame number of the reception order and the event information;
Data reproduction means for reproducing the reception data recorded by the reception data recording means based on the frame number and event information;
A client device of a server-based computing system, comprising: Furthermore, a recording mode setting means for setting a recording mode according to a user operation is provided,
The received data recording means records the data received by the data receiving means in association with the frame number in the receiving order and the event information in a state where the recording mode is set by the recording mode setting means.
The client device of the server-based computing system according to claim 1. The received data recording means is
Recording event information determining means for determining whether or not event information corresponding to the data received last time by the data receiving means matches event information corresponding to the data received this time;
If the event information corresponding to the currently received data is determined to be the same event information as the event information corresponding to the previously received data by the recording event information determining means, Record the received data in association with the same frame number as the previous received data,
The client device of the server-based computing system according to claim 1 or 2, wherein the client device is a server device. The data reproducing means includes
Number direction reproducing means for reproducing the received data recorded by the received data recording means in the direction of sending the frame number associated with the received data;
Replay event information determining means for determining whether or not the event information of the frame number associated with the reception data reproduced last time and the event information of the frame number associated with the reception data to be reproduced this time are the same. Have
If the event information associated with the reception data to be reproduced this time is determined to be different from the event information associated with the reception data reproduced last time by the reproduction event information determination means, Play the received data to be played this time according to the input event according to the user operation.
The client device of the server-based computing system according to any one of claims 1 to 3, wherein the client device is a server device.   5. The server-based computing system according to claim 4, wherein the number direction reproduction means includes a transmission degree variable means for changing a transmission degree of a frame number when reproducing the received data. Client device. The data reproducing means includes
A return frame number setting means for returning and setting the frame number of received data to be reproduced according to a return input event according to a user operation;
In the case where the number direction reproducing means returns and sets the frame number of the reception data to be reproduced by the return frame number setting means, the frame set by returning the reception data recorded by the reception data recording means. Play from the number in the direction of sending the frame number,
The client device of the server-based computing system according to claim 4.   The return frame number setting means comprises return degree variable means for making variable the degree of return of the frame number when setting the frame number of the reception data to be reproduced. A client device of the described server-based computing system. The data reproducing means includes
It further has a time measuring means for measuring a preset time,
When the event information associated with the reception data to be reproduced this time is determined by the reproduction event information determination means to be different from the event information associated with the reception data reproduced last time, When the time set in advance by the time measuring means is timed or according to the input event according to the user operation, the received data to be reproduced this time is reproduced.
The client device of the server-based computing system according to any one of claims 4 to 7, wherein the client device is a server device.   9. The client device of a server-based computing system according to claim 8, wherein the time measuring means measures a time set to infinity in advance. A client control program for controlling a computer of a client device of a server-based computing system,
The computer,
Event transmission control means for transmitting event information input in response to a user operation to the server device via the communication control unit;
Data reception control means for receiving data generated by the server device in accordance with the event information transmitted by the event transmission control means via the communication control unit;
Received data recording control means for recording data received by the data reception control means in a memory in association with the frame number of the reception order and the event information,
Data reproduction control means for reproducing the reception data recorded in the memory by the reception data recording control means by the data output unit based on the frame number and event information;
Server-based computing system client control program that functions as a server.

Images