JP2016057965A - Server, storage determination program, and storage determination method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently store information on a screen of a client terminal, for efficient support.SOLUTION: A thin client system includes a client terminal 50 and a server 100. The server 100 detects an update area, on the basis of a screen operated by a user. The server 100 performs video-compression on a first update area having an update frequency equal to or higher than a predetermined threshold, and performs still-image compression on a second update area having an update frequency less than the predetermined threshold, on the basis of the update frequency of the update areas. The server 100 determines whether to store the second update area, on the basis of the size of the first update area and the size of the second update area, when a new update area includes both first and second update areas. The server 100 stores the second update area, on the basis of a determination result.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a server and the like.

  In recent years, there is a technique using a thin client system as a conventional technique for assisting a user in operating a client terminal. In this prior art, screen information while a user operates a client terminal is stored in a server, and the server notifies the supporter's terminal of the screen information. As a result, the supporter can check the user's operation status and the like, and can smoothly support the user's operation.

JP 2007-87023 A JP 2012-14533 A JP 2010-55280 A JP 2006-179990 A

  However, the above-described conventional technology has a problem that information on the screen of the client terminal cannot be efficiently stored and cannot be efficiently supported.

  For example, in the prior art, since the screen information while the user operates the client terminal is stored in the server, the data capacity of the screen information increases when the user operates the client terminal for a long time. End up. Further, when the screen information has a large capacity, referring to all the screen information may put a burden on the supporter.

  An object of one aspect is to provide a server, a storage determination program, and a storage determination method capable of efficiently storing information on a screen of a client terminal and efficiently supporting the information.

  In the first plan, the server includes a detection unit, a compression unit, a determination unit, and a storage determination unit. The detection unit detects an update area of the distribution screen. When the detection unit detects a new update area, the determination unit separates the new update area into the first update area and the second update area based on the occurrence frequency of the past update area. The compression unit compresses the first update area in which the update frequency is equal to or higher than a predetermined threshold based on the occurrence frequency of the past update area, and compresses the second update area in which the update frequency is lower than the predetermined threshold. To do. When the new update area includes both the first update area and the second update area, the storage determination unit determines whether the second update area is based on the size of the first update area and the size of the second update area. Whether or not to save is determined.

  According to one embodiment of the present invention, the information on the screen of the client terminal can be efficiently stored and can be efficiently supported.

FIG. 1 is a diagram illustrating the configuration of the thin client system according to the first embodiment. FIG. 2 is a functional block diagram illustrating the configuration of the client terminal according to the first embodiment. FIG. 3 is a functional block diagram illustrating the configuration of the server according to the first embodiment. FIG. 4 is a diagram illustrating an example of the data structure of the screen information. FIG. 5 is a diagram illustrating an example of the data structure of the non-storage area information. FIG. 6 is a diagram for supplementing the description of the process for determining the first update region. FIG. 7 is a diagram illustrating an example of the first update area and the second update area. FIG. 8 is a diagram for explaining the processing of the storage determination unit. FIG. 9 is a diagram illustrating an example of restoration processing of a client terminal. FIG. 10 is a flowchart (1) illustrating a processing procedure when the server according to the first embodiment generates screen information. FIG. 11 is a flowchart (2) illustrating a processing procedure when the server according to the first embodiment generates screen information. FIG. 12 is a flowchart illustrating the processing procedure of the update area processing according to the first embodiment. FIG. 13 is a flowchart of a process procedure performed when the server according to the first embodiment receives control information from the client terminal. FIG. 14 is a functional block diagram illustrating the configuration of the server according to the second embodiment. FIG. 15 is a flowchart illustrating the processing procedure of the update area processing according to the second embodiment. FIG. 16 is a functional block diagram illustrating the configuration of the server according to the third embodiment. FIG. 17 is a flowchart illustrating the processing procedure of the update area processing according to the third embodiment. FIG. 18 is a functional block diagram illustrating the configuration of the server according to the fourth embodiment. FIG. 19 is a flowchart illustrating a processing procedure when the server according to the fourth embodiment generates screen information. FIG. 20 is a functional block diagram illustrating the configuration of the server according to the fifth embodiment. FIG. 21 is a flowchart illustrating a processing procedure when the server according to the fifth embodiment receives control information from the client terminal. FIG. 22 is a diagram illustrating an example of a computer that executes a storage determination program.

  Hereinafter, embodiments of a server, a storage determination program, and a storage determination method disclosed in the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  FIG. 1 is a diagram illustrating the configuration of the thin client system according to the first embodiment. As shown in FIG. 1, this thin client system includes client terminals 50 a and 50 b and a server 100. The client terminals 50a and 50b and the server 100 are connected to each other via the network 40. Here, as an example, the client terminals 50a and 50b and the server 100 are shown, but the thin client system may include other client terminals and other servers 100.

  For example, the client terminal 50a is a terminal device used by a user who receives support from a supporter. The client terminal 50b is a terminal device used by a supporter. For example, the user uses the client terminal 50 a to access the server 100 and operate application software managed by the server 100. The server 100 receives access from the client terminal 50a, executes processing corresponding to the user's operation command, and notifies the client terminal 50a of screen information. As described above, in the thin client system, the client terminal 50a and the server 100 cooperate to allow the user to operate the application software even when the application software is not installed in the client terminal 50a.

  In addition, the supporter operates the client terminal 50b to access the server 100, and refers to information on the screen operated by the user. In the following description, the client terminals 50a and 50b are collectively referred to as the client terminal 50 as appropriate.

  As an example, the client terminal 50 and the server 100 according to the first embodiment store screen information in the first storage mode or the second storage mode when storing information on the screen operated by the user. The user of the client terminal 50 selects whether to save the screen information in the first saving mode or to save the screen information in the second saving mode, and notifies the server 100 of it.

  An example of processing of the server 100 when screen information is stored in the first storage mode will be described. The server 100 detects the update area based on the screen operated by the user. The server 100 compresses and saves an update area whose update frequency is equal to or higher than a predetermined threshold based on the past occurrence frequency of the update area. The server 100 compresses and saves an update area whose update frequency is less than a predetermined threshold based on the occurrence frequency of the past update area.

  An example of processing of the server 100 when screen information is stored in the second storage mode will be described. The server 100 detects the update area based on the screen operated by the user. Based on the frequency of occurrence of past update areas, the server 100 compresses an update area in which the update frequency is equal to or higher than a predetermined threshold, and compresses an update area in which the update frequency is lower than the predetermined threshold. In the following description, a screen area whose update frequency is equal to or higher than a predetermined threshold is referred to as a first update area as appropriate. A screen area in which the update frequency is less than a predetermined threshold is referred to as a second update area.

  Further, when a new update area is detected, the server 100 separates the new update area into the first update area and the second update area based on the occurrence frequency of the past update area. When the new update area includes both the first update area and the second update area, the server 100 determines the size of the first update area included in the new update area and the second update area included in the new update area. Whether to save the second update area is determined based on the size of the update area. The server 100 stores the second update area based on the determination result.

  Next, an example of the configuration of the client terminal 50 illustrated in FIG. 1 will be described. FIG. 2 is a functional block diagram illustrating the configuration of the client terminal according to the first embodiment. As illustrated in FIG. 2, the client terminal 50 includes a communication unit 51, an input unit 52, a display unit 53, a storage unit 54, and a control unit 55.

  The communication unit 51 is a processing unit that performs data communication with the server 100 and other devices via the network 40. For example, the communication unit 51 corresponds to a communication device or the like. A control unit 55 described later exchanges information with the server 100 via the communication unit 51.

  The input unit 52 is an input device that inputs various data to the client terminal 50. For example, the input unit 52 corresponds to a keyboard, a mouse, a touch panel, or the like.

  The display unit 53 is a display device that displays various data output from the control unit 55. For example, the display unit 53 corresponds to a liquid crystal display, a touch panel, or the like.

  The storage unit 54 has screen information 54a. The storage unit 54 corresponds to a storage device such as a semiconductor memory element such as a random access memory (RAM), a read only memory (ROM), and a flash memory.

  The screen information 54 a is screen information received from the server 100. For example, the screen information 54a includes screen information obtained by compressing a moving image and screen information obtained by expanding information on a screen compressed by a still image.

  The control unit 55 includes a screen reception unit 55a, a request unit 55b, and an expansion unit 55c. The control unit 55 corresponds to an integrated device such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 55 corresponds to an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit).

  The screen receiving unit 55 a is a processing unit that receives screen information from the server 100. The screen receiving unit 55a stores the received screen information in the storage unit 54 as screen information 54a.

  The request unit 55b is a processing unit that notifies the server 100 of save mode information indicating whether to save screen information in the first save mode or to save screen information in the second save mode. For example, the user operates the input unit 52 to input save mode information.

  In addition, the request unit 55b notifies the server 100 of the specified time information. For example, when the user obtains information about the screen to be used when requesting assistance from the supporter from the server 100, the user operates the input unit 52 with designated time information indicating when the screen information is to be obtained. And enter. The request unit 55b notifies the server 100 of this designated time information.

  The decompressing unit 55c is a processing unit that decompresses moving image compressed screen information and still image compressed screen information included in the image information 54a. The decompression unit 55 c outputs the decompressed screen information to the display unit 53 and causes the display unit 53 to display a screen image corresponding to the screen information.

  Next, an example of the configuration of the server 100 illustrated in FIG. 1 will be described. FIG. 3 is a functional block diagram illustrating the configuration of the server according to the first embodiment. As illustrated in FIG. 3, the server 100 includes a communication unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150.

  The communication unit 110 is a processing unit that performs data communication with the client terminal 50 and other devices via the network 40. For example, the communication unit 110 corresponds to a communication device or the like. A control unit 150 described later exchanges information with the client terminal 50 via the communication unit 110.

  The input unit 120 is an input device that inputs various data to the server 100. For example, the input unit 120 corresponds to a keyboard, a mouse, a touch panel, or the like.

  The display unit 130 is a display device that displays various data output from the control unit 150. For example, the display unit 130 corresponds to a liquid crystal display, a touch panel, or the like.

  The storage unit 140 includes history information 141, storage mode information 142, designated time information 143, screen information 144, and non-storage area information 145. The storage unit 140 corresponds to a storage device such as a semiconductor memory element such as a RAM, a ROM, or a flash memory.

  The history information 141 is history information on a screen displayed on the client terminal 50 when the user operates the application software of the server 100 using the client terminal 50. The history information 141 is a moving image in which a plurality of screen frames are connected at each time. The server 100 may temporarily store the history information 141 and delete the history information 141 after a predetermined time has elapsed.

  The storage mode information 142 is information indicating whether the server 100 operates in the first storage mode or the second storage mode. The storage mode information 142 is notified from the client terminal 50.

  The specified time information 143 is information notified from the client terminal 50 described above. The response unit 154 described later transmits the moving image compressed data and the still image compressed data of the screen information 144 to the client terminal 50 based on the designated time information.

  The screen information 144 is information on a screen generated by a first storage control unit 152 or a second storage control unit 153 described later. FIG. 4 is a diagram illustrating an example of the data structure of the screen information. As shown in FIG. 4, the screen information 144 associates time, stream ID, data name, length, codec, and coordinates.

  The time is information indicating the time when the moving image compressed information or the still image compressed information on the screen is stored, and is represented by the elapsed time from the reference time. For example, the reference time corresponds to the time when access from the client terminal 50 is accepted and the thin client service is started. Note that the reference time may be set to another time.

  The stream ID is information for uniquely identifying information that has been compressed. The data name is a name given to information compressed with moving images or information compressed with still images. The length is information indicating the data capacity of information compressed with moving images or information compressed with still images. The codec is information indicating the type of compression method. For example, when the codec is “JPEG (Joint Photographic Experts Group)”, it indicates that the area of the screen has been compressed. When the codec is “MPEG4 (Moving Picture Experts Group)”, it indicates that the screen area is compressed.

  The coordinates are information indicating the coordinates of an area on the screen that has been compressed with a moving image or a still image. For example, the coordinates are represented by lower left coordinates and upper right coordinates of the region. The compressed data is moving image compressed information or still image compressed information corresponding to coordinates. In the following description, the information compressed by moving image is appropriately expressed as moving image compressed data, and the information compressed by still image is expressed by still image compressed data.

  In the record in the first line in FIG. 4, information of still image compressed data compressed with the codec “JPEG” is stored at time “0 ms”. The data name of the still image compressed data is “0xaaaaaa” and the length is “12214 bytes”. In the still image compressed area, the lower left coordinates are (0, 0), and the upper right coordinates are (1280, 1024).

  In the record in the second row in FIG. 4, the information of the moving image compressed data compressed by the codec “MPEG4” is stored at the time “1266 ms”. The stream ID of the moving image compressed data is “1”, the data name is “0xbbbbbb”, and the length is “4289 bytes”. In the moving image compressed area, the lower left coordinates are (360, 250), and the upper right coordinates are (560, 550). In the case of moving image compressed data, the start time and end time of the moving image may be registered as time information.

  Returning to the description of FIG. The non-saved area information 145 is information indicating the coordinates of the area that is not saved for each screen. In the following description, an area that is not saved on the screen is appropriately referred to as a non-saved area. FIG. 5 is a diagram illustrating an example of the data structure of the non-storage area information. As shown in FIG. 5, this non-storage area information 145 associates time with coordinates. For example, the time is information indicating the time when the non-storage area appears on the screen, and is represented by, for example, an elapsed time from the reference time. The coordinates indicate coordinates on the screen of the non-saved area. For example, the coordinates are represented by lower left coordinates and upper right coordinates of the region.

  Returning to the description of FIG. The control unit 150 includes a screen control unit 151, a first storage control unit 152, a second storage control unit 153, and a response unit 154. The control unit 150 corresponds to, for example, an integrated device such as an ASIC or FPGA. The control unit 150 corresponds to an electronic circuit such as a CPU or MPU.

  The screen control unit 151 is a processing unit that provides a thin client environment to the client terminal 50. For example, the screen control unit 151 acquires an operation command from the client terminal 50 and executes application software based on the acquired operation command. The screen control unit 151 generates a screen as a result of executing the application software, notifies the client screen 50 of the generated screen, displays the generated screen, and registers the generated screen information in the history information 141.

  When the storage mode information is received from the client terminal 50, the screen control unit 151 updates the storage mode information 142 with the received storage mode information. Further, when the designated time information is received from the client terminal 50, the screen control unit 151 updates the designated time information 143 with the received designated time information.

  The screen control unit 151 receives the storage mode information from the client terminal 50, and when the storage mode information is changed from the first storage mode to the second storage mode, the current full screen is compressed as a still image. To do. The screen control unit 151 registers the current time and information obtained by compressing the entire screen as still images in the screen information 144. The description regarding the screen information 144 will be described later.

  The first storage control unit 152 refers to the storage mode information 142 and operates when the storage mode information 142 includes information indicating that screen information is stored in the first storage mode. The first storage control unit 152 refers to the history information 141, compares adjacent screens in time series, and detects an update region. The first storage control unit 152 calculates the update frequency of each update area, compresses the first update area where the update frequency is equal to or greater than the threshold, and registers the compressed information in the screen information 144. The first storage control unit 152 performs still image compression on the second update area where the update frequency is less than the threshold value, and registers the still image compressed information in the screen information 144.

  The first storage control unit 152 registers the time-compressed information in the screen information 144 in association with the time, stream ID, data name, length, codec “MPEG4”, coordinates, and video compression data. The first storage control unit 152 registers time information, data name, length, codec “JPEG”, coordinates, and still image compressed data in the image information 144 in association with the still image compressed information.

  Here, an example of processing in which the first storage control unit 152 determines the first update area will be described. FIG. 6 is a diagram for supplementing the description of the process for determining the first update region. As illustrated in FIG. 6, the first storage control unit 152 divides the screen 10a into a plurality of meshes.

  The 1st preservation | save control part 152 collects the information of each mesh updated on the screen 10a. For example, the first storage control unit 152 compares the temporally changing screens and determines whether or not each mesh has been updated. The 1st preservation | save control part 152 specifies the mesh from which the update frequency in fixed time becomes more than a threshold value among the updated meshes. For example, the 1st preservation | save control part 152 specifies the mesh 12 and each mesh contained in the area | region 11 as a mesh whose update frequency is higher than a threshold value on the screen 10a.

  The 1st preservation | save control part 152 specifies the mesh from which update frequency becomes more than a threshold value as a 1st update area | region. If the meshes whose update frequency is equal to or greater than the threshold value are adjacent to each other, the first storage control unit 152 synthesizes the adjacent meshes. For example, the 1st preservation | save control part 152 synthesize | combines each mesh contained in the area | region 11 of the screen 10a, and makes it the mesh 13 shown on the screen 10b. In the example illustrated in FIG. 6, the first storage control unit 152 specifies the mesh 11 and the mesh 13 as the first update region.

  Next, an example of processing in which the first storage control unit 152 determines the second update area will be described. The first storage control unit 152 divides the screen into a plurality of meshes in the same manner as the screen 10a in FIG. The 1st preservation | save control part 152 collects the information of each mesh updated on a screen. For example, the first storage control unit 152 compares the temporally changing screens and determines whether or not each mesh has been updated. The 1st preservation | save control part 152 specifies the mesh from which the update frequency in a fixed time becomes less than a threshold value among the updated meshes. Of the updated meshes, meshes other than the first update region become the second update region.

  The 1st preservation | save control part 152 specifies the mesh from which update frequency is less than a threshold value as a 2nd update area | region. If the meshes whose update frequency is less than the threshold value are adjacent to each other, the first storage control unit 152 synthesizes the adjacent meshes.

  Returning to the description of FIG. The second storage control unit 153 refers to the storage mode information 142 and operates when the storage mode information 142 includes information indicating that screen information is stored in the second storage mode. The second storage control unit 153 refers to the history information 141, compares adjacent screens in time series, and detects an update region. The second storage control unit 153 calculates the update frequency of each update region, compresses the update region where the update frequency is equal to or higher than a predetermined threshold, and compresses the update region where the update frequency is lower than the predetermined threshold, still image compression .

  When the second storage control unit 153 detects a new update area, the second storage control unit 153 separates the new update area into the first update area and the second update area based on the history information 141. When the new update area includes both the first update area and the second update area, the second storage control unit 153 sets the size of the first update area included in the new update area and the new update area. Whether to save the second update area is determined based on the size of the second update area included. Based on the determination result, the second storage control unit 153 compresses the second update area as a still image and registers it in the screen information 144.

  The second storage control unit 153 will be specifically described. The second storage control unit 153 includes a detection unit 153a, a compression unit 153b, a determination unit 153c, and a storage determination unit 153d.

  The detection unit 153a is a processing unit that refers to the history information 141 and detects an update area for each image. For example, the detection unit 153a divides the screen into a plurality of meshes in the same manner as the first storage control unit 152. The detection unit 153a compares the meshes on the screens that move back and forth in time, and determines for each mesh whether or not the image has been updated. The detection unit 153a detects a mesh area in which the image is updated as an update area. The detection unit 153a outputs information on the update area of each screen to the compression unit 153b.

  Based on the update frequency of the update area, the compression unit 153b compresses the first update area where the update frequency is equal to or higher than a predetermined threshold, and compresses the second update area where the update frequency is lower than the predetermined threshold. It is a processing unit. The process of specifying the first update area and the process of specifying the second update area by the compression unit 153b is the same as that of the first storage control unit 152.

  For example, the compression unit 153b compresses the first update area in a moving image in accordance with the MPEG4 standard. The compression unit 153b compresses the still image in the second update area in accordance with the JPEG standard.

  The compression unit 153b outputs information on the compression result of the moving image and information on the compression result of the still image to the determination unit 153c. The video compression result information includes the video start time and end time, video area coordinates, and video compression data. The still image compression result information includes still image time, still image area coordinates, and still image compression data.

  The determination unit 153c is a processing unit that separates a new update area into a first update area and a second update area. The determination unit 153c outputs the determination result, the information on the compression result of the moving image, and the information on the compression result of the still image to the storage determination unit 153d.

An example of processing of the determination unit 153c will be described. FIG. 7 is a diagram illustrating an example of the first update area and the second update area. For example, the screen 20a of FIG. 7 is a screen image of time _{t 1, the} screen 20b is a screen image of time _{t 2.} In the screen 20a, an area 21 is an area determined as an area with a high update frequency based on the occurrence frequency of past update areas. In Figure 20b, the area 22, the newly generated update area at time _{t 2.} For example, in the new update area 22, the determination unit 153c determines, for example, an area 30a that overlaps an area with a high update frequency as the first update area. Further, in the new update area 22, an area 30b that does not overlap with an area with a high update frequency is determined as the second update area. Therefore, the determination unit 153c outputs a determination result indicating that both the first update region 30a and the second update region 30b are included in the new update region to the storage determination unit 153d.

  When both the first update area and the second update area exist in the new update area, the storage determination unit 153d determines the second update area based on the size of the first update area and the size of the second update area. It is a processing unit that determines whether or not to save an update area.

  In the example illustrated in FIG. 7, the area 30 a is the first update area, and the area 30 b is the second update area. In a new update area, the storage determination unit 153d determines that the second update area is a non-storage area when the relationship between the area of the first update area and the area of the second update area satisfies the expression (1). As specified. The threshold value B shown in Expression (1) is a threshold value set in advance by the administrator.

  Area of second update region <area of first update region × threshold value B (1)

  The storage determination unit 153d registers the information on the non-storage area in the non-storage area information 145. For example, the storage determination unit 153d registers the time of the screen where the non-storage area exists and the coordinates of the non-storage area in association with the non-storage area information 145.

  The storage determination unit 153d registers the information about the first update area and the information about the second update area other than the non-storage area in the screen information 144. The storage determination unit 153d registers the time information, the stream ID, the data name, the length, the codec “MPEG4”, the coordinates, and the moving image compressed data in the screen information 144 in association with the information of the first update area. The storage determination unit 153d registers time information, a data name, a length, a codec, coordinates, and still image compressed data in the screen information 144 in association with the information in the second update region other than the non-storage region.

  Next, for the screen at a certain time t, the storage determination unit 153d determines that the relationship between the area of the update region, the area of the non-storage region, and the total screen area of the screen at time t satisfies the expression (2). In this case, a screen in which the non-storage area is filled with a predetermined color is generated for the screen at time t. The threshold A shown in Expression (2) is a threshold set in advance by the administrator. For example, the storage determination unit 153d may fill the non-storage area with any color and image as long as the information amount is smaller than the information amount of the original image included in the non-storage area.

  Sum of the area of the update area and the area of the non-storage area> total screen area × threshold A (2)

  A supplementary description will be given of the update region of Expression (2) using the screen 20b of FIG. The update areas are area 22 and area 23. The non-storage area corresponds to the non-storage area up to the previous frame before the non-storage area is detected from the new update.

  FIG. 8 is a diagram for explaining the processing of the storage determination unit. In the screen 40a of FIG. 8, an area 41 is an update area. Areas 42, 43, 44, 45, and 46 are non-storage areas. When the relationship between the area of the update region 41 and the areas of the non-storage regions 42 to 46 satisfies the condition of Expression (2), the storage determination unit 153d sets the non-storage regions 42 to 46 in a predetermined color. A filled screen 40b is generated. For example, the storage determination unit 153d generates the screen 40b by blackening the non-storage areas 42 to 46.

  The storage determination unit 153d compresses the still image of the entire screen including the blackened area of the non-storage area. The still image compressed data obtained by compressing the still image of the entire screen is defined as a key frame. The storage determination unit 153d registers the key frame information in the screen information 144. For example, the save determination unit 153d registers the time frame information, the stream ID, the data name, the length, the codec “MPEG4”, the coordinates, and the moving image compression data in the screen information 144 in association with the key frame information.

  Returning to the description of FIG. The response unit 154 is a processing unit that transmits moving image compressed data and still image compressed data corresponding to the designated time information 143 to the client terminal 50. For example, the designated time of the designated time information is T1. The response unit 154 compares the time of the screen information 144 with the designated time T1, and specifies the latest key frame time T2 before the designated time T1.

  The response unit 154 detects moving image compressed data and still image compressed data from time T2 to time T1 from the screen information 144, and transmits the detected moving image compressed data and still image compressed data to the client terminal 50. The client terminal 50 uses the key frame at time T2 and the moving image compression data and still image compression data between time T2 and time T1 to restore the screen excluding the non-storage area at time T1. Can do.

  FIG. 9 is a diagram illustrating an example of restoration processing of a client terminal. For example, a key frame at time T2 is set as a key frame 45a, and moving image compressed data and still image compressed data from time T2 to time T1 are set as compressed data 45b. The client terminal 50 can restore the screen 45c at the instruction time T1 by superimposing the compressed data 45b on the key frame 45a. In addition, in the screen 45c, the non-storage areas 46a and 46b are painted black, but the black painted portion is often information on a screen that is not effective for support, and there is no problem when the supporter performs support.

  Subsequently, the response unit 154 detects moving image compressed data and still image compressed data after time T <b> 1 from the screen information 144, and transmits the detected moving image compressed data and still image compressed data to the client terminal 50.

  Next, a processing procedure of the server 100 according to the first embodiment will be described. 10 and 11 are flowcharts illustrating a processing procedure when the server according to the first embodiment generates screen information. As illustrated in FIG. 10, the control unit 150 of the server 100 acquires screen information (step S101) and detects an update area (step S102).

  The control unit 150 determines whether or not the storage mode is the second storage mode (step S103). When the storage mode is not the second storage mode (No at Step S103), the control unit 150 proceeds to Step S108 in FIG.

  On the other hand, when the storage mode is the second storage mode (step S103, Yes), the control unit 150 determines whether or not the condition of Expression (2) is satisfied (step S104). When the condition of Expression (2) is not satisfied (No at Step S104), the control unit 150 proceeds to Step S108 in FIG.

  On the other hand, when the condition of Expression (2) is satisfied (Yes in step S104), the control unit 150 paints a non-storage area of the full screen image and generates a key frame by compressing the full screen image. (Step S105). The control unit 150 stores the key frame information in association with the time (step S106), sets the stored flag to 1 (step S107), and proceeds to step S108 in FIG.

  The description shifts to the description of FIG. The control unit 150 separates the update area into a first update area and a second update area (step S108). The control unit 150 executes update area processing (step S109). The control unit 150 compresses the first update area as a moving image, compresses the second update area as a still image (step S110), and transmits the moving image compressed data and the still image compressed data to the client terminal 50 (step S111).

  The control unit 150 determines whether or not the saved flag is 1 (step S112). When the saved flag is 1 (step S112, Yes), the control unit 150 ends the process. If the saved flag is not 1 (No at Step S112), the control unit 150 proceeds to Step S113.

  The control unit 150 selects an unselected separated update area and determines whether or not the update area has been registered in the non-storage area (step S113). If the update area has already been registered in the non-storage area (step S113, Yes), the control unit 150 proceeds to step S116.

  When the update area has not been registered in the non-storage area (No at Step S113), the control unit 150 stores the moving image compressed data or the still image compressed data in association with the time (Step S114). The control unit 150 deletes the area where the moving image compressed data or the still image compressed data is saved from the non-saved area (step S115). The control unit 150 determines whether all the update areas have been selected (step S116).

  When all the update areas have not been selected (No at Step S116), the control unit 150 proceeds to Step S113. If all the update areas have been selected (step S116, Yes), the control unit 150 ends the process.

  Next, the processing procedure of the update area process shown in step S109 of FIG. 11 will be described. FIG. 12 is a flowchart illustrating the processing procedure of the update area processing according to the first embodiment. As illustrated in FIG. 12, the control unit 150 selects a new change area, and calculates the area of the first update area and the area of the second update area in the new update area (step S201).

  The control unit 150 determines whether or not the condition of Expression (1) is satisfied (step S202). When the condition of Expression (1) is not satisfied (No at Step S202), the control unit 150 proceeds to Step S204.

  On the other hand, when the condition of Expression (1) is satisfied (Yes in Step S202), the control unit 150 registers the second update area in the non-storage area (Step S203).

  The control unit 150 determines whether all update areas have been selected (step S204). If all the update areas have been selected (step S204, Yes), the control unit 150 ends the update area process. On the other hand, when all the update regions are not selected (step S204, No), the control unit 150 selects a new update region that has not been selected, and determines the areas of the first update region and the second update region. Is calculated (step S205), and the process proceeds to step S202.

  Next, a processing procedure when the server 100 according to the first embodiment receives a control command from the client terminal 50 will be described. FIG. 13 is a flowchart of a process procedure performed when the server according to the first embodiment receives control information from the client terminal.

  As illustrated in FIG. 13, the control unit 150 of the server 100 receives control information from the client terminal 50 (step S301). The control unit 150 determines whether or not the control information is storage mode information (step S302). If the control information is not storage mode information (No at Step S302), the control unit 150 determines whether the control information is designated time information (Step S303).

  When the control information is not the specified time information (No at Step S303), the control unit 150 ends the process. On the other hand, when the control information is designated time information (Yes at Step S303), the control unit 150 determines whether or not the storage mode is the second storage mode (Step S304).

  If the storage mode is not the second storage mode (No at Step S304), the control unit 150 ends the process. When the storage mode is the second storage mode (Yes at Step S304), the control unit 150 acquires the designated time T1 (Step S305). The control unit 150 acquires the latest time T2 of the key frame before the designated time T1 (step S306).

  The control unit 150 transmits the moving image compressed data and the still image compressed data between the time T2 and the specified time T1 to the client terminal 50 (step S307). The control unit 150 sequentially transmits the moving image compressed data and the still image compressed data after the designated time T1 to the client terminal 50 (step S308).

  By the way, when the control information is the storage mode information in step S302 (step S302, Yes), the control unit 150 updates the storage mode information (step S309). The controller 150 determines whether or not the storage mode is the second storage mode (step S310).

  If the storage mode is not the second storage mode (No at Step S310), the control unit 150 ends the process. On the other hand, when the storage mode is the second storage mode (step S310, Yes), the control unit 150 generates a key frame by compressing the full-screen image (step S311) and corresponds to the time. Then, the key frame information is stored (step S312).

  Next, effects of the server 100 according to the first embodiment will be described. The server 100 detects the update area of the screen, and based on the update frequency of the update area, compresses the first update area in which the update frequency is equal to or greater than a predetermined threshold, and the update frequency is less than the predetermined threshold. 2. Compress still image in the update area. In addition, when both the first update area and the second update area exist in the new update area, the server 100 determines whether the second update area depends on the size of the first update area and the size of the second update area. Whether or not to save is determined. As a result, it is possible to efficiently save the information on the screen of the client terminal 50 and efficiently support it.

  For example, the server 100 determines not to save the second update area when the second update area is smaller than the first update area. For example, the information in the second update area smaller than the first update area included in the new update area is often not effective information when the supporter supports. For example, when a figure is drawn by CAD (Computer Aided Design) software or the like, the moving area of the mouse temporarily out of the operation range becomes the second update area. Therefore, the server 100 prevents storing information that is unlikely to be used for support by not storing the second update area according to the comparison result between the second update area and the first update area. be able to.

  When the sum of the areas of the update area and the second update area determined not to be stored occupies a predetermined ratio or more of the area of the entire screen, the server 100 determines the second update area determined not to be stored as a predetermined value. Save the keyframe filled with the color. As a result, it is possible to generate a screen at a time designated by a designated time with a key frame having a minimum necessary data amount.

  The server 100 registers the data obtained by compressing the first update area with moving images, the data obtained by compressing the second update area with still images, and the data of the entire screen painted with a predetermined color in the screen information 144 in association with each time. To do. For this reason, the information on the screen of the client terminal 50 can be efficiently saved.

  Here, an advantage of the present embodiment compared with general video distribution will be described. In general video distribution technology, each frame is compared for each pixel to identify the difference portion, and the video is distributed by a predetermined video compression process, but in order to perform the comparison process for all the pixels of each frame, The load on the server is large. On the other hand, in the technology premised on the present invention, since it is only necessary to compress a moving image or a still image based on the update frequency based on the user's operation based on the update frequency, each frame is processed for each pixel. The processing load can be reduced as compared with general video distribution technology. Furthermore, in the present embodiment, as described above, the server resources can be efficiently used by appropriately not storing the second update area.

  Next, a thin client system according to the second embodiment will be described. In the client system according to the second embodiment, when the area of the second update area is equal to or less than a predetermined area for the second update area, the second update area is not stored, so that the amount of data to be stored is reduced. Reduce further.

  In the thin client system according to the second embodiment, the client terminal is the same as the client terminal 50 according to the first embodiment, and thus the description thereof is omitted.

  An example of the configuration of the server according to the second embodiment will be described. FIG. 14 is a functional block diagram illustrating the configuration of the server according to the second embodiment. As illustrated in FIG. 14, the server 200 includes a communication unit 210, an input unit 220, a display unit 230, a storage unit 240, and a control unit 250.

  The description regarding the communication unit 210, the input unit 220, and the display unit 230 is the same as the description regarding the communication unit 110, the input unit 120, and the display unit 130 illustrated in FIG.

  The storage unit 240 includes history information 241, storage mode information 242, designated time information 243, screen information 244, and non-storage area information 245. The storage unit 240 corresponds to a storage device such as a semiconductor memory element such as a RAM, a ROM, or a flash memory.

  The history information 241, the storage mode information 242, the specified time information 243, the screen information 244, and the non-storage area information 245 will be described with respect to the history information 141, the storage mode information 142, the specified time information 143, the screen information 144 shown in FIG. This is the same as the description regarding the non-storage area information 145.

  The control unit 250 includes a screen control unit 251, a first storage control unit 252, a second storage control unit 253, and a response unit 254. The control unit 250 corresponds to, for example, an integrated device such as an ASIC or FPGA. Moreover, the control part 250 respond | corresponds to electronic circuits, such as CPU and MPU, for example.

  The description regarding the screen control unit 251, the first storage control unit 252, and the response unit 254 is the same as the description regarding the screen control unit 151, the first storage control unit 152, and the response unit 154 shown in FIG.

  The second storage control unit 253 includes a detection unit 253a, a compression unit 253b, a determination unit 253c, and a storage determination unit 253d.

  The description regarding the detection unit 253a, the compression unit 253b, and the determination unit 253c is the same as the description regarding the detection unit 153a, the compression unit 153b, and the determination unit 153c illustrated in FIG.

  When both the first update area and the second update area exist in the new update area, the storage determination unit 253d performs the second update according to the size of the first update area and the size of the second update area. It is a processing unit that determines whether or not to save an area.

  Furthermore, when the area of the second update region is less than a predetermined threshold for the second update region, the storage determination unit 253d registers information on the second update region in the non-storage region information 245. For example, the storage determination unit 253d specifies the second update area as a non-storage area when the condition of Expression (3) is satisfied.

  Area of second update region <threshold value C (3)

  The storage determination unit 253d registers information on the non-storage area in the non-storage area information 245. For example, the storage determination unit 253d registers the time of the screen where the non-storage area exists and the coordinates of the non-storage area in association with each other in the non-storage area information 245. Other processing relating to the storage determination unit 253d is the same as that of the storage determination unit 153d illustrated in FIG.

  Next, a processing procedure of the server 200 according to the second embodiment will be described. In the processing procedure of the server 200 according to the second embodiment, the update area process shown in step S110 of FIG. FIG. 15 is a flowchart illustrating the processing procedure of the update area processing according to the second embodiment.

  As illustrated in FIG. 15, the control unit 250 of the server 200 selects a new update region, and calculates the area of the first update region and the area of the second update region in the new update region (step S401). . The control unit 250 determines whether or not the area of the first update region is 0 in the update region (step S402).

  When the area of the first update region is not 0 (No at Step S402), the control unit 250 determines whether or not the condition of Expression (1) is satisfied (Step S403). When the condition of Expression (1) is not satisfied (No at Step S403), the control unit 250 proceeds to Step S405.

  On the other hand, when satisfy | filling the conditions of Formula (1) (step S403, Yes), the control part 250 registers a 2nd update area | region in a non-storage area | region (step S404). If the second update area does not overlap with the first update area in step S404, the second update area is registered in the non-storage area.

  The control unit 250 determines whether or not all update areas have been selected (step S405). When all the update areas have been selected (step S405, Yes), the control unit 250 ends the update area process. On the other hand, if all the update regions are not selected (step S405, No), the control unit 250 selects an unselected update region and calculates the area of the first update region and the area of the second update region. (Step S406), the process proceeds to step S402.

  Meanwhile, in step S402, when the area of the first update region is 0 (step S402, Yes), the control unit 250 determines whether or not the condition of the expression (3) is satisfied (step S407). If the condition of Expression (3) is satisfied (step S407, Yes), the control unit 250 proceeds to step S404. On the other hand, the control part 250 transfers to step S405, when the conditions of Formula (3) are not satisfy | filled (step S407, No).

  Next, effects of the server 200 according to the second embodiment will be described. When the area of the second update region is equal to or less than the threshold value C, the server 200 can further reduce the amount of data to be stored by not storing the second update region.

  Next, a thin client system according to the third embodiment will be described. In the thin client system according to the third embodiment, even in the first update area, when the area of the first update area is equal to or smaller than a predetermined area, the data amount to be saved is not saved by saving the first update area. Is further reduced.

  In the thin client system according to the third embodiment, the client terminal is the same as the client terminal 50 according to the first embodiment, and thus the description thereof is omitted.

  An example of the configuration of the server according to the third embodiment will be described. FIG. 16 is a functional block diagram illustrating the configuration of the server according to the third embodiment. As illustrated in FIG. 16, the server 300 includes a communication unit 310, an input unit 320, a display unit 330, a storage unit 340, and a control unit 350.

  The description regarding the communication unit 310, the input unit 320, and the display unit 330 is the same as the description regarding the communication unit 110, the input unit 120, and the display unit 130 illustrated in FIG.

  The storage unit 340 includes history information 341, storage mode information 342, designated time information 343, screen information 344, and non-storage area information 345. The storage unit 340 corresponds to a storage device such as a semiconductor memory element such as a RAM, a ROM, or a flash memory.

  The history information 341, the storage mode information 342, the specified time information 343, the screen information 344, and the non-storage area information 345 are described with respect to the history information 141, the storage mode information 142, the specified time information 143, the screen information 144 shown in FIG. This is the same as the description regarding the non-storage area information 145.

  The control unit 350 includes a screen control unit 351, a first storage control unit 352, a second storage control unit 353, and a response unit 354. The control unit 350 corresponds to an integrated device such as an ASIC or FPGA, for example. The control unit 350 corresponds to an electronic circuit such as a CPU or MPU.

  The descriptions about the screen control unit 351, the first storage control unit 352, and the response unit 354 are the same as the descriptions about the screen control unit 151, the first storage control unit 152, and the response unit 154 shown in FIG.

  The second storage control unit 253 includes a detection unit 353a, a compression unit 353b, a determination unit 353c, and a storage determination unit 353d. The description regarding the detection unit 353a, the compression unit 353b, and the determination unit 353c is the same as the description regarding the detection unit 153a, the compression unit 153b, and the determination unit 153c illustrated in FIG.

  When both the first update area and the second update area exist in the new update area, the storage determination unit 353d performs the second update according to the size of the first update area and the size of the second update area. It is a processing unit that determines whether or not to save an area.

  Furthermore, the storage determination unit 353d also registers information on the first update region in the non-storage region 345 when the area of the first update region is less than a predetermined threshold for the first update region. For example, the storage determination unit 353d specifies the first update area as a non-storage area when the condition of Expression (4) is satisfied.

  Area of first update region <threshold value D (4)

  The storage determination unit 353d registers information on the non-storage area in the non-storage area information 345. For example, the storage determination unit 353d registers the time of the screen where the non-storage area exists and the coordinates of the non-storage area in association with the non-storage area information 345. Other processes relating to the storage determination unit 353d are the same as those of the storage determination unit 153d illustrated in FIG.

  Next, a processing procedure of the server 300 according to the third embodiment will be described. In the processing procedure of the server 300 according to the third embodiment, the update area process shown in step S110 of FIG. FIG. 17 is a flowchart illustrating the processing procedure of the update area processing according to the third embodiment.

  As illustrated in FIG. 17, the control unit 350 of the server 300 selects a new update region, and calculates the area of the first update region and the area of the second update region in the new update region (step S501). . The control unit 350 determines whether or not the condition of Expression (4) is satisfied (step S502).

  If the condition of Expression (4) is satisfied (step S502, Yes), the control unit 350 registers the first update area as a non-storage area (step S503), and proceeds to step S506.

  On the other hand, when the condition of Expression (4) is not satisfied (No at Step S502), the control unit 350 determines whether or not the condition of Expression (1) is satisfied (Step S504). If the condition of Expression (1) is satisfied (step S504, Yes), the control unit 350 registers the second update area as a non-storage area (step S505), and proceeds to step S506. When the condition of Expression (1) is not satisfied (No at Step S504), the control unit 350 proceeds to Step S506.

  The control unit 350 determines whether or not all update areas have been selected (step S506). When all the update areas are selected (step S506, Yes), the control unit 350 ends the update area process.

  When all the update regions are not selected (step S506, No), the control unit 350 selects an unselected update region and calculates the area of the first update region and the area of the second update region ( Step S507) and the process proceeds to Step S502.

  Next, effects of the server 300 according to the third embodiment will be described. The server 300 can further reduce the amount of data to be stored by not storing the first update area when the area of the first update area is equal to or less than the threshold D area for the first update area. .

  Next, a thin client system according to the fourth embodiment will be described. In the thin client system according to the fourth embodiment, when the relationship among the area of the non-storage area, the area of the update area, and the total screen area satisfies the condition of Expression (2), a screen in which the non-storage area is blackened is displayed. Generate, compress still image, and save. Furthermore, when the relationship between the area of the non-storage area, the area of the update area, and the total screen area satisfies the condition of formula (5), the thin client system does not paint the non-storage area black. Is compressed and saved. Here, the threshold value E is a value larger than the threshold value A. As a result, it is possible to prevent keyframes that have only been painted black and to refresh the keyframes.

  Sum of the area of the update area and the area of the non-storage area> total screen area × threshold E (5)

  In the thin client system according to the fourth embodiment, the client terminal is the same as the client terminal 50 according to the first embodiment, and thus the description thereof is omitted.

  An example of the configuration of the server according to the fourth embodiment will be described. FIG. 18 is a functional block diagram illustrating the configuration of the server according to the fourth embodiment. As illustrated in FIG. 18, the server 400 includes a communication unit 410, an input unit 420, a display unit 430, a storage unit 440, and a control unit 450.

  The description regarding the communication unit 410, the input unit 420, and the display unit 430 is the same as the description regarding the communication unit 110, the input unit 120, and the display unit 130 illustrated in FIG.

  The storage unit 440 includes history information 441, storage mode information 442, designated time information 443, screen information 444, and non-storage area information 445. The storage unit 440 corresponds to a storage device such as a semiconductor memory element such as a RAM, a ROM, or a flash memory.

  The history information 441, the storage mode information 442, the specified time information 443, the screen information 444, and the non-storage area information 445 are described with respect to the history information 141, the storage mode information 142, the specified time information 143, the screen information 144 shown in FIG. This is the same as the description regarding the non-storage area information 145.

  The control unit 450 includes a screen control unit 451, a first storage control unit 452, a second storage control unit 453, and a response unit 454. The control unit 450 corresponds to, for example, an integrated device such as an ASIC or FPGA. The control unit 450 corresponds to an electronic circuit such as a CPU or MPU.

  The descriptions about the screen control unit 451, the first storage control unit 452, and the response unit 454 are the same as the descriptions about the screen control unit 151, the first storage control unit 152, and the response unit 154 shown in FIG.

  The second storage control unit 453 includes a detection unit 453a, a compression unit 453b, a determination unit 453c, and a storage determination unit 453d. The description regarding the detection unit 453a, the compression unit 453b, and the determination unit 453c is the same as the description regarding the detection unit 153a, the compression unit 153b, and the determination unit 153c illustrated in FIG.

  When both the first update area and the second update area exist in the new update area, the storage determination unit 453d performs the second update according to the size of the first update area and the size of the second update area. It is a processing unit that determines whether or not to save an area.

  When the relationship between the area of the update region, the area of the non-save region, and the total screen area of the screen at time t satisfies the condition of Expression (2) for the screen at a certain time t, the storage determination unit 453d For the screen at time t, a screen in which the non-storage area is filled with a predetermined color is generated. The storage determination unit 453d generates a key frame by compressing the entire screen including the area where the non-storage area is blacked out. The storage determining unit 453d registers the key frame information in the screen information 444.

  Note that the save determination unit 453d determines that the relationship between the area of the update area, the area of the non-save area, and the total screen area of the screen at time t satisfies the condition of Expression (5) for a screen at a certain time t. In addition, a non-preserved area is not painted black, and the entire frame at time t is compressed as a still image to generate a key frame. The storage determining unit 453d registers the key frame information in the screen information 444.

  Other processes relating to the storage determination unit 453d are the same as those of the storage determination unit 153d illustrated in FIG.

  Next, a processing procedure of the server 400 according to the fourth embodiment will be described. FIG. 19 is a flowchart illustrating a processing procedure when the server according to the fourth embodiment generates screen information. Note that the processing after Steps S603, 604, and 608 in FIG. 19 is the same as the processing after Step S108 shown in FIG. As shown in FIG. 19, the control unit 450 of the server 400 acquires screen information (step S601) and detects an update area (step S602).

  The controller 450 determines whether or not the storage mode is the second storage mode (step S603). When the storage mode is not the second storage mode (step S603, No), the control unit 450 proceeds to step S108 in FIG.

  On the other hand, when the storage mode is the second storage mode (Yes at Step S603), the control unit 450 determines whether or not the condition of Expression (2) is satisfied (Step S604). If the condition of Expression (2) is not satisfied (step S604, No), the control unit 450 proceeds to step S108 in FIG.

  On the other hand, when the condition of Expression (2) is satisfied (Step S604, Yes), the control unit 450 determines whether or not the condition of Expression (5) is satisfied (Step S605). When the condition of Expression (5) is satisfied (step S605, Yes), the controller 450 compresses the full-screen image as a still image and generates a key frame (step S609). The controller 450 deletes all the information in the non-storage area and resets it (step S610). The control unit 450 stores the key frame information in association with the time (step S607), sets the stored flag to 1 (step S608), and proceeds to step S108 in FIG.

  On the other hand, if the condition of Expression (5) is not satisfied (No in step S605), the control unit 450 paints a non-storage area of the full-screen image and generates a key frame by compressing the full-screen image still. (Step S606), the process proceeds to Step S607.

  Next, effects of the server 400 according to the fourth embodiment will be described. When the relationship between the area of the non-storage area, the area of the update area, and the total screen area satisfies the condition of Expression (5), the server 400 displays the screen as a still image without painting the non-storage area black. Compress and save. For this reason, it is possible to prevent the keyframes that have only been painted black from being saved and refresh the keyframes.

  Next, a thin client system according to the fifth embodiment will be described. In the thin client system according to the fifth embodiment, when the designated time information is received, the latest key frame before the designated time is specified. The client system sequentially transmits only the key frames after this key frame from the identified key frame. As a result, the screen information can be fast-forwarded, and the supporter can refer only to the main points of the screen information.

  In the thin client system according to the fifth embodiment, the client terminal is the same as the client terminal 50 according to the first embodiment, and thus the description thereof is omitted.

  An example of the configuration of the server according to the fifth embodiment will be described. FIG. 20 is a functional block diagram illustrating the configuration of the server according to the fifth embodiment. As illustrated in FIG. 20, the server 500 includes a communication unit 510, an input unit 520, a display unit 530, a storage unit 540, and a control unit 550.

  The description regarding the communication unit 510, the input unit 520, and the display unit 530 is the same as the description regarding the communication unit 110, the input unit 120, and the display unit 130 illustrated in FIG.

  The storage unit 540 includes history information 541, storage mode information 542, designated time information 543, screen information 544, and non-storage area information 545. The storage unit 540 corresponds to a storage device such as a semiconductor memory element such as a RAM, a ROM, or a flash memory.

  Regarding the history information 541, the storage mode information 542, the specified time information 543, the screen information 544, and the non-storage area information 545, the history information 141, the storage mode information 142, the specified time information 143, the screen information 144 shown in FIG. This is the same as the description regarding the non-storage area information 145.

  The control unit 550 includes a screen control unit 551, a first storage control unit 552, a second storage control unit 553, and a response unit 554. The control unit 550 corresponds to, for example, an integrated device such as an ASIC or FPGA. Moreover, the control part 550 respond | corresponds to electronic circuits, such as CPU and MPU, for example.

  The description regarding the screen control unit 551 and the first storage control unit 552 is the same as the description regarding the screen control unit 251 and the first storage control unit 152 illustrated in FIG. 3.

  The second storage control unit 553 includes a detection unit 553a, a compression unit 553b, a determination unit 553c, and a storage determination unit 553d. The description regarding the detection unit 553a, the compression unit 553b, the determination unit 553c, and the storage determination unit 553d is the same as the description regarding the detection unit 153a, the compression unit 153b, the determination unit 153c, and the storage determination unit 153d illustrated in FIG.

  The response unit 554 is a processing unit that transmits a key frame corresponding to the specified time information 543 to the client terminal 50. For example, the designated time of the designated time information is T1. The response unit 554 compares the time of the screen information 544 with the designated time T1, and specifies the latest key frame time T2 before the designated time T1.

  The response unit 554 detects a key frame after the time T <b> 1 from the screen information 544 and transmits the detected key frame to the client terminal 50.

  Next, a processing procedure of the server 500 according to the fifth embodiment will be described. FIG. 21 is a flowchart illustrating a processing procedure when the server according to the fifth embodiment receives control information from the client terminal.

  As shown in FIG. 21, the control unit 550 of the server 500 receives control information from the client terminal 50 (step S701). The control unit 550 determines whether or not the control information is storage mode information (step S702). If the control information is not storage mode information (No in step S702), the control unit 550 determines whether the control information is designated time information (step S703).

  If the control information is not the specified time information (step S703, No), the control unit 550 ends the process. On the other hand, when the control information is designated time information (step S703, Yes), the control unit 550 determines whether or not the storage mode is the second storage mode (step S704).

  When the storage mode is not the second storage mode (step S704, No), the control unit 550 ends the process. On the other hand, when the storage mode is the second storage mode (step S704, Yes), the control unit 550 acquires the specified time T1 (step S705). The control unit 550 obtains the latest key frame time T2 before the designated time T1 (step S706).

  The control unit 550 transmits the key frame at the designated time T2 to the client terminal 50 (step S707). The client terminal 50 sequentially transmits to the client terminal 50 key frames (still image compressed data whose area is equal to the entire screen area) after the specified time T1 (step S708).

By the way, when the control information is storage mode information in Step S702 (Yes in Step S702), the control unit 550 updates the storage mode information (Step S7).
09). The control unit 550 determines whether or not the storage mode is the second storage mode (step S710).

  When the storage mode is not the second storage mode (No at Step S710), the control unit 550 ends the process. On the other hand, when the storage mode is the second storage mode (step S710, Yes), the control unit 550 generates a key frame by compressing the full-screen image (step S711) and corresponds to the time. Then, the key frame information is stored (step S712).

  Next, effects of the server 500 according to the fifth embodiment will be described. When the server 500 receives the specified time information from the client terminal 50, the server 500 specifies the latest key frame before the specified time. Then, the server 500 sequentially transmits the key frames after this key frame to the client terminal 50 from the identified key frames. As a result, the screen information can be fast-forwarded, and the supporter can refer only to the main points of the screen information.

  Next, an example of a computer that executes an image processing program that implements the same functions as those of the image processing apparatuses 100 to 500 described in the above embodiment will be described. FIG. 22 is a diagram illustrating an example of a computer that executes a storage determination program.

  As illustrated in FIG. 22, the computer 600 includes a CPU 601 that executes various arithmetic processes, an input device 602 that receives input of data from a user, and a display 603. The computer 600 includes a reading device 604 that reads a program and the like from a storage medium, and an interface device 605 that exchanges data with another computer via a network. The computer 600 also includes a RAM 606 that temporarily stores various types of information and a hard disk device 607. The devices 601 to 607 are connected to the bus 608.

  The hard disk device 607 includes a detection program 607a, a compression program 607b, a determination program 607c, and a storage determination program 607d. The CPU 601 reads out the detection program 607a, the compression program 607b, the determination program 607c, and the storage determination program 607d and expands them in the RAM 606. The detection program 607a functions as a detection process 606a. The compression program 607b functions as a compression process 606b. The determination program 607c functions as a determination process 606c. The storage determination program 607d functions as a storage determination process 606d.

  For example, the detection process 606a corresponds to the detection units 153a, 253a, 353a, 453a, and 553a. The compression process 606b corresponds to the compression units 153b, 253b, 353b, 453b, and 553b. The determination program 606c corresponds to the determination units 153c, 253c, 353c, 453c, and 553c. The storage determination process 606d corresponds to the storage determination units 153d, 253d, 353d, 453d, and 553d.

  Note that the detection program 607a, the compression program 607b, the determination program 607c, and the storage determination program 607d are not necessarily stored in the hard disk device 607 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, or an IC card inserted into the computer 600. Then, the computer 600 may read and execute each of the programs 607a to 607d.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary note 1) a detection unit for detecting an update area of the distribution screen;
A compression unit that compresses a first update area in which the update frequency is equal to or higher than a predetermined threshold based on the occurrence frequency of the past update area, and compresses a second update area in which the update frequency is less than the predetermined threshold; ,
When a new update area is detected in the detection unit, the new update area is defined as a first update area and a second update area in the new update area based on the occurrence frequency of the past update area. A determination unit that separates into
When both the first update area and the second update area are included in the new update area, the size of the first update area included in the new update area and the second update included in the new update area And a storage determination unit that determines whether or not to store the second update area based on the size of the area.

(Additional remark 2) The said storage determination part determines not to preserve | save the said 2nd update area, when the 2nd update area contained in the said new update area is smaller than the said 1st update area, It is characterized by the above-mentioned. The server according to attachment 1.

(Additional remark 3) The said preservation | save determination part does not preserve | save, when the sum of the area of the said new update area | region and the 2nd update area | region determined not to preserve | save occupies more than the predetermined ratio of the area of a delivery screen 3. The server according to appendix 2, wherein the distribution screen data in which the second update area determined to be filled with an image having an information amount smaller than the information amount of the original image is stored.

(Additional remark 4) The said preservation | save determination part is based on the data which carried out the moving image compression of the said 1st update area | region, the data which carried out the still image compression of the said 2nd update area | region, and the image of information amount smaller than the information amount of the said original image. The server according to appendix 3, wherein the filled distribution screen data is stored in a storage device in association with each time.

(Additional remark 5) It further has a communication part which receives a transmission request from a client terminal, and when the said communication part receives a transmission request from a client terminal, among the data of the delivery screen memorize | stored in the said memory | storage device, the said transmission A response for transmitting the latest distribution screen data before the specified time included in the request to the client terminal and transmitting the compressed video data and the still image compressed data corresponding to the time after the latest time to the client terminal The server according to appendix 4, further comprising: a section.

(Supplementary note 6) The server according to supplementary note 1, wherein the storage determination unit determines that the second update region is not stored when the area of the second update region is smaller than a predetermined area.

(Supplementary note 7) The server according to supplementary note 1, wherein the storage determination unit determines that the first update region is not stored when the area of the first update region is smaller than a predetermined area.

(Additional remark 8) The said storage determination part is 2nd in which the sum of the area of the said 2nd update area determined not to preserve | save and the area of the said new update area is still larger than the 1st ratio of the area of a delivery screen. 4. The server according to appendix 3, wherein the data on which the distribution screen is compressed as a still image is stored when the ratio is larger than the ratio.

(Additional remark 9) It further has a communication part which receives a transmission request from a client terminal, and when the said communication part receives a transmission request from a client terminal, among the data of the delivery screen memorize | stored in the said memory | storage device, the said transmission It further includes a response unit that transmits the data of the delivery screen at the latest time before the specified time included in the request to the client terminal, and transmits the still image compressed data corresponding to the time after the latest time to the client terminal. The server according to appendix 4, characterized by:

(Appendix 10)
Detect the update area of the distribution screen,
Based on the occurrence frequency of the past update area, the first update area where the update frequency is equal to or higher than a predetermined threshold is compressed into a moving image, the second update area where the update frequency is lower than the predetermined threshold is compressed as a still image,
In the detection process, when a new update area is detected, the new update area is changed to the first update area and the second update area in the new update area based on the occurrence frequency of the past update area. Separated into areas,
When both the first update area and the second update area are included in the new update area, the size of the first update area included in the new update area and the second update included in the new update area A storage determination program for executing a process of determining whether or not to store the second update area based on the size of the area.

(Supplementary Note 11) The process of determining whether or not to save the second update area is performed when the second update area included in the new update area is smaller than the first update area. The storage determination program according to appendix 10, wherein it is determined that the data is not stored.

(Additional remark 12) The process which determines whether the said 2nd update area | region is preserve | saved is the predetermined | prescribed area | region of a delivery screen as the sum of the area of the said new update area | region and the 2nd update area | region determined not to preserve | save The distribution screen data in which the second update area determined not to be stored is filled with an image having an information amount smaller than the information amount of the original image is stored. The storage judgment program described in 1.

(Additional remark 13) The process which determines whether the said 2nd update area is preserve | saved is the data which carried out the moving image compression of the said 1st update area, the data which compressed the said 2nd update area, and the said original image 13. The storage determination program according to appendix 12, wherein the distribution screen data painted with an image having an information amount smaller than the information amount is stored in the storage device in association with the time.

(Additional remark 14) When the transmission request is received from the client terminal, the distribution screen data at the latest time before the specified time included in the transmission request among the data of the distribution screen stored in the storage device is stored in the client terminal. 14. The storage determination program according to claim 13, further causing the computer to execute processing for transmitting to the client terminal the data compressed with moving image and the data compressed with still image corresponding to the time after the latest time .

(Supplementary Note 15) The process of determining whether or not to save the second update area is to determine not to save the second update area when the area of the second update area is smaller than a predetermined area. The storage determination program according to appendix 10, characterized by:

(Additional remark 16) The process which determines whether the said 2nd update area | region is preserve | saved determines that the said 1st update area | region is not preserve | saved when the area of the said 1st update area | region is smaller than a predetermined area. The storage determination program according to appendix 10, characterized by:

(Supplementary Note 17) In the process of determining whether or not to save the second update area, the sum of the area of the second update area and the area of the new update area determined not to save is the area of the distribution screen 13. The storage determination program according to appendix 12, wherein the distribution screen is stored as still image compressed data when the second ratio is larger than the second ratio.

(Supplementary Note 18) When a transmission request is received from a client terminal, the distribution screen data at the latest time before the specified time included in the transmission request among the data of the distribution screen stored in the storage device is The storage determination program according to appendix 13, further comprising: causing the computer to further execute a process of transmitting the still image compressed data corresponding to the time after the latest time to the client terminal.

(Supplementary note 19) A storage determination method executed by a computer,
Detect the update area of the distribution screen,
Based on the occurrence frequency of the past update area, the first update area where the update frequency is equal to or higher than a predetermined threshold is compressed into a moving image, the second update area where the update frequency is lower than the predetermined threshold is compressed as a still image,
In the detection process, when a new update area is detected, the new update area is changed to the first update area and the second update area in the new update area based on the occurrence frequency of the past update area. Separated into areas,
When both the first update area and the second update area are included in the new update area, the size of the first update area included in the new update area and the second update included in the new update area A storage determination method characterized by executing a process of determining whether or not to store the second update area based on the size of the area.

(Supplementary note 20) The process of determining whether or not to save the second update area is performed when the second update area included in the new update area is smaller than the first update area. 20. The storage determination method according to appendix 19, wherein the storage is determined not to be stored.

(Supplementary Note 21) In the process of determining whether or not to save the second update area, the sum of the areas of the new update area and the second update area determined not to be saved is a predetermined area of the distribution screen. The distribution screen data, in which the second update area determined not to be stored when it occupies at least the above-mentioned ratio, is filled with an image having an information amount smaller than the information amount of the original image, is stored. The preservation | save determination method of description.

(Supplementary Note 22) The process of determining whether or not to save the second update area includes data obtained by compressing the first update area as a moving image, data obtained by compressing the second update area as a still image, and the original image. The storage determination method according to appendix 21, wherein the distribution screen data filled with an image having an information amount smaller than the information amount is stored in the storage device in association with the time.

(Supplementary Note 23) When a transmission request is received from a client terminal, the distribution screen data of the latest time before the specified time included in the transmission request among the distribution screen data stored in the storage device is stored in the client terminal 24. The storage determination method according to claim 22, further comprising: causing the computer to further execute a process of transmitting to the client terminal the data compressed with moving image and the data compressed with still image corresponding to the time after the latest time. .

(Supplementary Note 24) The process of determining whether or not to save the second update area is to determine not to save the second update area when the area of the second update area is smaller than a predetermined area. The storage determination method according to appendix 19, characterized by:

(Supplementary Note 25) The process of determining whether or not to save the second update area is to determine not to save the first update area when the area of the first update area is smaller than a predetermined area. The storage determination method according to appendix 19, characterized by:

(Supplementary Note 26) In the process of determining whether or not to save the second update area, the sum of the area of the second update area and the area of the new update area determined not to save is the area of the distribution screen The storage determination method according to appendix 21, wherein data obtained by still image compression of the distribution screen is stored when the second ratio is larger than the first ratio.

(Supplementary Note 27) When a transmission request is received from a client terminal, the distribution screen data of the latest time before the specified time included in the transmission request among the distribution screen data stored in the storage device is 24. The storage determination method according to appendix 22, further comprising causing the computer to further execute a process of transmitting the still image compressed data corresponding to the time after the latest time to the client terminal.

50 Client terminal 100, 200, 300, 400, 500 Server

Claims (11)

A detection unit for detecting an update area of the distribution screen;
A compression unit that compresses a first update area in which the update frequency is equal to or higher than a predetermined threshold based on the occurrence frequency of the past update area, and compresses a second update area in which the update frequency is less than the predetermined threshold; ,
In the detection unit, when a new update region is detected, a determination unit that separates the new update region into a first update region and a second update region based on the occurrence frequency of the past update region;
When both the first update area and the second update area are included in the new update area, the size of the first update area included in the new update area and the second update included in the new update area And a storage determination unit that determines whether or not to store the second update area based on the size of the area.   The storage determination unit determines not to store the second update area when the second update area included in the new update area is smaller than the first update area. The listed server.   The storage determination unit determines not to store when the sum of the areas of the new update area and the second update area determined not to save occupies a predetermined ratio or more of the area of the distribution screen. 3. The server according to claim 2, wherein the distribution screen data in which the update area is filled with an image having an information amount smaller than the information amount of the original image is stored.   The storage determination unit is a distribution screen in which the first update area is compressed with moving image data, the second update area is compressed with still image compression, and an image having an information amount smaller than the information amount of the original image The server according to claim 3, wherein the data is stored in the storage device in association with each time.   A communication unit that receives a transmission request from the client terminal; and when the communication unit receives a transmission request from the client terminal, the transmission request includes the distribution screen data stored in the storage device. It further includes a response unit that transmits the latest delivery screen data before the specified time to the client terminal, and transmits the compressed video data and the still image compressed data corresponding to the time after the latest time to the client terminal. The server according to claim 4.   The server according to claim 1, wherein the storage determination unit determines that the second update region is not stored when the area of the second update region is smaller than a predetermined area.   The server according to claim 1, wherein the storage determination unit determines that the first update region is not stored when the area of the first update region is smaller than a predetermined area.   The storage determination unit determines that the sum of the area of the second update region determined not to be stored and the area of the new update region is larger than a second ratio that is larger than the first ratio of the area of the distribution screen. 4. The server according to claim 3, wherein when the size is large, data obtained by compressing the still image on the distribution screen is stored.   A communication unit that receives a transmission request from the client terminal; and when the communication unit receives a transmission request from the client terminal, the transmission request includes the distribution screen data stored in the storage device. It further includes a response unit that transmits data of a delivery screen at the latest time before the specified time to the client terminal and transmits still image compressed data corresponding to the time after the latest time to the client terminal. The server according to claim 4. On the computer,
Detect the update area of the distribution screen,
Based on the occurrence frequency of the past update area, the first update area where the update frequency is equal to or higher than a predetermined threshold is compressed into a moving image, the second update area where the update frequency is lower than the predetermined threshold is compressed as a still image,
In the detection process, when a new update area is detected, the new update area is changed to the first update area and the second update area in the new update area based on the occurrence frequency of the past update area. Separated into areas,
When both the first update area and the second update area are included in the new update area, the size of the first update area included in the new update area and the second update included in the new update area A storage determination program for executing a process of determining whether or not to store the second update area based on the size of the area. A storage determination method executed by a computer,
Detect the update area of the distribution screen,
Based on the occurrence frequency of the past update area, the first update area where the update frequency is equal to or higher than a predetermined threshold is compressed into a moving image, the second update area where the update frequency is lower than the predetermined threshold is compressed as a still image,
In the detection process, when a new update area is detected, the new update area is changed to the first update area and the second update area in the new update area based on the occurrence frequency of the past update area. Separated into areas,
When both the first update area and the second update area are included in the new update area, the size of the first update area included in the new update area and the second update included in the new update area A storage determination method characterized by executing a process of determining whether or not to store the second update area based on the size of the area.

Images