JP2011192229A - Server device and information processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a server device that prevents operability and convenience from being reduced even when a frame rate is reduced. <P>SOLUTION: The server device includes: an operation information processing part for receiving operation information from a thin client, and for transmitting the received operation information to an application; a screen information generating part for generating screen information, based on processing of the application by the operation information; an encoding processing part for receiving the generated screen information, to be encoded as a video image; a transmission part for transmitting the encoded screen information to the thin client; and a frame rate changing part for changing the frame rate of the screen information encoded by the encoding processing part. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

    The present invention relates to a server device and an information processing method.

  Thin client technology for operating a remote device via a network has been put into practical use. This is because a server device is installed at a remote location, connected to a thin client device at hand via a network, operation information from an input device connected to the thin client device is transmitted to the server device, and this is operated on the server device. As an input to the application, at the same time, the screen information output by the application operating on the server device is transmitted to the thin client device, and this is output to the output device connected to the thin client device.

  In addition to this, there is a publication of JP 2007-214808 (Patent Document 1). In this publication, in the TV conference system, the use status of the electronic conference function such as whiteboard sharing is determined, and the image capturing process, the transmission / reception process, and the image display process are automatically performed so as to reduce the load on the terminal during drawing. It is described to change.

JP2007-214808

  It is an object to provide a server device that does not impair the operability and convenience of a thin client terminal.

  Therefore, when the server device encodes the screen information as a moving image, input information from the thin client device, information on the status of the application being executed, screen information generated by the application, or encoding processing The frame rate of the moving image to be encoded is controlled by one or a combination of information such as the above information.

  Since the server apparatus controls the frame rate of the moving image to be encoded, the encoding process can be reduced while ensuring operability and convenience.

It is an example of a block diagram of a server device It is a figure which shows the relationship of each process part of the server apparatus 100 in Example 1. FIG. It is a figure which shows the processing flow which the frame rate change part 135 performs with a fixed period. It is an example of a frame rate change table It is an example of an elapsed time judgment table It is an example of the CPU time of the server device occupied by each user's application program and screen information generation and encoding processing when the frame rate is fixed It is an example of the CPU time of the server apparatus occupied by one user's application program and screen information generation and encoding processing when the frame rate is variable It is an example of the CPU time of the server device occupied by each user's application program and screen information generation and encoding processing when the frame rate is variable This is a case where the CPU time of the server device occupied by each user's application program and screen information generation and encoding processing does not exceed the processing capability when the frame rate is variable It is an example of the figure which shows the relationship of each process part of the server apparatus 100 in Example 2. FIG. It is an example of an elapsed time judgment table It is an example of the figure which shows the relationship of each process part of the server apparatus 100 in Example 3. It is an example of the figure which shows the relationship of each process part of the server apparatus 100 in Example 4. It is an example of the block diagram of the server apparatus 100 in case a some user connects to the same server apparatus. It is an example of a block diagram of a thin client device It is an example of a figure at the time of comprising a thin client apparatus with a television apparatus.

In a normal thin client terminal, a still image processed by a terminal server device (hereinafter also referred to as a server device) is transmitted to the thin client terminal by IP (Internet Protocol).
On the other hand, in this embodiment, in order to reduce the amount of screen information that the server device outputs to the thin client device, the server device encodes and outputs the screen information as a moving image instead of a still image, and the thin client device It is possible to decode the moving image and output the decoded screen information to the output device.

  In such a method, when the server apparatus encodes the screen information as a moving image, the processing load of the server apparatus may increase because the calculation amount of the moving image encoding process is large. As a result, when a plurality of thin client devices are connected to the server device, the server device exceeds the amount of encoding processing that can be processed per unit time, for example, the screen information output of the thin client device is delayed, There may be a problem that processing of an application executed on the apparatus is delayed.

  Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is an example of a configuration diagram of a server device. The server apparatus 100 includes a processor 110 and a memory 120, and is connected to a network 141 such as a LAN or a WAN via a communication interface 140.

  The processor 110 executes various programs 130 such as an application program 131, an operation information processing program 132, a screen information generation program 133, an encoding processing program 134, and a frame rate change program 135 stored in the memory 120. Processing, screen information generation, and encoding processing are performed.

Some of these may be integrated or integrated into a processing unit that performs each process, such as the application unit 131, the operation information processing unit 132, the screen information generation unit 133, the encoding processing unit 134, and the frame rate changing unit 135. All can be realized by hardware.
In order to simplify the description, each processing unit realized by executing the various programs 130 by the processor 110 will be described as the subject of each processing. When each processing unit is realized by hardware, each processing unit mainly performs each process.

FIG. 15 is an example of a configuration diagram of a thin client device. The thin client device 142 includes a processor 210 and a memory 220, and is connected to a network 141 such as a LAN or a WAN via a communication interface 240.
The processor 210 executes various programs 230 such as an application program 231, an operation information processing program 232, a screen information processing program 233, and a decryption processing program 234 stored in the memory 220, so that applications, operation information processing, screen information generation, Each process of a decoding process is performed.

A part or all of these are realized by hardware by integrating them as processing units for performing each process such as the application unit 231, the operation information processing unit 232, the screen information processing unit 233, and the decoding processing unit 234. You can also.
The display unit 250 is a display unit that displays a moving image transmitted from the server device. In FIG. 14, the thin client device is configured to include the display unit 250, but the display unit 250 may be an external display unit outside the thin client device.
The input unit 260 receives input from the user. For example, operation information such as mouse operation, keyboard input, and remote control operation is received.

  An outline of the operation of the thin client system will be described. The input unit 260 of the thin client device 142 receives an operation from the user. The operation information processing unit 232 processes the coordinate information and the keyboard input information when the mouse is clicked by the user's mouse operation or keyboard input, and transmits the processed operation information to the server device 100 via the communication interface 240.

FIG. 2 is a diagram showing the relationship between the processing units of the server device 100 when the operation information processing unit 132 controls the frame rate changing unit 135 mainly in the server device 100 of FIG. The arrows connecting the processing units indicate the flow of processing.
The server apparatus 100 receives operation information from the thin client apparatus 142 via the communication interface 140. The received operation information is transmitted to the operation information processing unit 132, and the operation information processing unit 132 inputs the operation information to the application unit 131 as operation information for the server device.

  The application unit 131 transmits drawing information to the screen information generation unit 133 when the screen information of the application is changed due to input of operation information from the operation information processing unit 132 or a temporal change. The drawing information is, for example, information such as writing a character on a screen or writing a rectangle. The screen information generation unit 133 generates screen information for the received drawing information at a constant cycle. The encoding processing unit 134 acquires screen information from the screen information generation unit 133 at a constant cycle, encodes the screen information, and transmits the screen information to the thin client device 142 via the communication interface 140.

  The thin client device 140 receives the encoded screen information and inputs it to the decoding processing unit 234. The decryption processing unit 234 decrypts the screen information and displays it on the display unit 250.

  In this embodiment, the screen information processing is performed by the encoding processing unit 134 of the server device 100 and the decoding processing unit 234 of the thin client device 142. By using the moving image processing method in this manner, the decoding processing unit of the television device can be used as the thin client device 142.

  FIG. 16 is an example of a diagram in which the thin client device 142 is configured by a television device.

  The thin client device 142 includes a tuner 101, a descrambler 102, a demultiplexer 103, a decoder 234, a display unit / speaker 250, an input unit 106, a memory 220, an application unit 231, an operation information processing unit 232, a screen information processing unit 233, a recording Unit 115, encryption / decryption processing unit 112, communication processing unit 104, communication interface 240, and control unit 210.

  The tuner 101 selects a desired channel from a plurality of channels received from the broadcasting station via the antenna 10, and demodulates the digitally modulated program.

  The descrambler 102 de-scrambles the program that is being played in order to be able to receive only the channel contracted with the service provider.

  The demultiplexer 103 extracts audio data and video data from the broadcast program.

  The decoder 234 decodes the compressed and encoded audio data and video data received from the broadcast program and the communication interface 240, and expands them to the original audio and video signals.

  The display unit / speaker 250 reproduces the output signal from the decoder 234. It may be external rather than internal.

  The input unit 106 inputs operation information such as a mouse operation from a user, a remote control operation, an input from a touch panel, and a keyboard input.

  The processes of the application unit 231, the operation information processing unit 232, and the screen information processing unit 233 are the same as those described above.

  The recording unit 115 is a built-in memory that records a broadcast program. In addition, a removable HDD (Hard Disk Drive), an optical disk, a memory card, and a hybrid form combining these may be used.

  The encryption / decryption processing unit 112 encrypts or decrypts the content received via the digital input / output terminal 114 via the broadcast program or the network, using a key shared with the server device 100.

  The communication processing unit 104 transmits / receives moving image information and control commands to / from other server devices connected via a network via the communication interface 240. When transmitting / receiving content, identification codes such as “Copy free”, “Copy one generation”, “No more copies”, and “Copy never” indicating how to handle the content are added.

  The control unit 210 comprehensively controls the operation of each unit in the thin client device 142.

  The frame rate conversion will be described. The operation information processing unit 132 transmits operation information to the frame rate changing unit 135. The frame rate changing unit 135 determines to lower, maintain, or increase the frame rate based on the received operation information. If it is determined to increase or decrease the frame rate, the frame rate changing unit 135 notifies the encoding processing unit 134 of the new frame rate.

  When the encoding processing unit 134 is notified of the frame rate from the frame rate changing unit 135, the encoding processing unit 134 acquires the screen information from the screen information generating unit 133, and is notified of the cycle for encoding the screen information from the frame rate changing unit 135. Change to a different frame rate.

  For example, when the encoding format is the MPEG2 format, one frame is handled as one picture and a plurality of pictures are handled in units of GOP (Group of Pictures), but the GOP configuration is 15 pictures of IBBPBBPBBPBBPBB. When one GOP is 0.5 seconds and the frame rate is changed to 10 fps, the GOP configuration is composed of five pictures of IPPPP and encoded. As a result, 0.5 seconds is expressed by five pictures, so that 10 fps is realized. However, in a GOP configuration such as IBBPBBPBBPBBPBB, when the frame rate is lowered, there is a limit to the realizable frame rate.

  Therefore, if the GOP configuration is such that, for example, IPPPPPPPPP... And P continues for 15 sheets or more, an arbitrary frame rate can be set because an arbitrary P can be omitted and a GOP can be configured. The method for changing the frame rate may be other than the method described above.

  Next, processing in which the frame rate changing unit 135 lowers the frame rate step by step with time will be described. FIG. 3 shows a processing flow periodically performed by the frame rate changing unit 135. The frame rate changing unit 135 stores the reference frame rate as the current frame rate (S501). In this embodiment, the reference frame rate is 30 fps.

  The frame rate changing unit 135 reads the timer counter value built in the processor 110 and stores it as a reference time (S502). This timer counter value may be realized by other hardware instead of the processor 110. Further, the read timer counter value may be converted from a time unit unique to the processor to a second unit. In the present embodiment, all the timer counter values are converted into seconds and handled.

  The frame rate changing unit 135 periodically reads the timer counter value and obtains the elapsed time from the previously stored reference time (S503). The frame rate changing unit 135 incorporates a frame rate changing table 600 shown in FIG. The frame rate change table 600 may be built in the program, or may be stored in another hardware such as a flash ROM. The frame rate changing unit 135 obtains a frame rate corresponding to the elapsed time based on the frame rate changing table 600 (S504) and sets it as the new frame rate.

  The obtained new frame rate is compared with the current frame rate (S505). If the obtained new frame rate is different from the current frame rate, the frame rate changing unit 135 newly holds the new frame rate as the current frame rate (S506). ) And notifies the encoding processing unit 134 of the frame rate (S507). For example, when the elapsed time is 100, the new frame rate is 15 fps. Thereafter, the process returns to S503 again.

  Next, processing in which the frame rate changing unit 135 immediately increases the frame rate will be described. When the frame rate changing unit 135 receives operation information from the operation information processing unit 132, the frame rate changing unit 135 reads the timer counter value and stores it as a reference time. After that, when the new frame rate is calculated by the periodic processing, the elapsed time becomes the elapsed time between the reference time and the timer counter value read in the next cycle after storing the reference time. This corresponds to the column of 0 to 99 seconds in the table 600. Therefore, the new frame rate becomes the reference frame rate. As a result, when any operation information is received, it is possible to immediately return to the reference frame rate.

  On the other hand, as another method, the frame rate changing unit 135 can increase the frame rate step by step using the following method. When the frame rate changing unit 135 receives the operation information from the operation information processing unit 132, the new elapsed time is obtained from the elapsed time determination table 601 shown in FIG. 5 based on the frame rate held by the frame rate changing unit 135. Further, the timer counter value is read, and the sum of the timer counter value and the new elapsed time is stored as a reference time. The correspondence between the new elapsed time and the frame rate in the elapsed time determination table 601 is that the new elapsed time obtained from the frame rate change table 600 when the new elapsed time is selected based on the frame rate held by the frame rate changing unit 135. The new elapsed time in the elapsed time determination table 601 is set so that the new elapsed time becomes such that the frame rate corresponding to the time is larger than the frame rate held by the frame rate changing unit 135.

  After that, when the new frame rate is calculated by the periodic processing, the elapsed time becomes the elapsed time between the reference time and the timer counter value read in the next cycle after storing the reference time. The frame rate obtained from the table 600 is larger than the frame rate held by the frame rate changing unit 135.

  For example, if the frame rate changing unit 135 stores 10 fps, the new elapsed time obtained from the elapsed time determination table 601 is 100. Therefore, if the sum of this and the timer counter value is stored as the reference time, The new frame rate obtained from the frame rate change table 600 by the periodic processing is 15 fps.

  According to this method, the frame rate can be increased step by step when any operation information is received.

  According to the present embodiment, if the time during which the thin client device does not transmit the operation information continues for a certain period of time, the server device 100 can decrease the frame rate step by step. Further, when the thin client device transmits some operation information, the server device 100 can return the frame rate to the reference frame rate immediately or stepwise. Based on the user's operation information, if there is operation information, the frame rate is increased, and if there is no operation information, the frame rate is reduced, so that the operability and convenience of the application are not impaired.

Next, an effect obtained by this embodiment when a plurality of users are simultaneously connected to one server apparatus using a thin client terminal will be described.
FIG. 14 shows an example of the configuration of the server apparatus 100 when a plurality of users connect to the same server apparatus 100 simultaneously using a thin client apparatus. In the server apparatus 100, the processor 110 processes the program 130 at the same time.

  When a plurality of users simultaneously connect to the same server device 100 using a thin client device and operate an application program on the server device 100, the CPU occupation time of the server device 100 is the application program executed by each user. And the total CPU time occupied by screen information generation and encoding processing.

  In the case of a conventional server device, for example, as shown in FIG. 6, the CPU time occupied by each user's application program, screen information generation and encoding processing is 30%, and the connected users are user A and user B. And the user C, the total CPU occupation ratio is always 90%. At this time, if another user D connects to the same server device 100 and tries to operate an application program, the CPU occupation rate reaches 100%, which exceeds the processing capability of the server device 100. This may cause a problem that, for example, the operation of the application program becomes slow.

  Therefore, with the server device 100 according to the present embodiment, it is possible to reduce the frame rate for the application program executed by each user, so that screen information generation and encoding processing can be reduced. For this reason, the CPU occupancy of each user's application program, screen information generation, and encoding process varies with time, so the CPU occupancy of the entire server device 100 is lower than when the frame rate is fixed.

  For example, FIG. 7 shows that the encoding rate is reduced and the CPU occupancy rate is lowered by lowering the frame rate according to this embodiment. The same applies to user B and user C. As shown in FIG. 8, since the CPU occupancy rates of user A, user B, and user C fluctuate with time, their total is lower than the CPU occupancy rate of server device 100 shown in FIG. Therefore, even if the user D is connected to the same server device 100 and operates the application program, the CPU occupation rate does not exceed 100% as shown in FIG.

  In the server device 100 according to the present embodiment, the factor for lowering the frame rate depends on the operation information from the user. However, if the number of users connected to the same server device 100 increases, the timing at which each user inputs the operation information is statistical. Therefore, the CPU occupancy rate of the server apparatus 100 can always be kept low.

  By keeping the CPU occupancy rate of the server device 100 always low, even when a plurality of users are connected to the same server device 100 at the same time, the operation of the application program is hardly delayed. Furthermore, since the CPU occupation rate is always low, the power consumption of the server apparatus 100 can be reduced. Since the CPU occupancy rate is low, the performance of the processor 110 of the server device 100 can be lowered, and the hardware cost of the server device 100 can be reduced.

  Further, with the server device 100 according to the present embodiment, the frame rate of the application program executed by each user can be lowered, so that the amount of code output from the encoding processing unit 134 can be reduced. is there.

  Therefore, since the code amount of the screen information of each user varies with time, the total code amount output by the server apparatus 100 is lower than when the frame rate is fixed. As with the CPU occupancy rate, in the server device 100 according to the present embodiment, the factor for lowering the frame rate depends on the operation information from the user. However, if the number of users connected to the same server device 100 increases, The timing at which information is input is statistically smoothed, and the total amount of codes output by the server apparatus 100 can always be kept low.

  Since the amount of data transmitted / received on the network can be reduced by keeping the total amount of codes output by the server device 100 always low, congestion on the network is less likely to occur. In addition, since the amount of data transmitted and received on the network can be reduced, in the case of a network system that charges according to the amount of data, it is possible to reduce the charging fee.

Hereinafter, embodiments will be described with reference to the drawings. In addition, description is abbreviate | omitted about the part same as Example 1, and a different part is demonstrated in detail.
FIG. 10 is a diagram showing the relationship between the processing units of the server device 100 when the application unit 131 controls the frame rate changing unit 135 mainly in the server device 100 of FIG. The arrows connecting the processing units indicate the flow of processing.

  Since the processing of the operation information processing unit 132, the screen information generation unit 133, and the encoding processing unit 134 is the same as the processing described in the first embodiment, description thereof is omitted. The periodic processing performed by the frame rate changing unit 135 is the same as the processing described in the first embodiment, and thus description thereof is omitted. Since the processing when the frame rate changing unit 135 receives the operation information from the operation information processing unit 132 is the same as the processing described in the first embodiment, the description thereof is omitted.

  Next, processing in which the application unit 131 requests the frame rate changing unit 135 to change the frame rate will be described. The application unit 131 increases the frame rate to the frame rate changing unit 135 or sets the frame rate to the frame rate changing unit 135 when it is desired to explicitly lower the frame rate or explicitly increase the frame rate for the currently executed application program. A request to lower can be sent.

  For example, there is a control such as lowering the frame rate when continuing to display still images and increasing the frame rate when continuing to display moving images. The application program can freely determine whether to increase the frame rate or decrease the frame rate.

  Next, processing in which the frame rate changing unit 135 lowers the frame rate step by step with time will be described. When the frame rate changing unit 135 receives a request to increase the frame rate from the application unit 131, the frame rate changing unit 135 reads the timer counter value and stores it as a reference time. After that, when the new frame rate is calculated by the periodic processing, the elapsed time becomes the difference between the reference time and the timer counter value read in the next cycle after storing the reference time. This corresponds to a column of 0 to 99 seconds of 600. Therefore, the new frame rate becomes the reference frame rate. As a result, when a request to increase the frame rate is received, it is possible to immediately return to the reference frame rate.

  Next, processing in which the frame rate changing unit 135 immediately decreases the frame rate will be described. When the frame rate changing unit 135 receives a request to lower the frame rate from the application unit 131, the frame rate changing unit 135 determines that the elapsed time corresponding to the lowest new frame rate in the frame rate changing table 600 shown in FIG. Save a large value as the reference time.

  After that, when the new frame rate is calculated by the periodic processing, the elapsed time becomes the difference between the reference time and the timer counter value read in the next cycle after storing the reference time. 600 corresponds to a 400 to 499 second column. Therefore, the new frame rate is the lowest frame rate in the frame rate change table 600. Thus, when a request for lowering the frame rate is received, the lowest frame rate in the frame rate change table 600 can be immediately set.

  On the other hand, as another method, the frame rate changing unit 135 can increase or decrease the frame rate step by step using the following method. When the frame rate changing unit 135 receives a request to increase or decrease the frame rate from the application unit 131, the elapsed time determination table shown in FIG. 11 is based on the frame rate held by the frame rate changing unit 135. The reference time is obtained from 602 and the elapsed time determination table 603, and the reference time is stored as the reference time.

  After that, when the new frame rate is calculated by the periodic processing, the elapsed time becomes the difference between the reference time and the timer counter value read in the next cycle after storing the reference time. The frame rate obtained from 600 is larger or smaller than the frame rate held by the frame rate changing unit 135.

  For example, when the frame rate changing unit 135 saves 10 fps, the reference time obtained from the elapsed time determination table 601 is 100. If this is saved as the reference time, the frame rate is changed by the periodic processing. The new frame rate obtained from table 600 is 15 fps.

According to this method, when a request for lowering the frame rate is received, the frame rate can be lowered step by step.
According to the present embodiment, when the application program explicitly sends a request for lowering the frame rate or raising the frame rate, the server apparatus 100 can lower or lower the frame rate stepwise or immediately. .

An effect obtained by increasing or decreasing the frame rate mainly by the application unit will be described.
A request to increase the frame rate when an application program running in the application section requires real-time performance such as agile response and fast screen switching for operability and convenience By transmitting, the frame rate increases in a situation where real-time performance is required, and operability and convenience are not impaired. For example, in an application program that performs moving image reproduction, the moving image is smoothly reproduced by increasing the frame rate when reproduction is started.

  Conversely, when the application program running in the application section does not require real-time performance, it sends a request to lower the frame rate, so that the frame rate does not require real-time performance. And the load on the encoding processing unit can be reduced. For example, in an application program that performs moving image playback, the frame rate is lowered when playback is stopped, so that the frame rate is lowered when the moving image is not played back, thereby reducing the load on the encoding processing unit. it can.

Hereinafter, embodiments will be described with reference to the drawings. In addition, description is abbreviate | omitted about the part same as Example 1, and a different part is demonstrated in detail.
FIG. 12 is a diagram illustrating the relationship between the processing units of the server apparatus 100 when the screen information generation unit 133 controls the frame rate changing unit 135 mainly in the server apparatus 100 of FIG. The arrows connecting the processing units indicate the flow of processing.

  Since the processing of the operation information processing unit 132, the application unit 133, and the encoding processing unit 134 is the same as the processing described in the first embodiment, description thereof is omitted. The periodic processing performed by the frame rate changing unit 135 is the same as the processing described in the first embodiment, and thus description thereof is omitted. Since the processing when the frame rate changing unit 135 receives the operation information from the operation information processing unit 132 is the same as the processing described in the first embodiment, the description thereof is omitted.

Next, a process in which the screen information generation unit 133 requests the frame rate change unit 135 to change the frame rate will be described. The screen information generation unit 133 determines whether to increase or decrease the frame rate, and transmits a request to increase or decrease the frame rate to the frame rate changing unit 135.
There are the following methods for determining whether the screen information generation unit 133 increases or decreases the frame rate. The screen information generation unit 133 compares the screen information generated this time with the screen information generated last time, and when the change in the screen information is small, transmits a request to lower the frame rate. Conversely, the screen information generation unit 133 compares the screen information generated this time with the screen information generated last time, and transmits a request to lower the frame rate if the screen information changes greatly.

  As another method, a request to lower the frame rate is transmitted when the drawing information from the application unit 131 within a certain time is less than the threshold, and a request to increase the frame rate is transmitted when the drawing information is greater than or equal to the threshold. There is a technique. The method by which the screen information generation unit 133 determines to increase or decrease the frame rate may not be based on the above-described method.

The processing when the frame rate changing unit 135 receives a request to increase the frame rate from the screen information generating unit 133 is performed when the frame rate changing unit 135 according to the second embodiment receives a request to increase the frame rate from the screen information generating unit 133. This is the same as the case where the application unit 131 is replaced with the screen information generation unit 133 in the above process, and thus the description thereof is omitted.
According to the present embodiment, when the screen information generation unit 133 transmits a request for decreasing the frame rate or increasing the frame rate, the server apparatus 100 can decrease or decrease the frame rate stepwise or immediately. .

An effect obtained by increasing or decreasing the frame rate mainly by the screen information generation unit will be described.
In the method according to the first embodiment, when the operation information is not input from the thin client terminal, the frame rate is decreased. However, even when there is no operation information, it is not possible to cope with the situation where the frame rate is desired to be increased. For example, in the case of a graphical screen using animation on a screen waiting for input from the user, there are cases where it is desired to increase the frame rate in order for the animation to move smoothly. According to the method according to the third embodiment, the frame rate can be increased or decreased from the magnitude of the change in the screen information even when no operation information is input. This makes it possible to further improve operability and convenience.

Hereinafter, embodiments will be described with reference to the drawings. In addition, description is abbreviate | omitted about the part same as Example 1, and a different part is demonstrated in detail.
FIG. 13 is a diagram showing the relationship between the processing units of the server device 100 when the encoding processing unit 134 mainly controls the frame rate changing unit 135 in the server device 100 of FIG. The arrows connecting the processing units indicate the flow of processing.

  Since the processing of the operation information processing unit 132, the application unit 133, and the screen information generation unit 133 is the same as the processing described in the first embodiment, description thereof is omitted. The periodic processing performed by the frame rate changing unit 135 is the same as the processing described in the first embodiment, and thus description thereof is omitted. Since the processing when the frame rate changing unit 135 receives the operation information from the operation information processing unit 132 is the same as the processing described in the first embodiment, the description thereof is omitted.

  Next, a process in which the encoding processing unit 134 requests the frame rate changing unit 135 to change the frame rate will be described. The encoding processing unit 134 determines whether to increase or decrease the frame rate, and transmits a request to increase or decrease the frame rate to the frame rate changing unit 135.

There are the following methods for determining whether the encoding processing unit 134 increases or decreases the frame rate. The encoding processing unit 134 compares the screen information acquired from the current screen information generation unit 133 with the screen information acquired from the previous screen information generation unit 133, and transmits a request to lower the frame rate when the change in the screen information is small. To do.
Conversely, the screen information generation unit 133 compares the screen information acquired from the screen information generation unit 133 with the screen information acquired from the screen information generation unit 133, and if the screen information changes greatly, a request to reduce the frame rate is issued. Send.

  As another method, for example, when the encoding processing unit 134 is an MPEG2 encoding method, the residual of motion compensation between the current frame and the preceding and succeeding frames in the course of the encoding process is calculated as a discrete cosine. The DCT (Discrete Cosine Transform) coefficient is calculated by conversion. When the calculated DCT coefficient residual is small, it is considered that the change in the screen information is small, and a request for lowering the frame rate is transmitted. Conversely, when the residual is large, the change in the screen information is considered large, and the frame rate Send a request to raise. Any encoding format that calculates a DCT coefficient residual in the encoding process may not be MPEG2. Further, in the process of calculating the residual, a calculation method not based on DCT may be used.

  As another method, for example, when the encoding processing unit 134 is an MPEG2 encoding method, the ratio of the number of intra macroblocks in the screen to the total intra macroblocks in the screen, that is, the intra rate is low. The request for lowering the frame rate is transmitted assuming that the change in the screen information is small. If the intra rate is high, the request for increasing the frame rate is transmitted assuming that the change in the screen information is large. An MPEG2 system is not necessary as long as it is an encoding format in which macroblocks are distinguished and encoded as intra and non-intra in the process of encoding.

The processing when the frame rate changing unit 135 receives a request to increase the frame rate from the encoding processing unit 134 is performed when the frame rate changing unit 135 according to the second embodiment receives a request from the encoding processing unit 134 to increase the frame rate. In this process, the application unit 131 is the same as the case where the encoding processing unit 134 is replaced, and thus the description thereof is omitted.
According to the present embodiment, when the encoding processing unit 134 transmits a request for decreasing the frame rate or increasing the frame rate, the server apparatus 100 can decrease or decrease the frame rate stepwise or immediately. .

An effect obtained by increasing or decreasing the frame rate mainly by the encoding processing unit will be described.
In the method according to the fourth embodiment, similarly to the effect of the third embodiment, even when no operation information is input, the magnitude of the screen change can be determined from the residual of the DCT coefficient and the intra rate, and the frame rate can be increased or decreased. . This makes it possible to further improve operability and convenience.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. In addition, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of a certain embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function can be stored in a recording device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

  Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

DESCRIPTION OF SYMBOLS 100 Server apparatus 110 Processor 120 Memory 130 Program 131 Application part 132 Operation information processing part 133 Screen information generation part 134 Coding process part 135 Frame rate change part 140 Communication interface 141 Network 142 Thin client apparatus 600, 602, 603 Frame rate change table 601 Elapsed time judgment table

Claims (14)

In the server device,
An operation information processing unit that receives operation information from a thin client and transmits the received operation information to an application;
A screen information generation unit that generates screen information based on processing of the application by the operation information;
An encoding processing unit that receives the generated screen information and performs encoding as a moving image;
A transmission unit for transmitting the encoded screen information to the thin client;
A frame rate changing unit that changes a frame rate of the screen information encoded by the encoding processing unit;
The server apparatus characterized by having. The server device according to claim 1,
The frame rate changing unit is configured to frame the screen information encoded by the encoding processing unit when a time during which the operation information processing unit does not receive the operation information from the thin client exceeds a predetermined time. A server apparatus characterized by lowering a rate. In the server apparatus according to claim 1 or 2,
The frame rate changing unit increases the frame rate of the screen information encoded by the encoding processing unit when the operation information processing unit receives the operation information from the thin client. In the server apparatus according to any one of claims 1 to 3,
The screen information generation unit compares the previous screen information with the current screen information, and if the amount of change in the screen information is less than a threshold, the frame rate changing unit is encoded by the encoding processing unit. A server device characterized by lowering a frame rate of the screen information. In the server apparatus according to any one of claims 1 to 4,
The screen information generation unit compares the previous screen information with the current screen information, and if the amount of change in the screen information is greater than or equal to a threshold, the frame rate changing unit is encoded by the encoding processing unit. A server device characterized by increasing the frame rate of the screen information. In the server apparatus according to any one of claims 1 to 5,
The frame rate changing unit lowers the frame rate of the screen information encoded by the encoding processing unit when a drawing request from the application to the screen information generating unit is less than a threshold value for a predetermined time. A server device. In the server apparatus according to any one of claims 1 to 6,
The frame rate changing unit increases the frame rate of the screen information encoded by the encoding processing unit when a drawing request from the application to the screen information generating unit is equal to or greater than a threshold value for a predetermined time. A server device. In the server apparatus according to any one of claims 1 to 7,
When the encoding processing unit compares the screen information received from the previous screen information generation unit with the screen information received by the encoding processing unit from the current screen information generation unit, and the amount of change in the screen information is less than the threshold The frame rate changing unit lowers the frame rate of the screen information encoded by the encoding processing unit. In the server apparatus according to any one of claims 1 to 8,
When the encoding processing unit compares the screen information received from the previous screen information generation unit with the screen information received by the encoding processing unit from the current screen information generation unit, and the amount of change in the screen information is greater than or equal to the threshold value The frame rate changing unit lowers the frame rate of the screen information encoded by the encoding processing unit. In the server apparatus according to any one of claims 1 to 9,
When the residual of the DCT coefficient is less than a threshold value, the frame rate changing unit lowers the frame rate of the screen information encoded by the encoding processing unit. In the server apparatus of any one of Claims 1-10,
When the residual of the DCT coefficient is equal to or greater than a threshold, the frame rate changing unit increases the frame rate of the screen information encoded by the encoding processing unit. In the server apparatus according to any one of claims 1 to 11,
When the ratio of the number of intra macroblocks in the screen to the total intra macroblocks in the screen is less than the threshold, the frame rate changing unit lowers the frame rate of the screen information encoded by the encoding processing unit. The server apparatus characterized by the above-mentioned. In the server apparatus according to any one of claims 1 to 12,
When the ratio of the number of intra macroblocks in the screen to the total intra macroblocks in the screen is equal to or greater than the threshold, the frame rate changing unit increases the frame rate of the screen information encoded by the encoding processing unit. The server apparatus characterized by the above-mentioned. In the information processing method in the server device,
Receives operation information from the thin client,
Send the received operation information to the application,
Generate screen information based on the processing of the application by the operation information,
The generated screen information is received and encoded as a moving image,
Transmitting the encoded screen information to the thin client;
Changing the frame rate of the screen information to be encoded;
The server method characterized by the above-mentioned.

Images