KR102247889B1 - System for cloud streaming service, method of cloud streaming service to save system resources and apparatus for the same - Google Patents

Abstract

Disclosed are a cloud streaming service system, a cloud streaming service method for saving system resources, and an apparatus therefor. Captures the execution screen of the application running for the cloud streaming service, extracts the key frame from the execution screen, generates still image compression information corresponding to the key frame, and other frames and key frames excluding the key frame from the execution screen Encoding a still image by applying a still image compression technique based on still image compression information to one or more of them, and transmitting the still image compression-encoded execution screen to the user's terminal device to provide a still image-based cloud streaming service. You can do it. It is possible to provide a cloud streaming service to more users by saving system resources for generating sub-frame still image compression information.

Description

Cloud streaming service system, cloud streaming service method to save system resources, and device therefor {SYSTEM FOR CLOUD STREAMING SERVICE, METHOD OF CLOUD STREAMING SERVICE TO SAVE SYSTEM RESOURCES AND APPARATUS FOR THE SAME}

The present invention is a cloud streaming service system capable of saving system resources by dividing a similar scene into a key frame and a sub frame in a cloud streaming service, generating compressed still image information corresponding to a key frame, and applying it to a sub frame to encode, It relates to a cloud streaming service method for saving system resources and an apparatus therefor.

Recently, as applications driven in a mobile communication terminal such as a smart phone have been developed in various ways, many virtualization technologies for driving applications that require high performance for execution in a user terminal of relatively low performance have been proposed. Among them, a cloud streaming technology that compresses the screen into video and provides it to the user's terminal by streaming it, and that reproduces the streamed screen in the terminal as if running an application on the terminal is in the spotlight.

In the case of such a cloud streaming-based service, it is necessary to process requests from many users at the same time, and as many applications as many as the number of users connected must be run on the server side. It can be a key part of providing it.

Korean Patent Publication No. 10-2014-0041317, published on April 4, 2014 (Name: Transmission of video using static content information from a video source)

An object of the present invention is to prevent a transmission delay that may occur during still image-based cloud streaming by increasing a compression rate of data by using a suitable still image compression technique when providing a still image-based cloud streaming service.

It is also an object of the present invention to pre-configure still image compression information for an important frame among similar parts of a service scene, and reuse it when encoding a frame similar to an important frame, thereby maximizing system resources required for generating still image compression information. It is to save.

A cloud streaming server according to the present invention for achieving the above object includes: a capture unit for capturing an execution screen of an application executed for a cloud streaming service; A key frame extraction unit for extracting a key frame from the execution screen; Generates still image compression information corresponding to the key frame, and applies a still image compression technique based on the still image compression information to at least one of the key frames and the remaining frames of the execution screen excluding the key frame. An encoding unit that encodes the execution screen as a still image; And a sending unit that transmits the still image-encoded execution screen to the user's terminal device to perform a still image-based cloud streaming service.

In this case, the encoding unit may compare the key frame among the remaining frames to extract a sub-frame whose change value is less than a preset reference value, and compress the sub-frame using the still image compression technique.

In this case, the key frame may be a frame in which a change value of the frame is greater than or equal to the preset reference value compared to a previous frame among the execution screens.

In this case, the still image compression information may include one or more of a pallet, a color replacement table, and an optimal filter.

In this case, the encoding unit may compress the subframe using the still image compression technique based on the still image compression information until a next key frame of the key frame appears.

In this case, the encoding unit may store the key frame and the still image compression information in a database by interlocking with each other.

In this case, the key frame extractor may calculate the change value of the frame in consideration of the change of the pixel value of the execution screen.

In this case, the key frame extracting unit may extract the key frame in consideration of at least one of the input signal of the application and the number of additional colors.

In this case, the encoding unit may encode the still image using the palletized PNG (portable network graphics) still image compression technique using the still image compression information.

In addition, the cloud streaming service method for saving system resources according to the present invention includes: capturing an execution screen of an application executed for the cloud streaming service; Extracting a key frame from the execution screen; Generates still image compression information corresponding to the key frame, and applies a still image compression technique based on the still image compression information to at least one of the key frames and the remaining frames of the execution screen excluding the key frame. Encoding the execution screen as a still image; And transmitting the still image-encoded execution screen to the user's terminal device to perform a still image-based cloud streaming service.

In this case, the encoding step includes extracting a subframe whose change value is less than a preset reference value compared to the key frame among the remaining frames, and compresses the subframe using the still image compression technique. can do.

In this case, the key frame is a frame in which a change value of the frame is greater than or equal to the preset reference value compared to a previous frame among the execution screens, thereby saving system resources.

In this case, the still image compression information may include one or more of a pallet, a color replacement table, and an optimal filter.

In this case, the encoding step may compress the subframe using the still image compression technique based on the still image compression information until a next key frame of the key frame appears.

In this case, the encoding step may save system resources, characterized in that the key frame and the still image compression information are interlocked and stored in a database.

In this case, in the step of extracting the key frame, the change value of the frame may be calculated in consideration of the change of the pixel value of the execution screen.

In this case, in the step of extracting the key frame, the key frame may be extracted in consideration of at least one of the input signal of the application and the number of additional colors.

In this case, in the encoding step, the execution screen may be still image encoded using a palletized PNG (portable network graphics) still image compression technique using the still image compression information.

In addition, as another means for solving the problem of the present invention, a computer program stored in a medium is provided in order to execute the above-described method.

In addition, the cloud streaming service system according to the present invention captures an execution screen of an application executed for a cloud streaming service, extracts a key frame from the execution screen, Encoding the execution screen as a still image by applying a still image compression technique based on still image compression information corresponding to the key frame to at least one of the frames, and transmitting the execution screen encoded with the still image to the user's terminal device. A cloud streaming server that performs a still image-based cloud streaming service; And a terminal receiving an application execution result screen corresponding to the cloud streaming service from the cloud streaming server.

According to the present invention, when a still image-based cloud streaming service is provided, a transmission delay that may occur during still image-based cloud streaming can be prevented by increasing a compression rate of data by using a suitable still image compression technique.

In addition, the present invention pre-configures still image compression information for an important frame among similar parts of a service scene, and reuses it when encoding a frame similar to an important frame, thereby saving system resources required for generating still image compression information as much as possible. I can.

1 is a block diagram showing a cloud streaming service system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating an example of a cloud streaming server shown in FIG. 1.
3 is a diagram illustrating a key frame and a sub frame according to an embodiment of the present invention.
4 is a flowchart illustrating a cloud streaming service method for saving system resources according to an embodiment of the present invention.
5 is a detailed operation flowchart illustrating a cloud streaming service method for saving system resources according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, in the following description and the accompanying drawings, detailed descriptions of known functions or configurations that may obscure the subject matter of the present invention will be omitted. In addition, it should be noted that the same components are denoted by the same reference numerals as much as possible throughout the drawings.

The terms or words used in the present specification and claims described below should not be construed as being limited to their usual or dictionary meanings, and the inventors are appropriate as the concept of terms for describing their own invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention on the basis of the principle that it can be defined. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only the most preferred embodiments of the present invention, and do not represent all of the technical spirit of the present invention. It should be understood that there may be equivalents and variations. In addition, terms such as first and second are used to describe various elements, and are only used for the purpose of distinguishing one element from other elements, and are not used to limit the elements.

1 is a block diagram showing a cloud streaming service system according to an embodiment of the present invention.

Referring to FIG. 1, a cloud streaming service system according to an embodiment of the present invention includes a cloud streaming server 110, terminal devices 120-1, ..., 120-N, and a network 130. .

The cloud streaming server 110 captures an execution screen of an application executed for a cloud streaming service.

In addition, the cloud streaming server 110 extracts a key frame from the execution screen. In this case, the key frame may be a frame in which a change value of the frame is greater than or equal to a preset reference value compared to a previous frame among the execution screens. In this case, the change value of the frame may be calculated in consideration of the change of the pixel value of the execution screen. In this case, the key frame may be extracted in consideration of one or more of the number of additional colors and the input signal of the application.

In addition, the cloud streaming server 110 generates still image compression information corresponding to the key frame, and compresses still images based on still image compression information in at least one of the remaining frames and key frames excluding the key frame among the execution screens. A still image is encoded by applying the technique. In this case, a subframe in which a change value of the frame is less than a preset reference value may be extracted from among the remaining frames, and the subframe may be compressed using a still image compression technique. In this case, the still image compression information may include one or more of a pallet, a color replacement table, and an optimal filter. In this case, the subframe may be compressed using a still image compression technique based on still image compression information until a next key frame of the key frame appears. At this time, the key frame and still image compression information may be linked and stored in the database. In this case, a still image may be encoded using a palletized PNG (portable network graphics) still image compression technique using still image compression information.

In addition, the cloud streaming server 110 transmits the still image-encoded execution screen to the user's terminal devices 120-1, ..., 120-N to perform a still image-based cloud streaming service.

The terminal devices 120-1, ..., 120-N receive an application execution result screen corresponding to the cloud streaming service from the cloud streaming server 110 and provide it to the user.

Terminal devices 120-1, ..., 120-N are devices that are connected to a communication network and can execute applications based on a cloud computing system, and are not limited to mobile communication terminals, and all information and communication devices and multimedia terminals , A wired terminal, a fixed terminal, and an Internet Protocol (IP) terminal. In addition, the terminal devices 120-1, ..., 120-N are mobile phones, portable multimedia played (PMP), mobile internet devices (MID), smart phones, desktops, and tablet computers, respectively. (Tablet PC), notebook (Note book), net book (Net Book), personal digital assistant (PDA), smart TV and information communication devices, such as mobile devices with various mobile communication specifications have.

The network 130 provides a path for transferring data between the cloud streaming server 110 and the terminal devices 120-1, ..., 120-N, and is a network that is used in the past and a network that can be developed in the future. It is a concept that encompasses all. For example, the network 130 is a wired/wireless local area communication network that provides communication of various information devices within a limited area, a mobile communication network that provides communication between mobiles and the mobile and the outside of the mobile, and communicates between the earth station and the earth station using satellites. It may be a satellite communication network, a wired or wireless communication network, or a combination of two or more. Meanwhile, the transmission method standard of the network 130 is not limited to the existing transmission method standard, and may include all transmission method standards to be developed in the future. In addition, a network used between the cloud streaming server 110 and the terminal devices 120-1, ..., 120-N in FIG. 1 is the terminal devices 120-1, ..., 120-N. It may be different from a network used for each other, or may be the same.

FIG. 2 is a block diagram illustrating an example of a cloud streaming server shown in FIG. 1.

Referring to FIG. 2, the cloud streaming server 110 shown in FIG. 1 includes a capture unit 210, a key frame extraction unit 220, an encoding unit 230, a sending unit 240, and a storage unit 250. Includes.

The capture unit 210 captures an execution screen of an application executed for a cloud streaming service. For example, when an application execution request is received by one or more terminal devices connected to the cloud streaming server 110 through a network, the cloud streaming server 110 executes the application in response to the request of the terminal device, An execution screen corresponding to an application execution result may be captured to deliver to the terminal device.

The key frame extraction unit 220 extracts a key frame from the execution screen.

The execution screen may be composed of several frames, and among the several frames, there may be a number of similar frames because the change is not large. At this time, in order to perform a still image-based cloud streaming service, all frames must be compressed with a still image compression method and provided to the user's terminal device, and still image compression information for each of all frames is configured according to the still image compression method. Needs to be. If there are many similar frames, the consumption rate of system resources increases due to composing still image compression information for all frames, and thus a problem may arise in providing a service smoothly. Accordingly, it is possible to save system resources by extracting a key frame from among similar frames, and compressing still image compression information generated based on the key frame by applying it to the remaining similar frames.

In this case, the key frame may be a frame in which a change value of the frame is greater than or equal to a preset reference value compared to a previous frame among the execution screens. For example, when about 60% of the entire area of one frame is changed compared to the previous frame, the corresponding frame may be extracted as a key frame. Such a preset reference value can be freely set and modified by a developer.

In addition, if the reference value is set too low, the number of extracted key frames may increase, and the consumption of system resources may increase. If the reference value is set too high, the number of extracted key frames may be reduced, leading to severe deterioration in quality of the encoded execution screen. Accordingly, in order to prevent such a problem, a maximum reference value and a minimum reference value may be set, and correction may be made within a range.

In this case, the change value of the frame may be calculated in consideration of the change of the pixel value of the execution screen. For example, a change value of a frame may be calculated by calculating a change of all pixels included in a frame, and determining a degree of change by comparing it with a pixel value of a previous frame. If it is assumed that there are 10000 pixels constituting the frame and the preset reference value is 60%, the corresponding frame can be determined and extracted as a key frame when there is a change of 6000 pixels or more compared to the previous frame.

In this case, the key frame may be extracted in consideration of one or more of the number of additional colors and the input signal of the application. For example, since the input signal of the application may be a signal that changes the execution screen of the application, it is possible to detect the input signal of the application and determine the first frame of the execution screen generated by the input signal as a large key frame. . In addition, when the number of additional colors is reduced by more than a preset number compared to the previous frame, the corresponding frame may be determined as a key frame and may be extracted.

The encoding unit 230 generates still image compression information corresponding to the key frame, and applies a still image compression technique based on the still image compression information to at least one of the remaining frames and key frames excluding the key frame among the execution screens. The execution screen is encoded as a still image. For example, after generating still image compression information for one or a plurality of key frames among the frames constituting the execution screen, the remaining frames are still images by applying a still image compression technique using the already generated still image compression information. By performing encoding, it is possible to encode a still image of an execution screen composed of a key frame and the remaining frames.

In this case, a subframe in which a change value of the frame is less than a preset reference value may be extracted from among the remaining frames, and the subframe may be compressed using a still image compression technique. For example, since a plurality of key frames may exist in the execution screen, similar frames corresponding to each key frame may exist among the remaining frames. When compared with each of the key frames as described above, a similar frame with a change value of the frame less than a preset reference value may be extracted as a sub-frame for the key frame and compressed using a still image compression technique in which the key frame is compressed.

In this case, the still image compression information may include one or more of a pallet, a color replacement table, and an optimal filter. For example, when still image-based cloud streaming is performed by applying PNG still image compression technique to perform still image encoding, the compression efficiency is lower than that of video-based cloud streaming, so transmission delay may occur when network bandwidth is limited. There is a problem. Therefore, in order to increase the compression efficiency of the PNG still image compression technique, the width of colors constituting the image can be reduced by using a pallet, a color substitution table, and an optimal filter.

In general, in the PNG format, image color information can be expressed in RGB format and Indexed Color format. The RGB format can use red, green, and blue values for each pixel. However, in the Indexed Color format, colors used to compose an image can be pre-written in a table called a palette, and information on the number of colors in the palette table can be included for each pixel. In this case, the RGB format expresses red, green, and blue in 8 bits each, and 24 bits per pixel is required. However, since only 8 bits are required in the Indexed Color format, when converting the RGB format to the Indexed Color format, the size of the data is 3 It can be reduced to a quarter.

In addition, since the Indexed Color format uses 8 bits to represent information, only a maximum of 256 pieces of information can be expressed. However, since more colors can be expressed in the RGB format than in the Indexed Color format, the RGB format colors can be replaced with the colors included in the Indexed Color format palette using a color substitution table. In this case, since the number of colors that can be used is inevitably reduced compared to when expressed in the RGB format, the image quality may be deteriorated.

In addition, since it is inefficient to treat information indicating which color for each pixel in the Indexed Color format as it is, a structure in which data such as a filter is arranged for each row constituting a pixel can be applied. For example, if one row consists of 5 pixels, and all 5 pixels contain information that says red, the information of the pixel is converted to'red, red, red, red, red'. It can be more concise to indicate with '5 red' than to indicate. Therefore, it is possible to display more concisely when handling data by applying an optimum filter to each row of a pixel.

At this time, the subframe may be compressed using a still image compression technique based on still image compression information until a next key frame of the key frame appears. For example, still image compression information for a first key frame extracted from an execution screen may be configured, and subsequent subframes may be still image encoded using this. At this time, if still image encoding of the subframes for the first key frame is completed and the second key frame is extracted, still image compression information is again configured for the second key frame, and the second key frame is Still image encoding may be performed by applying a still image compression technique using still image compression information of a key frame.

In this case, the key frame and still image compression information may be linked and stored in the database. For example, before performing still image encoding, all key frames included in the execution screen may be extracted in advance, and still image compression information for each key frame may be generated and stored in a database. At this time, by interlocking and storing the key frame and still image compression information, it is possible to easily find and use the stored still image compression information when the key frame and sub-frame are compressed.

In this case, a still image may be encoded using a palletized PNG (portable network graphics) still image compression technique using still image compression information. Compared to the PNG still image compression method, the palletized PNG still image compression method can cause image quality deterioration, but it can reduce the size of the data to one third, resulting in transmission delay in environments with limited network bandwidth. Can be prevented.

The sending unit 240 transmits the still image-encoded execution screen to the user's terminal device to perform a still image-based cloud streaming service. By completing encoding of the still image and transmitting the compressed execution screen to the user's terminal device, the user can feel as if the application is being executed on the user's terminal device.

As described above, the storage unit 250 stores various types of information generated in the process of a cloud streaming service according to an embodiment of the present invention.

According to an embodiment, the storage unit 250 may be configured independently of the cloud streaming server 110 to support a function for a cloud streaming service. In this case, the storage unit 250 may operate as a separate mass storage, and may include a control function for performing the operation.

In addition, the cloud streaming server 110 configured as described above may be implemented as one or more servers.

Meanwhile, the cloud streaming server 110 is equipped with a memory to store information in the device. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in another implementation, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may include, for example, a hard disk device, an optical disk device, or some other mass storage device.

In addition, the storage unit 250 may interlock and store the key frame and still image compression information instead of the database of the present invention.

3 is a diagram illustrating a key frame and a sub frame according to an embodiment of the present invention.

Referring to FIG. 3, a plurality of frames constituting an execution screen of an application may be divided into key frames 310 and 320 and subframes 310-1, 310-2, 320-1, and 320-N.

In this case, the key frame may be a frame in which a change value of the frame is greater than or equal to a preset reference value compared to a previous frame among the execution screens. For example, in the case of the key frame 320, the change value of the frame may be greater than or equal to a preset reference value when compared with the sub-frame 310-2 corresponding to the previous frame.

As such, it is highly possible that the key frames 310 and 320 and the subframes 310-1, 310-2, 320-1, and 320-N corresponding to the key frames 310 and 320 are sequentially arranged. Accordingly, when extracting the sub-frames 310-1, 310-2, 320-1, and 320-N, frames positioned between each key frame 310 and 320 may be extracted.

In addition, when applying the still image compression technique using still image compression information for the key frames 310 and 320, the sub-frame is generated by applying the still image compression technique using the same still image compression information until the next key frame appears. Can be compressed. For example, the sub-frame 310-1 and the sub-frame 310-2 may be compressed by a still image compression technique using still image compression information for the key frame 310. After that, when the next key frame, the key frame 320 appears, the still image compression information for the key frame 310 is not used, and the still image compression method using the still image compression information for the key frame 320 is used. The frame 320-1 can be compressed. Thereafter, in the same manner as described above, compression may be performed using a still image compression technique using still image compression information of the key frame 320 until the next key frame appears.

4 is a flowchart illustrating a cloud streaming service method for saving system resources according to an embodiment of the present invention.

Referring to FIG. 4, the cloud streaming service method for saving system resources according to an embodiment of the present invention captures an execution screen of an application executed for the cloud streaming service (S410). For example, when an application execution request is received by one or more terminal devices connected to the cloud streaming server through a network, the application is executed in the cloud streaming server in response to the request of the terminal device, and the result corresponding to the application execution result. The execution screen can be captured for delivery to the terminal device.

In addition, the cloud streaming service method for saving system resources according to an embodiment of the present invention extracts a key frame from an execution screen (S420).

The execution screen may be composed of several frames, and among the several frames, there may be a number of similar frames because the change is not large. At this time, in order to perform a still image-based cloud streaming service, all frames must be compressed with a still image compression method and provided to the user's terminal device, and still image compression information for each of all frames is configured according to the still image compression method. Needs to be. If there are many similar frames, the consumption rate of system resources increases due to composing still image compression information for all frames, and thus a problem may arise in providing a service smoothly. Accordingly, it is possible to save system resources by extracting a key frame from among similar frames, and compressing still image compression information generated based on the key frame by applying it to the remaining similar frames.

In this case, the key frame may be a frame in which a change value of the frame is greater than or equal to a preset reference value compared to a previous frame among the execution screens. For example, when about 60% of the entire area of one frame is changed compared to the previous frame, the corresponding frame may be extracted as a key frame. Such a preset reference value can be freely set and modified by a developer.

In addition, if the reference value is set too low, the number of extracted key frames may increase, and the consumption of system resources may increase. If the reference value is set too high, the number of extracted key frames may be reduced, leading to severe deterioration in quality of the encoded execution screen. Accordingly, in order to prevent such a problem, a maximum reference value and a minimum reference value may be set, and correction may be made within a range.

In this case, the change value of the frame may be calculated in consideration of the change of the pixel value of the execution screen. For example, a change value of a frame may be calculated by calculating a change of all pixels included in a frame, and determining a degree of change by comparing it with a pixel value of a previous frame. If it is assumed that there are 10000 pixels constituting the frame and the preset reference value is 60%, the corresponding frame can be determined and extracted as a key frame when there is a change of 6000 pixels or more compared to the previous frame.

In this case, the key frame may be extracted in consideration of one or more of the number of additional colors and the input signal of the application. For example, since the input signal of the application may be a signal that changes the execution screen of the application, it is possible to detect the input signal of the application and determine the first frame of the execution screen generated by the input signal as a large key frame. . In addition, when the number of additional colors is reduced by more than a preset number compared to the previous frame, the corresponding frame may be determined as a key frame and may be extracted.

In addition, the cloud streaming service method for saving system resources according to an embodiment of the present invention generates still image compression information corresponding to a key frame, and at least one of the remaining frames excluding the key frame among the execution screens and key frames. The still image is encoded by applying a still image compression technique based on the still image compression information to the still image (S430). For example, after generating still image compression information for one or a plurality of key frames among the frames constituting the execution screen, the remaining frames are still images by applying a still image compression technique using the already generated still image compression information. By performing encoding, it is possible to encode a still image of an execution screen composed of a key frame and the remaining frames.

In this case, a subframe in which a change value of the frame is less than a preset reference value may be extracted from among the remaining frames, and the subframe may be compressed using a still image compression technique. For example, since a plurality of key frames may exist in the execution screen, similar frames corresponding to each key frame may exist among the remaining frames. When compared with each of the key frames as described above, a similar frame with a change value of the frame less than a preset reference value may be extracted as a sub-frame for the key frame and compressed using a still image compression technique in which the key frame is compressed.

In this case, the still image compression information may include one or more of a pallet, a color replacement table, and an optimal filter. For example, when still image-based cloud streaming is performed by applying PNG still image compression technique to perform still image encoding, the compression efficiency is lower than that of video-based cloud streaming, so transmission delay may occur when network bandwidth is limited. There is a problem. Therefore, in order to increase the compression efficiency of the PNG still image compression technique, the width of colors constituting the image can be reduced by using a pallet, a color substitution table, and an optimal filter.

In general, in the PNG format, image color information can be expressed in RGB format and Indexed Color format. The RGB format can use red, green, and blue values for each pixel. However, in the Indexed Color format, colors used to compose an image can be pre-written in a table called a palette, and information on the number of colors in the palette table can be included for each pixel. In this case, the RGB format expresses red, green, and blue in 8 bits each, and 24 bits per pixel is required. However, since only 8 bits are required in the Indexed Color format, when converting the RGB format to the Indexed Color format, the size of the data is 3 It can be reduced to a quarter.

In addition, since the Indexed Color format uses 8 bits to represent information, only a maximum of 256 pieces of information can be expressed. However, since more colors can be expressed in the RGB format than in the Indexed Color format, the RGB format colors can be replaced with the colors included in the Indexed Color format palette using a color substitution table. In this case, since the number of colors that can be used is inevitably reduced compared to when expressed in the RGB format, the image quality may be deteriorated.

In addition, since it is inefficient to treat information indicating which color for each pixel in the Indexed Color format as it is, a structure in which data such as a filter is arranged for each row constituting a pixel can be applied. For example, if one row consists of 5 pixels, and all 5 pixels contain information that says red, the information of the pixel is converted to'red, red, red, red, red'. It can be more concise to indicate with '5 red' than to indicate. Therefore, it is possible to display more concisely when handling data by applying an optimum filter to each row of a pixel.

At this time, the subframe may be compressed using a still image compression technique based on still image compression information until a next key frame of the key frame appears. For example, still image compression information for a first key frame extracted from an execution screen may be configured, and subsequent subframes may be still image encoded using this. At this time, if still image encoding of the subframes for the first key frame is completed and the second key frame is extracted, still image compression information is again configured for the second key frame, and the second key frame is Still image encoding may be performed by applying a still image compression technique using still image compression information of a key frame.

In this case, the key frame and still image compression information may be linked and stored in the database. For example, before performing still image encoding, all key frames included in the execution screen may be extracted in advance, and still image compression information for each key frame may be generated and stored in a database. At this time, by interlocking and storing the key frame and still image compression information, it is possible to easily find and use the stored still image compression information when the key frame and sub-frame are compressed.

In this case, a still image may be encoded using a palletized PNG (portable network graphics) still image compression technique using still image compression information. Compared to the PNG still image compression method, the palletized PNG still image compression method can cause image quality deterioration, but it can reduce the size of the data to one third, resulting in transmission delay in environments with limited network bandwidth. Can be prevented.

In addition, the cloud streaming service method for saving system resources according to an embodiment of the present invention performs a still image-based cloud streaming service by transmitting a still image-encoded execution screen to the user's terminal device (S440). By completing encoding of the still image and transmitting the compressed execution screen to the user's terminal device, the user can feel as if the application is being executed on the user's terminal device.

5 is an operation flowchart illustrating in detail a cloud streaming service method for saving system resources according to an embodiment of the present invention.

5, the cloud streaming service method for saving system resources according to an embodiment of the present invention first captures an execution screen of an application executed for the cloud streaming service (S510).

Thereafter, in order to extract a key frame, a change value of the frame is calculated by comparing the current frame with the previous frame of the frames constituting the execution screen (S520).

After that, it is determined whether the change value of the frame is equal to or greater than a preset reference value (S525).

If the determination result in step S525 is greater than or equal to a preset reference value, the current frame is extracted as a key frame (S530).

If the determination result in step S525 is less than the preset reference value, the current frame is determined as the remaining frames other than the key frame (S540).

After that, still image compression information for the extracted key frame is generated (S550). In this case, the still image compression information may include one or more of a pallet, a color replacement table, and an optimal filter.

Thereafter, a frame change value is calculated by comparing the key frame with the remaining frames (S560).

After that, it is determined whether or not the frame change value of the remaining frames compared with the key frame is less than a preset reference value (S565).

If the determination result in step S565 is less than the preset reference value, the corresponding remaining frame is extracted as a subframe of the key frame (S570).

If the determination result in step S565 is greater than or equal to a preset reference value, it may be determined again whether the corresponding remaining frame is a key frame, and may be classified as one of a key frame and a subframe of another key frame.

After that, a still image is encoded on an execution screen composed of a key frame and a sub frame (S580). For example, encoding of a still image may be performed by compressing a sub-frame using a still image compression technique using still image compression information of a key frame.

Thereafter, the execution screen in which still image encoding is completed is transmitted to the user's terminal device to perform a still image-based cloud streaming service (S590).

The cloud streaming service method for saving system resources according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes magneto-optical media, and any type of hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of the program instructions may include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

As described above, the cloud streaming service system according to the present invention, the cloud streaming service method for saving system resources, and the apparatus therefor are not limited to the configuration and method of the embodiments described as described above, but the The embodiments may be configured by selectively combining all or part of each of the embodiments so that various modifications may be made.

According to the present invention, an execution screen of an application executed for a cloud streaming service is captured, a key frame is extracted from the execution screen, still image compression information corresponding to the key frame is generated, and the remaining frames excluding the key frame from the execution screen Encoding a still image by applying a still image compression technique based on still image compression information to at least one of the fields and key frames, and transmitting the still image-encoded execution screen to the user's terminal device to provide a still image-based cloud. Streaming service can be performed. Furthermore, by saving system resources for generating still image compression information for a subframe of a key frame, the cloud streaming server can provide a service to more users at the same time.

110: cloud streaming server 120-1 to 120-N: terminal devices
130: network 210: capture unit
220: key frame extraction unit 230: encoding unit
240: sending unit 250: storage unit
310, 320: key frame
310-1, 310-2, 320-1, 320-N: sub-frame

Claims (20)

A capture unit for capturing an execution screen of an application executed for a cloud streaming service;
A key frame extraction unit for extracting a key frame from the execution screen;
Generates still image compression information corresponding to the key frame, and applies a still image compression technique based on the still image compression information to at least one of the key frames and the remaining frames of the execution screen except for the key frame. An encoding unit that encodes the execution screen as a still image; And
Sending unit to perform a still image-based cloud streaming service by transmitting the still image-encoded execution screen to the user's terminal device
Including,
The still image compression information
Contains information of one or more of a palette, a color substitution table, and an optimal filter,
The encoding unit
Comparing with the key frame from among the remaining frames, extracting a sub-frame whose change value is less than a preset reference value, and encoding the sub-frame with a still image compression technique to which still image compression information corresponding to the key frame is applied Cloud streaming server, characterized in that. delete The method according to claim 1,
The key frame is
Cloud streaming server, characterized in that the frame compared to the previous frame of the execution screen, the change value of the frame is equal to or greater than the preset reference value. delete delete delete delete delete delete Capturing an execution screen of an application executed for a cloud streaming service;
Extracting a key frame from the execution screen;
Generates still image compression information corresponding to the key frame, and applies a still image compression technique based on the still image compression information to at least one of the key frames and the remaining frames of the execution screen except for the key frame. Encoding the execution screen as a still image; And
Transmitting the still image-encoded execution screen to a user's terminal device to perform a still image-based cloud streaming service
Including,
The still image compression information
Contains information of one or more of a palette, a color substitution table, and an optimal filter,
The step of encoding the still image
Comparing with the key frame from among the remaining frames, extracting a sub-frame whose change value is less than a preset reference value, and encoding the sub-frame with a still image compression technique to which still image compression information corresponding to the key frame is applied Cloud streaming service method for saving system resources, characterized in that. delete delete delete delete delete delete delete delete delete Captures the execution screen of an application executed for the cloud streaming service, extracts a key frame from the execution screen, and corresponds to the key frame in at least one of the key frames and the remaining frames excluding the key frame among the execution screens. A cloud that performs a still image-based cloud streaming service by applying a still image compression technique based on still image compression information to encode the execution screen as a still image and transmit the execution screen encoded with the still image to the user's terminal device Streaming server; And
Terminal receiving an application execution result screen corresponding to the cloud streaming service from the cloud streaming server
Including,
The still image compression information
Contains information of one or more of a palette, a color substitution table, and an optimal filter,
The cloud streaming server
Comparing with the key frame from among the remaining frames, extracting a sub-frame whose change value is less than a preset reference value, and encoding the sub-frame with a still image compression technique to which still image compression information corresponding to the key frame is applied Cloud streaming service system, characterized in that.

Images