KR102232899B1 - System for cloud streaming service, method of cloud streaming service based on type of image and apparatus for the same - Google Patents

Abstract

Disclosed are a cloud streaming service system, a cloud streaming service method according to an image type, and an apparatus therefor. Comparing to the previous frame, capturing the change region included in the changed frame, and selecting a still image compression technique according to the image type determined by considering at least one of the number of colors of the image corresponding to the change region and the standard deviation of the image pixel value. , A still image-based cloud streaming service can be performed by transmitting a change region encoded by a still image compression technique to a user. When providing a cloud streaming service, it is possible to improve the compression efficiency of still images and the speed of the cloud streaming service by using a still image compression technique suitable for the image type.

Description

Cloud streaming service system, cloud streaming service method according to image type, and device therefor {SYSTEM FOR CLOUD STREAMING SERVICE, METHOD OF CLOUD STREAMING SERVICE BASED ON TYPE OF IMAGE AND APPARATUS FOR THE SAME}

The present invention is a cloud streaming service system capable of improving image compression efficiency and service speed by determining an image type of a changed area during a cloud streaming service and performing encoding using a still image compression technique according to the image type, according to image type. It relates to a cloud streaming service method and an apparatus therefor.

Due to the rapid development of the modern Internet, the communication speed of individuals has rapidly improved. With this increase in communication speed, access to a remote computer and download or upload large amounts of data, or log in locally to the remote computer using a remote computer control program. As previously described, an environment where remote computers can be used has been created.

In addition, the application is run on the server, and the driving screen is compressed through video encoding and transmitted to the client, and the client plays the transmitted video to create the same effect as running the application on its own terminal based on screen virtualization. Cloud streaming service is in the spotlight.

Cloud streaming services such as this basically applied a cloud streaming technique based on a video codec. However, even in the case of servicing a static screen such as a menu display, since the entire screen is unnecessarily captured and operated using a video codec, there is an inefficient part in operating the cloud streaming service system.

Korean Patent Publication No. 10-2014-0027040, published on March 6, 2014 (Name: Adaptive image compression system and method thereof)

An object of the present invention is to improve compression efficiency and cloud streaming service speed by using a still image compression technique suitable for an image type when providing a still image-based cloud streaming service.

In addition, an object of the present invention is to maximize the streaming extrusion efficiency through a still image-based cloud streaming service to provide a cloud streaming service that is faster and targets more users through the same band.

Cloud streaming server according to the present invention for achieving the above object, the capture unit for capturing a change area included in the changed frame compared to the previous frame; A still image compression technique is selected according to an image type determined in consideration of one or more of the number of colors of the image corresponding to the change area and the standard deviation of image pixel values, and one An encoding unit encoding a still image on the change region corresponding to a frame; And a sending unit that transmits the still image-encoded change region to the user to perform a still image-based cloud streaming service.

In this case, the encoding unit may select the still image compression method according to an image type of a natural image or a synthetic image.

In this case, the encoding unit may determine the image type as the natural image when the number of colors of the image corresponding to the change region exceeds a preset reference number.

In this case, when the number of colors of the image corresponding to the change region is less than or equal to the preset reference number, the encoding unit may determine the image type in consideration of a standard deviation of the image pixel values corresponding to the change region.

In this case, the encoding unit determines the image type when the number of colors of the image corresponding to the change area is less than or equal to the preset reference number, and the standard deviation of the image pixel values corresponding to the change area is less than or equal to a preset reference value. It can be determined as a composite image.

In this case, the encoding unit may determine the image type as the natural image when the standard deviation of the image pixel value corresponding to the change region exceeds a preset reference value.

In this case, the encoding unit may encode the still image on the change region using any one of a portable network graphics (PNG) and a joint photography experts group (JPEG) according to the image type.

In this case, the capture unit may capture a changed area as the change area compared to a screen area corresponding to the previous frame among the screen areas corresponding to the changed frame.

In this case, the capture unit may detect a section in which the frame change is small among the entire frame section to be transmitted to the user's terminal device, and capture the change area in the section in which the frame change is small.

In this case, the capture unit may detect an event type of an application generated from the user's terminal device to detect a section in which the frame change is small.

In addition, a cloud streaming service method according to an image type according to the present invention includes: capturing a change area included in a changed frame compared to a previous frame; A still image compression technique is selected according to an image type determined in consideration of one or more of the number of colors of the image corresponding to the change area and the standard deviation of image pixel values, and one Encoding the change region corresponding to the frame as a still image; And transmitting the still image-encoded change region to the user to perform a still image-based cloud streaming service.

In this case, in the encoding step, the still image compression technique may be selected according to an image type of either a natural image or a synthetic image.

In this case, in the encoding step, when the number of colors of the image corresponding to the change area exceeds a preset reference number, the image type may be determined as a natural image.

In this case, in the encoding step, when the number of colors of the image corresponding to the change region is less than or equal to the preset reference number, the image type may be determined in consideration of a standard deviation of the image pixel values corresponding to the change region.

In this case, in the encoding step, when the number of colors of the image corresponding to the change area is less than or equal to the preset reference number, and the standard deviation of the image pixel values corresponding to the change area is less than or equal to a preset reference value, the image type Can be determined as a composite image.

In this case, the encoding step may determine the image type as the natural image when a standard deviation of the image pixel value corresponding to the change region exceeds a preset reference value.

In this case, in the encoding step, the still image may be encoded using one of a still image compression technique of PNG (portable network graphics) and JPEG (joint photography experts group) according to the image type.

In this case, in the capturing step, the changed area may be captured as the change area by comparing with the screen area corresponding to the previous frame among the screen areas corresponding to the changed frame.

In addition, as another means for solving the problem of the present invention, a computer program stored in a medium is provided in order to execute the above-described method.

In addition, the cloud streaming service system according to the present invention captures the change region included in the changed frame compared to the previous frame, and considers at least one of the number of colors of the image corresponding to the change region and the standard deviation of the image pixel values. Selecting a still image compression technique according to the determined image type, encoding the change area corresponding to one frame by using the still image compression technique, and encoding the change area encoded with the still image by the user A cloud streaming server that transmits to and performs a still image-based cloud streaming service; And a terminal receiving an application execution result screen corresponding to the cloud streaming service from the cloud streaming server.

According to the present invention, when a still image-based cloud streaming service is provided, image compression efficiency and cloud streaming service speed can be improved by using a still image compression technique suitable for an image type.

In addition, the present invention maximizes streaming compression efficiency through a still image-based cloud streaming service to provide a cloud streaming service that is faster and targets more users through the same band.

1 is a block diagram showing a cloud streaming service system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating an example of a cloud streaming server shown in FIG. 1.
3 is a diagram showing a still image-based cloud streaming system.
4 is a diagram illustrating a process of capturing a change area in a still image-based cloud streaming service according to an embodiment of the present invention.
5 is a flowchart illustrating a cloud streaming service method according to an image type according to an embodiment of the present invention.
6 is an operation flowchart showing in detail a cloud streaming service method according to an image type according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, in the following description and the accompanying drawings, detailed descriptions of known functions or configurations that may obscure the subject matter of the present invention will be omitted. In addition, it should be noted that the same components are denoted by the same reference numerals as much as possible throughout the drawings.

The terms or words used in the present specification and claims described below should not be construed as being limited to their usual or dictionary meanings, and the inventors are appropriate as the concept of terms for describing their own invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention on the basis of the principle that it can be defined. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only the most preferred embodiments of the present invention, and do not represent all of the technical spirit of the present invention. It should be understood that there may be equivalents and variations. In addition, terms such as first and second are used to describe various elements, and are only used for the purpose of distinguishing one element from other elements, and are not used to limit the elements.

1 is a block diagram showing a cloud streaming service system according to an embodiment of the present invention.

Referring to FIG. 1, a cloud streaming service system according to an embodiment of the present invention includes a cloud streaming server 110, terminal devices 120-1, ..., 120-N, and a network 130. .

The cloud streaming server 110 executes an application by receiving a request for a cloud streaming service from the terminal devices 120-1, ..., 120-N, and displays an execution result screen corresponding to the application execution result to the terminal devices 120 -1, ..., 120-N).

Cloud streaming service basically applies a cloud streaming technique based on a video codec, so even when a cloud streaming technique is applied to a static screen such as a menu display, it is an inefficient method that unnecessarily captures the entire screen and encodes it with a video codec. Cloud streaming service was performed. In order to solve this problem, the present invention provides a cloud streaming system in which a cloud streaming server detects a change between frames and encodes only the detected change region using a still image compression technique when the change between frames is not large, such as a menu display screen. do.

The cloud streaming server 110 captures a change area included in the changed frame compared to the previous frame. In this case, among the screen areas corresponding to the changed frame, the changed area may be captured as the change area compared with the screen area corresponding to the previous frame. In this case, a section in which a frame change is small among the entire frame section to be transmitted to the terminal devices 120-1, ..., 120-N may be detected, and a change region may be captured in a section in which the frame change is small. In this case, an event type of an application generated from the terminal devices 120-1, ..., 120 -N may be detected to detect a section in which the frame change is small.

In addition, the cloud streaming server 110 selects a still image compression technique according to an image type determined in consideration of one or more of the number of colors of the image corresponding to the change area and the standard deviation of image pixel values, and A still image is encoded using a compression technique for a region of change corresponding to one frame. In this case, a still image compression technique may be selected according to an image type of either a natural image or a synthetic image. In this case, when the number of colors of the image corresponding to the change area exceeds a preset reference number, the image type may be determined as a natural image. In this case, when the number of colors of the image corresponding to the change region is less than or equal to a preset reference number, the image type may be determined in consideration of a standard deviation of image pixel values corresponding to the change region. In this case, when the number of colors of the image corresponding to the change area is less than or equal to the preset reference number, and the standard deviation of the image pixel values corresponding to the change area is less than or equal to the preset reference value, the image type may be determined as a composite image. In this case, when the standard deviation of the image pixel value corresponding to the change region exceeds a preset reference value, the image type may be determined as a natural image. In this case, the change region may be encoded using any one of a portable network graphics (PNG) and a joint photography experts group (JPEG) according to the image type.

In addition, the cloud streaming server 110 performs a still image-based cloud streaming service by transmitting the still image-encoded change region to the user.

The terminal devices 120-1, ..., 120-N receive an application execution result screen corresponding to the cloud streaming service from the cloud streaming server 110 and provide it to the user.

Terminal devices 120-1, ..., 120-N are devices that are connected to a communication network and can execute applications based on a cloud computing system, and are not limited to mobile communication terminals, and all information and communication devices and multimedia terminals , A wired terminal, a fixed terminal, and an Internet Protocol (IP) terminal. In addition, the terminal devices 120-1, ..., 120-N are mobile phones, portable multimedia played (PMP), mobile internet devices (MID), smart phones, desktops, and tablet computers, respectively. (Tablet PC), notebook (Note book), net book (Net Book), personal digital assistant (PDA), smart TV and information communication devices, such as mobile devices with various mobile communication specifications have.

The network 130 provides a path for transferring data between the cloud streaming server 110 and the terminal devices 120-1, ..., 120-N, and is a network that is used in the past and a network that can be developed in the future. It is a concept that encompasses all. For example, the network 130 is a wired/wireless local area communication network that provides communication of various information devices within a limited area, a mobile communication network that provides communication between mobiles and the mobile and the outside of the mobile, and communicates between the earth station and the earth station using satellites. It may be a satellite communication network, a wired or wireless communication network, or a combination of two or more. Meanwhile, the transmission method standard of the network 130 is not limited to the existing transmission method standard, and may include all transmission method standards to be developed in the future. In addition, a network used between the cloud streaming server 110 and the terminal devices 120-1, ..., 120-N in FIG. 1 is the terminal devices 120-1, ..., 120-N. It may be different from a network used for each other, or may be the same.

FIG. 2 is a block diagram illustrating an example of a cloud streaming server shown in FIG. 1.

Referring to FIG. 2, the cloud streaming server 110 illustrated in FIG. 1 includes a capture unit 210, an encoding unit 220, a sending unit 230, and a storage unit 240.

Cloud streaming service basically applies a cloud streaming technique based on a video codec, so even when a cloud streaming technique is applied to a static screen such as a menu display, it is an inefficient method that unnecessarily captures the entire screen and encodes it with a video codec. Cloud streaming service was performed. In order to solve this problem, the present invention provides a cloud streaming server 110 that detects a change between frames and encodes only the detected change region using a still image compression technique when the inter-frame change is not large, such as a menu display screen. ).

The capture unit 210 captures a change area included in the changed frame compared to the previous frame. The video codec-based cloud streaming technique can perform encoding by capturing all frames corresponding to a screen on which an application is running. However, if the change between frames on the screen on which the application is running is not large, the still image encoding is performed by capturing only the change area of the changed frame compared to the previous frame, and the parts except the change area are the same when displayed on the user's terminal device. It can be displayed in such a way that it is displayed in a way that changes only the change area.

In this case, among the screen areas corresponding to the changed frame, the changed area may be captured as the change area compared with the screen area corresponding to the previous frame. For example, if the area A is the changed area compared to the screen area corresponding to the previous frame among the screen area corresponding to the changed frame, only the area A of the screen area corresponding to the changed frame may be captured as the change area. The change region may vary in size or image properties corresponding to the change region according to an input signal input from the user's terminal device.

In this case, a section in which a frame change is small among the entire frame section to be transmitted to the user's terminal device may be detected, and a change region may be captured in a section in which the frame change is small.

In this case, by detecting an event type of an application generated from the user's terminal device, a section in which the frame change is small may be detected. For example, the event type of the application may be determined according to an input signal generated from the user's terminal device. Depending on the input signal, when an application is executed, such as a menu, an option, and a guide window, a static event with little change in frame or a dynamic event in which the frame frequently changes such as video playback and game play may occur. Accordingly, when the user detects an event type with an input signal generated through the terminal device, and if the event type is a static event, an interval in which the frame change is small among the entire frame interval corresponding to the event may be detected.

The encoding unit 220 selects a still image compression technique according to an image type determined in consideration of at least one of the number of colors of the image corresponding to the change area and the standard deviation of the image pixel values, and selects a still image compression technique. A still image is encoded using a change area corresponding to one frame. For example, since the image type of the captured change area may be a type such as a single photo or picture, or an image created by combining multiple images, determine the image type and use a still image compression technique suitable for each image type. By selecting and performing still image encoding, it is possible to increase the compression efficiency of the change region.

In this case, a still image compression technique may be selected according to an image type of either a natural image or a synthetic image. The natural image may mean an image of a naturally occurring object. For example, it may mean not only an optical image such as a photograph or picture, but also images of different wavelengths such as X-rays and infrared rays. Unlike natural images, the composite image may mean an image created or synthesized through a computer or artificial means.

In this case, when the number of colors of the image corresponding to the change area exceeds a preset reference number, the image type may be determined as a natural image. Since a natural image may be an image of a naturally occurring object, changes in brightness and saturation applied to the color representing the image may be more elaborate than a composite image. Accordingly, since the number of colors expressed by pixels configured to represent a natural image may be more diverse than that of a composite image, an image type corresponding to the change region can be determined in consideration of the number of colors of the image corresponding to the change region. For example, if the preset number of criteria is set to 100,000, if the number of colors constituting the image corresponding to the change area exceeds 100,000, the image type is determined as a natural image, and if it is less than 100,000, the image The type can be determined as a composite image.

In this case, when the number of colors of the image corresponding to the change region is less than or equal to a preset reference number, the image type may be determined in consideration of a standard deviation of image pixel values corresponding to the change region. The standard deviation of the pixel value may be a distribution diagram of color values indicated by pixels constituting an image corresponding to the change area. If the distribution of color values is 0, it may mean that all colors are the same value, so that the distribution of color values, that is, a small standard deviation of the pixel values, may be similar to the meaning that the number of colors constituting the image is small. have. Accordingly, the image type of the image corresponding to the change area may be determined in consideration of the standard deviation of the image pixel values.

In this case, when the number of colors of the image corresponding to the change area is less than or equal to the preset reference number, and the standard deviation of the image pixel values corresponding to the change area is less than or equal to the preset reference value, the image type may be determined as a composite image. For example, it may be assumed that a preset reference number for the number of colors in an image is 100,000, and a preset reference value for a standard deviation of an image pixel value is N. In this case, if the number of colors constituting the image corresponding to the change area is less than 100,000 and the distribution of color values represented by the pixels constituting the image corresponding to the change area is less than N, the image corresponding to the change area is The image type can be determined as a composite image.

In this case, when the standard deviation of the image pixel value corresponding to the change region exceeds a preset reference value, the image type may be determined as a natural image. For example, if the standard deviation of the image pixel values corresponding to the change area exceeds a preset reference value, it may mean that the number of colors constituting the image is larger. Accordingly, if the standard deviation of the image pixel values exceeds a preset reference value, it may be determined that the image corresponding to the change area is a natural image composed of many colors.

In this case, the change region may be encoded using any one of a portable network graphics (PNG) and a joint photography experts group (JPEG) according to the image type.

The PNG still image compression technique has good image quality when encoding, but the transmission speed may decrease when the bandwidth of the transmission network is low due to the large data size, and the JPEG still image compression technique has good compression efficiency, so the transmission data is transmitted during encoding. Although the amount can be significantly reduced, it is a technique that can generate a lot of system load when performing encoding and decoding as the compression efficiency is high. In addition, the JPEG still image compression technique can be more effective in compression efficiency when natural images in which similar colors are gathered around one pixel are compressed. Accordingly, when the image type corresponding to the change region is a natural image, the change region may be encoded using a JPEG still image compression technique.

In addition, when the change area is a composite image, it may be difficult to expect compression efficiency like a natural image even if a JPEG still image compression technique is used. Accordingly, when the image type of the image corresponding to the change region is a composite image, a PNG still image compression technique capable of enhancing the image quality may be selected to perform still image encoding of the change region.

In addition, when the change region is a composite image and the size of the change region is large and data is large, the change region may be encoded using a palletized PNG (portable network graphics) still image compression technique capable of reducing the size of the data. The palletized PNG still image compression technique reduces the data size of the image during encoding, so the transmission speed can be guaranteed, but since it expresses colors in 8 bits, it may deteriorate the image quality.

The sending unit 230 transmits the still image-encoded change region to the user to perform a still image-based cloud streaming service. By completing the encoding and transmitting the compressed change area to the user's terminal device, a cloud streaming service can be performed so that the user's terminal device can feel the application being executed.

As described above, the storage unit 240 stores various types of information generated in the process of a cloud streaming service according to an embodiment of the present invention.

According to an embodiment, the storage unit 240 may be configured independently of the cloud streaming server 110 to support a function for a cloud streaming service. In this case, the storage unit 240 may operate as a separate mass storage, and may include a control function for performing the operation.

In addition, the cloud streaming server 110 configured as described above may be implemented as one or more servers.

Meanwhile, the cloud streaming server 110 is equipped with a memory to store information in the device. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in another implementation, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may include, for example, a hard disk device, an optical disk device, or some other mass storage device.

3 is a diagram showing a still image-based cloud streaming system.

Referring to FIG. 3, the server 310 may perform screen capture by executing a program screen 310 corresponding to a user input generated from a user's terminal device. In this case, the server 300 may be a server collectively referred to as a cloud streaming server and a web application server. In addition, a process of performing a screen capture by executing the program screen 310 may correspond to an operation flow of the capture unit 210 illustrated in FIG. 2.

The server 300 may encode (Image Encoding) 320 the captured program screen 310 through a still image compression technique. In this case, the process of performing the encoding (Image Encoding) 320 may correspond to the operation flow of the encoding unit 220 shown in FIG. 2.

In this case, only the changed area may be captured by comparing the changed frame with the previous frame. In addition, it is possible to select any one of portable network graphics (PNG) and joint photography experts group (JPEG) in consideration of the image type of the changed region, and encode the changed region with the selected still image compression scheme.

Thereafter, the server 300 may provide a still image-based cloud streaming service by streaming 330 of the changed region encoded with the still image to the user's terminal device. In this case, the process of performing the streaming 330 may correspond to the operation flow of the sending unit 230 shown in FIG. 2.

Thereafter, the user's terminal device renders the received data (Image Rendering) 340 so that the program screen 310 executed in the application by the user input is sent to the user as if it was executed on the user's terminal device. Can be displayed.

4 is a diagram illustrating a process of capturing a change area in a still image-based cloud streaming service according to an embodiment of the present invention.

Referring to FIG. 4, a still image-based cloud streaming service according to an embodiment of the present invention may compare a first frame 410 and a second frame 420 to capture and encode only a changed region 430.

For example, assuming that the areas B, C, D, and E in the first frame 410 and the second frame 420 are menu displays, in the first frame 410, the cursor is positioned in the C area, so that the C area is different. It has been shaded so that it can be distinguished from areas B, D, and E, and in the second frame 420, the cursor is moved from area C to area D, so that area D is shaded so that it can be distinguished from other areas B, C, and E. I can.

In this case, when comparing the first frame 410 and the second frame 420, only the changed regions C and D regions may be captured as the change region 430. In addition, a still image compression technique may be selected in consideration of the size of the change area 430 and image properties corresponding to the change area 430, and the change area 430 may be encoded using the selected still image compression technique.

As described above, the first frame 410 and the second frame 420 are frames that are clearly changed, and in the case of the existing cloud streaming service, it may be necessary to capture all frames corresponding to the frame rate including two frames to perform encoding. have. However, since the still image-based cloud streaming service captures only the change area 430 and encodes it with a still image compression technique, the service efficiency can be improved compared to the existing cloud streaming service method that unnecessarily captures many frames and the entire screen I can.

5 is a flowchart illustrating a cloud streaming service method according to an image type according to an embodiment of the present invention.

Referring to FIG. 5, the cloud streaming service method according to an image type according to an embodiment of the present invention captures a change area included in a changed frame compared to a previous frame (S510). The video codec-based cloud streaming technique can perform encoding by capturing all frames corresponding to a screen on which an application is running. However, if the change between frames on the screen on which the application is running is not large, the still image encoding is performed by capturing only the change area of the changed frame compared to the previous frame, and the parts except the change area are the same when displayed on the user's terminal device. It can be displayed in such a way that it is displayed in a way that changes only the change area.

In this case, among the screen areas corresponding to the changed frame, the changed area may be captured as the change area compared with the screen area corresponding to the previous frame. For example, if the area A is the changed area compared to the screen area corresponding to the previous frame among the screen area corresponding to the changed frame, only the area A of the screen area corresponding to the changed frame may be captured as the change area. The change region may vary in size or image properties corresponding to the change region according to an input signal input from the user's terminal device.

In this case, a section in which a frame change is small among the entire frame section to be transmitted to the user's terminal device may be detected, and a change region may be captured in a section in which the frame change is small.

In this case, by detecting an event type of an application generated from the user's terminal device, a section in which the frame change is small may be detected. For example, the event type of the application may be determined according to an input signal generated from the user's terminal device. Depending on the input signal, when an application is executed, such as a menu, an option, and a guide window, a static event with little change in frame or a dynamic event in which the frame frequently changes such as video playback and game play may occur. Accordingly, when the user detects an event type with an input signal generated through the terminal device, and if the event type is a static event, an interval in which the frame change is small among the entire frame interval corresponding to the event may be detected.

In addition, the cloud streaming service method according to the image type according to an embodiment of the present invention is a still image (still image) according to the image type determined in consideration of at least one of the number of colors of the image corresponding to the change area and the standard deviation of the image pixel value. image) compression scheme is selected, and a still image encoding a change region corresponding to one frame is performed using a still image compression scheme (S520). For example, since the image type of the captured change area may be a type such as a single photo or picture, or an image created by combining multiple images, determine the image type and use a still image compression technique suitable for each image type. By selecting and performing still image encoding, it is possible to increase the compression efficiency of the change region.

In this case, a still image compression technique may be selected according to an image type of either a natural image or a synthetic image. The natural image may mean an image of a naturally occurring object. For example, it may mean not only an optical image such as a photograph or picture, but also images of different wavelengths such as X-rays and infrared rays. Unlike natural images, the composite image may mean an image created or synthesized through a computer or artificial means.

In this case, when the number of colors of the image corresponding to the change area exceeds a preset reference number, the image type may be determined as a natural image. Since a natural image may be an image of a naturally occurring object, changes in brightness and saturation applied to the color representing the image may be more elaborate than a composite image. Accordingly, since the number of colors expressed by pixels configured to represent a natural image may be more diverse than that of a composite image, an image type corresponding to the change region can be determined in consideration of the number of colors of the image corresponding to the change region. For example, if the preset number of criteria is set to 100,000, if the number of colors constituting the image corresponding to the change area exceeds 100,000, the image type is determined as a natural image, and if it is less than 100,000, the image The type can be determined as a composite image.

In this case, when the number of colors of the image corresponding to the change region is less than or equal to a preset reference number, the image type may be determined in consideration of a standard deviation of image pixel values corresponding to the change region. The standard deviation of the pixel value may be a distribution diagram of color values indicated by pixels constituting an image corresponding to the change area. If the distribution of color values is 0, it may mean that all colors are the same value, so that the distribution of color values, that is, a small standard deviation of the pixel values, may be similar to the meaning that the number of colors constituting the image is small. have. Accordingly, the image type of the image corresponding to the change area may be determined in consideration of the standard deviation of the image pixel values.

In this case, when the number of colors of the image corresponding to the change area is less than or equal to the preset reference number, and the standard deviation of the image pixel values corresponding to the change area is less than or equal to the preset reference value, the image type may be determined as a composite image. For example, it may be assumed that a preset reference number for the number of colors in an image is 100,000, and a preset reference value for a standard deviation of an image pixel value is N. In this case, if the number of colors constituting the image corresponding to the change area is less than 100,000 and the distribution of color values represented by the pixels constituting the image corresponding to the change area is less than N, the image corresponding to the change area is The image type can be determined as a composite image.

In this case, when the standard deviation of the image pixel value corresponding to the change region exceeds a preset reference value, the image type may be determined as a natural image. For example, if the standard deviation of the image pixel values corresponding to the change area exceeds a preset reference value, it may mean that the number of colors constituting the image is larger. Accordingly, if the standard deviation of the image pixel values exceeds a preset reference value, it may be determined that the image corresponding to the change area is a natural image composed of many colors.

In this case, the change region may be encoded using any one of a portable network graphics (PNG) and a joint photography experts group (JPEG) according to the image type.

The PNG still image compression technique has good image quality when encoding, but the transmission speed may decrease when the bandwidth of the transmission network is low due to the large data size, and the JPEG still image compression technique has good compression efficiency, so the transmission data is transmitted during encoding. Although the amount can be significantly reduced, it is a technique that can generate a lot of system load when performing encoding and decoding as the compression efficiency is high. In addition, the JPEG still image compression technique can be more effective in compression efficiency when natural images in which similar colors are gathered around one pixel are compressed. Accordingly, when the image type corresponding to the change region is a natural image, the change region may be encoded using a JPEG still image compression technique.

In addition, when the change area is a composite image, it may be difficult to expect compression efficiency like a natural image even if a JPEG still image compression technique is used. Accordingly, when the image type of the image corresponding to the change region is a composite image, a PNG still image compression technique capable of enhancing the image quality may be selected to perform still image encoding of the change region.

In addition, when the change region is a composite image and the size of the change region is large and data is large, the change region may be encoded using a palletized PNG (portable network graphics) still image compression technique capable of reducing the size of the data. The palletized PNG still image compression technique reduces the data size of the image during encoding, so the transmission speed can be guaranteed, but since it expresses colors in 8 bits, it may deteriorate the image quality.

In addition, the cloud streaming service method according to an image type according to an embodiment of the present invention performs a still image-based cloud streaming service by transmitting a still image-encoded change region to a user (S530). By completing the encoding and transmitting the compressed change area to the user's terminal device, a cloud streaming service can be performed so that the user's terminal device can feel the application being executed.

In addition, although not shown in FIG. 5, the cloud streaming service method according to an image type according to an embodiment of the present invention stores various information generated during a cloud streaming service process according to an embodiment of the present invention.

6 is an operation flowchart showing in detail a cloud streaming service method according to an image type according to an embodiment of the present invention.

Referring to FIG. 6, in the cloud streaming service method according to an image type according to an embodiment of the present invention, a change area included in a changed frame is first compared with a previous frame (S610). In this case, among the screen areas corresponding to the changed frame, the changed area may be captured as the change area compared with the screen area corresponding to the previous frame.

After that, it is determined whether the number of colors of the image corresponding to the change area exceeds a preset reference number (S615).

If the determination result in step S615 exceeds the preset reference number, it is determined that the image type of the change area is a natural image (S620).

If the determination result in step S615 does not exceed the preset reference number, it is determined whether the standard deviation of the pixel values of the image corresponding to the change area exceeds the preset reference value (S625).

If the determination result in step S625 exceeds a preset reference value, the image type of the change area is determined as a natural image (S620).

If the determination result in step S625 does not exceed a preset reference value, the image type of the change area is determined as a composite image (S630).

Thereafter, a still image compression technique is selected according to the determined image type of the change area (S640). In this case, any one still image compression technique may be selected from portable network graphics (PNG) and joint photography experts group (JPEG). In addition, when the image type is a composite image, a palletized PNG (portable network graphics) still image compression technique may be used.

After that, a still image is encoded using the selected still image compression technique (S650).

After that, the still image-encoded change region is transmitted to the user's terminal device (S660). By encoding the still image and transmitting the compressed change region to the user's terminal device, a cloud streaming service may be performed so that the user's terminal device may feel that the application is running.

The cloud streaming service method according to the image type according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes magneto-optical media, and any type of hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of the program instructions may include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

As described above, the cloud streaming service system according to the present invention, the cloud streaming service method according to the image type, and the apparatus therefor are not limited to the configuration and method of the embodiments described as described above, but the above embodiments. They may be configured by selectively combining all or part of each of the embodiments so that various modifications may be made.

According to the present invention, a change region included in a frame that has been changed compared to a previous frame is captured, and a still image according to an image type determined in consideration of at least one of the number of colors of the image corresponding to the change region and the standard deviation of image pixel values ( Still image) compression method is selected, the change area corresponding to one frame is encoded using the selected still image compression method, and the still image-encoded change area is transmitted to the user to provide a still image-based cloud streaming service. You can do it. Furthermore, the cloud streaming service according to the image type maximizes the streaming compression efficiency, and the cloud streaming service can be provided faster and target more users through the same band. It can provide efficient service to users while saving.

110: cloud streaming server 120-1 to 120-N: terminal devices
130: network 210: capture unit
220: encoding unit 230: sending unit
240: storage unit 300: server
310: Program Screen 320: Image Encoding
330: streaming 340: image rendering
410: first frame 420: second frame
430: area of change

Claims (20)

A capture unit for capturing a change area included in the changed frame compared to the previous frame;
A still image compression technique is selected according to an image type determined in consideration of one or more of the number of colors of the image corresponding to the change area and the standard deviation of image pixel values, and one An encoding unit for encoding a still image on the change region corresponding to a frame; And
Sending unit for performing a still image-based cloud streaming service by transmitting the still image-encoded change area to the user
Including,
The encoding unit
Firstly, the image type is determined in consideration of the number of colors of the image, and when the number of colors of the image is less than or equal to a preset reference number, the image type is secondarily determined by considering the standard deviation of the image pixel values. ,
And determining the image type as a composite image when the number of colors of the image is less than or equal to the preset reference number and a standard deviation of the image pixel values is less than or equal to a preset reference value. The method according to claim 1,
The encoding unit
A cloud streaming server, characterized in that the still image compression technique is selected according to an image type of one of a natural image and a synthetic image. The method according to claim 2,
The encoding unit
And determining the image type as the natural image when the number of colors of the image corresponding to the change area exceeds a preset reference number. delete delete delete delete delete delete delete Capturing a change area included in the changed frame compared to the previous frame;
A still image compression technique is selected according to an image type determined in consideration of one or more of the number of colors of the image corresponding to the change area and the standard deviation of image pixel values, and one Encoding the change region corresponding to the frame as a still image; And
Transmitting the still image-encoded change region to a user to perform a still image-based cloud streaming service
Including,
The step of encoding the still image
Firstly, the image type is determined in consideration of the number of colors of the image, and when the number of colors of the image is less than or equal to a preset reference number, the image type is secondarily determined by considering the standard deviation of the image pixel values. ,
Cloud according to an image type of a cloud streaming service, characterized in that when the number of colors of the image is less than the preset reference number and the standard deviation of the image pixel value is less than or equal to a preset reference value, the image type is determined as a composite image. Streaming service method. delete delete delete delete delete delete delete delete A still image according to an image type determined in consideration of at least one of the number of colors of the image corresponding to the change area and the standard deviation of image pixel values by capturing the change region included in the changed frame compared to the previous frame Selecting a compression method, encoding the change region corresponding to one frame using the still image compression method, and transmitting the still image-encoded change region to the user to perform a still image-based cloud streaming service A cloud streaming server; And
Terminal receiving an application execution result screen corresponding to the cloud streaming service from the cloud streaming server
Including,
The cloud streaming server
Firstly, the image type is determined in consideration of the number of colors of the image, and when the number of colors of the image is less than or equal to a preset reference number, the image type is secondarily determined by considering the standard deviation of the image pixel values. ,
And determining the image type as a composite image when the number of colors of the image is less than or equal to the preset reference number and a standard deviation of the image pixel value is less than or equal to a preset reference value.

Images