KR20200097497A - Apparatus and method for encoding of cloud streaming - Google Patents

Abstract

Disclosed are a device and a method for encoding of cloud streaming. According to an embodiment of the present invention, the device for encoding of cloud streaming includes: an image change detector which detects changes in images provided through cloud streaming to check the presence of global cache data; and a post-processing unit determines whether re-encoding is necessary according to a preset condition based on reference data required for post-processing of the global cache data, wherein the global cache data is re-encoded based on whether the re-encoding is required, and the global cache data is updated using a re-encoded result, thereby ensuring better data size and quality in cloud streaming.

Description

Cloud streaming encoding device and method {APPARATUS AND METHOD FOR ENCODING OF CLOUD STREAMING}

The present invention relates to a cloud streaming technology, and more particularly, to an encoding technology of cloud streaming.

The rapid development of the Internet has resulted in a rapid increase in the communication speed of individuals, and this increase in communication speed is like downloading or uploading large amounts of data by accessing a remote computer, or using a remote computer control program. It provides an environment where you can use a remote computer, such as logging in locally to the computer.

In addition, as applications driven in mobile communication terminals such as smart phones have been developed in various ways, many virtualization technologies have been proposed for driving applications that require high performance in user terminals with relatively low performance.

Among them, the application is run on the server, the driving screen is compressed through video encoding and transmitted to the client, and the client plays the transmitted video to create the same effect as running the application on its own terminal based on screen virtualization. The cloud streaming service of is in the spotlight.

Cloud streaming service basically applied a cloud streaming technique based on a video codec. However, even in the case of servicing a static screen such as a menu display, the entire screen was unnecessarily captured and operated using a video codec, resulting in inefficient aspects of the system as a whole.

In addition, when providing a service to a terminal with low network performance during a cloud streaming service using a cache, even if high-quality data is acquired through the cache, in order to provide it to the terminal again, inefficient processing that lowers the quality of the data is performed. I can.

Therefore, there is an urgent need for an image-based cloud streaming service technology capable of capturing only the static image of the application execution screen and providing it to the user's terminal, but providing the service through a cache corresponding to the performance of the terminal. do.

In this cloud streaming service, a cloud streaming technique based on a video codec is basically applied. However, even in the case of servicing a static screen such as a menu display, since the entire screen is unnecessarily captured and operated using a video codec, there is an inefficient part in operating the cloud streaming service system.

In image cloud streaming, selection of an appropriate encoding method is an important factor in quality, data size, and real-time processing. (There is a requirement that it is difficult to satisfy all that the encoding method should be selected quickly, the encoding speed should be fast, and the data size should be small but the quality should be good.)

In the cloud streaming service system, a cloud streaming service method using source information, and a device therefor (10-2014-0137733), a proposal for acquiring image characteristic information or determining an appropriate encoding method based on the characteristic information analyzing image data I did.

In addition, a method has been proposed to perform simultaneous encoding for high-definition/low-quality images, process only low-quality encoding if the number of colors is less than or equal to the number of colors, and deliver high-quality encoded data when the number of colors is greater than or equal to the number of reference colors to ensure image quality and speed. (Cloud streaming service system, image cloud streaming service using simultaneous encoding 10-2015-0014861)

However, since these methods must guarantee real-time performance, they can be processed in a limited encoding mode (high quality/low quality) (actually, there are various encoding variables such as encoding format and detailed options depending on the format), and image characteristic information is converted into natural, or low-quality encoding. It classifies the degree of synthesis and proposes an encoding method suitable for this characteristic.

Korean Patent Publication No. 10-2015-0014861, published on April 12, 2016 (Name: Cloud Streaming Service System, Image Cloud Streaming Service Using Simultaneous Encoding)

An object of the present invention is to ensure better data size and quality in cloud streaming provided in real time.

In addition, an object of the present invention is to efficiently provide cloud streaming by performing encoding with optimal data.

In order to achieve the above object, the cloud streaming encoding apparatus according to an embodiment of the present invention includes an image change detector configured to detect a change in an image provided through cloud streaming in order to check the existence of global cache data, and the global cache. Determine whether re-encoding is necessary according to a preset condition based on reference data required for post-processing of data, re-encode the global cache data based on whether re-encoding is necessary, and use the re-encoded result to the global cache It includes a post-processing unit for updating data.

In this case, when there is no global cache image corresponding to a result of detecting the change in the image, the image change detection unit may store reference data required for post-processing.

In this case, the reference data necessary for the post-processing may include at least one of a size of data for determining the image quality and information on a peak signal-to-noise ratio (PSNR).

In this case, the post-processor may perform encoding with at least one encoding option for global cache data that is larger than the size of data for determining the image quality.

In this case, the post-processor may compare global cache data encoded with the at least one encoding option and select one of the encoded global cache data to update the global cache data.

In addition, the cloud streaming encoding method according to an embodiment of the present invention for achieving the above object is provided through cloud streaming in order to check the existence of global cache data in the cloud streaming encoding method of the cloud streaming encoding device. Detecting a change in the image being changed; Determining whether re-encoding is necessary according to a preset condition based on reference data required for post-processing of the global cache data, re-encoding the global cache data based on the need for re-encoding, and using the re-encoded result And a post-processing step of updating the global cache data.

In this case, the step of detecting the change of the image may store reference data necessary for post-processing when there is no global cache image corresponding to a result of detecting the change of the image.

In this case, the reference data necessary for the post-processing may include at least one of a size of data for determining the image quality and information on a peak signal-to-noise ratio (PSNR).

In this case, in the post-processing step, encoding may be performed using at least one encoding option for global cache data that is larger than the size of data for determining the image quality.

In this case, in the post-processing step, the global cache data may be updated by comparing global cache data encoded with the at least one encoding option and selecting any one of the encoded global cache data.

The present invention can ensure better data size and quality in cloud streaming provided in real time.

In addition, the present invention can efficiently provide cloud streaming by performing encoding with optimal data.

1 is a block diagram showing a cloud streaming service system according to an embodiment of the present invention.
2 is a block diagram illustrating an example of a cloud streaming router shown in FIG. 1.
3 is a block diagram showing in detail a cloud streaming service system according to an embodiment of the present invention.
4 is a block diagram showing a cloud streaming encoding apparatus according to an embodiment of the present invention.
5 is a flowchart illustrating a cloud streaming encoding method according to an embodiment of the present invention.
6 is an operation flow diagram illustrating a cloud streaming encoding method in a process of detecting image change according to an embodiment of the present invention.
7 is an operation flow diagram illustrating a cloud streaming encoding method in a process of post-processing a global cache according to an embodiment of the present invention.
8 is a diagram showing a computer system according to an embodiment of the present invention.

The present invention will be described in detail with reference to the accompanying drawings as follows. Here, repeated descriptions, well-known functions that may unnecessarily obscure the subject matter of the present invention, and detailed descriptions of configurations are omitted. Embodiments of the present invention are provided to more completely explain the present invention to those with average knowledge in the art. Therefore, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

Throughout the specification, when a certain part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

In the present invention, various transformations may be applied and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail. Effects and features of the present invention, and a method of achieving them will be apparent with reference to the embodiments described later in detail together with the drawings. Here, repeated descriptions, well-known functions that may unnecessarily obscure the subject matter of the present invention, and detailed descriptions of configurations are omitted. Embodiments of the present invention are provided to more completely explain the present invention to those with average knowledge in the art. Therefore, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

However, the present invention is not limited to the embodiments disclosed below, and may be implemented in various forms by selectively combining all or part of each embodiment. In the following embodiments, terms such as first and second are used for the purpose of distinguishing one constituent element from other constituent elements rather than a limiting meaning. In addition, expressions in the singular include plural expressions unless the context clearly indicates otherwise. In addition, terms such as include or have means that the features or elements described in the specification are present, and do not preclude the possibility of adding one or more other features or elements in advance.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, detailed descriptions of known functions or configurations that may obscure the subject matter of the present invention will be omitted in the following description and accompanying drawings. In addition, it should be noted that the same components are indicated by the same reference numerals as possible throughout the drawings. Embodiments of the present invention are provided to more completely explain the present invention to those with average knowledge in the art. Therefore, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

The terms or words used in the present specification and claims described below should not be construed as being limited to a conventional or dictionary meaning, and the inventor is appropriate as a concept of terms for describing his own invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention on the basis of the principle that it can be defined. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only the most preferred embodiments of the present invention, and do not represent all the technical spirit of the present invention, and thus various alternatives that can be substituted for them at the time of application It should be understood that there may be equivalents and variations. In addition, terms such as first and second are used to describe various elements, and are used only for the purpose of distinguishing one element from other elements, and are not used to limit the elements.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, identical or corresponding constituent elements are denoted by the same reference numerals, and redundant descriptions thereof will be omitted. .

1 is a block diagram showing a cloud streaming service system according to an embodiment of the present invention.

1, a cloud streaming service system according to an embodiment of the present invention includes a cloud streaming router 110, a cloud streaming server 120-1 to 120-N, a terminal 130-1 to 130-N, and Includes network 140.

The cloud streaming router 110 determines the network performance of at least one terminal 130-1 to 130-N for session connection to any one of the plurality of cloud streaming servers 120-1 to 120-N.

In this case, in consideration of at least one of the size of the bandwidth and the read/write performance of the network, the network performance may be determined as one of a first level corresponding to a predetermined reference performance or higher and a second level having a lower performance than the first level. .

In this case, the cache image quality level may correspond to one of a first level corresponding to the original quality of the application execution screen and a second level corresponding to a quality deteriorated than the first level.

In this case, the plurality of cloud streaming servers 120-1 to 120 -N have a first cloud streaming server having a cache image quality level corresponding to the first level and a second cloud streaming server having a cache image quality level corresponding to the second level. It can correspond to any of the servers.

In this case, the first cloud streaming server may encode the image stored in the cache according to the first level and transmit the still image to the at least one terminal 130-1 to 130 -N.

In this case, the second level may correspond to one of a palletized PNG and a JPEG with deteriorated image quality.

In this case, when at least one terminal (130-1 to 130-N) corresponds to a personal computer (PC) connected through the Internet, it is determined that the network performance is at the first level, and at least one terminal ( When 130-1 to 130-N) correspond to a low-performance set-top box connected through a cable modem network, the network performance may be determined to be at the second level.

In addition, the cloud streaming router 110 includes a plurality of cloud streaming servers 120-1 to 120-N, in which the cache image quality level of the image stored in the cache corresponds to the network performance. At least one terminal (130-1 to 130-N) is connected to N).

At this time, when at least one terminal (130-1 to 130-N) corresponds to the first level, at least one terminal (130-1 to 130-N) is connected to the first cloud streaming server, and at least one When the terminals 130-1 to 130-N of correspond to the second level, at least one terminal 130-1 to 130-N may be connected to the second cloud streaming server.

The cloud streaming servers 120-1 to 120-N provide an image-based cloud streaming service to the session-connected terminals 130-1 to 130-N.

In this case, an image corresponding to the application execution screen may be obtained by using the image stored in the cache.

In this case, the still image may be encoded and transmitted to the terminals 130-1 to 130-N according to the cache image quality level of the image stored in the cache.

In this case, the cache image quality level may correspond to one of a first level corresponding to the original quality of the application execution screen and a second level corresponding to a quality deteriorated than the first level.

In this case, the cloud streaming servers 120-1 to 120 -N may include at least one of a cache having a cache image quality level corresponding to the first level and a cache having a cache image quality level corresponding to the second level.

In this case, the cloud streaming servers 120-1 to 120-N may provide an image-based cloud streaming service through a pipeline process corresponding to rendering, capturing, still image encoding, and sending.

The terminals 130-1 to 130-N receive an application execution screen corresponding to the cloud streaming service from the cloud streaming server 110 and provide it to the user.

In addition, the terminals 130-1 to 130-N are devices that are connected to a communication network and can execute applications based on a cloud computing system, and are not limited to mobile communication terminals, and all information and communication devices, multimedia terminals, wired terminals, It may be a variety of terminals such as fixed terminal and IP (Internet Protocol) terminal. In addition, the terminals 130-1 to 130-N are mobile phones, Portable Multimedia Played (PMP), Mobile Internet Devices (MID), smart phones, desktops, tablet computers, and notebook computers, respectively. It may be a mobile terminal having various mobile communication specifications, such as (Note book), Net Book, Personal Digital Assistant (PDA), smart TV and information communication device.

In addition, the terminals 130-1 to 130-N receive various information such as number and character information, set various functions, and transmit signals input in connection with function control of the terminals 130-1 to 130-N. It can be transmitted to the control unit through the input unit. In addition, the input unit of the terminals 130-1 to 130 -N may include at least one of a keypad and a touch pad that generate an input signal according to a user's touch or manipulation. At this time, the input unit of the terminals 130-1 to 130-N is configured in the form of a single touch panel (or touch screen) together with the display unit of the terminals 130-1 to 130-N, The display function can be performed simultaneously. In addition, the input unit of the terminals 130-1 to 130-N may use all types of input means that may be developed in the future in addition to input devices such as a keyboard, keypad, mouse, and joystick. In particular, the input unit of the terminals 130-1 to 130-N according to the present invention may transmit an input signal for uploading or downloading content based on cloud computing to the controllers of the terminals 130-1 to 130-N.

In addition, the display unit of the terminals 130-1 to 130-N may display information about a series of operation states and operation results that occur while performing functions of the terminals 130-1 to 130-N. Also, the display unit of the terminals 130-1 to 130-N may display a menu of the terminals 130-1 to 130-N and user data input by the user. Here, the display of the terminals 130-1 to 130-N is a liquid crystal display (LCD), an ultra-thin liquid crystal display (TFT-LCD, Thin Film Transistor LCD), and a light emitting diode (LED). , Organic light-emitting diodes (OLED, Organic LED), active organic light-emitting diodes (AMOLED, Active Matrix OLED), Retina Display, Flexible display, and 3D display, etc. have. At this time, when the display unit of the terminals 130-1 to 130-N is configured in the form of a touch screen, the display unit of the terminals 130-1 to 130-N functions as an input unit of the terminals 130-1 to 130-N. Some or all of them may be performed. In particular, the display unit of the terminals 130-1 to 130-N according to the present invention may display information related to execution of content provided based on cloud computing on a screen.

In addition, the storage unit of the terminals 130-1 to 130-N is a device for storing data, including a main memory device and an auxiliary memory device, and is an application required for the functional operation of the terminals 130-1 to 130-N. You can save the program. The storage of the terminals 130-1 to 130-N may largely include a program area and a data area. Here, when the terminals 130-1 to 130-N activate each function in response to a user's request, the terminals 130-1 to 130-N execute the corresponding application programs under the control of the controller to provide each function. In particular, the storage unit of the terminals 130-1 to 130-N according to the present invention can store an operating system for booting the terminals 130-1 to 130-N, a program for uploading or downloading content based on cloud computing. have. Further, the storage unit of the terminals 130-1 to 130-N may store a content DB for storing a plurality of contents and information of the terminals 130-1 to 130-N. In this case, the content DB may include execution data for executing the content and attribute information on the content, and may store content usage information according to content execution. In addition, the information of the terminals 130-1 to 130-N may include terminal specification information.

In addition, the communication unit of the terminals 130-1 to 130 -N may perform a function of transmitting and receiving data through the cloud streaming server 110 and the network 130. Here, the communication unit of the terminals 130-1 to 130-N may include an RF transmission unit for up-converting and amplifying the frequency of the transmitted signal, and an RF receiving unit for low-noise amplification and down-converting the received signal. . The communication unit of the terminals 130-1 to 130-N may include at least one of a wireless communication module and a wired communication module. And, the wireless communication module is a configuration for transmitting and receiving data according to a wireless communication method, and when the terminals 130-1 to 130-N use wireless communication, a wireless network communication module, a wireless LAN communication module, and a wireless fan communication module Data can be transmitted and received to the cloud streaming server 110 using any one of them. In addition, the wired communication module is for transmitting and receiving data by wire. The wired communication module may connect to the network 130 through a wire and transmit/receive data to the cloud streaming server 110. That is, the terminals 130-1 to 130-N access the network 130 using a wireless communication module or a wired communication module, and may transmit and receive data to and from the cloud streaming server 110 through the network 130. In particular, the network 130 according to the present invention may communicate with the cloud streaming server 110 or other terminals 130-1 to 130-N to transmit and receive data necessary for uploading or downloading content based on cloud computing.

In addition, the control unit of the terminal devices 130-1 to 130-N may be an operating system (Operation System) and a process device that drives each component. For example, the control unit is the cloud streaming server 110 When accessing the cloud streaming server 110 through a separate service application, the entire process of executing the service application can be controlled according to the user's request, and at the same time, the cloud It is possible to control so that a service use request is transmitted to the streaming server 110, and at this time, information of the terminals 130-1 to 130-N required for user authentication can be controlled to be transmitted together.

In addition, the controller of the terminals 130-1 to 130-N may execute specific content stored in the storage of the terminals 130-1 to 130-N according to a user's request. In this case, the controller may store the content use history according to the content execution as content use information.

In addition, the control unit of the terminals 130-1 to 130-N is cloud streaming with execution data for executing the content, content information including attribute information on the content, and content usage information, which is information according to the content usage history. It can be uploaded by transmitting it to the server 110. Thereafter, the control unit may delete the content uploaded by transmitting to the cloud streaming server 110 from the storage unit of the terminals 130-1 to 130-N according to the user's request, and access the cloud streaming server 110 The content may be executed and used through the cloud streaming server 110.

In addition, the control unit of the terminals 130-1 to 130-N may access the cloud streaming server 110 to download content from other terminals 130-1 to 130-N and then control to be stored in the storage unit. , When executing content through the cloud streaming server 110, it is possible to control the content to be executed after receiving only data necessary for execution.

The network 140 provides a path for transmitting data between the cloud streaming router 110 and the terminals 130-1 to 130-N, and is a concept encompassing both an existing network and a network that can be developed in the future. . For example, the network 140 is a wired/wireless local area communication network that provides communication of various information devices within a limited area, a mobile communication network that provides communication between mobiles and the mobile and the outside of the mobile, and the earth station and the earth station communicate using satellites. It may be a satellite communication network, a wired or wireless communication network, or a combination of two or more. Meanwhile, the transmission method standard of the network 140 is not limited to the existing transmission method standard, and may include all transmission method standards to be developed in the future. In addition, the network used between the cloud streaming router 110 and the terminals 130-1 to 130-N in FIG. 1 may be different from the network used between the terminals 130-1 to 130-N, It could be the same thing.

2 is a block diagram illustrating an example of a cloud streaming router shown in FIG. 1.

Referring to FIG. 2, the cloud streaming router 110 illustrated in FIG. 1 includes a communication unit 210, a network performance determination unit 220, a connection unit 230, and a storage unit 240.

The communication unit 210 serves to transmit and receive information related to a plurality of terminals through a communication network such as the network shown in FIG. 1. In particular, the communication unit 210 according to an embodiment of the present invention may receive performance information of the terminal from the terminal together with a request for a cloud streaming service. For example, information such as a device type or model name of a terminal requesting a session connection, a network type and a network speed used by the terminal may be provided. The performance of the terminal or the network performance information of the terminal may be freely set according to the administrator setting of the cloud streaming service system.

In this case, the application execution result may be received from the cloud streaming server for the cloud streaming service requested by the terminal.

In addition, an execution result of an application corresponding to the cloud streaming service requested by the terminal may be provided to the terminal. For example, an application execution screen corresponding to a service requested by the terminal may be received from a cloud streaming server connected to the terminal and transmitted to the terminal. In addition, if the cloud streaming server and the terminal are already in a session-connected state through the cloud streaming router 110, then data can be directly transmitted and received through a communication network between the cloud streaming server and the terminal without going through the cloud streaming router 110.

The network performance determination unit 220 determines the network performance of at least one terminal for session connection to any one of a plurality of cloud streaming servers. For example, it may be determined by receiving a message requesting a session connection from the terminal together with information on a network being used by the terminal or performance information of a network. In this way, a server capable of providing an appropriate service to the terminal among a plurality of cloud streaming servers may be searched for and connected to the terminal using information related to the terminal network.

In this case, in consideration of at least one of the size of the bandwidth and the read/write performance of the network, the network performance may be determined as one of a first level corresponding to a predetermined reference performance or higher and a second level having a lower performance than the first level. .

The bandwidth can be said to be a value indicating how much transmission capability a channel for transmitting data through a network has. For example, if the road is wide, many cars can run at once, and if the bandwidth is large, there is a lot of data that can be transmitted at one time, so it can be determined that the transmission capacity of the channel is high. That is, if the bandwidth is large, the capacity of data that can be transmitted at one time may be large.

Therefore, since the plurality of cloud streaming servers according to the present invention transmit data with a large capacity mainly for picture files to the terminal, the performance of the network can be determined by considering at least one of the bandwidth of the network or the performance of reading or writing the data. I can.

In this case, the preset reference performance for determining the first level may correspond to a degree to which an image corresponding to an application execution screen provided from the cloud streaming server can be received without a problem. For example, when a cloud streaming server encodes an image corresponding to the application execution screen to a terminal and transmits it to a terminal, the network level capable of receiving an image without delay in the terminal side is determined as a preset reference performance. Can be set.

Accordingly, the second level, which is lower in performance than the first level corresponding to the preset reference performance, may correspond to a level at which a delay occurs when processing an image having an image quality corresponding to an application execution screen.

In this case, the preset standard performance, which can be the standard of the first level and the second level, may be set according to the image quality of the application execution screen provided from the cloud streaming server, or freely set by the administrator of the cloud streaming service system. And may be changed.

In this case, the cache image quality level may correspond to one of a first level corresponding to the original quality of the application execution screen and a second level corresponding to a quality deteriorated than the first level. In other words, the cache image quality level may correspond to the quality level of images stored in the cache provided in the cloud streaming server.

A plurality of cloud streaming servers according to an embodiment of the present invention may store only an image corresponding to one of the first level and the second level in the cache. That is, only an image corresponding to the original image quality may be stored or only a deteriorated image may be stored, and this may be to connect a session by matching the image quality information stored in the cache with the network performance of the terminal.

In addition, according to various embodiments, there may be a case in which a single cloud streaming server includes a cache for storing an image corresponding to a first level and an image corresponding to a second level.

In this case, the plurality of cloud streaming servers may correspond to any one of a first cloud streaming server having a cache image quality level corresponding to the first level and a second cloud streaming server having a cache image quality level corresponding to the second level. . That is, by classifying the cloud streaming server into two types according to the image quality stored in the cache, the cloud streaming router can easily search for a cloud streaming server to connect to a session with the terminal according to the network performance of the terminal. For example, a server identifier capable of distinguishing between a first cloud streaming server or a second cloud streaming server may be assigned to each of the plurality of cloud streaming servers, and the server type may be easily distinguished using the server identifier. In addition, even if it is not a method using a server identifier, it is possible to provide a faster session connection to a terminal by easily displaying two types of cloud streaming servers using various methods.

In this case, the first cloud streaming server may encode the still image corresponding to the first level and transmit the image stored in the cache to at least one terminal. That is, if the terminal is session-connected to the first cloud streaming server through the cloud streaming router, it may mean that there is no difficulty in receiving an image corresponding to the first level of the network performance of the terminal. Accordingly, the image stored in the cache is encoded and transmitted to the terminal so that the image quality does not deteriorate, so that the original quality service can be provided to the user through the terminal.

In this case, the second level may correspond to one of a palletized PNG and a JPEG with deteriorated image quality. For example, the second level, in which the image quality is degraded according to the network performance of the terminal, may be generated by reducing the number of colors or lowering the resolution of the image.

In this case, in the case of the palletized PNG, 1 byte of information corresponding to a color index per pixel constituting the image may be included. This may correspond to a quarter of the data size per pixel than the PNG32bit method, which has relatively good image quality. Therefore, since the data capacity of the entire image is also reduced to a quarter, it may be suitable for an environment with poor network performance.

In addition, it is possible to reduce the data size by lowering the image quality of a normal JPEG image using a method such as lossy compression.

As described above, a service can be provided to a terminal with poor network performance without delay by using a cache image of the second level, which has lowered the image quality, that is, a data capacity corresponding to the image is reduced. In addition, even in the cloud streaming server, the image stored in the cache does not need to be re-rendered according to the network performance, thus saving resources accordingly.

At this time, when at least one terminal corresponds to a personal computer (PC) connected through the Internet, it is determined that the network performance is at the first level, and at least one terminal is connected through a cable modem network. When it corresponds to a connected low-performance set-top box, it can be determined that the network performance is at the second level.

A cable modem may be a device used when accessing the Internet through a cable TV network. Such a cable modem can be installed at the end of a hybrid fiber coaxial (HFC) network in which an optical cable and a coaxial cable are mixed, that is, in a subscriber's house to a cable network and connected to a set-top box. At this time, the set-top box connected through a cable modem refers to a receiver for digital satellite broadcasting, and is named after a box installed on a television. When a set-top box is installed, digital television broadcasts can be received through analog television, and a set-top box is essential for realizing interactive television or video on demand. These set-top boxes may be of a wide variety according to the subscriber's household, and performance may vary depending on the type.

Therefore, depending on the type of set-top box, in the case of a low-performance set-top box, there may be a delay in processing the image provided in the original quality from the cloud streaming server. Therefore, it is determined as the second level and the image quality is deteriorated corresponding to the second level. Service can be provided.

The connection unit 230 connects at least one terminal to a cloud streaming server in which a cache image quality level of an image stored in a cache among the plurality of cloud streaming servers corresponds to network performance. For example, assuming that there is no problem in receiving a high-quality image due to high network performance of the terminal, the terminal can be session-connected to a cloud streaming server having a high cache image quality level. That is, if the terminal's network performance is good and high-quality service can be provided, it is possible to provide high-quality service by connecting a cloud streaming server that stores high-quality images in a cache.

However, when a terminal with low network performance is connected to a cloud streaming server that stores high-quality images in a cache, there is a situation in which the cloud streaming server provides an image with deteriorated image quality through the process of deteriorating the image quality. Can occur. This may be a cause of inefficient use of the resources of the cloud streaming server because it is not possible to provide a high-definition service, and the cloud streaming server has to unnecessarily process the image quality deterioration process.

Therefore, when the network performance of the terminal is not high, it is possible to provide a faster and more efficient cloud streaming service by connecting a cloud streaming server that stores an image in a cache at a low quality accordingly.

In this case, when at least one terminal corresponds to the first level, at least one terminal is connected to the first cloud streaming server, and when at least one terminal corresponds to the second level, at least one terminal is connected to the second level. You can connect to a cloud streaming server. That is, a terminal having a network performance corresponding to the first level connects a session with a first cloud streaming server capable of providing a cache image of the first level, and a terminal having a network performance corresponding to the second level has a second level cache. Session connection with a second cloud streaming server capable of providing images.

In this case, the first cloud streaming server and the second cloud streaming server may correspond to a cloud streaming server that provides an image-based cloud streaming service. Accordingly, a service can be provided to the terminal through a pipeline process corresponding to rendering, capturing, still image encoding and sending of an application execution screen.

The video codec-based cloud streaming technique can perform encoding by capturing all frames corresponding to the screen on which the application is running. However, if the change between frames on the screen on which the application is running is not large, as in the image-based cloud streaming technique, when compared with the previous frame, only the changed area of the changed frame is captured and still image encoded and displayed on the user's terminal. Except for the change area, it can be displayed in the same way and only the change area is changed.

In this case, among frames corresponding to the execution screen of the rendered application, sections with little change between frames may be detected, and a frame to capture an image may be detected in sections with little change between frames. For example, in a section in which there are many changes between frames, it may be inefficient to provide an image-based cloud streaming service because the number of images to be captured is large. Accordingly, in such a period with many changes, the service is performed through a video codec-based cloud streaming technique, and an image can be captured from the changed frame in order to perform the image-based cloud streaming service only in the section with little inter-frame change.

In this case, the image corresponding to the changed portion may be captured compared to the previous frame of the frame. For example, among many frames constituting an application execution screen, a current frame and a previous frame may be compared, and a part having a difference between the two frames may be determined as a changed part and captured as an image. In addition, when the changed portion is detected as a plurality of regions on the frame, one region including the plurality of regions may be captured as an image or the plurality of regions may be captured as multiple images, respectively.

In this case, when the image requested from the terminal exists in the cache, the image may be obtained from the cache without capturing the image from the application execution screen. In addition, the image captured and encoded on the application execution screen and provided to the terminal is stored in a cache, so that the next request by the terminal can be quickly provided through the cache.

In this case, the image to be transmitted to the terminal may be compressed using a still image encoding method, and still image encoding may be performed by selecting a method corresponding to the image among several still image encoding methods.

In this case, in the present invention, since the cloud streaming server is selected according to the network capability of the terminal and connected to the session, when encoding the still image stored in the cache, the still image encoding method may be selected according to the image quality. In addition, when the image requested by the terminal does not exist in the cache and the application execution screen is captured and still image encoded, the still image encoding method may be selected according to the performance of the terminal and the network performance of the terminal. For example, in order to improve the speed of a cloud streaming service, images can be compressed using a pallet PNG encoding method that has a high compression rate and low load. In addition, if you want to provide good image quality regardless of the amount of load, you can compress the image using a PNG32bit encoding method or a JPEG encoding method.

As described above, the storage unit 240 stores various types of information generated during a cloud streaming service process according to an embodiment of the present invention.

According to an embodiment, the storage unit 240 may be configured independently of the cloud streaming router 110 to support a function for a cloud streaming service. In this case, the storage unit 240 may operate as a separate mass storage, and may include a control function for performing the operation.

In addition, the cloud streaming router 110 configured as described above may be implemented as one or more devices.

Meanwhile, the cloud streaming router 110 is equipped with a memory and can store information within the device. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in another implementation, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may include, for example, a hard disk device, an optical disk device, or some other mass storage device.

3 is a block diagram showing in detail a cloud streaming service system according to an embodiment of the present invention.

3, a cloud streaming service system according to an embodiment of the present invention includes a cloud streaming router 310, a first cloud streaming server 320, a second cloud streaming server 330, and a PC corresponding to the terminal ( 340) and STB (Set Top Box) 350.

At this time, the cache 321 provided in the first cloud streaming server 320 stores an image having a cache image quality level corresponding to the first level, and the cache 331 provided in the second cloud streaming server 330 May store an image whose cache image quality level corresponds to the second level.

In this case, the cloud streaming router 310 may receive a session connection request to the cloud streaming server from the PC 340 through the Internet. The cloud streaming router 310 may determine the network performance of the PC 340. If the cloud streaming router 310 determines the network performance of the PC 340 to be a first level corresponding to a predetermined reference performance or higher, the cloud streaming router 310 connects the PC 340 to the first cloud streaming server ( 320). That is, the PC 340 determines that there is no problem in receiving the image corresponding to the first level stored in the cache 321, and can connect to receive a service of the quality corresponding to the first level from the first cloud streaming server. .

In addition, the cloud streaming router 310 may receive a session connection request to the cloud streaming server from the STB 350 through a cable modem. The cloud streaming router 310 may determine the network performance of the STB 350. If the cloud streaming router 310 determines the network performance of the STB 350 as a second level lower than the first level, the cloud streaming router 310 transfers the STB 350 to the second cloud streaming server 330. Session can be connected. That is, it may be determined that the network performance is low for the STB 350 to receive the image stored in the cache 321 of the first cloud streaming server 320. Accordingly, the STB 350 may connect to the second cloud streaming server so that the image corresponding to the second level stored in the cache 331 can be received.

Meanwhile, the cloud streaming router is equipped with a memory and can store information within the device. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in another implementation, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may include, for example, a hard disk device, an optical disk device, or some other mass storage device.

4 is a block diagram showing a cloud streaming encoding apparatus according to an embodiment of the present invention.

Referring to FIG. 4, a cloud streaming encoding apparatus according to an embodiment of the present invention includes an image change detection unit 11 and a post-processing unit 12.

In this case, the cloud streaming encoding apparatus may correspond to the cloud streaming servers 120-1, 120-2, and 120-N described in FIGS. 1 to 3.

The image change detection unit 11 may detect a change in an image provided through cloud streaming in order to check whether global cache data exists.

In this case, when there is no global cache image corresponding to a result of detecting the change in the image, the image change detection unit 11 may store reference data required for post-processing.

In this case, the image change detection unit 11 may transmit data to the user's terminal device when a global cache image corresponding to a result of detecting the change in the image exists.

In this case, the reference data required for post-processing may include at least one of a size of data for determining the image quality and information on a peak signal-to-noise ratio (PSNR).

The post-processing unit 12 determines whether re-encoding is necessary according to a preset condition based on reference data required for post-processing of the global cache data, re-encodes the global cache data based on whether the re-encoding is necessary, and re-encodes the global cache data. The global cache data may be updated using the encoded result.

In this case, the post-processor 12 may perform encoding with at least one or more encoding options for global cache data that is larger than the size of the data for determining the image quality.

In this case, the post-processing unit 12 may compare the global cache data encoded with the at least one encoding option and select any one of the encoded global cache data to update the global cache data.

In this case, the post-processing unit 12 may encode global cache data for various encoding options compared to real-time processing in a predetermined order, and update and maintain data re-encoded as optimal data as global cache data.

5 is a flowchart illustrating a cloud streaming encoding method according to an embodiment of the present invention.

Referring to FIG. 5, the cloud streaming encoding method according to an embodiment of the present invention may first detect an image change (S21).

That is, in step S21, a change in an image provided through cloud streaming may be detected to check whether the global cache data exists.

In this case, in step S21, when there is no global cache image corresponding to a result of detecting the change in the image, reference data required for post-processing may be stored.

In this case, in step S21, when a global cache image corresponding to a result of detecting the change in the image exists, data may be delivered to the user's terminal device.

In this case, the reference data required for post-processing may include at least one of a size of data for determining the image quality and information on a peak signal-to-noise ratio (PSNR).

In addition, the cloud streaming encoding method according to an embodiment of the present invention may post-process global cache data (S22).

That is, step S22 determines whether re-encoding is necessary according to a preset condition based on reference data required for post-processing of the global cache data, and re-encodes the global cache data based on whether or not re-encoding is necessary, The global cache data may be updated using the re-encoded result.

In this case, in step S22, as a result of determining whether re-encoding of the global cache data is required, if re-encoding is not required, the procedure may be terminated without performing post-processing.

In this case, in step S22, encoding may be performed using at least one or more encoding options for global cache data that is larger than the size of the data for determining the image quality.

In this case, in step S22, the global cache data may be updated by comparing global cache data encoded with the at least one encoding option and selecting any one of the encoded global cache data.

In this case, in step S22, global cache data may be encoded for various encoding options compared to real-time processing in a predetermined order, and data re-encoded as optimal data may be updated and maintained as global cache data.

6 is an operation flow diagram illustrating a cloud streaming encoding method in a process of detecting image change according to an embodiment of the present invention.

Referring to FIG. 6, the cloud streaming encoding method according to an embodiment of the present invention may first detect an image change (S31).

That is, in step S31, a change in an image provided through cloud streaming may be detected in order to check whether the global cache data exists.

In this case, in step S32, when there is no global cache image corresponding to a result of detecting the change in the image, reference data required for post-processing may be stored (S33).

In this case, in step S32, when a global cache image corresponding to a result of detecting the change in the image exists, data may be delivered to the user's terminal device (S34).

In this case, the reference data required for post-processing may include at least one of a size of data for determining the image quality and information on a peak signal-to-noise ratio (PSNR).

7 is an operation flow diagram illustrating a cloud streaming encoding method in a process of post-processing a global cache according to an embodiment of the present invention.

Referring to FIG. 7, the cloud streaming encoding method according to an embodiment of the present invention may post-process global cache data (S41).

In this case, in step S42, it may be determined whether re-encoding is necessary according to a preset condition based on reference data required for post-processing of the global cache data.

In this case, as a result of determining whether re-encoding of the global cache data is required in step S42, if re-encoding is not required, the procedure may be terminated without performing post-processing.

In this case, in step S43, the global cache data may be re-encoded based on whether the re-encoding is required.

In this case, in step S43, encoding may be performed with at least one or more encoding options for global cache data that is larger than the size of the data for determining the image quality.

In this case, in step S44, the global cache data may be updated using the re-encoded result.

In this case, in step S44, the global cache data may be updated by comparing global cache data encoded with the at least one encoding option and selecting any one of the encoded global cache data.

In this case, in step S44, global cache data may be encoded for various encoding options compared to real-time processing in a certain order, and data re-encoded as optimal data may be updated and maintained as global cache data.

8 is a diagram showing a computer system according to an embodiment of the present invention.

Referring to FIG. 8, the cloud streaming encoding apparatus according to an embodiment of the present invention may be implemented in a computer system 1100 such as a computer-readable recording medium. As shown in FIG. 8, the computer system 1100 includes one or more processors 1110, a memory 1130, a user interface input device 1140, and a user interface output device 1150 communicating with each other through a bus 1120. And a storage 1160. Also, the computer system 1100 may further include a network interface 1170 connected to the network 1180. The processor 1110 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 1130 or the storage 1160. The memory 1130 and the storage 1160 may be various types of volatile or nonvolatile storage media. For example, the memory may include a ROM 1131 or a RAM 1132.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the computer-readable recording medium may be specially designed and configured for the present invention or may be known and usable to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magnetic-optical media such as floptical disks. medium), and a hardware device specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device can be changed to one or more software modules to perform the processing according to the present invention, and vice versa.

The specific implementations described in the present invention are examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections, and in an actual device, various functional connections that can be replaced or additionally Connections, or circuit connections, may be represented. In addition, if there is no specific mention such as “essential” or “importantly”, it may not be an essential component for the application of the present invention.

Accordingly, the spirit of the present invention is limited to the above-described embodiments and should not be defined, and all ranges equivalent to or equivalently changed from the claims to be described later as well as the claims to be described later are the scope of the spirit of the present invention. It will be said to belong to.

As described above, the cloud streaming encoding apparatus and method according to an embodiment of the present invention are not limited to the configuration and method of the embodiments described as described above, but the above embodiments may be variously modified. All or part of each of the embodiments may be selectively combined and configured.

As such, this specification is not intended to limit the invention to the specific terms presented. Accordingly, although the present invention has been described in detail with reference to the above-described examples, those skilled in the art can make modifications, changes, and modifications to these examples without departing from the scope of the present invention. The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

According to the present invention, the cloud streaming encoding apparatus and method detects a change in an image provided through cloud streaming in order to check the existence of global cache data, and a preset condition based on reference data required for post-processing of the global cache data. It is possible to determine whether or not re-encoding is required according to, re-encode the global cache data based on whether or not re-encoding is necessary, and update the global cache data using the re-encoded result.

110, 310, 710, 810: Cloud Streaming Router
120-1~ 120-N: Cloud streaming server
130-1~ 130-N, 740, 840: terminal 140: network
210: communication unit 220: network performance determination unit
230: connection unit 240: storage unit
320, 720, 820: first cloud streaming server
330, 730, 830: second cloud streaming server
321, 331: cache 340: PC
350: STB
11: image change detection unit 12: post-processing unit
13: encoding unit
1100: computer system
1110: processor
1120: bus
1130: memory
1131: ROM
1132: RAM
1140: user interface input device
1150: user interface output device
1160: storage
1170: network interface
1180: network

Claims (10)

An image change detector configured to detect a change in an image provided through cloud streaming to check whether global cache data exists; And
Determine whether or not re-encoding is necessary according to a preset condition based on reference data required for post-processing of the global cache data, re-encode the global cache data based on whether or not re-encoding is necessary, and use the re-encoded result A post-processing unit for updating the global cache data;
Cloud streaming encoding device comprising a. The method according to claim 1,
The image change detection unit
And storing reference data necessary for post-processing when there is no global cache image corresponding to a result of detecting the change of the image. The method according to claim 2,
The reference data required for the post-processing is
A cloud streaming encoding apparatus comprising at least one of information on a size of data for determining the image quality and a peak signal-to-noise ratio (PSNR). The method of claim 3,
The post-processing unit
And performing encoding with at least one encoding option for global cache data that is larger than the size of the data for determining the image quality. The method of claim 4,
The post-processing unit
And updating the global cache data by comparing the global cache data encoded with the at least one encoding option and selecting any one of the encoded global cache data. In the cloud streaming encoding method of a cloud streaming encoding device,
Detecting a change in an image provided through cloud streaming to check whether global cache data exists;
Determining whether re-encoding is required according to a preset condition based on reference data required for post-processing of the global cache data; And
A post-processing step of re-encoding the global cache data based on whether the re-encoding is necessary, and updating the global cache data using the re-encoded result;
Cloud streaming encoding method comprising a. The method of claim 6,
The step of detecting the change of the image
And storing reference data required for post-processing when there is no global cache image corresponding to a result of detecting the change in the image. The method of claim 7,
The reference data required for the post-processing is
A cloud streaming encoding method comprising at least one of information on a size of data for determining the image quality and a peak signal-to-noise ratio (PSNR). The method of claim 8,
The post-treatment step
And performing encoding with at least one encoding option for global cache data that is larger than the size of the data for determining the image quality. The method of claim 9,
The post-treatment step
And updating the global cache data by comparing global cache data encoded with the at least one encoding option and selecting any one of the encoded global cache data.

Images