KR101888846B1

KR101888846B1 - System for cloud streaming service, method of image cloud streaming service using reciprocal cache and apparatus for the same

Info

Publication number: KR101888846B1
Application number: KR1020150121135A
Authority: KR
Inventors: 배태면; 나현식
Original assignee: 에스케이테크엑스 주식회사
Priority date: 2015-05-07
Filing date: 2015-08-27
Publication date: 2018-08-16
Also published as: KR20160131832A

Abstract

A cloud streaming service system, an image cloud streaming service method using a non-recursive cache, and an apparatus therefor are disclosed. The cloud streaming server of the present invention includes: a plurality of reciprocal caches corresponding to each of a plurality of terminals; An execution unit for executing an application corresponding to one of the terminals; And delivering the execution result of the application to one of the terminals through a pipeline process corresponding to rendering, capturing, still image encoding and sending, and if the captured image is stored in a reciprocal cache corresponding to one of the terminals, And a streaming unit for transmitting the index.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a cloud streaming service system, a cloud streaming service system, an image cloud streaming service method using a non-broadcast cache,

The present invention relates to a cloud streaming service system, an image cloud streaming service method, and an apparatus therefor, which can reduce the amount of data transmitted using a cache and a server load.

The rapid development of the Internet has resulted in a rapid increase in the communication speed of an individual. The improvement in the communication speed can be achieved by downloading or uploading a large amount of data by accessing a computer located at a remote place, or by using a remote computer control program, As well as providing an environment in which a remote computer can be used.

In addition, as a variety of applications have been developed for mobile communication terminals such as smart phones, many virtualization technologies have been proposed for driving applications requiring high performance in relatively low performance user terminals.

Among them, the application is driven on the server, the driving screen is compressed through video encoding and transmitted to the terminal, and the terminal reproduces the transmitted video, thereby generating a screen virtualization Cloud streaming service is becoming popular.

The cloud streaming service basically adopts the cloud streaming method based on the video codec. However, even when a static screen such as a menu display is serviced, the entire screen is unnecessarily captured and operated using a video codec, which makes the overall system ineffective.

In addition, in the cloud streaming service, repeatedly encoding the same image as the image previously captured and encoded not only increases the server load but also increases the amount of data transmitted.

Therefore, image-based cloud streaming service technology that operates more efficiently using cache is urgently required.

Korean Registered Patent No. 10-1088022, registered on November 23, 2011 (name: Multimedia streaming system and method for content distributed storage that reflects VCR function operation as user preference)

It is an object of the present invention to improve the compression rate by removing temporal redundancy while utilizing an image codec in a cloud streaming service.

In addition, an object of the present invention is to provide a cloud streaming service more efficiently by operating a server and a terminal using a recursive cache in an image-based cloud streaming service.

It is another object of the present invention to provide an efficient streaming transmission of a screen not stored in a recursive cache by using a global cache in addition to a shared cache shared by a server and a terminal.

According to an aspect of the present invention, there is provided a cloud streaming server including: a plurality of recipient caches corresponding to a plurality of terminals; An execution unit for executing an application corresponding to one of the terminals; And delivering the execution result of the application to one of the terminals through a pipeline process corresponding to rendering, capturing, still image encoding and sending, and if the captured image is stored in a reciprocal cache corresponding to one of the terminals, And a streaming unit for transmitting the index.

In this case, the streaming unit may skip the still image encoding and sending when transmitting the cache index, and may transmit the cache index.

At this time, the cloud streaming server may further include a global cache shared by the terminals, and the streaming unit may transmit the stored image to one of the terminals if the captured image is not stored in the recipient cache and stored in the global cache have.

At this time, the streaming unit may determine whether the captured image is stored in the recipient caches and the global cache, considering at least one of the position, the width, the data size, and the history information of the captured region It can be judged.

At this time, the non-persistent caches can be updated on a first-in-first-out (FIFO) basis.

At this time, the global cache may store data that is excluded from the recipient caches by an update within the memory size limit.

According to another aspect of the present invention, there is provided an image cloud streaming service method comprising: storing reciprocal data in reciprocal caches corresponding to a plurality of terminals, respectively; Executing an application corresponding to one of the terminals; Capturing a transmission object screen corresponding to an execution result of the application and generating a captured image; And transmitting the cache index if the captured image is stored in a reciprocal cache corresponding to one of the terminals.

In this case, the image cloud streaming service method may further include storing global data in a global cache shared by the terminals.

At this time, the image cloud streaming service method may further include transmitting the stored image to one of the terminals if the captured image is not stored in the recipient cache and stored in the global cache.

At this time, the recipient caches are updated on a first-in first-out (FIFO) basis, and the global cache can store the reciprocal data that is excluded from the recipient caches by update within the memory size limit.

At this time, the image cloud service method may skip the still image encoding and sending of the captured image when transmitting the cache index.

At this time, the image cloud service method may further include a step of determining whether or not the captured image is one of the reciprocal caches and the global cache, considering at least one of a position, an area, a data size and a history information of the captured region It can be determined whether or not it is stored.

According to the present invention, it is possible to improve the compression rate by eliminating temporal redundancy while utilizing an image codec in a cloud streaming service.

In addition, the present invention can provide a cloud streaming service more efficiently by operating the server and the terminal using the recursive cache in the image-based cloud streaming service.

In addition, in the present invention, a global cache is used in addition to the shared cache shared by the server and the terminal, so that a screen that is not stored in the recipient cache can be efficiently streamed.

1 is a block diagram illustrating a cloud streaming service system according to an embodiment of the present invention.
2 is a block diagram illustrating an example of the cloud streaming server shown in FIG.
3 is a diagram showing reciprocal caches and a global cache according to an embodiment of the present invention.
4 is a diagram illustrating a process of capturing an execution result of an application according to an embodiment of the present invention.
5 is a block diagram showing an example of a terminal shown in FIG.
6 is a flowchart illustrating an image cloud streaming service method according to an exemplary embodiment of the present invention.
7 is an operation flowchart illustrating a process of updating a cache according to an image cloud streaming service method according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art. Accordingly, the shapes and sizes of the elements in the drawings and the like can be exaggerated for clarity.

The terms and words used in the present specification and claims should not be construed to be limited to ordinary or dictionary meanings and the inventor is not limited to the concept of terminology for describing his or her invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible. Also, the terms first, second, etc. are used for describing various components and are used only for the purpose of distinguishing one component from another component, and are not used to define the components.

1 is a block diagram illustrating a cloud streaming service system according to an embodiment of the present invention.

1, a cloud streaming service system according to an exemplary embodiment of the present invention includes a cloud streaming server 110, terminals 120-1 to 120-N, a web server 130, and a network 140 do.

The cloud streaming server 110 includes reciprocal caches corresponding to each of the plurality of terminals 120-1 to 120-N, and when streaming application execution results through a streaming pipeline process , And may transmit a cache index if the captured image is stored in a reciprocal cache corresponding to one of the terminals 120-1 to 120-N.

At this time, the streaming pipeline process may include rendering, capturing, still image encoding and sending.

At this time, the cloud streaming server 110 may skip the still image encoding and sending and transmit the cache index when transmitting the cache index.

At this time, the cloud streaming server 110 further includes a global cache shared by the terminals 120-1 to 120-N, and the capture image is not stored in the recipient cache, The cache may transmit the stored image to one of the terminals when stored.

At this time, the cloud streaming server 110 determines whether or not the captured image is one of the reciprocal caches and the global caches, considering at least one of the position, the area, the data size, and the history information of the captured region It can be judged whether or not it is stored in one.

At this time, the non-volatile cache can be updated on a first-in-first-out (FIFO) basis.

At this time, the cloud streaming server 110 executes the requested application from the user's terminals 120-1 to 120-N and transmits the requested application to browsers corresponding to the terminals 120-1 to 120-N It is possible to access the web server 130 and execute the application.

The cloud streaming server 110 delivers the execution result of the application to the user's terminals 120-1 to 120-N through a pipeline process corresponding to rendering, capturing, still image encoding and sending.

The terminals 120-1 to 120-N receive an application execution screen corresponding to the cloud streaming service from the cloud streaming server 110 and provide them to the user.

At this time, the terminals 120-1 to 120-N may include a reciprocal cache corresponding to one of the recipient caches included in the cloud streaming server 110, respectively. Therefore, even when the cloud streaming server 110 transmits only the cache index without still image encoding and transmitting the captured image, the terminals 120-1 to 120-N can determine that the server Images can be obtained.

The terminals 120-1 to 120-N are connected to a communication network and are capable of executing applications on the basis of a cloud computing system. The terminals 120-1 to 120-N are not limited to the mobile communication terminal but may be any information communication device, multimedia terminal, A fixed terminal and an IP (Internet Protocol) terminal. Each of the terminals 120-1 to 120-N may be a mobile phone, a portable multimedia player (PMP), a mobile Internet device (MID), a smart phone, a desktop, a tablet PC, A mobile terminal having various mobile communication specifications such as a notebook computer, a notebook computer, a netbook, a personal digital assistant (PDA), a smart TV, and an information communication device.

The terminals 120-1 to 120-N receive various kinds of information such as numeric and character information, set various functions, and input signals related to the function control of the terminals 120-1 to 120-N And can be transmitted to the control unit through the input unit. The input unit of the terminals 120-1 to 120-N may include at least one of a keypad and a touchpad for generating an input signal according to a user's touch or operation. At this time, the input units of the terminals 120-1 to 120-N are configured in the form of one touch panel (or touch screen) together with the display units of the terminals 120-1 to 120-N, Display function can be performed at the same time. The input unit of the terminals 120-1 to 120-N may be any type of input unit that can be developed in addition to an input unit such as a keyboard, a keypad, a mouse, a joystick, and the like. In particular, the input units of the terminals 120-1 to 120-N according to the present invention can transmit input signals for uploading or downloading contents based on cloud computing to the control units of the terminals 120-1 to 120-N.

In addition, the display unit of the terminals 120-1 to 120-N can display information on a series of operation states, operation results, and the like that occur during the performance of the functions of the terminals 120-1 to 120-N. In addition, the display units of the terminals 120-1 to 120-N can display menus of the terminals 120-1 to 120-N and user data input by the user. The display units of the terminals 120-1 to 120-N may be a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), a light emitting diode (LED) An organic light emitting diode (OLED), an organic light emitting diode (AMOLED), an active matrix OLED, a retina display, a flexible display, and a three-dimensional display have. At this time, when the display units of the terminals 120-1 to 120-N are configured as touch screens, the display units of the terminals 120-1 to 120-N function as the input units of the terminals 120-1 to 120- Some or all of them. In particular, the display unit of the terminals 120-1 to 120-N according to the present invention can display information related to execution of content provided on a cloud computing basis on a screen.

The storage units of the terminals 120-1 to 120-N are devices for storing data, and include a main storage device and an auxiliary storage device, and are used for applications required for functional operations of the terminals 120-1 to 120- You can save the program. The storage units of these terminals 120-1 to 120-N may largely include a program area and a data area. Here, when the terminals 120-1 to 120-N activate the respective functions in response to a request from the user, the terminals 120-1 to 120-N provide the respective functions by executing the corresponding application programs under the control of the controller. In particular, the storage units of the terminals 120-1 to 120-N according to the present invention may store an operating system for booting the terminals 120-1 to 120-N, a program for uploading or downloading contents on the basis of cloud computing have. In addition, the storage units of the terminals 120-1 to 120-N may store information of a content DB storing a plurality of contents and information of the terminals 120-1 to 120-N. At this time, the content DB includes execution data for executing the content and attribute information about the content, and content usage information according to the execution of the content can be stored. The information of the terminals 120-1 to 120-N may include terminal specification information.

In addition, the communication units of the terminals 120-1 to 120-N may perform a function of transmitting and receiving data through the cloud streaming server 110 and the network 130. [ Here, the communication unit of the terminals 120-1 to 120-N may include RF transmitting means for up-converting and amplifying the frequency of the transmitted signal, RF receiving means for low-noise amplifying the received signal and down-converting the frequency . The communication units of the terminals 120-1 to 120-N may include at least one of a wireless communication module and a wired communication module. In the case where the terminals 120-1 to 120-N use wireless communication, the wireless communication module is a module for transmitting and receiving data according to a wireless communication method. The wireless communication module, the wireless LAN communication module, The data can be transmitted / received to / from the cloud streaming server 110 by using any one of them. The wired communication module is for transmitting / receiving data by wire. The wired communication module may connect to the network 130 through a wired network and transmit / receive data to / from the cloud streaming server 110. That is, the terminals 120-1 to 120-N access the network 130 using a wireless communication module or a wired communication module and transmit and receive data to and from the cloud streaming server 110 through the network 130. [ In particular, the network 130 according to the present invention can communicate with the cloud streaming server 110 or other terminals 120-1 to 120-N to transmit / receive data necessary for uploading or downloading content based on cloud computing.

The control unit of the terminal devices 120-1 to 120-N may be an operating system (OS) and a process unit for driving each configuration. For example, the control unit may include a cloud streaming server 110, When accessing the cloud streaming server 110 through a separate service application, it is possible to control the entire process of executing the service application according to the request of the user, and at the same time, It is possible to control the streaming server 110 to transmit the service use request and control the information of the terminals 120-1 to 120-N necessary for user authentication to be transmitted together.

In addition, the control units of the terminals 120-1 to 120-N can execute the specific contents stored in the storage units of the terminals 120-1 to 120-N according to the request of the user. At this time, the control unit may store the content usage history according to the execution of the content as the content usage information.

In addition, the control unit of the terminals 120-1 to 120-N may execute execution data for executing the content, content information including attribute information about the content, and content usage information, which is information according to the content usage history, And uploaded to the server 110 for uploading. Then, the controller transmits the uploaded content to the cloud streaming server 110, deletes the uploaded content from the storage unit of the terminals 120-1 to 120-N according to the user's request, connects to the cloud streaming server 110 The content may be executed through the cloud streaming server 110 and used.

In addition, the control units of the terminals 120-1 to 120-N may access the cloud streaming server 110 to download the content from the other terminals 120-1 to 120-N and control the content to be stored in the storage unit , It is also possible to control the content to be executed after receiving only data necessary for execution at the time of executing the content through the cloud streaming server 110.

The web server 130 is a means for executing an application according to a user's request through the terminals 120-1 to 120-N. The web server 130 is a web server or a JSP , a web application server that can execute programs such as servlets to provide dynamic results.

The network 140 provides a path for transferring data between the cloud streaming server 110 and the terminals 120-1 to 120-N or between the cloud streaming server 110 and the web server 130, It is a concept that encompasses both the network used and the network that can be developed in the future. For example, the network 140 may be a wired or wireless local area network that provides communication of various information devices within a limited area, a mobile communication network that provides communication between mobile objects and mobile objects outside the mobile object, Or a wired / wireless communication network, or a combination of two or more. Meanwhile, the transmission standard of the network 140 is not limited to the existing transmission standard, and may include all transmission standard to be developed in the future. In addition, the network used between the cloud streaming server 110 and the terminals 120-1 to 120-N in FIG. 1 may be different from the network used between the terminals 120-1 to 120-N, It may be the same.

2 is a block diagram illustrating an example of the cloud streaming server shown in FIG.

2, the cloud streaming server 110 shown in FIG. 1 includes a communication unit 210, a global cache 220, reciprocal caches 230, an execution unit 240, a streaming unit 250, And a storage unit 260.

The communication unit 210 transmits and receives information related to a plurality of terminals through a communication network such as the network shown in FIG. In particular, the communication unit 210 according to an embodiment of the present invention can receive a request for a cloud streaming service from the terminal and provide the terminal with the execution result of the application corresponding to the cloud streaming service requested by the terminal.

At this time, the terminal can receive the application execution result from the cloud streaming server with respect to the requested cloud streaming service.

The global cache 220 stores global data in the cache shared by the UEs.

Since the size of the cloud streaming server and the terminal are different from each other, the size of the cache may be different. The cloud streaming server includes the same cache (recipient cache) as the terminal, and additionally has a global cache, thereby storing caching data shared by a plurality of terminals.

At this time, it is preferable that the data (global data) stored in the global cache is information independent of the personal information of users. Accordingly, the cloud streaming server can determine whether or not the cached object stores the cached data in the corresponding page in the global cache according to the address or property of the web page.

For example, a screen captured on a page after a user's login on a banking website can be prevented from being cached in the global cache.

At this time, the global cache can recognize and store the data included in the global data list among the plurality of data corresponding to the application as global data. That is, the global data list may be a means for classifying and storing the global data commonly used for a plurality of terminals with individual data such as user's personal information. At this time, the global data list may be data that can be stored in the global cache in a concept corresponding to a white list. For example, among the plurality of data that can be provided by the application, the data other than the individual data may be included in the global data list. Alternatively, among the data other than the individual data, only data having a certain number of times or more requested from the user's terminal may be included.

At this time, data having the same URL as a URL (Uniform Resource Locator) included in the global data list among a plurality of data can be recognized as global data. That is, the URL can be used as a means of distinguishing whether specific data is data that can be stored in the global cache. For example, if the URL of data A is the same as any one of a plurality of URLs included in the global data list, data A can be recognized and stored as data that can be stored in the global cache.

Also, when the global data is stored in the global cache, the URL can be also used as a means for checking whether or not the requested data from the user is stored in the global cache. For example, if it is assumed that data B is requested from a terminal of a user, the URL of the data B can be used to retrieve whether global data corresponding to the data B is stored in the global cache. At this time, if there is no global data corresponding to the data B in the global cache but the same URL as the URL of the data B exists in the global data list, the data B is obtained from the web server, It can be stored in the cache.

At this time, the global data list includes the global data reference count corresponding to the number of times the global data is referenced, and the storage space of the global data can be managed based on the order of the global data reference count. At this time, the storage space of the global data may correspond to the global memory of the global cache.

The cache memory may correspond to a device that takes some of the instructions and data in the main memory and temporarily stores them so that the CPU can provide that information quickly when needed. These cache memories are designed to solve the problem of not using the high performance CPU because the data processing speed difference between the CPU and the main memory is large, and the processing speed of the cache memory is about 5 to 10 times faster than the main memory By placing the cache memory between the CPU and the main memory, it is possible to overcome the difference in data processing speed between the CPU and the main memory. Such a cache memory can be classified into a level 1 cache, a level 2 cache, and the like depending on the position of the cache memory and the type of data to be stored. For example, a Level 1 cache is the fastest in the CPU chip, and the Level 2 cache is slower than the Level 1 cache, but has a larger memory size and can be equipped with a separate chip.

According to the characteristic of the cache, since the global data having a high global data reference count is frequently used from a plurality of terminals, it can be stored in a cache having a high processing speed. In addition, global data with a low global data reference count is often used, so it can be stored in a cache that has a slow processing speed but a large memory space.

At this time, global data can be stored in the order corresponding to the memory cache, RAM disk space, and general disk space in the ascending order of the global data reference count. For example, the memory cache may correspond to a Level 1 cache with the fastest processing speed, a level 2 cache of RAM disk space, and a level 3 cache, or a hard disk or solid state drive (SSD). In other words, the memory cache is the fastest, the ramdisk space is the next, and finally the normal disk space is the slowest.

Accordingly, the higher the global data reference count, that is, the more global data actively used from a plurality of terminals, the faster the processing speed can be stored in the memory cache.

At this time, if the storage space is insufficient, global data whose global data reference count is equal to or less than a preset reference count can be deleted. For example, if you need to store global data in addition to the global cache and there is insufficient storage space corresponding to the global cache, you can delete global data already stored in the global cache based on the global data reference count. At this time, if it is assumed that the predetermined reference count is 5, the global data corresponding to the global data reference count of 5 or less among the at least one global data stored in the global cache may be retrieved and deleted.

Also, even if the storage space of the global cache is not short, if the global cache stores too much global data, it may take a long time to retrieve the global data corresponding to the requested data, thereby slowing the service speed. Accordingly, when global data is stored in a global cache beyond a preset reference storage amount, global data reference counts can be used to delete infrequently used global data.

In addition, it is possible to set a certain period and delete global data that is not frequently used in the global cache at each cycle.

At this time, the global cache 220 may store data that is excluded from the recursive cache 230 by updating, within a memory size limit.

The non-persistent cache 230 corresponds to each of a plurality of terminals.

That is, the cloud streaming server optimizes the data transmission between the server and the terminals by providing a recursive cache 230 modeling the cache of the terminal (the reciprocal cache).

The cloud streaming server skips the still image encoding and sending procedure by sending only the cache index when the captured image is stored in the recipient cache 230 to reduce the server load and the network traffic load .

At this time, whether or not a specific captured image is stored in the cache can be determined considering at least one of the position, the area, the data size, and the history information of the area (change area) where the capturing is performed.

At this time, the history information may include information for constructing a screen such as a menu ID, a focus position, and the like.

At this time, the reciprocal data stored in the reciprocal cache can be retrieved through the URL in the same way as global data. You can also manage the recursive cache by creating a recursive data reference count for the reciprocal data.

At this time, the reciprocal cache can be updated on a first-in first-out (FIFO) basis.

The execution unit 240 executes an application corresponding to one of the terminals. At this time, the execution unit 240 can access the web server by using a browser corresponding to the terminal among a plurality of browsers to execute the application.

Also, after connecting to the web server and executing the application, the data received from the web server can be stored in the recipient cache and / or the global cache. By adding new data to either the recyclable cache or the global cache and continuously managing the recursive cache and the global cache, the cloud streaming service can be more efficiently provided.

The streaming unit 250 delivers the execution result of the application to the user terminal through a pipeline process corresponding to rendering, capturing, still image encoding, and sending.

The cloud streaming technique based on the video codec can capture and encode all the frames corresponding to the screen on which the application operates. However, when the frame-to-frame change on the screen on which the application operates is not large, the still-image encoding is performed by capturing only the change area of the changed frame as compared with the previous frame like the image-based cloud streaming technique, The part except for the change area can be displayed in the same manner, and the change area can be displayed in a manner of changing only.

Accordingly, the execution result of the application can be output to the execution screen through the rendering process, and the still image encoding process can be performed by capturing only the image corresponding to the change area in the frames constituting the execution screen.

In this case, it is possible to detect intervals in which frames among the frames corresponding to the execution result of the application are small, and to capture the images in a section where there is little change between frames. For example, it may be inefficient to provide an image-based cloud streaming service because there are a large number of images to be captured with a large number of frames between frames. Accordingly, in such a period, a service can be performed through a cloud streaming technique based on a video codec, and an image of a changed capture area can be captured in order to perform an image-based cloud streaming service only in a period in which a change between frames is small .

At this time, the changed result can be detected by comparing the frames of the execution result of the application, and the changed part can be captured. For example, it is possible to compare a current frame with a previous frame among a large number of frames constituting an application execution screen, and to detect a difference part between two frames as a changed part. Further, when the changed portion is detected as a plurality of regions on the frame, one region including the plurality of regions may be captured or each of the plurality of regions may be captured.

In this case, the still image encoding method can be determined in consideration of the load generated in the cloud streaming server 110, the service speed, the image quality of the image, or the performance of the terminal receiving and rendering the encoded image. For example, to improve the speed of the cloud streaming service, you can compress the image with a palette PNG encoding method that has a high compression ratio and low load. In addition, if it is desired to provide a good image quality without regard to the load, the image can be compressed using the PNG 32-bit encoding method or the JPEG encoding method.

The still image encoding method compresses the captured image and transmits the compressed image to the terminal of the user through the sending process.

As described above, the streaming unit 250 can transmit a cache index when a captured image is stored in a reciprocal cache corresponding to one of the terminals.

In this case, the streaming unit 250 may transmit the cache index by omitting the still image encoding and sending when transmitting the cache index.

At this time, the streaming unit 250 may transmit the stored image to one of the terminals if the captured image is not stored in the recipient cache, but is stored in the global cache.

In this case, the streaming unit 250 may store the captured image in one of the reciprocal caches and the global caches in consideration of at least one of the position, the width, the data size, and the history information of the captured region. It can be determined whether or not it is stored.

The storage unit 260 stores various information generated in the cloud streaming service process according to the embodiment of the present invention as described above.

According to an embodiment, the storage unit 260 may be configured independently of the cloud streaming server 110 to support a function for the cloud streaming service. At this time, the storage unit 260 may operate as a separate mass storage and may include a control function for performing operations.

In addition, the cloud streaming server 110 configured as described above may be implemented as one or more servers.

On the other hand, the cloud streaming server 110 can store information in the memory on which the memory is mounted. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in other embodiments, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may comprise, for example, a hard disk device, an optical disk device, or any other mass storage device.

Through the cloud streaming server 110, the server and the client can share the cache in the image-based cloud streaming, thereby providing the cloud streaming service with the optimum efficiency.

3 is a diagram showing reciprocal caches and a global cache according to an embodiment of the present invention.

Referring to FIG. 3, the server cache 310 of the cloud streaming server 110 includes a global cache 311 and recipient caches 312-1,..., 312-N.

The terminals 120-1, ..., 120-N include reciprocal caches 320-1, 320-N.

The recipient cache 312-1 of the cloud streaming server 110 corresponds to the recipient cache 320-1 of the terminal 120-1 and the recipient cache 312-1 of the cloud streaming server 110 corresponds to the recipient cache 320-1 of the terminal 120-1, 2 corresponds to the recipient cache 320-2 of the terminal 120-2 and the recipient cache 312-N of the cloud streaming server 110 corresponds to the recipient cache 320-2 of the terminal 120- Lt; / RTI > cache 320-N.

The global cache 311 may be shared by the terminals 120-1, 120-2, ..., and 120-N.

For example, when a cloud streaming service is provided to the terminal 120-1 by executing an application, when the captured image is stored in the recipient cache 312-1, the cloud streaming server 110 transmits the cache index to the terminal 120-1, (120-1).

At this time, the cloud streaming server 110 can lower the server load and the network traffic load by omitting still image encoding and sending.

The terminal 120-1 searches the recipient cache 320-1 using the received cache index, loads the stored image, and generates a cloud streaming screen using the loaded image.

For example, when a cloud streaming service is provided to the terminal 120-2 by executing an application, when the captured image is stored in the recipient cache 312-2, the cloud streaming server 110 transmits the cache index to the terminal 120-2, (120-2).

The terminal 120-2 searches the recipient cache 320-2 using the received cache index, loads the stored image, and generates a cloud streaming screen using the loaded image.

For example, when a cloud streaming service is provided to the terminal 120-3 by executing an application, when the captured image is stored in the recipient cache 312-3, the cloud streaming server 110 transmits the cache index to the terminal 120-3, (120-3).

The terminal 120-3 searches the recipient cache 320-3 using the received cache index, loads the stored image, and generates a cloud streaming screen using the loaded image.

Each of the non-persistent caches 312-1, 312-2 ... 312-N, 320-1, 320-2, ..., 320-N may be updated on a first-in first- .

The global cache 311 can be utilized when a screen that is encoded once by a plurality of terminals connected to the cloud streaming server 110 at the same time is used for another terminal. At this time, the global cache 311 may store data that is excluded from the recursive caches 312-1, 312-2, ..., 312-N by updating within the memory size limit.

The global data list stored in the global cache 311 may be generated in a table format including the global data reference count and the URL of the global data.

At this time, the global data reference count may correspond to the number of times the global data reference count is used and referred to by the plurality of terminals driven in the cloud streaming server. That is, when the terminal refers to the global data corresponding to the number 1 of the table, the global data reference count can be modified from 15 to 16.

Such a global data reference count may be used to manage the storage space of the global cache. For example, since the global cache storage space is not infinite, it may be difficult to store all the global data provided by the application.

Accordingly, the global data frequently referred to and used by a plurality of terminals are left in the global cache based on the global data reference count, and the global cache having a relatively small number of times of being used is deleted to effectively use the global cache.

At this time, in addition to the case where the storage space of the global cache is insufficient, the global data whose global data reference count is equal to or less than a preset reference count is deleted at a predetermined interval in the global cache, thereby improving the data providing speed through the global cache in executing the cloud streaming service It is possible. That is, when the global data corresponding to the data requested from the user terminal is checked in the global cache, the search can be performed through the URL of the data. If too much global data is stored in the global cache, There is a possibility that the overall service speed is lowered.

Therefore, when the global data is retrieved from the global cache, the data storage amount enough to affect the speed is set. If the global data is stored in the global cache beyond the storage amount, the global cache is managed by deleting the less- can do.

Also, the global data list can manage the storage space in which the global data included in the global cache is stored in the ascending order of the global data reference count. For example, it can be assumed that the global cache consists of an L1 cache, an L2 cache, and an L3 cache. At this time, if the processing speed of the L1 cache is the fastest, then the L2 cache, and finally the processing speed of the L3 cache is the slowest, the global data with the highest global data reference count, that is, the most used global data from the terminals It can be stored in the L1 cache.

At this time, the global data reference count which is a reference for each cache can be set, and the global data corresponding to the reference global data reference count can be moved to the corresponding cache. For example, assuming that the reference of the global data reference count of the L1 cache is 100, the reference of the global data reference count of the L2 cache is 50, and the reference of the global data reference count of the L3 cache is 10, The global data corresponding to 70 times can be stored in the L2 cache. Also, if the global data reference count of the global data stored in the L2 cache is gradually accumulated and exceeds 100 times, the corresponding global data may be moved from the L2 cache to the L1 cache.

In addition, the global data list may correspond to a list of data that can be stored in the global cache in a concept corresponding to a white list. Therefore, the URL information of the data can be used to check whether the specific data is global data that can be stored in the global cache.

4 is a diagram illustrating a process of capturing an execution result of an application according to an embodiment of the present invention.

Referring to FIG. 4, in the process of capturing an application execution result according to an embodiment of the present invention, an interval in which a change between frames is small among a plurality of frames corresponding to an application execution result may be detected.

In this case, since the image-based cloud streaming service captures and transmits the image of the changed portion in the static service screen, it is possible to detect a portion of the execution result of the acquired application that is little changed, Should be extracted.

Thereafter, the changed portions can be detected by comparing the frames corresponding to the portions with little change. For example, if the previous frame 410 of FIG. 4 and the current frame 420 are compared, then the A region of the previous frame 410 does not change in the current frame 420, B ', respectively.

Accordingly, in the previous frame 410 and the current frame 420, a portion corresponding to B 'can be extracted to the capture image 430 to perform capturing.

If there are a plurality of changed portions, the plurality of changed images may be respectively extracted into the captured image 430, and each of the captured images may be captured. Alternatively, one new region including all the changed images may be created, and the generated new region may be extracted to the capture image 430 to perform capturing.

5 is a block diagram showing an example of a terminal shown in FIG.

Referring to FIG. 5, the terminal illustrated in FIG. 1 includes a communication unit 510, an execution unit 520, a reciprocal cache 530, and a storage unit 540.

The communication unit 510 may perform a function of transmitting and receiving data to and from the cloud streaming server 110 and the network 130. Here, the communication unit 510 may include RF transmitting means for up-converting and amplifying the frequency of the transmitted signal, RF receiving means for performing low-noise amplification on the received signal and down-converting the frequency. The communication unit 510 may include at least one of a wireless communication module and a wired communication module. When the communication unit 510 uses wireless communication, the wireless communication module uses any one of a wireless network communication module, a wireless LAN communication module, and a wireless fan communication module to transmit / receive data according to a wireless communication method Data can be transmitted and received to the cloud streaming server 110. The wired communication module is for transmitting / receiving data by wire. The wired communication module may connect to the network 130 through a wired network and transmit / receive data to / from the cloud streaming server 110. That is, the communication unit 510 can access the network 130 using a wireless communication module or a wired communication module, and can exchange data with the cloud streaming server 110 through the network 130. In particular, the network 130 according to the present invention can communicate with the cloud streaming server 110 or another terminal to transmit / receive data necessary for uploading or downloading content based on cloud computing.

The execution unit 520 may be an operating system (OS) and a process unit for driving each configuration. For example, the execution unit 520 may perform the entire process of connecting to the cloud streaming server 110 When connecting to the cloud streaming server 110 through a separate service application, the entire process of executing the service application can be controlled according to the request of the user, and at the same time, the cloud streaming server 110 The service use request can be controlled to be transmitted, and at this time, the information necessary for user authentication can be controlled to be transmitted together.

In addition, the execution unit 520 may execute the specific content stored in the storage unit 540 according to the request of the user. At this time, the execution unit 520 may store the content usage history according to the execution of the content as the content usage information.

The execution unit 520 also transmits the execution data for executing the content, the content information including the attribute information about the content, and the content usage information, which is information according to the content usage history, to the cloud streaming server 110 Can be uploaded. The execution unit 520 may delete the uploaded content from the storage unit of the terminals 120-1 to 120-N according to a request from the user and send the uploaded content to the cloud streaming server 110 And the content may be executed through the cloud streaming server 110 and used.

The execution unit 520 may access the cloud streaming server 110 to download the content from another terminal and store the downloaded content in the storage unit 540. When executing the content through the cloud streaming server 110, It is possible to control the content to be executed after receiving only data necessary for execution.

The storage unit 540 is a device for storing data, and includes a main storage device and an auxiliary storage device, and may store an application program required for functional operation of the terminals 120-1 to 120-N. The storage unit 540 may include a program area and a data area. Here, if the terminal activates each function corresponding to a request of the user, the terminal executes the corresponding application programs under the control of the execution unit 520 to provide each function. In particular, the storage unit 540 according to the present invention may store an operating system for booting the terminal, a program for uploading or downloading content based on cloud computing, and the like. In addition, the storage unit 540 may store a content DB storing a plurality of contents and information of the terminal. At this time, the content DB includes execution data for executing the content and attribute information about the content, and content usage information according to the execution of the content can be stored. The terminal information may include terminal specification information.

The recursive cache 530 corresponds to the reciprocal cache of the cloud streaming server as described above. When the cache index is transmitted from the cloud streaming server, the recipient cache 530 is utilized to generate an image using the received cache index.

6 is a flowchart illustrating an image cloud streaming service method according to an exemplary embodiment of the present invention.

Referring to FIG. 6, an image cloud streaming service method according to an exemplary embodiment of the present invention stores reciprocal data in reciprocal caches corresponding to a plurality of terminals, respectively (S610).

A non-persistent cache is provided in each of a plurality of terminals, and a cryptographic cache corresponding to a plurality of terminals is provided in a cloud streaming server.

That is, the cloud streaming server optimizes the data transmission between the server and the terminal by providing a recursive cache modeling the cache of the terminal (the reciprocal cache).

The cloud streaming server can reduce the server load and network traffic load by skipping the still image encoding and sending procedure by sending only the cache index when the captured image is stored in the reciprocal cache have.

At this time, the history information may include information for composing a screen such as a menu, an ID, and a focus position.

At this time, the reciprocal data stored in the recipient cache may be retrieved via the URL. You can also manage the recipient cache by creating a reference count on the reciprocal data.

In addition, the image cloud streaming service method according to an embodiment of the present invention stores global data in a global cache shared by the terminals (S612).

At this time, the global cache is a cache shared by the UEs, and stores global data so that an encoded image for one UE can be used in another UE.

At this time, the global cache can store data that is excluded from the recursive cache by the update within the memory size limit.

In addition, the image cloud streaming service method according to an embodiment of the present invention executes an application corresponding to one of the terminals (S613).

At this time, in step S613, an application can be executed by accessing a web server with a browser corresponding to the terminal among a plurality of browsers.

In addition, the image cloud streaming service method according to an embodiment of the present invention captures a transmission object screen corresponding to the execution result of the application and generates a capture image (S620).

Step S620 may be performed by the streaming unit. The streaming unit delivers the execution result of the application to the user terminal through a pipeline process corresponding to rendering, capturing, still image encoding and sending.

In addition, the image cloud streaming service method according to an embodiment of the present invention determines whether the captured image is stored in a reciprocal cache corresponding to one of the terminals (S630).

If it is determined in step S630 that the mobile terminal is stored in the reciprocal cache corresponding to one of the terminals, the cloud streaming service method according to an embodiment of the present invention transmits the cache index in operation S660.

In this case, step S660 may transmit the cache index by omitting the still image encoding and sending when transmitting the cache index.

If it is determined in step S630 that the content is not stored in the reciprocal cache corresponding to one of the terminals, the cloud streaming service method according to one embodiment of the present invention determines whether the captured image is stored in the global cache (S640).

The determination in steps S630 and S640 may be performed in consideration of at least one of the position, the area, the data size, and the history information of the captured region.

If it is determined in step S640 that the image data is stored in the global cache, the image cloud streaming service method according to an exemplary embodiment of the present invention transmits the stored image to one of the terminals in operation S670.

If it is determined in step S640 that the image data is not stored in the global cache, the image cloud streaming service method according to an exemplary embodiment of the present invention performs still image encoding and sending (S650).

Although not explicitly shown in FIG. 6, the image cloud streaming service method according to an embodiment of the present invention can store various information generated in a cloud streaming service process.

According to an embodiment, the storage module for storing information may be configured independently of the cloud streaming server to support a function for the cloud streaming service. At this time, the storage module may operate as a separate mass storage and may include control functions for performing operations.

Through this image cloud streaming service method, the server and the client can share the cache in the image based cloud streaming service, thereby providing the cloud streaming service with the optimum efficiency.

Also, although not shown in FIG. 6, the image cloud streaming service method according to an embodiment of the present invention transmits and receives information related to a plurality of terminals through a communication network such as the network shown in FIG. In particular, a request for the cloud streaming service can be received from the terminal and the execution result of the application corresponding to the cloud streaming service requested by the terminal can be provided to the terminal.

In addition, the cloud streaming server operated as described above may be implemented as one or more servers.

On the other hand, the cloud streaming server can store information in the device by loading the memory. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in other embodiments, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may comprise, for example, a hard disk device, an optical disk device, or any other mass storage device.

7 is an operation flowchart illustrating a process of updating a cache according to an image cloud streaming service method according to an embodiment of the present invention.

Referring to FIG. 7, an image cloud streaming service method according to an embodiment of the present invention captures a transmission object screen corresponding to an application execution result to generate a capture image (S710).

Step S710 may be performed by the streaming unit. The streaming unit delivers the execution result of the application to the user terminal through a pipeline process corresponding to rendering, capturing, still image encoding and sending.

In addition, the image cloud streaming service method according to an embodiment of the present invention determines whether a captured image exists in the cache (S720).

At this time, the cache may be a non-volatile cache or a global cache.

That is, the step S720 determines whether the captured image is stored in the reciprocal cache corresponding to the terminal or is stored in the global cache.

If it is determined to exist in the cache as a result of the determination in step S720, the image cloud streaming service method according to an exemplary embodiment of the present invention transmits the cache index (if stored in the non-recursive cache) or transmits the stored image If stored in the global cache).

In this case, the transmission efficiency in the case of transmitting only the cache index is higher than that in the case of transmitting the stored image.

If it is determined in step S720 that the image cloud streaming service does not exist in the cache, the image cloud streaming service method according to an embodiment of the present invention performs still image encoding on the captured image and then transmits the captured image to the terminal in operation S730.

If it is determined in step S720 that there is no cache, the image cloud streaming service method according to an embodiment of the present invention stores the encoded image in the reciprocal cache corresponding to the corresponding terminal (S740 ).

At this time, the non-volatile cache can be updated by first-in-first-out (FIFO).

In addition, the image cloud streaming service method according to an exemplary embodiment of the present invention determines whether data is deleted from the recipient cache by storing S740 in operation S750.

That is, since the reciprocal cache has a limited memory space, if the new data is stored while the data is full, the first stored data is deleted, and the step S750 detects such deletion .

If it is not detected that deletion has occurred as a result of the determination in step S750, the image cloud streaming service method according to an exemplary embodiment of the present invention terminates the process and performs processing for the next captured image.

If it is detected in step S750 that deletion has occurred, the image cloud streaming service method according to an embodiment of the present invention determines whether there is free space in the global cache (S760).

If it is determined in step S760 that there is no free space in the global cache, the image cloud streaming service method according to an exemplary embodiment of the present invention terminates the process and performs processing on the next captured image.

If it is determined in step S760 that there is free space in the global cache, the image cloud streaming service method according to an embodiment of the present invention stores the deleted data in the global cache (S770).

The image cloud streaming service method according to the present invention can be implemented in the form of a program command that can be executed through various computer means and recorded in a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Includes all types of hardware devices that are specially configured to store and execute magneto-optical media and program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the cloud streaming service system, the image cloud streaming service method using the recursive cache, and the apparatus therefor are not limited to the configuration and method of the embodiments described above, The embodiments may be configured so that all or some of the embodiments may be selectively combined so that various modifications may be made.

According to the present invention, there is provided a reciprocal cache in which a server and a terminal are associated with each other, so that it is possible to transmit only the cache index without redundant image encoding and data transmission, thereby reducing the load on the server in the case of an image- based cloud streaming service, .

110: Cloud streaming servers 120-1 to 120-N:
130: Web server 140: Network
210: communication unit 220: global cache
230: Reciprocal Cache 240: Execution Unit
250: streaming unit 260: storage unit

Claims

1. A cloud streaming server for providing an image-based cloud streaming service,
Reciprocal caches corresponding to each of a plurality of terminals, respectively;
An execution unit for executing an application corresponding to one of the terminals; And
Transfers the execution result of the application to one of the terminals through a pipeline process corresponding to rendering, capturing, still image encoding and sending, and if the captured image is stored in a reciprocal cache corresponding to one of the terminals, A streaming unit
Lt; / RTI >
The streaming unit includes:
Wherein when the cache index is transmitted, the still image encoding and sending are skipped and the cache index is transmitted.

delete

The method according to claim 1,
The cloud streaming server
Wherein the streaming unit transmits the stored image to one of the terminals when the captured image is not stored in the recipient caches but is stored in the global cache. server.

◈ Claim 4 is abandoned due to the registration fee.

The method of claim 3,
The streaming unit
And determines whether the captured image is stored in the recipient caches or the global cache in consideration of at least one of the position, the width, the data size, and the history information of the captured region. A cloud streaming server.

◈ Claim 5 is abandoned due to the registration fee.

The method of claim 3,
The recipient caches
(FIFO) -based cloud streaming server.

◈ Claim 6 is abandoned due to the registration fee.

The method of claim 5,
The global cache
And stores data that is excluded from the recipient caches by an update within a memory size limit.

A method of providing an image cloud streaming service by a cloud streaming server providing an image-based cloud streaming service,
Storing reciprocal data in recipient caches corresponding to each of a plurality of terminals, respectively;
Executing an application corresponding to one of the terminals;
Capturing a transmission object screen corresponding to an execution result of the application and generating a captured image; And
Transmitting the cache index if the captured image is stored in a reciprocal cache corresponding to one of the terminals
Lt; / RTI >
Wherein the transmitting the cache index comprises:
And skipping still image encoding and sending for the captured image and transmitting the cache index.

◈ Claim 8 is abandoned due to the registration fee.

The method of claim 7,
Further comprising the step of storing global data in a global cache shared by the terminals.

◈ Claim 9 is abandoned upon payment of registration fee.

The method of claim 8,
Further comprising transmitting the stored image to one of the terminals if the captured image is not stored in the recipient cache and stored in the global cache. &Lt; RTI ID = 0.0 > 31. < / RTI >

◈ Claim 10 is abandoned due to the registration fee.

The method of claim 8,
Wherein the recipient caches are updated on a first-in first-out (FIFO) basis, and the global cache stores the reciprocal data that is excluded from the recipient caches by an update within a memory size limit Image cloud streaming service method.