KR20160110844A - Method for providing of cloud streaming service, and apparatus thereof - Google Patents

Abstract

The present invention relates to a method for providing cloud streaming service and devices therefor, and more particularly to a method for providing cloud streaming service and devices therefor, which can provide information about an address, at which a video or image object in a content execution screen has been stored, and image data, in which a region exclusive of the video or image object has been rendered and encoded, to a user device. For this purpose, a cloud streaming server according to an embodiment of the present invention processes a region, on which a video or image object is displayed, in a content execution screen into white or black color having a small data amount, renders and encodes the content execution screen, and provides the rendered, encoded execution screen, together with information about an address at which the video or image object has been stored, to a user device. The user device receives the video or image object from a resource server based on the received information about the address, performs overlay processing by inserting the received video or image object into the received rendered, encoded execution screen, and then transmits a final output screen to an output device. Accordingly, PIP requirements and full browsing are not required, and the amount of data can be minimized, thereby providing an economical effect.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a cloud streaming service providing method,

The present invention relates to a method and a device for providing a cloud streaming service and, when content is provided using cloud streaming, a video object or an execution image other than an area in which an image object is displayed is encoded in a content execution screen, The image object or the video object is directly received from the resource server based on the address information received from the user device and is inserted into a region where the video or image object is displayed, A method for providing a streaming service, and an apparatus therefor.

The contents described in this section merely provide background information on the present embodiment and do not constitute the prior art.

With the advancement of computer networks, existing computing environments that rely on the independent hardware performance of each terminal evolve into a form of cloud computing that provides the services according to the request of terminals by utilizing all the computing resources on the network. .

Cloud computing can be defined as an on demand outsourcing service of computing resources through an information communication network such as the Internet. In a cloud computing environment, service providers integrate distributed data centers with virtualization technologies to provide services to users. Service users do not install and use computing resources such as application, storage, OS, security, etc. in the terminals owned by each user, but instead use services in the virtual space created through virtualization technology At a desired point in time. In other words, cull computing is a computing service in which computing resources such as hardware / software existing in an intangible form, such as a cloud, is borrowed and paid for as much as it is needed, Refers to a technology that integrates computing resources existing in a location with virtualization technology.

According to such cloud computing, a user accesses a cloud network through a terminal that performs only network connection and basic computation functions, performs tasks requiring a large-capacity storage device, a high-performance computing resource, and receives various services.

The cloud streaming service is a service based on such a cloud computing environment. It executes content on a server side by utilizing server resources and captures and encodes the execution screen as a GUI screen and transmits the execution screen to the terminal in real time And only the input and output are performed on the terminal side, thereby making it possible for the user to use the application or the contents requiring high performance computing resources regardless of the terminal specifications.

In the current cloud streaming service, content such as an application is executed on the server side, the execution screen is rendered in a GUI (Graphic User Interface) screen and transmitted in real time, and input and output are performed on the terminal side. However, the conventional cloud streaming service encodes the GUI screen rendered in the cloud streaming server and transmits the encoded data, so that only the remaining area excluding the video object or the image object in the execution screen is rendered, and the video object or the image object There is a problem that it is impossible to directly receive the stored address information and receive it directly from the user device.

Korean Patent No. 10-0925042 filed on Jul. 28, 2009 (name: VOD information providing method of IPTV)

It is an object of the present invention to provide a method and apparatus for processing a video object or an image object into a white or black color having a small data capacity, And provides the video object or the image object together with the address information of the image object to the user device. The user device receives the video object or the image object directly from the resource server through the received address information and inserts the video object or the image object into the execution screen, The present invention provides a cloud service providing method and apparatus for the same that can reduce overhead of a server and perform overlay processing more conveniently.

However, the object of the present invention is not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood from the following description.

In order to achieve the above object, a cloud streaming server according to an exemplary embodiment of the present invention includes a screen for generating first image data by encoding a video object or a remaining region excluding an image object on an execution screen of a content requested by a user apparatus, Processing module; A content transmission module for providing the user device with address information in which the first image data and the video object or the image image generated by the screen processing module are stored; As shown in FIG.

At this time, the screen processing module may encode the execution screen by processing the area in which the video object or the image object is displayed in white or black, according to the coding scheme of the first image data.

The cloud streaming server may further include a content execution module for executing the content.

The cloud streaming server may further include a parsing module for parsing the content and generating address information in which the video object or the image object is stored.

At this time, the parsing processing module may further generate tag information including at least one of a position, size, color, shape, and screen output format in which the video object or the image object is displayed on the execution screen And the content transmission module may further transmit the tag information together with the first image data and the address information.

According to an aspect of the present invention, there is provided a user equipment connected to an external server including a cloud streaming server through a communication network, the communication module transmitting and receiving data to and from the external server. The first video data in which the video object or the remaining region excluding the image object is encoded in the execution screen of the content in response to the request for the content, and the first video data in which the video object or the image object is stored A control module for receiving the video object or the image object based on the received address information; A first decoding module for decoding the received first image data; A second decoding module for decoding the received video object or image object; A synthesis module for overlaying the decoded first video data and the decoded video object or image object to generate a final output screen; . ≪ / RTI >

According to an aspect of the present invention, there is provided a cloud streaming service method, wherein a cloud streaming server encodes a remaining region excluding a video object or an image object from an execution screen of a content requested by a user apparatus, Generating image data; The cloud streaming server parsing the content to generate address information in which a video object or an image object is stored; And the cloud streaming server providing address information of the first image data and the video object or image object to the user device; . ≪ / RTI >

The generating of the first data may encode the execution image by processing the video object or the region in which the image object is displayed in white or black according to the coding scheme of the first image data.

The step of generating address information may further include parsing the content to generate tag information including at least one of a position, size, color, shape, and screen output format of the video object or the image object displayed on the execution screen And providing the tag information to the user device may further transmit the tag information together with the first image data and the address information.

According to an aspect of the present invention, there is provided a method of providing a service in a user device, the method comprising: requesting a content to a cloud streaming server; Receiving, from the cloud streaming server, first image data obtained by encoding a video object or an area other than an area in which an image object is displayed in an execution screen of the content, and address information in which the video object or the image object is stored; Receiving a video object or an image object based on the received address information; Decoding the received first image data; Decoding the received video object or image object; And overlaying the decoded first video data and the decoded video object or image object to generate a final output screen; . ≪ / RTI >

According to the method and apparatus for providing a cloud streaming service of the present invention, only a remaining area excluding a video object or an image object in an execution screen of a content is rendered and encoded in the cloud streaming server and transmitted together with address information in which a video object or an image object is stored The user device directly receives a video object or an image object directly from the resource server based on the address information, inserts the video object or the image object into the rendered and encoded data, and outputs the inserted video object or image object, thereby reducing the load on the cloud streaming server have.

In addition, the cloud streaming server of the present invention can reduce the data capacity by processing a video object or an image object in the execution screen of the content as white or black.

In addition, since the rendering and encoding data and the video object or the image object are overlaid and synthesized by the user device of the present invention, there is no PIP (Picture in Picture) requirement and it is economical compared to the existing method since full browsing is not required.

1 is a diagram illustrating a schematic configuration of a cloud streaming service system according to an embodiment of the present invention.
2 is a diagram illustrating a main configuration of a cloud streaming server according to an embodiment of the present invention shown in FIG.
FIG. 3 is a diagram illustrating a main configuration of a user apparatus according to an embodiment of the present invention shown in FIG. 1. Referring to FIG.
4 is a flowchart illustrating a cloud streaming service method according to an embodiment of the present invention.
5 is a flowchart illustrating a service method in a user equipment according to an embodiment of the present invention.
6 is a diagram illustrating a procedure of a cloud streaming service according to an embodiment of the present invention.
FIG. 7A is an exemplary diagram illustrating a cloud streaming service method according to an embodiment of the present invention. Referring to FIG.
7B is an exemplary diagram illustrating a cloud streaming service method according to an embodiment of the present invention.
8 is a diagram illustrating an operating environment of an apparatus for providing a cloud streaming service according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS For a more complete understanding of the nature and advantages of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings, in which:

In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted.

It is to be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The terms and words used in the following description and drawings are not to be construed in an ordinary sense or a dictionary, and the inventor can properly define his or her invention as a concept of a term to be described in the best way It should be construed as meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

Also, terms including ordinal numbers such as first, second, etc. are used to describe various elements, and are used only for the purpose of distinguishing one element from another, Not used. For example, without departing from the scope of the present invention, the second component may be referred to as a first component, and similarly, the first component may also be referred to as a second component.

In addition, when referring to an element as being "connected" or "connected" to another element, it means that it can be connected or connected logically or physically. In other words, it is to be understood that although an element may be directly connected or connected to another element, there may be other elements in between, or indirectly connected or connected.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise.

It is also to be understood that the terms such as " comprising "or" having ", as used herein, are intended to specify the presence of stated features, integers, It should be understood that the foregoing does not preclude the presence or addition of other features, numbers, steps, operations, elements, parts, or combinations thereof.

In addition, embodiments within the scope of the present invention include computer-readable media having computer-executable instructions or data structures stored on computer-readable media. Such computer-readable media can be any available media that is accessible by a general purpose or special purpose computer system. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or in the form of computer- But is not limited to, a physical storage medium such as any other medium that can be used to store or communicate certain program code means of the general purpose or special purpose computer system, .

In the following description and claims, the term "network" is defined as one or more data links that enable electronic data to be transmitted between computer systems and / or modules. When the information is transmitted or provided to a computer system via a network or other (wired, wireless, or a combination of wired or wireless) communication connection, the connection may be understood as a computer-readable medium.

Computer readable instructions include, for example, instructions and data that cause a general purpose computer system or special purpose computer system to perform a particular function or group of functions. The computer executable instructions may be binary, intermediate format instructions, such as, for example, assembly language, or even source code.

In addition, the invention may be practiced with other computer systems, including personal computers, laptop computers, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, a pager, and the like. < RTI ID = 0.0 > [0040] < / RTI > The invention may also be practiced in distributed systems environments where both local and remote computer systems linked by a combination of wired data links, wireless data links, or wired and wireless data links over a network perform tasks. In a distributed system environment, program modules may be located in local and remote memory storage devices.

Prior to the description, the present invention relates to a cloud streaming service that allows a user to use a predetermined service or function regardless of the performance of a terminal through screen virtualization based on a cloud computing environment.

The data described in the specification of the present invention basically means a web application for a UI (User Interface). The web application at this time is composed of a plurality of web pages and can be captured on a browser basis.

However, the data described in the present invention is not limited to the UI web application, and may include various types of content data such as real-time broadcasting, VOD service, and game service, or may be a set of a plurality of data.

The data according to the embodiment of the present invention can be applied to the data of the present invention as long as it can be provided to the client device according to the cloud streaming service method.

Now, a method and a device for providing a cloud streaming service according to an embodiment of the present invention will be described in detail with reference to the drawings.

First, the overall structure of a cloud streaming service system according to an embodiment of the present invention will be schematically described.

FIG. 1 is a diagram for schematically explaining a structure of a cloud streaming service system according to an embodiment of the present invention.

1, a cloud streaming system 1000 according to an exemplary embodiment of the present invention includes a cloud streaming server 100, a user device 200, an output device 400 for receiving and outputting data from the user device 200, A resource server 300, and a communication network 500 that supports transmission and reception of information between the user device 200 and an external server including the cloud streaming server 100.

First of all, the user device 200 of the present invention refers to a user device capable of transmitting and receiving various data via the communication network 500 according to a user's operation.

The user device 200 may perform voice or data communication through the communication network 500 and may access the cloud streaming server 100 to receive various information provided by the cloud streaming server 100 in a cloud streaming service mode And transmit the user's input to the cloud streaming server 100.

The user device 200 of the present invention may include a browser for transmitting and receiving information, a memory for storing a program and a protocol, and a microprocessor for executing and controlling various programs.

In particular, the user device 200 according to an embodiment of the present invention accesses the cloud streaming server 100 via the communication network 500 and requests execution of specific data among a plurality of data provided by the cloud streaming server 100 And receives result information according to data execution from the cloud streaming server 100 in a streaming manner, so that the user of the user device 200 can utilize the data.

At this time, the execution of the actual data is executed in the cloud streaming server 100 located at a remote location of the user device 200, but the user of the user device 200 can recognize that the corresponding data is executed in the user device 200 The application is equally available. The data at this time may be a concept including a UI application such as a menu application.

The user apparatus 200 of the present invention can basically have a function of receiving and decoding encoded streaming data, and may further include a function of rendering and outputting decoded video data.

The user device 200 according to the embodiment of the present invention may be implemented in various forms. For example, the user device 200 described herein may be a mobile terminal such as a smart phone, a tablet PC, a PDA (personal digital assistant), a portable multimedia player (PMP) Of course, fixed terminals such as smart TVs, desktop computers, set-top boxes, etc. may be used.

In addition, the user device 200 according to the embodiment of the present invention can be divided into a device having an output function and a device having no output function, and the device having no output function can be divided into a TV, a monitor, It is possible to reproduce and output data received from the cloud streaming server 100 through the output device 400 in a streaming manner by transmitting data to the output device 400 by being connected to the same output device 400 .

The user device 200 according to the embodiment of the present invention may include an image processing unit for generating a content execution screen for executing a cloud streaming service from the cloud streaming server 100 by using image data processed in accordance with an embodiment of the present invention, Related data including at least one of information and tag information, and directly request or receive a video object or an image object to be displayed on the content execution screen to the resource server 300 based on the data.

In addition, the user device 200 may perform the overlay process by inserting the received video object or image object into the image data received from the cloud streaming server 100.

At this time, the combination of the image data and the video object or the image object may be a method of combining not only overlay processing but also combining two image data, and may include various synthesis methods and techniques.

The user device 200 according to the present invention can not enumerate all of the variations of the portable device according to the convergence trend of the digital device. However, since the units equivalent to the above- A device that can be used as the user device 200 and transmits and receives information to and from the cloud streaming server 100 via the communication network 500 and can decode and reproduce encoded data transmitted in a streaming manner from the cloud streaming server 100 , Any device may be used as the user device 200 according to an embodiment of the present invention.

The cloud streaming server 100 provides a cloud streaming service according to an embodiment of the present invention and provides the service or function requested by the user device 200 to the user device 200 through the screen virtualization function . In this case, the screen virtualization is implemented by the server side connected to the communication network 500, that is, the cloud streaming server 100, and the execution screen of specific data as implemented in the user device 200 is provided to the user device 200 it means.

For this purpose, the cloud streaming server 100 executes various data such as a UI application, a game application, a movie viewing application, etc., which is captured and used by the cloud streaming server 100, And transmits the encoded data to the user device 200 in a streaming manner. At this time, the cloud streaming server 100 drives the data according to a user request signal transmitted from the user device 200, and transmits the changed execution screen to the user device 200 by the streaming method, The user of the user device 200 may be provided with the same user experience as executing the data directly at his or her user device 200. [

Particularly, the cloud streaming server 100 according to the embodiment of the present invention executes content according to a request signal of the user device 200, and renders the remaining area except the screen on which the video object or the image object is displayed , The first video data can be generated by processing the video object or the area in which the video object is displayed as white or black having the smallest data capacity according to a coding scheme.

In addition, the cloud streaming server 100 parses the content executed according to a request from the user device 200, and stores the address information in which the video object or the image object is stored, and the state in which the video object or the image object is displayed on the execution screen of the content Can be generated.

At this time, the address information may be URL (Uniform Resource Locator) information indicating both a website address and resources on a computer network.

The cloud streaming server 100 provides the first image data, the address information, and the tag information to the user device 200, thereby making it possible to support the cloud streaming service which is more economical and has less server load.

The resource server 300 according to the embodiment of the present invention can receive various data by receiving a data request signal from the cloud streaming server 100 and the user device 200. In particular, (Web Application Server) that can provide a set of web applications. The resource server 300 may be an IPTV head-end system or a cable TV QAM transmitter. In this case, when the resource server 300 is connected to the cloud streaming server 100, May transmit data to the cloud streaming server 100 and support the cloud streaming service of the data provided by the cloud streaming server 100.

In particular, in the present invention, the resource server 300 receives data corresponding to the address information based on the address information stored in the video object or the audio object from the user device 200, Data can be transmitted.

The output device 400 according to the embodiment of the present invention displays information on a series of operation states and operation results generated during the execution of the functions of the user device 200. [ In addition, the output device 400 may display a menu of the user device 200 and user data input by the user. Here, the output device 400 may be an LCD (Liquid Crystal Display), a TFT-LCD (Thin Film Transistor LCD), an OLED (Organic Light Emitting Diodes), an LED, an AMOLED (Actice Matrix Organic LED) A flexible display and a three-dimensional display. When the output device 400 is formed as a touch screen, the output device 400 may be connected to an input device (not shown) Some or all of the functions can be performed.

Particularly, the output apparatus 400 according to the embodiment of the present invention can receive and output the decoded video data from the user apparatus 200. The user device 200 decodes the first image data received from the cloud streaming server 100 and the video object or the image object received from the resource server 300, And output it via the device 400. [

At this time, the output apparatus 400 may be mounted inside the user apparatus 200, and if not installed therein, the user apparatus 200 may be connected to the external apparatus having the output function through the communication network 500 And output the image data.

The communication network 500 according to the embodiment of the present invention may be a communication network using a wireless communication scheme such as a WLAN (Wireless LAN), a Wi-Fi, a Wibro, a WiMAX, and a HSDPA (High Speed Downlink Packet Access) (Fiber to the Curb), FTTH (Fiber To The Home), and the like. However, the present invention is not limited thereto. Or the like may be used.

In addition, the communication network 500 of the present invention includes, for example, a plurality of access networks (not shown) and a core network (not shown), and may include an external network such as an Internet network (not shown). Here, the access network (not shown) is an access network that performs wired / wireless communication with the user equipment 200 and includes a plurality of base stations such as a base station (BS), a base transceiver station (BTS), a NodeB, an eNodeB, (Base Station Controller), and an RNC (Radio Network Controller).

The digital signal processing unit and the radio signal processing unit, which are integrally implemented in the base station, are divided into a digital unit (hereinafter, referred to as a DU) and a radio unit (hereinafter referred to as an RU) A plurality of RUs (not shown) may be installed, and a plurality of RUs (not shown) may be connected to a centralized DU (not shown).

A core network (not shown) constituting a mobile network together with an access network (not shown) performs a role of connecting an access network (not shown) and an external network (not shown), for example, an Internet network (not shown).

As described above, the core network (not shown) is a network system that performs main functions for mobile communication services such as mobility control and switching between access networks (not shown), and includes a circuit switching or a packet exchange and manages and controls the packet flow in the mobile network. The core network (not shown) manages inter-frequency mobility and plays a role for interworking with traffic in an access network (not shown) and a core network (not shown) and other networks such as the Internet It is possible. Such a core network (not shown) may further include an SGW (Serving Gate Way), a PGW (PDN GateWay), an MSC (Mobile Switching Center), a HLR (Home Location Register), a MME (Mobile Mobility Entity) .

The Internet network (not shown) refers to a public network, that is, a public network, in which information is exchanged according to the TCP / IP protocol. The Internet network is connected to the cloud streaming server 100 and the resource server 300, The information provided from the streaming server 100 and the resource server 300 may be provided to the user device 200 through a core network (not shown) and an access network (not shown) To the cloud streaming server 100 and the resource server 300 through an access network (not shown) and a core network (not shown). However, the present invention is not limited thereto, and the cloud streaming server 100 and the resource server 300 may be integrated with a core network (not shown).

In addition to the above-described communication methods, other widely known or later-developed communication methods may be included.

2 is a diagram illustrating a main configuration of a cloud streaming server according to an embodiment of the present invention.

2, the cloud streaming server 100 of the present invention may include a screen processing module 101, a content transmission module 102, a content execution module 103, and a parsing processing module 104. [

More specifically, the screen processing module 101 of the cloud-streaming server 100, in response to a request from the user apparatus 200, selects, from among the execution screens of the content executed by the command of the content execution module 103, Alternatively, the first image data may be generated by rendering and encoding the remaining region except for the image object. At this time, the first image data may include both a still image and a moving image.

In addition, the screen processing module 101 may process the area in which the video object or the image object is displayed in white or black according to the coding scheme of the first image data.

The coding method is a method of compressing and storing an image, that is, an image formatting method. Typically, JPEG (Joint Photographic Experts Group, JPG), Graphics Interchange Format (GIF), Portable Network Graphics (PNG) And may be the above-mentioned coding method as long as it is an image storing type.

In the image formatting method, the capacity of the image file may increase together with the number of kinds of colors used in the image file. Further, in the image formatting method, the capacity of the image file may be smaller when the image file has a monochrome color. Also, the capacity of the image file may be different for each color.

That is, according to the coding scheme, an area for displaying the video object or the image object can be processed into black or white which can store the data size the smallest. The first image data may be generated by rendering and encoding the remaining region except for the region together with the processed region.

The content execution module 103 of the cloud streaming apparatus 100 according to the embodiment of the present invention may execute the content at the request of the user device 200. [

For example, the content execution module 103 may be an HTML 5 browser, and may be various versions of other browsers.

At this time, the parsing module 104 of the cloud streaming apparatus 100 can parse the content. The parsing is a technique for extracting specific data desired by a user. The parsing processing module 104 parses the content and stores address information in which a video object or an image object to be displayed on the content execution screen is stored Can be generated.

The parsing module 104 may further generate tag information including at least one of a position, a size, a color, a shape, and a screen output format in which the video object or the image object is displayed by parsing the content.

The content transmission module 102 of the cloud streaming server 100 according to the present invention receives the content request signal from the user device 200 through the communication network 500 and the content execution module 103 receives the content request signal from the user device 200, The first video data generated by the screen processing module 101, the address information stored in the video object or the image object generated by the parsing processing module 104, the video object or the video object, Information including at least one of the tag information indicating how the image object is displayed on the execution screen can be transmitted to the user device 200.

In addition, in the cloud streaming server 100 according to the embodiment of the present invention, the tag information and the address information generated by the parsing processing module 104 are one of languages for specifying the structure of data using tags and the like It can be described as an attribute of the tag through a markup language. Typically, the markup language includes Hypertext Markup Language (HTML), Extensible Hypertext Markup Language (XHTML), Extensible Markup Language (XML), and Javascript Object Notation (JSON) Tag information can be provided to the user device 200.

In addition, the first image data may be converted into a string format, inserted into any one of the markup formats, and provided to the user device 200.

The cloud streaming server 100 according to the embodiment of the present invention has the same hardware configuration as that of a typical web server or a network server. However, the software includes a program module implemented through a language such as C, C ++, Java, Visual Basic, Visual C, or the like.

As described above, the cloud streaming server 100 may be implemented in the form of a web server or a network server. The web server is generally connected to an unspecified number of clients and / or other servers through an open computer network such as the Internet, (Web server program) installed in the computer system for receiving a request to perform a work of a client or another web server and deriving a result of the work for the work. However, in addition to the above-described web server program, it should be understood as a broad concept including a series of application programs running on the web server and various databases built up in some cases. Such a cloud streaming server 100 is a web server program that is variously provided according to operating systems such as DOS, Windows, Linux, UNIX, and Macintosh to general server hardware. (Internet Information Server) used in a window environment, CERN, NCSA, and APPACH used in a UNIX environment can be used as typical examples. In addition, the cloud streaming server 100 can classify service subscription information and store and manage it in a member database, which can be implemented inside or outside the cloud streaming server 100.

In addition, a cloud streaming server 100 according to an embodiment of the present invention may be implemented as a virtual machine (VM) that is independent of a plurality of users on a same hardware platform connected so that one supercomputer or several computers function as one computer, Which may be implemented in the form of a distributed computing system.

On the other hand, the memory mounted on the user device 200 or the cloud streaming server 100 stores information in the device. In one implementation, the memory is a computer-readable medium. In one implementation, the memory may be a volatile memory unit, and in other embodiments, the memory may be a non-volatile memory unit. In one implementation, the storage device is a computer-readable medium. In various different implementations, the storage device may include, for example, a hard disk device, an optical disk device, or any other mass storage device.

In addition, the term '~ module' used in the embodiment of the present invention means a software component, and '~ module' performs certain roles. By way of example, '~ module' may include components such as software components, object-oriented software components, class components and task components, and processes, functions, attributes, procedures, Routines, segments of program code, drivers, data, databases, data structures, tables, arrays, and variables. In addition, the functions provided in the components and 'modules' may be combined into a smaller number of components and '~ modules' or further separated into additional components and 'modules'.

Although the present specification and drawings describe exemplary device configurations, the functional operations and subject matter implementations described herein may be embodied in other types of digital electronic circuitry, or alternatively, of the structures disclosed herein and their structural equivalents May be embodied in computer software, firmware, or hardware, including, or in combination with, one or more of the foregoing. Implementations of the subject matter described herein may be embodied in one or more computer program products, i. E. One for computer program instructions encoded on a program storage medium of the type for < RTI ID = 0.0 & And can be implemented as a module as described above. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects the machine readable propagation type signal, or a combination of one or more of the foregoing.

Hereinafter, the main configuration of the user apparatus 200 according to the embodiment of the present invention will be described.

FIG. 3 is a diagram showing a main configuration of a user apparatus 200 according to an embodiment of the present invention shown in FIG.

3, the user device 200 of the present invention includes a communication module 201, a control module 202, a synthesis module 203, a first decoding module 204, and a second decoding module 205 .

The communication module 201 of the user device 200 according to the embodiment of the present invention connects to an external server including the cloud streaming server 100 and the resource server 300 through the communication network 500 to transmit and receive data .

The communication module 201 of the present invention can transmit a content request signal of the user device 200 to the cloud streaming server 100. [

In addition, the communication module 201 may transmit a resource file request signal to the resource server 300 in order to receive a video object or an image object.

The control module 202 of the user device 200 according to the exemplary embodiment of the present invention requests content to the cloud streaming server 100 through the communication module 201 and transmits the content to the cloud streaming server 100 The first video data obtained by encoding the video object or the remaining area excluding the image object and the address information of the video object or the image object may be received from the execution screen of the content.

At this time, the first image data may be processed by the screen processing device 101 of the cloud streaming server 100 such that the video object or the image object is displayed in white or black.

Also, the control module 202 may receive the tag information including at least one of the location, size, color, shape, and screen output format of the video object or the image object displayed on the execution screen from the cloud streaming server 100 You can get more.

In addition, the control module 202 may request a resource file, that is, a video object or an image object, to the resource server 300 based on the received address information.

Then, the communication module 201 can transmit a request signal for requesting the resource file to the resource server 300 through the communication network 500.

The control module 202 transmits the received first image data to a first decoding module 204 for decoding and a second decoding module 205 for decoding the received video object or image object Lt; / RTI >

The first decoding module 204 of the user device 200 according to the embodiment of the present invention may decode the received first image data and transmit the decoded first image data to the combining module 203. [

Also, the second decoding module 205 of the user device 200 according to the embodiment of the present invention may decode the received video object or image object and transmit the decoded video object or the image object to the combining module 203.

The combining module 203 of the user device 200 according to the embodiment of the present invention may receive both the first image data and the decoded data of the video object or the image object and combine the two data.

The combining module 203 may perform overlay processing to combine the decoded first image data and the video object or the image object.

At this time, the method for combining the two data may include all the technical methods of combining images, and various technical methods for synthesizing image data may be used.

The synthesis module 203 can generate a final output screen to be output from the output device 400. The final output screen is connected to the communication network 500 by the communication module 203, Lt; RTI ID = 0.0 > 400 < / RTI >

The output device 400 according to the present invention may be installed in the user device 200 and may be connected to the communication network 500 through the communication module 201, And provide a final output screen to the external output device.

4 is a flowchart illustrating a service method in a cloud streaming server according to an embodiment of the present invention.

4, the cloud streaming server 100 receives the content request signal of the user device 200 and executes the content through the content execution module 103 (S101). After that, the screen processing module 101 of the cloud streaming server 100 performs a process of effectively rendering and encoding the execution screen of the content and transmitting the execution screen to the user device 200, May be processed in black or white according to the format format for compressing the execution screen of the content (S103). That is, the cloud streaming server 100 may process a video object or a region in which an image object is displayed in black or white according to a coding scheme of image data generated to provide a cloud streaming service.

In this case, the image format scheme, that is, the coding scheme may be any one of Joint Photographic Experts Group (JPG), GIF (Graphics Interchange Format), PNG (Portable Network Graphics) and BMP (Bitmap) If the format is stored, the coding scheme can be used.

In each coding scheme, the smaller the type of color used for the image, the smaller the data capacity, and the data capacity may decrease or increase depending on the color used.

Therefore, according to the embodiment of the present invention, the area in which the video object or the image object is displayed can be selected and processed according to the coding scheme out of black or white which can minimize the data capacity.

The cloud streaming server 100 processes a region in which a video object or an image object is displayed in black or white in an execution screen of the content, and renders and encodes an area excluding the video object or the image object, 1 image data can be generated (S105).

The parsing processing module 104 of the cloud streaming server 100 parses the content requested by the user device 200 in operation S107 and stores the address information in which the video object or the image object of the content is stored, The tag information including at least one of the position, size, color, shape, and screen output format displayed on the display screen (S109).

Thereafter, the cloud streaming server 100 may convert the address information and the tag information into a markup language including JSON and XML format (S111).

In step S111, the cloud streaming server 100 converts the first image data generated in step S105 into a string format, and inserts the converted image data into the address information and the tag information described in the markup language (step S113). Thereafter, the cloud streaming server 100 provides the converted information in the markup language to the user device 200 (S115).

5 is a flowchart illustrating a service method in a user equipment according to an embodiment of the present invention.

Referring to FIG. 5, the control module 202 of the user device 200 requests the cloud streaming server 100 to execute the content (S201). At this time, the content execution request signal of the user device 200 may be transmitted by the communication module 201 of the user device 200. [ Thereafter, the control module 202 of the user device 200 receives the first and second frames, which are obtained by rendering and encoding the remaining area excluding the video object or the image object among the execution screens of the content requested by the user device 200 from the cloud streaming server 100, And receives the video data and the address information in which the video object or the image object is stored (S203).

In addition, the control module 202 of the user device 200 receives the first image data and the address information from the cloud streaming server 100, together with the location and size of the video object and the image object displayed on the content execution screen , Shape, and screen output format.

The control module 202 of the user device 200 requests or receives a video object or an image object corresponding to the address information to the resource server 300 based on the received address information at step S205. At this time, a request signal may be transmitted by the communication module 201 of the user device 200. That is, the user device 200 can receive the video resource data or the image resource data to be displayed on the execution screen of the content from the resource server 300.

The user device 200 may decode the received first image data in the first decoding module 204 in step S207 and decode the received video object or image object in the second decoding module 205 S209).

Thereafter, the user device 200 synthesizes the decoded first video data, the video object, or the audio object through overlay processing (S211), and may generate a final output screen to be displayed on the content execution screen (S213) .

Through the above-described process, the user device 200 can transmit the final output screen generated in step S213 to the output device (S215).

6 is a diagram illustrating a service procedure of a cloud streaming server according to an embodiment of the present invention.

Referring to FIG. 6, in step S301, the user device 200 requests content from the cloud streaming server 100. Thereafter, the cloud streaming server 100 drives the content execution module 103 to execute the content requested by the user device 200 (S303). Then, through the screen processing module 101, (Step S305). The first image data is generated by rendering and encoding the remaining area except the part where the object or the image object is displayed.

Then, the content is parsed through the parsing module 104 (S307), and the address information in which the video object or the image object is stored is generated (S309).

The cloud streaming server 100 provides the first video data and the address information generated in steps S305 and S309 to the user device 200 in step S311.

Thereafter, the user device 200 may receive the first image data and the address information and request a video object or an image object to the resource server 300 based on the address information (S313).

In addition, the resource server 300 receives the request from the user device 200 and provides a video object or an image object corresponding to the address information to the user device 200 (S315).

The user device 200 decodes the first video data received in step S311 by the first decoding module 204 in step S317 and adds the video object or the image object received from the resource server 300 in step S315 2 decoding module 205 (S319). Thereafter, the user device 200 overlays and synthesizes the decoded first video data, the video object, or the image object (S321). Then, the user device 200 generates a final output screen with the synthesized data in step S321 and provides the final output screen to the output device 400 (S323).

FIG. 7A is an exemplary diagram illustrating a cloud streaming service method according to an embodiment of the present invention. Referring to FIG. FIG. 7A shows the first video data provided by the user device 200 output from the output device 400. FIG.

7A, an execution screen 30 of a content executed by the cloud streaming server 100 at the request of the user device 200 and areas 32a, 32b, 32c, and 32d in which video objects or image objects are displayed Fig.

At this time, the cloud streaming server 100 renders and encodes an area excluding the areas 32a, 32b, 32c, and 32d in which the video object or the image object is displayed on the execution screen 30 of the content to generate first image data And provides it to the user device 200.

At this time, the areas 32a, 32b, 32c, and 32d in which the video object or the image object is displayed may be processed in black or white according to the image compression format format of the first image data, that is, the coding format.

As described above, the cloud streaming server 100 according to the embodiment of the present invention processes the areas 32a, 32b, 32c, and 32d in which the video objects or the image objects are displayed in black or white, 1 image data can be generated.

7B is an exemplary diagram illustrating a cloud streaming service method according to an embodiment of the present invention.

Referring to FIG. 7B, different image data are inserted into the areas 33a, 33b, 33c and 33d in which a video object or an image object is displayed in the content execution screen of FIG. 7A.

The user device 200 requests data from the resource server 300 based on the address information stored in the video object or the image object to receive the video object or the image object and is decoded in the second decoding module of the user device 200 , An execution screen inserted in the first image data can be seen.

Referring to FIGS. 7A and 7B, the user device 200 receives the first image data and the address information of the video object or the image object, (32a, 32b, 32c, 32d) receiving a video object or an image object directly from the resource server (300) through the address information and displaying a video object or an image object in the first image data, FIG. 8 is a diagram illustrating an operating environment of an apparatus for providing a cloud streaming service according to an embodiment of the present invention. Referring to FIG.

Figure 8 and the following discussion are intended to provide a brief, general description of a suitable computing environment in which the invention may be implemented. Although not required, the invention may be described in connection with computer-executable instructions, such as program modules, being executed by a computer system.

Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer executable instructions, associated data structures, and program modules illustrate examples of program code means for carrying out the acts of the invention disclosed herein.

8, an exemplary computing system embodying the present invention includes a processing unit 11, a system memory 12, and various system components including the system memory 12 connected to the processing unit 11 And a system bus 10 for communicating with the system.

The processing unit 11 may execute computer-executable instructions designed to implement the features of the present invention.

The system bus 10 may be any of several types of bus structures including a local bus, a peripheral bus, and a memory bus or memory controller using any of a variety of bus architectures. The system memory 12 includes a ROM (Read Only Memory) 12a and a RAM (Random Access Memory) 12b. A basic input / output system (BIOS) 13a, containing the basic routines that help to transfer information between components within a computing system, such as during start-up, may generally be stored in ROM 12a.

The computing system may include storage means, for example, a hard disk drive 15 that reads information from, or writes information to, the hard disk, reads information from, or writes information to, And an optical disk drive 17 that reads information from, or writes information to, an optical disk such as, for example, a CD-ROM or other optical media . The hard disk drive 15, the magnetic disk drive 16 and the optical disk drive 17 are connected by a hard disk drive interface 18, a magnetic disk drive-interface 19 and an optical drive interface 20, respectively, And is connected to the bus 10.

Further, the computing system may further include an external memory 21 as a storage means. The external memory 21 may be connected to the system bus 10 through an input /

The above-described drives and their associated computer-readable media readable and writable by the drives provide non-volatile storage of computer-executable instructions, data structures, program modules and other data. The exemplary environment described herein illustrates a hard disk 15, magnetic disk 16 and optical disk 17, but may also include magnetic cassettes, flash memory cards, DVDs, Bernoulli cartridges, RAMs Other types of computer readable media for storing data, including ROM, ROM, etc., may be used.

Includes one or more program modules including an operating system 13b, one or more application programs 13c, other program modules 13d, and program data 13c that are loaded and executed by the processing unit 11 The program code means may be stored in the hard disk 15, the magnetic disk 16, the optical disk 17, the ROM 12a or the RAM 12b.

In addition, the computing system may receive commands and information from a user through other input devices 22 such as a keyboard, a pointing device, a microphone, a joystick, a game pad, a scanner,

These input devices 22 may be connected to the processing unit 11 via an input / output interface 24 connected to the system bus 10.

The input / output interface 24 may be, for example, a serial interface such as a serial port interface, a PS / 2 interface, a parallel port interface, a USB interface, an Institute of Electrical and Electronics Engineers (IEEE) 1394 interface (i.e., FireWire interface) Any of a variety of different interfaces can be represented logically, or even combinations of different interfaces can be represented logically.

In addition, the computing system to which the present invention is applied may further include a display device 26 such as a monitor or LCD or an audio device 27 such as a speaker or microphone, which may be connected via a video / audio interface 25 And is connected to the system bus 10.

For example, other peripheral output devices (not shown), such as speakers and printers, may also be connected to the computer system 420. The video / audio interface unit 25 may include a high definition multimedia interface (HDMI), a graphics device interface (GDI), or the like.

Further, the computing system embodying the invention is connectable to a network such as, for example, an office-wide or enterprise-wide computer network, a home network, an intranet, and / or the Internet. The computer system may exchange data with external sources such as, for example, a remote computer system, a remote application, and / or a remote database.

To this end, a computing system to which the present invention is applied includes a network interface 27 for receiving data from an external source and / or for transmitting data to an external source.

In the present invention, such a computing system can transmit and receive information to and from a remote device via the network interface 27. [

For example, when the computing system refers to the user device 200, it can transmit and receive information to and from the cloud streaming server 100 and the resource server 300 via the network interface 27. [ On the other hand, if the computing system refers to the cloud streaming server 100, it can send and receive information with the user device 200 via the network interface 27.

The network interface 27 may be represented by a logical combination of one or more software and / or hardware modules, such as, for example, a network interface card and a corresponding Network Driver Interface Specification ("NDIS") stack.

Likewise, the computer system receives data from an external source via the input / output interface 24 or transmits data to an external source. Output interface 24 may be coupled to a modem 23 (e.g., a standard modem, a cable modem, or a digital subscriber line (" DSL ") modem) Lt; / RTI > and / or transmit data to an external source.

Although FIG. 8 shows an operating environment suitable for the present invention, the principles of the present invention may be employed with any system that can implement the principles of the present invention, with appropriate modifications if necessary. The environment shown in Fig. 8 is merely exemplary and does not represent a small portion of a very diverse environment in which the principles of the present invention may be implemented.

In addition, various information generated when the cloud streaming service program of the present invention is executed can be stored and accessed on any computer-readable medium related to the computing system as shown in FIG.

For example, a portion of these program modules and a portion of the associated program data may be stored in the system memory 12, such as in the operating system 13b, the application program 13c, the program module 13d, and / (13e).

Further, when a mass storage device such as a hard disk is connected to the computing system, such program modules and related program data may be stored in the mass storage device.

In a networked environment, program modules associated with the present invention, or portions thereof, may be connected to a remote computer system, e.g., a user device 200 and a cloud streaming server, connected via a modem 23 or network interface 25 of the input / Such as system memory and / or mass storage devices associated with the computing system of the computer 100. The execution of such a module can be performed in a distributed environment as described above.

As described above, the present specification contains details of a number of specific implementations, but they should not be construed as being limitations on the scope of any invention or claimability, but rather on the particular embodiment of a particular invention But should be understood as an explanation of the features. Certain features described herein in the context of separate embodiments may be implemented in combination in a single embodiment.

Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

Likewise, although the operations are depicted in the drawings in a particular order, it should be understood that such operations must be performed in that particular order or sequential order shown to achieve the desired result, or that all illustrated operations should be performed. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products It should be understood.

Certain embodiments of the subject matter described herein have been described. Other embodiments are within the scope of the following claims. For example, the operations recited in the claims may be performed in a different order and still achieve desirable results. By way of example, the process illustrated in the accompanying drawings does not necessarily require that particular illustrated or sequential order to obtain the desired results. In certain implementations, multitasking and parallel processing may be advantageous.

The description sets forth the best mode of the invention, and is provided to illustrate the invention and to enable those skilled in the art to make and use the invention. The written description is not intended to limit the invention to the specific terminology presented. Thus, while the present invention has been described in detail with reference to the above examples, those skilled in the art will be able to make adaptations, modifications, and variations on these examples without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be limited by the described embodiments but should be defined by the claims.

It should be noted that the embodiments of the present invention disclosed in the present specification and drawings are only illustrative of specific examples for the purpose of understanding and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

The present invention relates to a method and a device for providing a cloud streaming service, and in providing a cloud streaming service, a method for rendering and encoding a video object or a remaining area excluding a screen on which an image object is displayed, Or the address information stored in the image object, thereby providing the video or image resource directly from the user device on the basis of the received address information, overlaying the inserted video or image resource, thereby eliminating the need for PIP and requiring no full browsing. This can contribute to the development of the cloud service industry.

In addition, since the present invention is not only possible to be marketed or operated, but also can be practically and practically carried out, it is industrially applicable.

100: Cloud streaming server 200: User device
300: Resource server 400: Output device
500: communication network 101: screen processing module
102: Content transmission module 103: Content execution module
104: parsing processing module 201: communication module
202: control module 203: synthesis module
204: first decoding module 205: second decoding module

Claims (10)

A screen processing module for generating first image data by encoding a video object or an area other than an image object on the execution screen of the content requested by the user apparatus;
A content transmission module for providing the user device with address information in which the first image data and the video object or the image image generated by the screen processing module are stored;
Wherein the server comprises: The method according to claim 1,
Wherein the screen processing module processes the area in which the video object or the image object is displayed in white or black according to a coding scheme of the first image data to encode the execution screen. The method according to claim 1,
And a content execution module for executing the content. The method according to claim 1,
A parsing module for parsing the content to generate address information in which the video object or the image object is stored;
Further comprising: a server for receiving the cloud streaming data. 5. The method of claim 4,
Wherein the parsing processing module further generates tag information including at least one of a position, size, color, shape, and screen output format of the video object or the image object displayed on the execution screen,
Wherein the content transmission module further transmits the tag information together with the first image data and the address information. A communication module connected to an external server including a cloud streaming server through a communication network to transmit and receive data;
The first video data in which the video object or the remaining region excluding the image object is encoded in the execution screen of the content in response to the request for the content, and the first video data in which the video object or the image object is stored A control module for receiving the video object or the image object based on the received address information;
A first decoding module for decoding the received first image data;
A second decoding module for decoding the received video object or image object; And
A synthesis module for overlaying the decoded first video data and the decoded video object or image object to generate a final output screen;
≪ / RTI > Generating a first image data by encoding a region other than a video object or an image object in an execution screen of a content requested by the user device;
The cloud streaming server parsing the content to generate address information in which a video object or an image object is stored; And
The cloud streaming server providing address information of the first image data and the video object or image object to the user device;
The method of claim 1, 8. The method of claim 7,
Wherein the step of generating the first data comprises:
Wherein the processing unit processes the area in which the video object or the image object is displayed in white or black according to the coding scheme of the first image data to encode the execution screen. 8. The method of claim 7,
Wherein the step of generating the address information comprises:
And further generates tag information including at least one of a position, a size, a color, a shape, and a screen output format in which the video object or the image object is displayed on the execution screen,
Wherein providing to the user device comprises:
And transmits the tag information together with the first image data and the address information. Requesting content to a cloud streaming server;
Receiving, from the cloud streaming server, first image data obtained by encoding a video object or an area other than an area in which an image object is displayed in an execution screen of the content, and address information in which the video object or the image object is stored;
Receiving a video object or an image object based on the received address information;
Decoding the received first image data;
Decoding the received video object or image object; And
Overlaying the decoded first video data and the decoded video object or image object to generate a final output screen;
The method comprising the steps of:

Images