KR102231875B1 - Apparatus and method for providing streaming video or application program - Google Patents

Abstract

An apparatus and method for providing streaming video of an application program are provided. In the method of providing a streaming image of an application program according to an embodiment of the present invention, in a method performed by a computing device, a predetermined condition among a plurality of streaming images outputted by executing a plurality of application programs according to a user's request is satisfied. Selecting two or more streaming images accordingly, generating one integrated streaming image by combining the selected two or more streaming images so that the selected two or more streaming images are output on one screen, encoding the integrated streaming image And providing the encoded integrated streaming image to the user terminal, extracting only the streaming image corresponding to an application program requested to be executed from the user terminal from the encoded integrated streaming image and providing it to the user terminal.

Description

Apparatus and method for providing streaming video of an application program {APPARATUS AND METHOD FOR PROVIDING STREAMING VIDEO OR APPLICATION PROGRAM}

The present invention relates to an apparatus and method for providing streaming video of an application program. In more detail, the present invention relates to an apparatus and method for providing a streaming image of an application program capable of encoding a streaming image outputted by executing a plurality of application programs and providing it to a user.

In recent years, with the development of computer-related information and communication technology, the game industry is also rapidly developing. Conventional computer games were mainly arcade games or video games of a simple plot, but with the development of multimedia technology and network technology, games of a completely different dimension from the previous one are being developed one after another. In particular, the spread of high-speed Internet and wireless Internet was an opportunity to revolutionize the game industry by creating a new game field called multi-user online game.

In addition, a technology that provides cloud game services in a streaming method for a number of user terminals supporting computing and networking functions is attracting attention, and the server performs game execution corresponding to the input signal by receiving an input signal from the user terminal. A technology has been developed to allow a user to enjoy a game by providing a result screen of the executed game to a user terminal.

Cloud games can be serviced without restrictions between devices such as PC, mobile, console, etc., without downloading and'owning' the content, and simply'subscribing' to the service, and operations on the game can be performed on the cloud game server. Because it is processed, there is an advantage that you can enjoy the latest games regardless of the performance of the connected PC or game console.

Unlike the existing video streaming service, the cloud game is important because the interaction that reflects the user's game control signal to the server while calculating and processing vast amounts of game data on the server continuously occurs, so the cloud and network environment are important.

In the case of a conventional cloud game service, when a cloud game is executed on a plurality of user terminals 40 as shown in FIG. 1, the main computing device 10 executes a cloud game corresponding to each user terminal, In order to provide a streaming image of an executed cloud game, a separate auxiliary computing device 20 was provided to encode each streaming image.

However, in the case of such a conventional cloud game service, since a separate auxiliary computing device 20 must be provided in addition to the main computing device 10 for executing the cloud game, a huge cost is required to build a server for providing the service. There is.

In addition, since the operation input from the user terminal 40 for the cloud game operation is transmitted to the main computing device 10 through the auxiliary computing device 20, the user's input is not quickly reflected in the game. There is a problem that) occurs.

In order to solve this problem, there has been an attempt to provide a cloud game service directly from the main computing device 10 to a plurality of user terminals 40 without going through the auxiliary computing device 20 shown in FIG. 1. However, when a plurality of cloud games are executed on the main computing device 40, an overlap phenomenon and noise occur between video, audio, and manipulation signals, and data has to be separated in order to eliminate this phenomenon. Therefore, in the prior art, in order to avoid such interference, a cloud game service has to be provided by driving a plurality of operating systems through a plurality of virtual machines. Accordingly, the main computing device 40 required to use a plurality of virtual machines, resulting in an overload, and sometimes wasted memory due to unnecessary driving of a virtual machine in which a cloud game is not executed. This soon became a factor that deteriorated the provision quality of cloud game services.

Therefore, there is a need to develop a streaming video providing method to overcome the limitations of the cloud game service and provide a high-quality cloud game service.

Korean Patent Publication No. 10-2015-0040393

The technical problem to be solved by the present invention is to combine and encode a plurality of streaming images output by executing a plurality of cloud games, so that an application program that does not require a separate auxiliary computing device for individually encoding each streaming image It is to provide an apparatus and method for providing streaming video.

Another technical problem to be solved by the present invention is an application program capable of encoding a plurality of streaming images with only one encoding operation by generating a single integrated streaming image by combining a plurality of streaming images and encoding the integrated streaming image. It is to provide an apparatus and method for providing streaming video of the.

Another technical problem to be solved by the present invention is to dramatically reduce the delay time for game manipulation by transferring the game manipulation input input from the user terminal to the main computing device providing the cloud game service without going through a separate auxiliary computing device. It is to provide an apparatus and method for providing streaming video of an application program that can be shortened to.

Another technical problem to be solved by the present invention is an application program for solving the conventional problem of providing low-quality streaming content through a capturing method as the main computing device receives and encodes streaming content directly from an external server. It is to provide an apparatus and method for providing streaming video of the.

Another technical problem to be solved by the present invention is to provide an apparatus and method for providing a streaming image of an application program capable of rendering a plurality of streaming contents that do not interfere with each other without the main computing device using several virtual machines. .

Another technical problem to be solved by the present invention is to provide an apparatus and method for providing a streaming image of an application program in which the main computing device can directly receive undecoded data from a server providing a game service in order to render the streaming content. It is to do. The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above technical problem, a method for providing a streaming image of an application program according to an embodiment of the present invention is a method performed by a computing device, wherein a plurality of outputted by executing a plurality of application programs according to a user's request Selecting two or more streaming images from among the streaming images of, generating one integrated streaming image by combining the selected two or more streaming images so that the selected two or more streaming images are output on one screen, the integrated streaming image Encoding and extracting only a streaming image corresponding to an application program requested to be executed from the user terminal from among the encoded integrated streaming images and providing them to the user terminal.

In an embodiment, the selecting of two or more streaming images may include determining the number of selected streaming images based on a resolution of a screen on which the integrated streaming image is output.

In one embodiment, the step of selecting two or more streaming images according to the resolution of the screen on which the integrated streaming image is output comprises: when the screen resolution is 4K resolution, selecting four streaming images from among the plurality of streaming images And when the resolution of the screen is 8K, selecting 16 streaming images from among the plurality of streaming images.

In one embodiment, the selecting of the two or more streaming images includes selecting two or more streaming images from among a plurality of streaming images output by executing a plurality of application programs having the same graphic specifications for executing the application program. can do.

In an embodiment, the selecting of two or more streaming images may include selecting two or more streaming images from among a plurality of streaming images output by executing a plurality of application programs having the same application category.

In one embodiment, the selecting of the two or more streaming images includes selecting two or more streaming images from among a plurality of streaming images output as the same application program is repeatedly executed in response to an execution request received from different user terminals. It may include the step of.

In one embodiment, further comprising the step of obtaining a user input for the first application program from the first user terminal, the step of generating the integrated streaming image, the first application program included in the integrated streaming image 1 It may include updating the streaming video to a streaming video that is changed according to the user input.

In one embodiment, the application program may be a game client program.

In an embodiment, the selecting of the two or more streaming images may include obtaining a streaming image of the application program from an image rendering buffer of the application program.

In one embodiment, the obtaining of the streaming image of the application program comprises: real-time updating a screen generated by the application program in the image rendering buffer according to an API call of the streaming library embedded in the application program, the According to an API call of the streaming library, obtaining an address of the image rendering buffer and obtaining a streaming image of the application program by using the address of the image rendering buffer.

In an embodiment, the image rendering buffer is formed on an internal memory of the graphic interface of the computing device, and may be formed separately from the basic image rendering buffer of the application program.

In an embodiment, the image rendering buffer is formed on the RAM of the computing device, and may be formed separately from the basic image rendering buffer of the application program.

In one embodiment, the step of extracting only a streaming image corresponding to an application program requested to be executed from the user terminal from among the encoded integrated streaming images and providing it to the user terminal comprises: rendering the sound of the application program requested to be executed from the user terminal Acquiring sound data of the application program from a buffer, generating an output sound by encoding sound data of the application program, and providing the output sound of the application program together with the extracted streaming image to the user terminal It may include.

In order to solve the above technical problem, a streaming image providing server of an application program according to another embodiment of the present invention includes a processor, a network interface, a memory, and a computer program loaded into the memory and executed by the processor. Including, wherein the computer program includes an instruction for selecting two or more streaming images according to a preset condition among a plurality of streaming images output by executing a plurality of application programs according to a user's request, and the selected two or more streaming images An instruction for generating one integrated streaming image by combining the selected two or more streaming images to be output on one screen, an instruction for encoding the integrated streaming image, and the encoded integrated streaming image are provided to the user terminal, wherein the It may include an instruction for extracting only the streaming image corresponding to the application program requested to be executed from the user terminal from the encoded integrated streaming image and providing it to the user terminal.

1 is a block diagram of a conventional cloud game service providing system.
2 is a block diagram of a system for providing streaming video of an application program according to an embodiment of the present invention.
3 is a flowchart illustrating a method of providing a streaming image of an application program according to another embodiment of the present invention.
4 is a flowchart illustrating a method of selecting a streaming image using a screen resolution in some embodiments.
5 is a flowchart illustrating a method of selecting a streaming video according to a graphic specification of an application program, in some embodiments.
6 is a flowchart illustrating a method of selecting a streaming video according to a category of an application program, in some embodiments.
7 is a diagram illustrating a form of an integrated streaming video when the screen resolution is 4K resolution, in some embodiments.
8 is a diagram illustrating a form of an integrated streaming video when the screen resolution is 8K, according to some embodiments.
9 is a diagram illustrating a form of providing an integrated streaming video to a user terminal in some embodiments.
10 is a block diagram of a system for providing streaming video of an application program according to another embodiment of the present invention.
11 is an exemplary diagram illustrating a method of executing a plurality of application programs by a streaming image providing server of a conventional application program.
12 is an exemplary diagram illustrating a method of executing a plurality of application programs by a streaming video providing server according to another embodiment of the present invention.
13 is a block diagram of a system for providing streaming video of an application program according to another embodiment of the present invention.
14 is a flowchart illustrating a method of providing a streaming image of an application program according to another embodiment of the present invention.
15 is a flowchart for explaining in detail some steps of FIG. 14.
16 is a hardware configuration diagram of a server for providing streaming video of an application program according to another embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments make the disclosure of the present invention complete, and the general knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to the possessor, and the invention is only defined by the scope of the claims. The same reference numerals refer to the same elements throughout the specification.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used with meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically. The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase.

In addition, in the present specification, the application program is a cloud game, and the streaming image output by executing the application program is described as a game streaming image output by executing the cloud game. However, it should be noted that this is only an example to aid understanding of the present invention, and is not limited thereto.

Hereinafter, some embodiments of the present invention will be described with reference to the drawings.

2 is a block diagram of a system for providing streaming video of an application program according to an embodiment of the present invention.

Referring to FIG. 2, a system for providing a streaming image of an application program according to an embodiment of the present invention may include a server 100 for providing a streaming image of an application program, a user terminal 200 and a network 300.

Here, the system for providing streaming video of the application program shown in FIG. 2 is according to an embodiment, and its components are not limited to the embodiment shown in FIG. 2, and may be added, changed, or deleted as necessary. Please note.

First, the streaming image providing server 100 of an application program may execute a plurality of cloud games according to an application program execution request received from the user terminal 200.

In addition, the streaming image providing server 100 of the application program selects two or more streaming images according to a preset condition among a plurality of cloud game streaming images output by executing a plurality of cloud games, and combines the selected two or more streaming images. One integrated streaming video can be created and encoded.

In addition, the streaming image providing server 100 of the application program provides the encoded integrated streaming image to the user terminal 200, but among the encoded integrated streaming images, the game streaming corresponding to the cloud game requested to be executed from the user terminal 200 Only images can be extracted and provided. Here, the streaming image providing server 100 of the application program may communicate with the plurality of user terminals 200 through the network 300.

The user terminal 200 may receive and output a game streaming image provided from the streaming image providing server 100 of an application program. To this end, the user terminal 200 may include a display for outputting an image. However, the present invention is not limited thereto, and the user terminal 200 performs wired/wireless communication with an external display device and transmits the game streaming image provided from the streaming image providing server 100 of the application program to the external display device, thereby providing a game streaming image. Can be output.

In addition, the user terminal 200 may input a user input for operating a cloud game. To this end, the user terminal 200 may further include an operation controller. For example, the manipulation controller may be a manipulation controller GUI that is output on a display included in the user terminal 200. However, the present invention is not limited thereto, and may be an external controller terminal capable of wired or wireless communication with the user terminal 200. Hereinafter, a method of providing a streaming image to a user by the streaming image providing server 100 of an application program will be described with reference to FIGS. 3 to 9.

The method of providing a streaming image of an application program may be performed in various systems and/or environments, but to provide convenience of understanding, it is assumed that it is the same environment as the environment illustrated in FIGS. 1 and 2.

In addition, unless otherwise noted, it is assumed that each step of a method for providing a streaming image of an application program is performed by the streaming image providing server 100 of an application program. Therefore, when the subject of each operation is omitted, it may be understood that each operation is performed by the streaming image providing server 100 of the application program. In addition, it is obvious that the order of each operation may be changed within a range in which the order of execution may be logically changed as necessary in the methods described below.

3 is a flowchart illustrating a method of providing a streaming image of an application program according to another embodiment of the present invention.

Referring to FIG. 3, first, a plurality of application programs are executed (S110). For example, when the application program streaming video providing server 100 receives an execution request for a first application program from a first user terminal, and receives an execution request for a second application program from the 32nd user terminal, The first application program and the second application program may be executed in response to the execution request. In addition, when the application program streaming video providing server 100 receives an execution request for the first application program, which is the same application program, from the first user terminal and the second user terminal, respectively, the first user terminal and the second user terminal The first application program, which is the same application program, may be executed twice according to the execution request of.

Next, in step S110, two or more streaming images are selected from among the plurality of streaming images output as the plurality of application programs are executed (S120). Hereinafter, various methods of selecting two or more streaming images from among a plurality of streaming images will be described with reference to FIGS. 4 to 6.

In an embodiment, the streaming image providing server 100 of an application program may select two or more streaming images according to a resolution of a screen on which an integrated streaming image is output.

Referring to FIG. 4, after step S110 of FIG. 3, a resolution of a screen on which an integrated streaming image is output is checked (S210).

Next, it is determined whether the screen resolution is a 4K resolution (S220), and if it is determined that the 4K resolution is in step S220, four streaming images are selected from among a plurality of streaming images output by executing a plurality of application programs. (S230).

Next, when it is determined in step S220 that the resolution of the screen is not the 4K resolution, it is determined whether the screen resolution is 8K resolution (S240), and in response to the determination as 8K resolution, a plurality of outputted by executing a plurality of application programs. Among the streaming images, 16 streaming images are selected (S250).

If, in steps S220 and S240, it is determined that the resolution of the screen does not match at least one of the 4K resolution and the 8K resolution, step S210 may be performed again. However, the present invention is not limited thereto, and when it is determined that the resolution of the screen does not match at least one of the 4K resolution and the 8K resolution, a guide message requesting to check and reset the resolution may be output to the outside.

In another embodiment, the streaming image providing server 100 of an application program may select two or more streaming images from among a plurality of streaming images output by executing a plurality of application programs having the same graphic specifications for executing the application program.

Referring to FIG. 5, after step S110 of FIG. 3, a plurality of application programs having the same graphic specifications for executing the application program are grouped (S310). For example, the streaming video providing server 100 of an application program is subdivided into five stages according to the graphic specifications, and provides a first application program group, which requires the highest level graphic specifications for a plurality of application programs, and a high level graphic specification. The second application program group that requires, the third application program group that requires a medium-level graphic specification, the fourth application program group that requires a low-level graphic specification, and the fifth application program group that requires the lowest level graphic specification. Can be grouped together. However, it is not limited thereto.

Next, two or more streaming images are selected for each group from among a plurality of streaming images output as the plurality of application programs are executed (S320). For example, when four streaming images are selected according to a preset condition, the streaming image providing server 100 of an application program includes a first application program group, a second application program group, a third application program group, and a fourth application program. Four streaming images may be selected for each group from the program group and the fifth application program group.

In another embodiment, the streaming image providing server 100 of an application program may select two or more streaming images from among a plurality of streaming images output by executing a plurality of application programs having the same application category.

Referring to FIG. 6, after step S110 of FIG. 3, a plurality of application programs having the same category of application programs are grouped (S410). For example, when the application is a cloud game, the category of the application may mean the genre of the cloud game, and the streaming video providing server 100 of the application may group cloud games having the same or similar genre with each other. have. However, it is not limited thereto.

Next, two or more streaming images are selected for each group from among a plurality of streaming images output as the plurality of application programs are executed (S420). For example, when four streaming images are selected according to a preset condition, the streaming image providing server 100 of the application selects four streaming images from among a plurality of streaming images for cloud games of a sports genre, Four streaming images can be selected from a plurality of streaming images for FPS genre cloud games.

In another embodiment, the streaming image providing server 100 of the application program is Two or more streaming images may be selected from a plurality of streaming images output as the same application program is repeatedly executed in response to an execution request received from different user terminals. When the plurality of streaming images includes four first streaming images for a first application program and four second streaming images for a second application program, the streaming image providing server 100 of the application program includes a plurality of streaming images. Among them, only four first streaming images or four second streaming images may be selected. However, it is not limited thereto.

Referring back to FIG. 3, after step S120, two or more streams selected in step S120 are combined to be output on one screen to generate one integrated streaming image (S130). For example, when the screen resolution is 4K, the streaming image providing server 100 of the application program combines four streaming images in a 2*2 format as shown in FIG. 7 to generate one integrated streaming image. can do. In addition, when the screen resolution is 8K resolution, the streaming image providing server 100 of the application program can generate one integrated streaming image by combining 16 streaming images in a 4*4 format as shown in FIG. have. However, it is not limited thereto. In this case, in another embodiment, when displaying a plurality of images, the streaming image providing server 100 of an application program may form a divide line capable of classifying each image as illustrated in FIGS. 7 and 8. Next, the integrated streaming video generated in step S130 is encoded (S140).

Here, with regard to the streaming video encoding method performed by the streaming video providing server 100 of the application program, "H.264-based video main road recognition encoding technique", Yong-Jun Jung, Seok-Gyu Lee, Hwang-Nam Kim, Korean Information Science Society, ITU-T H. Various documents such as 265 standard can be referenced.

Next, a part of the integrated streaming video encoded in step S140 is extracted and the extracted streaming video is provided to the user terminal (S150). For example, the streaming image providing server 100 of the application program provides the encoded integrated streaming image to the user terminal 200, but corresponds to the application program requested to be executed from the user terminal 200 among the encoded integrated streaming images. It is possible to extract only the streaming image to be provided to the user terminal 200.

In addition, in another embodiment, the streaming image providing server 100 of the application program identifies a divide line for classifying a plurality of images when extracting a part of the encoded image, and determines that the image is divided based on the divide line. Thus, each image can be classified and extracted.

Hereinafter, it will be described with reference to FIG. 9.

Referring to FIG. 9, when the screen resolution is 4K resolution, the streaming video providing server 100 of the application program may generate and encode a 2*2 integrated streaming video 1 as shown in FIG. 9. have. Thereafter, the streaming image providing server 100 of the application program provides the integrated streaming image 1 to the first user terminal 200-1 and the second user terminal 200-2, but among the integrated streaming image 1 Only the streaming image 2-1 of the first application program requested to be executed by the first user terminal 200-1 may be extracted and provided to the first user terminal 200-1. In addition, the streaming image providing server 100 of the application program extracts only the streaming image 2-2 of the second application program requested to be executed by the second user terminal 200-2 from the integrated streaming image 1 to be used by the second user. It can be provided to the terminal 200-2. However, it is not limited thereto.

In one embodiment, the streaming image providing server 100 of the application program may obtain a user input for the first application program from the first user terminal 200-1 through the network 300 and included in the integrated streaming image The first streaming image of the first application program may be updated to a streaming image that is changed according to a user input. However, it is not limited thereto.

In other words. The streaming image providing server 100 of an application program according to an embodiment of the present invention performs encoding on an integrated streaming image generated by combining a plurality of streaming images, thereby providing a streaming image to the user terminal 200. There is an advantage in that it is not necessary to have a plurality of auxiliary computing devices for encoding each of a plurality of streaming images.

In addition, the streaming image providing server 100 of an application program according to an embodiment of the present invention does not perform encoding and streaming image provision operation of the streaming image through a plurality of auxiliary computing devices, but itself encodes and streams the streaming image. Since the image providing operation is performed, there is an advantage in that the latency can be drastically reduced.

In addition, since the streaming image providing server 100 of an application program according to an embodiment of the present invention directly receives a user input for manipulating an application program input from the user terminal 200 without passing through a separate auxiliary computing device, There is an advantage in that the latency for the operation of an application program can be drastically reduced.

10 is a configuration diagram of a system for providing a streaming image of an application program according to another embodiment of the present invention, and FIG. 11 is a configuration diagram of a system for providing a streaming image of an application program according to another embodiment of the present invention.

Referring to FIG. 10, a system for providing a streaming image of an application program according to another embodiment of the present invention includes a content server 1, a server 100 for providing a streaming image of an application program, and a user terminal 200 and a network 300. Can include.

Here, the system for providing streaming video of the application program shown in FIG. 10 is according to an embodiment, and its components are not limited to the embodiment shown in FIG. 10, and may be added, changed, or deleted as necessary. Please note. Meanwhile, since the description of the user terminal 200 and the network 300 is the same as the above description, detailed information will be omitted.

The content server 1 may be a server that provides an application program as an external server, but is not limited thereto, and may be a service provider that provides various content such as games, video, or audio.

The streaming image providing server 100 of an application program may include an image rendering buffer 110 or a sound rendering buffer 120 in a memory (RAM) 103.

In another embodiment, the image rendering buffer 110 is formed on the RAM of the computing device and may be formed separately from the basic image rendering buffer of the application program.

In one embodiment, the image rendering buffer 110 is formed on an internal memory of a graphic interface of a computing device, and may be formed separately from a basic image rendering buffer of an application program.

When the streaming image providing server 100 of an application program acquires the streaming image of an application program, the streaming image providing server 100 may acquire the streaming image according to an API call of a streaming library embedded in the application program.

After obtaining the address of the image rendering buffer 110 according to the API call of the streaming library, the application program's streaming image providing server 100 uses the address of the image rendering buffer 110 to use the application program. It is possible to obtain a streaming image of. In this case, the image rendering buffer 110 may update an image generated by an application program in real time.

In addition, the streaming video providing server 100 of the application program controls the application program by directly acquiring a control signal for the manipulation of the application program from the user terminal 200 to the memory, and transmits the calculation result thereof to the content server 1 can do. Accordingly, the streaming image providing server 100 of the application program may receive an updated image according to a result calculated on the control signal from the content server 1 in real time.

In the conventional cloud game server, after decoding the encoded data to provide to the user terminal 200, there has been a hassle of transmitting an image in a capture method, but a streaming image of an application program according to another embodiment of the present invention The providing system captures because the streaming image providing server 100 of the application program can directly receive raw data from the image rendering buffer 110 and the sound rendering buffer 120 and process the data without capturing. You can get rid of the hassle of doing it.

11 is an exemplary diagram showing a method of a conventional streaming image providing server of an application program executing a plurality of application programs, and FIG. 12 is a streaming image providing server according to another embodiment of the present invention executing a plurality of application programs It is an exemplary diagram showing how to do it.

Referring to FIG. 11, a streaming image providing server of a conventional application program drives a plurality of virtual machines (VM#1 to VM#4) to execute a plurality of application programs, and drives the virtual machines (VM#1 to VM#4). The operating systems (OS1 to OS4) were executed in each of VM#4).

The streaming image providing server 100 of a conventional application program received encoded streaming content to process a plurality of images and a plurality of audio data at once. Therefore, the streaming image providing server of a conventional application program drives each virtual machine (VM#1 to VM#4) to provide streaming content to the user terminal 200, and each virtual machine (VM#1 to VM#4) VM#4) had to provide a cloud game service by capturing the already decoded streaming content.

Referring to FIG. 12, in the system for providing streaming video of an application program according to another embodiment of the present invention, since the streaming video providing server 100 of the application program directly receives raw data, various There is an advantage of being able to process multiple streaming contents without running multiple operating systems. The streaming image providing system of an application program according to another embodiment of the present invention drives one operating system as the streaming image providing server 100 of the application program directly processes the content provided from the image rendering buffer 110. Multiple videos can be combined and encoded at once.

Referring to FIG. 13, a system for providing a streaming image of an application program according to another embodiment of the present invention includes a content server 1, a server 100 for providing a streaming image of an application program, a user terminal 200 and a network 300. It may include.

Here, the system for providing streaming video of the application program shown in FIG. 13 is according to an embodiment, and its components are not limited to the embodiment shown in FIG. 13, and may be added, changed, or deleted as necessary. Please note. Meanwhile, in the case of the description of the content server 1, the user terminal 200, and the network 300, the descriptions of the content server 1, the user terminal 200, and the network 300 are the same as those described above, and thus detailed information will be omitted.

In this embodiment, the image rendering buffer 110 is formed on the VRAM 106, which is an internal memory of the graphic interface 105 of the streaming image providing server 100 of the application program, and is a basic image rendering buffer of the application program. Can be formed separately from. In this case, the image rendering buffer 110 may acquire an image through the memory 103, but in another embodiment, the image may be directly acquired from the content server 1.

Accordingly, in the system for providing streaming video of an application program according to another embodiment of the present invention, as the streaming video providing server 100 of the application program encodes a high-quality video using the VRAM 106, the user terminal 200 It has the advantage of providing high-quality images.

14 is a flowchart illustrating a method of providing a streaming image of an application program according to another embodiment of the present invention, and FIG. 15 is a flowchart illustrating some steps of FIG. 14 in detail.

The method of providing a streaming image of an application program according to another embodiment of the present invention may be performed by the streaming image providing server 100 of the application program, and the streaming image is generated from the image rendering buffer 110 in step S210. Can be obtained. Two or more of the plurality of streaming images obtained in step S220 are selected, and in step S230, an integrated streaming image is generated by combining the selected two or more streaming images, and in step S240, the entire integrated streaming image is It can be encoded. In step S250, some of the encoded integrated streaming video may be extracted and provided to the user terminal.

Referring to FIG. 15, in a method for providing a streaming image of an application program according to another embodiment of the present invention, in the step (S250) of extracting only a streaming image corresponding to an application program and providing it to the user terminal, Rendered audio may be extracted and provided together with the streaming image.

Specifically, in the step of extracting only the streaming image corresponding to the application program and providing it to the user terminal (S250), obtaining sound data of the application program from the sound rendering buffer 120 of the application program requested to be executed from the user terminal (S2501) may be performed, and a step (S2503) of generating an output sound by encoding sound data of an application program may be performed. Thereafter, in step S250, the output sound of the application program may be provided to the user terminal together with the extracted streaming image.

The methods according to the embodiments of the present invention described so far can be performed by executing a computer program implemented in computer-readable code. The computer program may be transmitted from a first computing device to a second computing device through a network such as the Internet and installed in the second computing device, thereby being used in the second computing device. The first computing device and the second computing device include all of a server device, a physical server belonging to a server pool for cloud services, and a fixed computing device such as a desktop PC.

The computer program may be stored in a recording medium such as a DVD-ROM or a flash memory device. Hereinafter, a hardware configuration of the server 100 providing a streaming image of an application program according to another embodiment of the present invention will be described with reference to FIG. 10.

16 is a hardware configuration diagram of a server for providing streaming video of an application program according to another embodiment of the present invention.

Referring to FIG. 16, the server 100 for providing a streaming image of an application program according to another embodiment of the present invention includes at least one processor 102, a first computer program 151 and a first computer program executed by the processor 102. 2 A memory 103 for loading a computer program 152, a bus 101, a network interface 104, and a storage 150 for storing the first computer program 151 and the second computer program 152 It may include. Here, only the components related to the embodiment of the present invention are shown in FIG. 16. Accordingly, those of ordinary skill in the art to which the present invention belongs may recognize that other general-purpose components may be further included in addition to the components shown in FIG.

The processor 102 controls the overall operation of each component of the streaming image providing server 100 of an application program. The processor 102 includes a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), or any type of processor well known in the technical field of the present invention. Can be. Further, the processor 102 may perform an operation on at least one application or program for executing the method according to the embodiments of the present invention. Here, the streaming image providing server 100 of the application program illustrated in FIG. 16 is illustrated as including one processor 102, but is not limited thereto and may include a plurality of processors.

The memory 103 stores various types of data, commands and/or information. The memory 103 may load the first computer program 151 and the second computer program 152 from the storage 150 in order to execute a method/operation according to various embodiments of the present disclosure. When the first computer program 151 and the second computer program 152 are loaded in the memory 103, the processor 102 performs one or more instructions constituting the first computer program 151 and the second computer program 152. By executing these, a method/operation of providing streaming video of an application program can be performed. The memory 103 may be implemented as a volatile memory such as RAM, but the technical scope of the present invention is not limited thereto.

The bus 101 provides a communication function between components of the server 100 providing a streaming image of an application program. The bus 101 may be implemented as various types of buses such as an address bus, a data bus, and a control bus.

The network interface 104 supports wired/wireless Internet communication of the streaming image providing server 100 of an application program. In addition, the network interface 104 may support various communication methods other than Internet communication. To this end, the network interface 104 may be configured to include a communication module well known in the art. In some embodiments, the network interface 104 may be omitted.

The storage 150 may non-temporarily store the first computer program 151 and the second computer program 152. When the application program is executed and manipulated through the streaming image providing server 100 of the application program, the storage 150 may store various data on the application program executed according to the execution and manipulation. For example, the storage 150 may store information on an application program to be executed, operation information on the application program, information on a user who has requested execution of the application program, and the like.

The storage 150 is a nonvolatile memory such as a ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, etc., a hard disk, a removable disk, or well in the technical field to which the present invention pertains. It may be configured to include any known computer-readable recording medium.

The first computer program 151 and the second computer program 152 include one or more instructions that when loaded into the memory 103 cause the processor 102 to perform a method/operation according to various embodiments of the present invention. can do. That is, the processor 102 may perform the method/operation according to various embodiments of the present disclosure by executing the one or more instructions.

In one embodiment, the first computer program 151 and the second computer program 152 may display two or more streaming images according to a preset condition among a plurality of streaming images output by executing a plurality of application programs according to a user's request. An instruction to select, an instruction for generating a single integrated streaming image by combining two or more selected streaming images so that two or more selected streaming images are displayed on a single screen, an instruction for encoding an integrated streaming image, and an encoded integrated streaming image on the user terminal However, it may include an instruction that extracts only the streaming image corresponding to the application program requested to be executed from the user terminal among the encoded integrated streaming images and provides it to the user terminal.

So far, various embodiments of the present invention and effects according to the embodiments have been mentioned with reference to FIGS. 1 to 16. The effects according to the technical idea of the present invention are not limited to the above-mentioned effects, and other effects that are not mentioned will be clearly understood by those skilled in the art from the following description.

Embodiments of the present invention have been described above with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features. I can understand. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting.

Claims (5)

In the method performed by the computing device,
In accordance with the API call of the streaming library embedded in the cloud game executed by the user's request, the unencoded row of the screen of the cloud game generated by the content server providing the cloud game to the video rendering buffer of the cloud game Data is updated in real time;
Obtaining an address of the image rendering buffer according to an API call of the streaming library;
Generating a streaming video of the cloud game using the unencoded raw data obtained by accessing the address of the video rendering buffer; And
Providing the generated streaming image of the cloud game to a terminal of a user requesting the execution of the cloud game,
Providing the streaming video of the cloud game to the terminal of the user who requested the execution of the cloud game,
Acquiring sound data of the cloud game from the sound rendering buffer of the cloud game from the user's terminal; And
Generating an output sound using sound data of the cloud game, and providing the generated output sound to the user's terminal together with the streaming image of the cloud game,
How the application provides streaming video. The method of claim 1,
The image rendering buffer is formed on an internal memory of the graphic interface of the computing device, and is formed separately from the basic image rendering buffer of the application program,
How the application provides streaming video. The method of claim 2,
The image rendering buffer is formed on the RAM of the computing device and is formed separately from the basic image rendering buffer of the application program,
How the application provides streaming video. delete Processor;
Network interface;
Memory; And
A computer program loaded into the memory and executed by the processor,
The computer program,
In accordance with the API call of the streaming library embedded in the cloud game executed by the user's request, the unencoded row of the screen of the cloud game generated by the content server providing the cloud game to the video rendering buffer of the cloud game An instruction in which data is updated in real time;
An instruction for obtaining an address of the image rendering buffer according to an API call of the streaming library;
An instruction for generating a streaming video of the cloud game using the unencoded raw data obtained by accessing the address of the video rendering buffer; And
Including an instruction for providing the generated streaming video of the cloud game to the terminal of the user requesting the execution of the cloud game,
The instruction for providing the streaming video of the cloud game to the terminal of the user who requested the execution of the cloud game,
An instruction for obtaining sound data of the cloud game from the sound rendering buffer of the cloud game from the user's terminal; And
Comprising an instruction for generating an output sound using sound data of the cloud game, and providing the generated output sound to the user's terminal together with the streaming image of the cloud game,
Server for providing streaming video of the application.

Images