KR20240018028A - 3D Contents Streaming System - Google Patents

Abstract

The present invention relates to a 3D content streaming system, which includes a plurality of content creation servers that run one or more server-side clients that generate original images of 3D content, and one that generates streaming images by encoding the original images of the generated 3D content. It includes a plurality of encoding servers that drive the above encoding process and a plurality of streaming servers that drive one or more streaming processes that stream the encoded streaming video to a web browser of the user terminal, and the server-side client is a web browser of the user terminal. A 3D content streaming system comprising a content logic processing unit that receives a user command from and configures 3D content according to the user command, and a rendering unit that renders the 3D content configured in the content logic processing unit to generate an original image of the 3D content. to provide.

Description

3D Contents Streaming System {3D Contents Streaming System}

The present invention relates to a 3D content streaming system, which allows 3D content services to be provided through a web browser using streaming technology, while also providing services by efficiently utilizing server resources even when a large number of users connect. It's about the system.

As the number of services using 3D graphics technology that is close to the real world increases, 3D content is becoming more diverse. In particular, as interactive services that allow various users to access and directly manipulate 3D content at the same time, rather than just providing and using the generated 3D content one-way, become active, it can be used in various fields such as games, metaverse, and smart factory. It is becoming.

The prior art, Korean Patent No. 10-2394158, “Metaverse Spatial Streaming System and Method,” is a technology that allows such 3D content services to be provided through a web browser using streaming technology, even if the specifications of the terminal are not high. It allows you to use 3D content simply by accessing the web page.

However, when providing 3D content using streaming like this, since all processing such as rendering is performed on the server, resource use on the server side increases, and efficient server management is required.

Figure 1 is a diagram showing the configuration of a 3D content streaming server according to the prior art.

As shown in the figure, operations such as content logic processing 110, rendering 120, encoding 130, and streaming 140 are performed in the 3D content streaming server 101. Conventionally, one server like this All operations were processed internally and streaming was carried out.

However, in the case of content logic processing (110) and streaming (140), CPU calculations are central, and in the case of rendering (120) and encoding (130), many calculations are performed through GPU, so such operations are not possible within one server. When processing all of them, there is difficulty in efficiently distributing resources such as CPU and GPU, resulting in waste of resources, and problems in processing when a large number of users connect.

Therefore, technology that enables efficient streaming of 3D content is required.

Korean Patent Publication No. 10-2394158

The purpose of the present invention is to efficiently utilize server resources while providing 3D content through streaming.

The present invention distributes the server-side client, which handles content logic and rendering, and encoding and streaming on separate servers, and increases and decreases each server as needed, thereby reducing idle time. The purpose is to minimize resources.

The purpose of the present invention is to enable response by immediately increasing available resources even when a large number of users access a 3D content service.

To achieve this purpose, the 3D content streaming system according to an embodiment of the present invention includes a plurality of content creation servers that run one or more server-side clients that generate the original image of the 3D content, and the original image of the generated 3D content. A plurality of encoding servers drive one or more encoding processes that encode and generate streaming video, and a plurality of streaming servers that drive one or more streaming processes that stream the encoded streaming video to a web browser of a user terminal, the server side A client receives a user command from a web browser of the user terminal, and renders a content logic processing unit that configures 3D content according to the user command and the 3D content configured in the content logic processing unit to generate an original image of the 3D content. It may be composed of parts.

At this time, a content creation server scaler that adjusts the number of content creation servers on which the server-side client runs according to the load of the plurality of content creation servers, and the number of encoding servers on which the encoding process is driven according to the load of the plurality of encoding servers. It may further include an encoding server scaler that adjusts and a streaming server scaler that adjusts the number of streaming servers on which the streaming process runs according to the load of the plurality of streaming servers.

In addition, the plurality of content creation servers are composed of servers including CPUs and GPUs, the plurality of encoding servers are composed of GPU clusters including GPUs, and the plurality of streaming servers are composed of servers including CPUs. You can.

In addition, it may further include a synchronization server that synchronizes information about the 3D content of a plurality of server-side clients.

In addition, it further includes a user session management server that manages the connection relationship between the web browser of the user terminal and the server-side client, and the web browser of the user terminal and the server-side client may be configured to correspond in N:1. there is.

In addition, it may further include a server session management server that manages the connection relationship between the server-side client, the encoding process, and the streaming process.

The present invention can achieve the effect of efficiently utilizing server resources while providing 3D content through streaming.

The present invention distributes the server-side client, which handles content logic and rendering, and encoding and streaming on separate servers, and increases and decreases each server as needed, thereby reducing idle time. The effect of minimizing resources can be achieved.

The present invention can achieve the effect of immediately increasing available resources to respond even when a large number of users access a 3D content service.

Figure 1 is a diagram showing the configuration of a 3D content streaming server according to the prior art.
Figure 2 is a configuration diagram showing the configuration of a 3D content streaming system according to an embodiment of the present invention.
Figure 3 is a configuration diagram showing the internal configuration of a 3D content streaming system according to an embodiment of the present invention.
Figure 4 is a diagram showing an example of a scaler that adjusts the number of each server in a 3D content streaming system according to an embodiment of the present invention.
Figure 5 is a diagram showing an example of a management server for managing each server in a 3D content streaming system according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. In describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted. Additionally, in describing embodiments of the present invention, specific numbers are only examples and the scope of the invention is not limited thereby.

The 3D content streaming system according to the present invention can be configured using a plurality of servers including a central processing unit (CPU), graphics processing unit (GPU), and memory, and in the present invention, the servers are physically separated. It may mean a hardware device that is installed, or it may mean a server where one or multiple hardware devices are connected using the cloud, etc. and managed as logically separate servers.

In the present invention, streaming of 3D content involves transmitting a 2D image generated by rendering 3D content to a terminal in real time through a protocol such as Web-RTC so that it can be viewed in real time. This means that when a user inputs an operation command for a streaming video displayed on a web browser, etc., it responds in real time, allowing the user to control 3D content.

Figure 2 is a configuration diagram showing the configuration of a 3D content streaming system according to an embodiment of the present invention.

As shown in the drawing, a 3D content streaming system according to an embodiment of the present invention may be configured to include a content creation server 201, an encoding server 202, and a streaming server 203.

The content creation server 201 generates a two-dimensional image by rendering information on the placement of 3D objects in virtual space. In the conventional technology, the resources of 3D objects were downloaded to the terminal and rendered by performing rendering on the graphics card of the terminal, but in the present invention, in order to minimize the load on the terminal and enable smooth 3D content service even in mobile terminals, etc. 3D content is created on the server.

In the content creation server 201, the logic processing process of placing 3D objects in virtual space is mainly performed by CPU operations, and rendering them into two-dimensional images is mainly performed using GPU.

The 3D content generated in the content creation server 201 is delivered to the encoding server 202, and the encoding server 202 encodes it into a format for streaming. Various video codecs can be used for encoding, and for efficient encoding, the encoding server 202 can be configured as a GPU cluster.

The video encoded in a streaming format in the encoding server 202 is delivered to the streaming server 203 and streamed from the streaming server to the user terminal web browser 102. The streaming server 203 mainly performs the role of transmitting data and can be operated using the CPU.

As shown in the figure, each server may be composed of a plurality, and processes for performing content creation, encoding, streaming, etc. within one server may be run by multiple servers.

If more users connect to the provided 3D content service and the number of tasks to be processed increases, the required servers can be increased to process the tasks. However, in the conventional method, even if CPU usage is low, if GPU is used a lot, the entire server must be increased. However, in the present invention, it is possible to efficiently manage servers by increasing only the necessary servers according to the characteristics of each task.

For example, as shown in the drawing, two content creation servers 201, four encoding servers 202, and three streaming servers 203 may be configured to operate as needed, where the GPU load of the encoding server If the number increases, smooth service becomes possible by increasing the number of encoding servers 202 to five.

In order to generate an image to be delivered to one user terminal web browser 102, one content creation server 201 is used, and in some cases, the encoding server 202 and streaming server 203 may be configured to provide multiple images for smooth parallel processing. It is also possible for tasks to be processed distributed across servers.

Figure 3 is a configuration diagram showing the internal configuration of a 3D content streaming system according to an embodiment of the present invention.

As described above, the 3D content streaming system according to an embodiment of the present invention may be composed of a plurality of content creation servers 201, a plurality of encoding servers 202, and a plurality of streaming servers 203.

The content creation server 201 runs one or more server-side clients (210, Server-Side Clients) that generate original images (raw texture streams) of 3D content. The server-side client 210 may include a content logic processing unit 211 (Contents Logic Process) and a rendering unit 212 (Rendering Module).

The content logic processing unit 211 receives a user command from the web browser 102 of the user terminal and configures 3D content according to the user command. The user command may be a command given by the user to manipulate content through actions such as clicking or touching the web browser, and according to such a command, the content logic processing unit 211 places 3D objects in the virtual space and Perform tasks such as moving. In this way, the task of placing and processing 3D objects in a virtual 3D space can use various technologies depending on the type of 3D content service, and is mainly performed through CPU calculation for logic processing.

If the 3D content service is a service such as a game or metaverse in which multiple users use one space or object together, the content logic processing unit 211 may exchange synchronization information to synchronize the screens viewed by the users. . Synchronization will be explained in more detail in FIG. 5.

The rendering unit 212 renders the 3D content configured in the content logic processing unit 211 to generate an original image of the 3D content. When 3D objects are placed in a 3D virtual space according to content, each user sees the 3D virtual space as seen from the user's viewpoint (view point) streamed through a web browser. Therefore, the image of the 3D virtual space seen from each user's perspective is created as a 2D image, and this process is called rendering.

As such, since different images must be generated according to each user's viewpoint, even if the 3D content service is a service where multiple users use the same 3D virtual space, such as a game or metaverse, one video is provided for each user terminal web browser 102. The server-side client 210 must operate to generate the original image of the 3D content.

The server-side client 210 is a function that operates in each user terminal (client) in the prior art and is located on the server, so that even terminals with low specifications can receive the same service through the web.

The encoding server 202 operates one or more encoding processes 220 that encode the original image of the generated 3D content and generate a streaming image. As described above, various streaming video codecs can be used for encoding, and the present invention is not limited by this encoding method.

Since GPU operations are mainly used in the operation of the encoding process 220, the encoding server 220 is preferably composed of a GPU cluster. A plurality of encoding processes 220 are used to encode the original image of one 3D content. can be used to process tasks in parallel, enabling more efficient encoding.

In particular, when attempting to provide a 3D content service through streaming as in the present invention, encoding without delay is essential to enable the screen to respond to user commands in real time. Therefore, by using a GPU cluster and operating a plurality of encoding processes 220, encoding can be achieved as quickly as possible.

The streaming server 203 drives one or more streaming processes 230 that stream the encoded streaming video to the web browser 102 of the user terminal.

The streaming process 230 transmits the encoded video to a web browser according to a streaming protocol such as Web-RTC, allowing processing through CPU association. Accordingly, the streaming server 203 may be configured as a CPU-oriented server, and the streaming process 230 may be configured to perform streaming tasks by being connected to one or more user terminal web browsers 102.

In this way, the content creation server 201 is configured using a server including a CPU and GPU, so that the content logic processing unit 211 uses the CPU, the rendering unit 212 uses the GPU, and the encoding server ( 202) is preferably configured as a GPU cluster including a GPU, and the streaming server 203 is preferably configured as a CPU-oriented server. In this way, each task is distributed and processed on a separate server, and by configuring a server suited to the characteristics of each task, efficient 3D content streaming is possible.

Figure 4 is a diagram showing an example of a scaler that adjusts the number of each server in a 3D content streaming system according to an embodiment of the present invention.

As shown in the drawing, the 3D content streaming system according to an embodiment of the present invention includes the number of content creation servers 201 on which the server-side client 210 runs according to the load of the plurality of content creation servers 201. A content creation server scaler 204 that adjusts, an encoding server scaler 205 that adjusts the number of encoding servers 202 on which the encoding process 220 runs according to the load of the plurality of encoding servers 202, and the plurality of encoding servers 202. It may be configured to include a streaming server scaler 206 that adjusts the number of streaming servers 203 on which the streaming process 230 runs depending on the load of the streaming server 206.

The content creation server scaler 204, encoding server scaler 205, and streaming server scaler 206 check the load status of the servers they each manage, and if the load increases and expansion of the server is necessary, expand the corresponding server. , by disabling the server when the load decreases, efficient server management can be made possible. In this way, cloud computing technology can be used to control servers, and the present invention is not limited to such specific server management technology, and conventional technologies that can efficiently and immediately control the number of servers can be applied. do.

Figure 5 is a diagram showing an example of a management server for managing each server in a 3D content streaming system according to an embodiment of the present invention.

As shown in the figure, the 3D content streaming system according to an embodiment of the present invention may further include a synchronization server 207 that synchronizes information about the 3D content of a plurality of server-side clients 210. there is.

The synchronization server 207 exchanges synchronization information with the content logic processing unit 211 of the server-side client 210 and allows multiple users to share the same environment. If the 3D content service provided is a service where multiple users are located at the same time in one virtual space and use the service, such as a game or metaverse, the actions of one user will affect the screens of other users, and this must be done in real time. Therefore, the content logic processing units 211 that manage the content of each user terminal web browser 102 must share information about the 3D virtual space with each other. Various methods can be applied to synchronize virtual space information, and the present invention is not limited to such specific synchronization methods.

In addition, the 3D content streaming system of the present invention may be configured to further include a user session management server 208 that manages the connection relationship between the web browser 102 of the user terminal and the server-side client 210. As described above, the server-side client 210 combines a number of tasks that were actually running on multiple user terminals and transferred them to the server side, and is configured to correspond 1:N with the user terminal web browser 102, thereby forming a single The server-side client 210 is configured to process the tasks of a predetermined number of user terminal web browsers 102.

The user transmits a user command to the content logic processing unit 211 through the user terminal web browser 102 and receives the corresponding video from the streaming process 230. Therefore, a function to manage the connection between the web browser 102 of the user terminal and the content logic processing unit 211 is required, and the user session management server 208 performs that role.

In addition, the 3D content streaming system of the present invention further includes a server session management server 209 that manages the connection relationship between the server-side client 210, the encoding process 220, and the streaming process 230. It can be. As described above, the original 3D content image generated by one server-side client 210 is processed by a plurality of encoding processes 220, and since each process operates on a different server, which data is stored in which process. You need to manage the information on how to process and transfer it to which process. As such, the server session management server 209 plays a role in managing which process on which server a task is processed and which task should be transferred to which process on which server.

Although the above has been described with reference to embodiments, those skilled in the art can make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the following patent claims.

102: User terminal web browser 201: Content creation server
202: encoding server 203: streaming server
210: Server-side client 220: Encoding process
230: Streaming process
211: Content logic processing unit 212: Rendering unit

Claims (6)

A plurality of content creation servers that run one or more server-side clients that generate original images of 3D content;
A plurality of encoding servers that drive one or more encoding processes to generate streaming video by encoding the original video of the generated 3D content; and
A plurality of streaming servers that run one or more streaming processes that stream the encoded streaming video to the web browser of the user terminal
Including,
The server-side client is
a content logic processing unit that receives a user command from a web browser of the user terminal and configures 3D content according to the user command; and
A rendering unit that renders the 3D content created in the content logic processing unit and generates an original image of the 3D content.
A 3D content streaming system comprising: According to paragraph 1,
a content creation server scaler that adjusts the number of content creation servers on which the server-side client runs according to the load of the plurality of content creation servers;
an encoding server scaler that adjusts the number of encoding servers on which the encoding process runs according to the load of the plurality of encoding servers; and
A streaming server scaler that adjusts the number of streaming servers on which the streaming process runs according to the load of the plurality of streaming servers.
A 3D content streaming system further comprising: According to paragraph 2,
The plurality of content creation servers are composed of servers including CPU and GPU,
The plurality of encoding servers are composed of GPU clusters including GPUs,
The plurality of streaming servers are composed of servers including a CPU.
A 3D content streaming system featuring. According to paragraph 3,
A synchronization server that synchronizes information about the 3D content of a plurality of server-side clients
A 3D content streaming system further comprising: According to clause 4,
A user session management server that manages the connection relationship between the web browser of the user terminal and the server-side client.
It further includes,
The web browser of the user terminal and the server-side client correspond in an N:1 manner.
A 3D content streaming system featuring. According to clause 5,
A server session management server that manages the connection relationship between the server-side client, the encoding process, and the streaming process.
A 3D content streaming system further comprising: