KR20230126062A - Network traffic reduction system and method for non-face-to-face online education - Google Patents

Abstract

A network traffic reduction system for non-face-to-face online education is disclosed. The system is equipped with a browser capable of executing WebRTC, implemented with a plurality of user nodes and a Janus WebRTC or TURN server that obtains each user video stream through a webcam, and user video obtained from each of the plurality of user nodes A relay server that receives streams and streams a single video stream of the received user video streams to the plurality of user nodes, and individually decodes, mixes, downscales, and encodes the user video streams received by the relay server and a process node generating the single video stream according to a procedure and transmitting the generated single stream to the relay server.

Description

Network traffic reduction system and network traffic reduction method for non-face-to-face online education {NETWORK TRAFFIC REDUCTION SYSTEM AND METHOD FOR NON-FACE-TO-FACE ONLINE EDUCATION}

An embodiment according to the concept of the present invention relates to a technique for reducing network traffic in multilateral video transmission such as video education or video conference, and more specifically, mixing individually decoded user images and downscaling the mixed images. The present invention relates to a network traffic reduction system and traffic reduction method for non-face-to-face online education that can significantly reduce multi-party network traffic by post-encoding.

Since the corona pandemic, the demand for non-face-to-face solutions continues to grow. A video solution, such as a video call or a video conference, uses a communication system composed of various terminals and lines to allow people who are far away to communicate while seeing each other through a video screen. Conventional 1:1 video solutions have been in various forms, but they are not suitable for schools or government offices that need to share multilateral video, but when the 1:1 video system is expanded and the service is provided, the number of participants increases linearly. This was because the increase in CPU usage caused by operating the media player to play each video stream and the amount of traffic to play each video stream acted as an insurmountable barrier.

A technical problem to be solved by the present invention is a network traffic reduction system for non-face-to-face online education that can dramatically reduce multi-party network traffic by mixing individually decoded user images, downscaling the mixed images, and then encoding them. And to provide a network traffic reduction method using the same.

A network traffic reduction system for non-face-to-face online education according to an embodiment of the present invention is equipped with a browser capable of running WebRTC, and includes a plurality of user nodes that acquire each user video stream through a webcam and a Janus WebRTC or TURN server and a relay server for receiving user video streams obtained from each of the plurality of user nodes and streaming a single video stream for the received user video streams to the plurality of user nodes. and a process node generating the single video stream by individually decoding and down-scaling the received user video streams, and transmitting the generated single video stream to the relay server.

According to an embodiment, the process node receives and individually decodes the user video streams from the relay server, generates a mixing stream by copying the individually decoded user video streams to a predetermined slot of a memory area for mixing, and The single video stream may be generated by down-scaling the generated mixing stream according to the qHD resolution and encoding the down-scaling mixing stream through a VP8 codec.

According to an embodiment, the process node detects that the relay server obtains a video stream from a new user node other than the plurality of user nodes, and requests the relay server to forward the video stream transmitted from the new user node. It may be characterized in that the single video stream is generated by doing so.

A method for reducing network traffic for non-face-to-face online education according to an embodiment of the present invention includes the steps of transmitting a user video stream acquired by each of a plurality of user nodes to a relay server using WebRTC, and the process node transmitting to the relay server. Requesting and receiving the forwarded user video streams, generating a mixing stream by decoding the received user video streams by the process node and inserting them into a predetermined memory slot, and the process node generating the mixing stream Generating a single stream by downscaling and encoding to a predetermined resolution, and streaming the generated single stream to the plurality of user nodes by a relay server.

According to an embodiment, the step of generating the single stream by the process node may include downscaling the generated mixing stream according to qHD resolution and encoding the downscaled mixing stream through a VP8 codec. there is.

As described above, the system and method for reducing network traffic for non-face-to-face online education according to an embodiment of the present invention have an effect of dramatically reducing traffic in a multilateral image transmission network by using a server mixing method.

Accordingly, the network traffic reduction system and network traffic reduction method for non-face-to-face online education according to an embodiment of the present invention have an effect of dramatically lowering the CPU usage rate of each user node.

A detailed description of each drawing is provided for a more complete understanding of the drawings cited in the detailed description of the present invention.
1 is a block diagram showing the internal configuration of a network traffic reduction system for non-face-to-face online education according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an example in which the process node shown in FIG. 1 individually decodes the received user image and inserts it into a slot.
3 is a flowchart illustrating a method for reducing network traffic for non-face-to-face online education according to an embodiment of the present invention.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are only illustrated for the purpose of explaining the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention It can be embodied in various forms and is not limited to the embodiments described herein.

Embodiments according to the concept of the present invention can apply various changes and can have various forms, so the embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosure forms, and includes all changes, equivalents, or substitutes included in the spirit and technical scope of the present invention.

Terms such as first or second may be used to describe various components, but the components should not be limited by the terms. The above terms are only for the purpose of distinguishing one component from another component, e.g., without departing from the scope of rights according to the concept of the present invention, a first component may be termed a second component, and similarly The second component may also be referred to as the first component.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle. Other expressions describing the relationship between elements, such as "between" and "directly between" or "adjacent to" and "directly adjacent to" should be interpreted similarly.

Terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this specification, terms such as "comprise" or "having" are intended to indicate that the described feature, number, step, operation, component, part, or combination thereof is present, but that one or more other features or numbers are present. However, it should be understood that it does not preclude the presence or addition of steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

Terms such as those defined in commonly used dictionaries should be interpreted as including a meaning consistent with the meaning in the context of the related art, and unless explicitly defined herein, interpreted in an ideal or excessively formal meaning. It doesn't work.

Hereinafter, the present invention will be described in detail by describing preferred embodiments of the present invention with reference to the accompanying drawings.

1 is a block diagram showing the internal configuration of a network traffic reduction system (hereinafter referred to as 'network traffic reduction system 10') for non-face-to-face online education according to an embodiment of the present invention.

Referring to FIG. 1 , the network traffic reduction system 10 includes a plurality of user nodes 100-1 to 100-n, a relay server 300, and a process node 500.

The plurality of user nodes 100-1 to 100-n in the detailed description of the present invention are devices having a communication function, and may be equipped with a web browser supporting WebRTC (Web-based Real Time Communication) service, It may be implemented in a smart phone, tablet PC, desktop PC, laptop PC, or the like.

The relay server 300 is a WebRTC server that relays stream data of a plurality of user nodes, and may be implemented as a Janus WebRTC server or a TURN server according to embodiments.

WebRTC is a communication standard technology that enables real-time voice and video calls based on a browser without plug-ins on the Web. ), if there is only one means of recognizing a specific person or application service, the person or the entire group can have a conversation with voice, video, or text, and send or receive work data together. , Sometimes you can hold a meeting while watching a video screen with several people.

In addition, if you want to find a specific person, you can access it only if that person exposes himself to the web environment (for example, a social network service application, etc.). Except for that, you don't need to install any software.

The relay server 300 provides basic functions for exchanging stream data between the user nodes 100-1 to 100-n, and individual functions may be provided as plug-ins.

For example, the relay server 300 provides various input/output formats and decoding/encoding functions through a media library such as FFmpeg.

In addition, the relay server 300 provides an individual function of forwarding the WebRTC stream in the RTP format through the VideoRoom plug-in, and provides an individual function of streaming the RTP stream in the WebRTC format through the Streaming plug-in. can do.

The relay server 300 requests a forward stream of the user nodes 100-1 to 100-n through the forward and streaming functions, receives the stream according to the request, and transmits the stream to the process node 500.

The process node 500 generates a new single stream by processing the stream of each user node 100-1 to 100-n transmitted through the relay server 300, and returns the generated single stream to the relay server 300. ) to each user node 100-1 to 100-n.

The process node 500 may provide a service in the form of a micro-service, and a user may request a function through a RESTful API.

At this time, the function may be composed of methods of POST and GET, functions may be branched according to URLs, the POST may be used for creation and change, and the GET may be configured to return the current state. can

The creation and destruction target two objects, and the two objects become a room and a participant included in the room.

If the process node 500 generates the single stream in more detail, the process node 500 individually decodes user video streams delivered through the relay server 300, and mixes each of the individually decoded videos A new mixing stream is created by copying to a predetermined slot of the memory area for.

FIG. 2 is a diagram illustrating an example in which the process node 500 shown in FIG. 1 individually decodes received user image frames and inserts them into slots.

Referring to FIGS. 1 and 2 , grid slots for mixing increase according to the number of reserved users. When the number of users is large, the output resolution may unnecessarily increase.

Accordingly, the process node 500 down-scaling the generated mixing stream according to a predetermined resolution and encodes the down-scaling mixing stream at regular frame rate intervals to generate a final single stream.

For example, if the number of participants is 100, and the number of grid slots is 10 each horizontally and vertically to include all the videos corresponding to the number of participants, and the resolution of the input source is 256 x 144, the total is 2560 x 1440 will have an output resolution of

As described above, a high resolution not only greatly affects the encoding speed, but also requires a high bit rate.

That is, the size of the output video has a great influence on the amount of traffic, and it is necessary to reduce the size of the video to a level where the quality can be tolerated.

According to an embodiment, the process node 500 may set the resolution for downscaling to qHD (960 X 540) so that there is no large delay in response speed (latency) and CPU usage can be implemented below a predetermined reference value.

Thereafter, the process node 500 generates the single stream by extracting and encoding the new data (downscaled mixing stream) obtained through the above process at regular frame rate intervals.

In the case of H.246, accelerated encoding is supported by many GPUs, but since VP8 is a codec supported by various operating systems and browsers, the process node 500 encodes the extracted frames using VP8.

Also, the process node 500 performs streaming through the RTP protocol.

The amount of video data may temporarily soar when the amount of change in video is large. In case of setting to VBR (Variable Bit Rate) in a restricted network environment, network connection failure may occur, so the process node (500 ) is set to CBR (Constant Bit Rate) to keep the stream data constant.

Meanwhile, in some network environments, when a UDP packet is fragmented, the corresponding packet is blocked to prevent smooth transmission, so the process node (500) adjusts the length of the packet so that it does not exceed the maximum transmission unit (MTU). do.

Thereafter, the process node 500 transmits the generated single stream to the relay server 300 again, and the relay server 300 transfers the single stream transmitted through the process node 500 to each of the user nodes 100-1 to 100. -n) to stream.

According to an embodiment, the process node 500 may detect whenever a new user participates and request the relay server 300 to forward a video data stream for the new participant, and accordingly, the relay server 300 may request The received video data stream for the new participant is transmitted to the process node 500 .

Thereafter, the process node 500 may create a new mixing frame by individually decoding the received video data stream for the new user and copying it to a predetermined reserved slot in the memory area.

As described above, the process node 500 may generate a final single stream by encoding the generated mixing stream at regular frame rate intervals.

As described above, the network traffic reduction system 10 according to an embodiment of the present invention can drastically reduce traffic through individual stream mixing in the server, and the user only needs to play one media player, so less CPU usage. can be expected

That is, compared to the full mesh method using only WebRTC or the existing MCU method using only Web Socket, it is possible to achieve an effect of drastic traffic reduction.

3 is a flowchart illustrating a method for reducing network traffic for non-face-to-face online education according to an embodiment of the present invention.

1 to 3, each of the user nodes 100-1 to 100-n obtains a user video stream using a built-in camera or a plug-in webcam, and transmits the obtained user video stream to a relay server ( 300) (S100).

Depending on the embodiment, the bit rate can be set to 512Kb, the codec to VP8, the FIR frequency to 3, and the resolution to 144p (256 X 144).

The process node 500 forwards the user video stream transmitted to the relay server 300 and receives the user video stream from the relay server 300 (S200).

At this time, the process node 500 detects each time a new user node participates in the relay server 300 and forwards the new user video stream transmitted to the relay server 300 and receives it (S300).

Sequentially, the process node 500 individually decodes all user video streams, including the received new user video stream (S330), and copies them to a reserved slot in the memory area for mixing (S350) to create a new mixing stream. (S400).

Meanwhile, the grid slots for mixing increase according to the number of reserved users, and when the number of users is large, the output resolution may unnecessarily increase.

The process node 500 sets the number of grids according to the number of participants. For example, when the number of participants is 100, the number of grids is set to 10 horizontally and vertically.

In addition, when the resolution of the input source video, that is, the resolution of the received user video stream is set to 256 X 144, the total output resolution is 2560 X 1440, so the process node 500 converts the mixing stream to qHD (960 X 540) Downscaling to the resolution (S500).

Sequentially, the process node 500 generates a single video stream by encoding (S530) the down-scaled mixing stream to VP8 (S550).

Thereafter, the process node 500 transmits the generated single video stream to the relay server 300 (S570).

The relay server 300 streams the transmitted single video stream to each of the user nodes 100-1 to 100-n (S600).

As described above, the method for reducing network traffic for non-face-to-face online education according to an embodiment of the present invention can drastically reduce traffic through individual stream mixing in the server, and the user only needs to play one media player. You can expect low CPU usage.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

10: Network traffic reduction system
100-1 to 100-n: user nodes
300: relay server
500: process node

Claims (5)

A plurality of user nodes equipped with a browser capable of executing WebRTC and acquiring each user video stream through a webcam;
A relay server implemented as a Janus WebRTC or TURN server, receiving user video streams obtained from each of the plurality of user nodes, and streaming a single video stream for the received user video streams to the plurality of user nodes ; and
A process node for processing the user video streams received by the relay server according to a predetermined procedure to generate the single video stream, and transmitting the generated single video stream to the relay server; for non-face-to-face online education including Network traffic reduction system. The method of claim 1, wherein the process node,
The user video streams are received from the relay server and individually decoded, and a mixing stream is generated by copying the individually decoded user video streams to a predetermined slot of a memory area for mixing, and the generated mixing stream is generated according to qHD resolution Network traffic reduction system for non-face-to-face online education, characterized in that for down-scaling and generating the single video stream by encoding the down-scaling mixing stream through a VP8 codec. The method of claim 1, wherein the process node,
Detecting that the relay server acquires a video stream from a new user node other than the plurality of user nodes, and generating the single video stream by requesting a forward video stream transmitted from the new user node to the relay server. Network traffic reduction system for non-face-to-face online education. Transmitting a user video stream acquired by each of a plurality of user nodes to a relay server using WebRTC;
Requesting and receiving, by a process node, forward user video streams transmitted to the relay server;
generating a mixing stream by decoding each of the received user video streams and inserting them into a predetermined memory slot by the process node;
Generating, by the process node, a single stream by downscaling and encoding the generated mixing stream to a predetermined resolution; and
Network traffic reduction method for non-face-to-face online education comprising; streaming the generated single stream to the plurality of user nodes by a relay server. The method of claim 4, wherein the step of generating the single stream by the process node comprises:
Downscaling the generated mixing stream according to qHD resolution;
Network traffic reduction method for non-face-to-face online education including; encoding the down-scaling mixing stream through a VP8 codec.