KR20200121596A - High Available Multi Encoder system for non-stop live video transmission - Google Patents

Abstract

The present invention relates to an encoder multiplexing system for non-stop live video transmission. The present invention provides an encoding multiplexing system encoding a video, comprising: a first and a second encoder encoding a video received from a broadcasting apparatus, which dually transmits a video of the same channel, to transmit the encoded video by the unit of a frame in real-time, while starting to encode the video at a setup time concurrently, adding a timestamp to transmit the same frame at the same time, and encoding the video in the same manner to perform real-time transmission; and a streaming server receiving the videos from the first and the second encoder to store the videos at a memory, allowing a regenerator to perform real-time streaming transmission while buffering, at the memory, the video of the first encoder having a priority between the first and the second encoder, and at the same time deleting the video of the second encoder from the memory. The present invention can offer a seamless streaming service of a live video of a corresponding channel based on an encoder duplexing structure which supports two encoders with respect to the video of the same channel.

Description

High Available Multi Encoder system for non-stop live video transmission

The present invention relates to an encoder multiplexing system for transmitting a non-stop live broadcast video, and more particularly, to an encoder multiplexing system for transmitting a non-stop live broadcast video capable of providing a live video to a player without interruption using duplication of an encoder.

In general, media content such as video and music is accessed and used by a plurality of user terminals through the Internet.

In recent years, the demand for live broadcasting streaming services is also gradually increasing due to the development of communication environment, diversification of contents, the appearance of various video services, and segmentation of service age groups.

In general, a user terminal such as a player accesses a streaming server to receive a real-time streaming service of a video of a desired channel. The streaming server receives the encoded video for the channel from the encoder and delivers it to the player.

However, if the connection between the encoder and the streaming server is interrupted due to a communication error, connection failure, or equipment failure, the player cannot receive the video any more. Therefore, there is a need for a new system capable of seamlessly streaming video while being robust against failures and errors.

The technology behind the present invention is disclosed in Korean Patent Publication No. 2016-00135461 (published on November 28, 2016).

An object of the present invention is to provide an encoder multiplexing system for transmitting a non-stop live broadcast video capable of providing a live video without interruption by duplexing an encoder.

In the present invention, in an encoding multiplexing system for encoding an image, an image received from a broadcasting device that transmits an image of the same channel is encoded and transmitted in real time in a frame unit, but encoding of the image is simultaneously started at a set time, First and second encoders for real-time transmission by assigning a time stamp to each frame to transmit the same frame at the same time and encoding the image in the same manner, and receiving images from the first and second encoders and storing them in memory, respectively And, encoding multiplexing including a streaming server for real-time streaming transmission to the player while buffering the image of the first encoder having a priority among the first and second encoders in the memory, and deleting the image of the second encoder from the memory. System.

Further, when the connection of the first encoder is interrupted, the streaming server may perform real-time streaming transmission to the player while buffering an image of the second encoder in a memory from a corresponding frame corresponding to the interruption point.

In addition, when the connection of the first encoder is interrupted, the streaming server extracts a target frame having the same time stamp as the corresponding frame corresponding to the interruption point from the image of the second encoder, and buffers the extracted target frame in real time. Streaming transmission is possible.

In addition, when the connection of the first encoder is resumed and the image of the first encoder starts to be input to the memory, the image of the first encoder is deleted from the memory while maintaining the streaming state of the image of the second encoder. I can make it.

In addition, the first and second encoders operate in synchronization with a time and a codec rule, and simultaneously start encoding an image at the set point based on the synchronized time, and a video based on the synchronized codec rule. Can be encoded in the same way and transmitted in real time.

In addition, the codec rule is a codec rule of video and audio required for encoding of the image, and in the case of the video, at least one of a key frame interval, whether a B frame is used, a bit rate, and a resolution A rule is included, and in the case of the audio, at least one rule of a sampling rate and a bit rate may be included.

In addition, any one of the first and second encoders is a master encoder, the others are slave encoders, and the master encoder provides the codec rule to the master encoder when information about the codec rule is set by the user. Codec rules can be synchronized between encoders.

In addition, any one of the first and second encoders is a master encoder, the others are slave encoders, and the master encoder sets an encoding start point when receiving an encoding start command for the video from a user and the slave encoder The master encoder and the slave encoder can simultaneously start the encoding operation for the image as soon as the encoding start time arrives using a time synchronized by a time server.

In addition, the streaming server initially assigns the priority to an encoder that has priority access to the streaming server and a network among the first and second encoders, and may change the priority according to a network connection state.

According to the present invention, a live video of a corresponding channel can be seamlessly streamed based on an encoder duplication structure that supports two encoders for an image of the same channel.

In addition, the present invention selects and streams the video of the first encoder having priority among the first and second encoders, but when the connection of the first encoder is interrupted, the video of the second encoder can be connected and transmitted from the stop frame. , You can stably stream the video of the channel from the player without interruption.

Moreover, in the present invention, since both the time and the codec setting are synchronized between the two encoders, even if the video of the second encoder is switched and transmitted due to the disconnection of the connection while streaming the video of the first encoder, it is naturally You can stream the connected live video.

1 is a diagram showing an encoder multiplexing system according to an embodiment of the present invention.
2 is a diagram showing the configuration of the streaming server shown in FIG.
FIG. 3 is a diagram illustrating a concept in which FIG. 1 is extended to images of multiple channels.

Then, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention.

The present invention is an encoder multiplexing system for transmitting non-stop live broadcast video, and based on a dual encoder structure supporting two encoders for video of the same channel, live video of the corresponding channel can be streamed to a player without interruption.

In an embodiment of the present invention, when the streaming server receives the video of the corresponding channel from the mutually synchronized first and second encoders, the video of the first encoder with priority is selected and streamed to the player, but the first encoder is connected. When this is stopped, the video from the second encoder is immediately successively transmitted from the frame at the time of the interruption, so that the live broadcast video can be stably streamed without interruption.

1 is a diagram showing an encoder multiplexing system according to an embodiment of the present invention.

As shown in Fig. 1, the encoder multiplexing system according to the embodiment of the present invention includes a first encoder 100, a second encoder 200, and a streaming server 300, and two encoders for an image of the same channel. The video is encoded based on the dualized encoding structure including, and streamed to the player 400.

The first and second encoders 100 and 200 are located between the broadcasting device 10 and the streaming server 300, and are network-connected to the broadcasting device 10 and the streaming server 300.

The broadcasting device 10 distributes an image of a predetermined channel acquired by using the camera 11 through the distributor 12 and transmits it in duplicate. Here, the image transmitted from the broadcasting device 10 may correspond to a live broadcast image (live image), but may correspond to a previously stored or recorded image. Hereinafter, an embodiment of the present invention will be described with a live broadcast video as a representative example.

The broadcasting device 10 may correspond to a broadcasting station equipment, a broadcasting station server, etc. that service images of multiple channels. Here, the camera 11 for photographing an image may be included in the broadcasting device 10, but may be connected to the outside of the broadcasting device 10.

In the case of FIG. 1, for convenience of explanation, an image of channel 100 (CH 100) is duplicated and transmitted among a plurality of channels.

The first and second encoders 100 and 200 receive the corresponding image from the broadcasting device 10 that double-transmits the image of the same channel (channel 100 in the case of FIG. 1), and encodes the received image in real time frame by frame. send.

The first and second encoders 100 and 200 simultaneously start encoding an image at a set time point. In addition, the first and second encoders 100 and 200 assign a time stamp to each frame so as to transmit the same frame at the same time, and encode the image in the same manner to transmit the same in real time.

Here, the first and second encoders 100 and 200 operate in synchronization with a time and a codec rule, and simultaneously start encoding an image at a set time based on the synchronized time, and perform the same image based on the synchronized codec rule. Encoding method.

Specifically, the first and second encoders 100 and 200 are time-synchronized with each other by a time server (for example, a Network Time Protocol (NTP) server), so that when the set point (encoding start point) arrives, the received image is At the same time, the encoding operation is started.

A specific example is as follows. In the embodiment of the present invention, one of the first and second encoders 100 and 200 corresponds to a master encoder and the other corresponds to a slave encoder.

If, assuming that the first encoder 100 is a master encoder, the first encoder 100 (master encoder) sets an encoding start time when receiving an encoding start command for an image from the user, and this is the second encoder 200. ) (Slave Encoder) to share. Here, the user may correspond to an administrator.

Then, the first and second encoders 100 and 200 simultaneously start encoding an image as soon as an encoding start time arrives based on the synchronized time.

In other words, if the user commands to start encoding, the master encoder delivers the exact start time in milliseconds or microseconds to the slave encoder, and both encoders start encoding at the same time. . For example, 5 seconds after the user presses the start button, the two encoders 100 and 200 simultaneously start the encoding operation.

In addition, the first and second encoders 100 and 200 may assign a time stamp synchronized to each frame in a predetermined order or number so as to transmit the same frame at the same time. Accordingly, frames to which the same time stamp is assigned can be transmitted at the same time. This operation can also be premised on mutual time synchronization.

Here, since image data generally includes audio and video, a synchronized time stamp value may be generated for each frame of audio and video. Since the synchronized time stamp is used, audio and video can be synchronized by creating the same time stamp in the same time frame.

In addition, the first and second encoders 100 and 200 synchronize codec rules in addition to time synchronization. Here, the codec rule represents a codec rule for video and audio required for video encoding.

The codec rule of the video may include at least one of a key frame interval, whether a B frame is used, a bit rate, and a resolution. In the case of audio, the codec rule may include at least one of a sampling rate and a bit rate.

Synchronization of codec rules can also be achieved based on information sharing between master and slave encoders, similar to synchronization of an encoding start point.

Assuming that the first encoder 100 is a master encoder, the first encoder 100 (master encoder) transmits and shares the codec rule to the master encoder when information about the codec rule is set by the user, thereby providing a codec between the two encoders. Synchronize the rules. Here, the user may correspond to an administrator.

In this way, the first and second encoders 100 and 200 operate in synchronization with the time and codec rules, and simultaneously start encoding the image received from the broadcasting device 10 and encode the image in the same manner to the streaming server ( 300) in real time.

In an embodiment of the present invention, the encoding operation of each of the encoders 100 and 200 may include a packetization operation. Packetization is the operation of dividing one encoded frame into a certain size. For example, an image frame of 64000 bytes may be divided into 100 640 bytes, and the order of packets may be recorded.

Since the two encoders have the same encoding start time, if the start order value is counted from 0, the recorded order always has a value. The two encoders share the size of each other's encoded frame bytes, so that the number of generated packets is matched with respect to frames encoded at the same time when packetization is performed.

The streaming server 300 receives images of the same address from the first and second encoders 100 and 200 and stores them in the memory 320, respectively, and stores the images of the first encoder having a priority among the first and second encoders. While buffering in 320, real-time streaming transmission is performed to the player 400, and the image of the second encoder 200 is deleted from the memory. Here, buffering refers to a buffer transmission operation of data normally performed in a streaming server.

The streaming server 300 initially gives priority to an encoder in which the streaming server 300 and the network connection have been made priority among the first and second encoders 100 and 200.

In addition, the streaming server 300 may change the priority according to the network connection state. For example, when both the first and second encoders 100 and 200 are initially connected, priority is given to the first encoder 100, which is confirmed to be connected first. Thereafter, when disconnection (connection interruption) of the first encoder 100 occurs, priority is given to the second encoder 200 currently maintaining the connection. Here, even if the connection of the first encoder 100 is resumed again later, the priority of the second encoder 200 is maintained since the connection of the second encoder 200 is maintained.

The streaming server 300 compares the access times of the two encoders 100 and 200, and gives priority to the first connected encoder. Hereinafter, it is assumed that the encoder connected first is the first encoder 100.

In this case, the streaming server 300 selects an image of the first encoder 100, which is connected first among the two encoders 100 and 200, and provides the image to the player 400, and at the same time, the image of the second encoder 200 is transmitted to the memory. 320) in real time. That is, the streaming server 300 only drops data while maintaining the connection with the second encoder 200. Of course, according to this operation, it is possible to efficiently use limited memory resources.

Here, when the connection of the first encoder 100 is interrupted, the streaming server 300 buffers the image of the second encoder 200 in the memory 320 from the frame corresponding to the interruption point in real time to the player 400 Streaming transmission.

When the connection of the first encoder 100 is interrupted, only the image of the second encoder 200 is input to the memory 320. in this case. The streaming server 300 extracts a frame having the same time stamp as the corresponding frame corresponding to the stop time, that is, the target frame from the image of the second encoder 200, buffers the extracted target frame and transmits real-time streaming.

For example, when the connection of the first encoder 100 is terminated, an I frame corresponding to the same time as the end point in the buffering data of the first encoder 100 is extracted from the image of the second encoder 200 and replaced. Through this, the data of the second encoder 200 is transmitted to the client.

Of course, after the connection of the first encoder 100 is resumed and the image of the first encoder 100 starts to be input to the memory 320, the second encoder 200 maintains the streaming state of the image. The image of the first encoder 100 is deleted from the memory 320. That is, the streaming server 300 drops only data while maintaining the connection with the first encoder 100.

As described above, in the embodiment of the present invention, since the time and codec settings are all synchronized between the first and second encoders 100 and 200, the connection of the first encoder 100 while streaming the image of the first encoder 100 Even if the video of the second encoder 200 is switched and transmitted due to the interruption, it is possible to stream the live broadcast video naturally connected from the frame of the interrupted time point.

This means that it is possible to prevent a phenomenon in which an image from a past or an image from a future point of view is unnaturally connected rather than the point where it was stopped.

As described above, in an embodiment of the present invention, an image of one encoder having a priority among the first and second encoders is first selected and streamed to the player 400, but the priority can be switched in real time according to the network connection state. Thus, even if an error occurs in one encoder, it is possible to adaptively cope with the error by using the other encoder.

2 is a diagram showing the configuration of the streaming server shown in FIG. The streaming server 300 includes a receiving unit 310, a memory 320, a setting unit 330, and a transmitting unit 340.

The receiving unit 310 receives an image from the first and second encoders 100 and 200 and transmits the image to the memory 320, and the memory 320 stores the transmitted image.

The setting unit 330 initially gives priority to an encoder in which a network connection with the streaming server 300 has been prioritized among the first and second encoders 100 and 200. Also, the setting unit 330 may change the priority according to the network connection state.

The transmission unit 340 controls the memory 320 and the setting unit 330, and the player 400 buffers the image of the first encoder having a priority among the first and second encoders 100 and 200 into the memory 320. Simultaneously with real-time streaming transmission, the image of the second encoder 200 is deleted from the memory.

FIG. 3 is a diagram illustrating a concept in which FIG. 1 is extended to images of multiple channels.

As shown in FIG. 3, the embodiment of the present invention can implement encoder duplication individually for images of a plurality of channels, and provide a streaming service based on this.

In the case of FIG. 3, images from channels 1 to N are illustrated, and the broadcasting device 10 may transmit images of each channel in duplicate using an individual distributor. In addition, the first and second encoders 100 and 200 are composed of N sets, and encode and transmit an image of a channel allocated to them.

The streaming server 300 may provide images of multiple channels including channels 1 to N to the user, and may provide the image of the channel selected by the user to the player 400. The player 400 may refer to a general user terminal such as a smart phone, PDA, desktop, pad (PAD), and notebook.

According to the present invention as described above, based on an encoder duplication structure supporting two encoders for an image of the same channel, a live image of a corresponding channel can be seamlessly streamed.

In addition, the present invention selects and streams the video of the first encoder having priority among the first and second encoders, but when the connection of the first encoder is interrupted, the video of the second encoder can be connected and transmitted from the stop frame. , You can stably stream the video of the channel from the player without interruption.

The present invention has been described with reference to the embodiments shown in the drawings, but these are only exemplary, and those of ordinary skill in the art will appreciate that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical scope of the present invention should be determined by the technical spirit of the appended claims.

100: first encoder 200: second encoder
300: streaming server 400: player

Claims (9)

In the encoder multiplexing system for encoding an image,
Encodes the image received from the broadcasting device that transmits the image of the same channel and transmits it in real time frame by frame, but at the set point, encoding of the image starts at the same time, and a time stamp is given to each frame so that the same frame is transmitted at the same time. And first and second encoders for encoding the image in the same manner and transmitting it in real time; And
Receives an image from the first and second encoders and stores them in a memory, respectively, buffers the image of the first encoder having a priority among the first and second encoders in the memory, and transmits real-time streaming to the player. 2 Encoder multiplexing system including a streaming server that deletes the video of the encoder from the memory. The method according to claim 1,
The streaming server,
When the connection of the first encoder is interrupted, an encoder multiplexing system for real-time streaming transmission to the player while buffering the image of the second encoder in a memory from a corresponding frame corresponding to the interruption point. The method according to claim 2,
The streaming server,
When the connection of the first encoder is interrupted, an encoder multiplexing system that extracts a target frame having the same time stamp as the frame corresponding to the interruption point from the image of the second encoder, buffers the extracted target frame and transmits real-time streaming. The method according to claim 2,
When the connection of the first encoder is resumed and the image of the first encoder starts to be input to the memory, an encoder that deletes the image of the first encoder from the memory while maintaining the streaming state of the image of the second encoder Multiplexing system. The method according to claim 1,
The first and second encoders,
Encoder multiplexing that operates in synchronization with the time and codec rules, and simultaneously starts encoding the video at the set point based on the synchronized time, and encodes the video in the same manner based on the synchronized codec rule and transmits it in real time. system. The method of claim 5,
The codec rule,
These are the video and audio codec rules required for encoding the video,
In the case of the video, at least one rule among a key frame interval, whether or not a B frame is used, a bit rate, and a resolution is included, and in the case of the audio, at least one of a sampling rate and a bit rate Encoder multiplexing system including the rules of. The method of claim 5,
Any one of the first and second encoders is a master encoder, and the other is a slave encoder,
The master encoder,
An encoder multiplexing system for synchronizing codec rules between encoders by providing the codec rule to the master encoder upon receiving information on the codec rule from a user. The method according to claim 1,
Any one of the first and second encoders is a master encoder, and the other is a slave encoder,
The master encoder,
When a user inputs an encoding start command for the video, an encoding start point is set and provided to the slave encoder,
The master encoder and slave encoder,
An encoder multiplexing system that simultaneously starts an encoding operation for the image as soon as the encoding start time arrives using a time synchronized by a time server. The method according to claim 1,
The streaming server,
Initially assigning the priority to an encoder in which the streaming server and the network connection have priority among the first and second encoders,
An encoder multiplexing system that changes the priority according to a network connection state.

Images