KR20110051733A - Network-adaptive variable stream layered method and apparatus for video streaming - Google Patents

Abstract

PURPOSE: A network-adaptive variable stream layered method and apparatus for video streaming are provided to convert a frame of video data transmitted to a reception terminal having a reduced available bandwidth among reception terminals receiving video data in an IP-based wired/wireless network environment in order to provide a continuous streaming service. CONSTITUTION: A network-adaptive variable stream layered apparatus(400) comprises: a stream analysis unit(440) which outputs NAL information of a video stream by analyzing a video stream compressed H.264/AVC; a map table generation unit which arranges a frame included in the video stream based on the outputted NAL information; and a rate control unit(430) which selectively removes or selects the arranged frame in consideration of the NAL information and the packet loss rate of the available bandwidth of each reception terminal.

Description

Network adaptive variable stream tiering device and method for video streaming {NETWORK-ADAPTIVE VARIABLE STREAM LAYERED METHOD AND APPARATUS FOR VIDEO STREAMING}

One embodiment of the present invention provides a frame rate of video data being transmitted to a receiving terminal whose available bandwidth is reduced among receiving terminals receiving high quality H.264 / AVC video data in an IP-based wired / wireless network environment. ) Can be provided in real time, thereby providing a seamless video streaming service.

When providing a high quality H.264 / AVC video stream in a broadcast service of an IP-based wired / wireless network environment, the available bandwidth of the receiving terminal is reduced due to network congestion and jitter, and loss of transmission video packets occurs. By doing so, we couldn't guarantee a seamless video streaming service.

In the related art, to solve this problem, a video stream is re-encoded using a bit-rate transcoding according to a dynamically changing network bandwidth loss rate, or SVC (scalable video), an international video compression standard. By coding and layering the video stream according to the performance of the receiving terminal in advance, the video stream being transmitted is selectively received according to the available bandwidth of the receiving terminal.

However, the method using the transform encoder has difficulty in adaptively converting and transmitting high-definition video data in real time to the available bandwidth reduced due to changes in the transmission environment of the network. This is because it is difficult to guarantee the QoS of the video data due to encoding delay and quality degradation which occur in the process of re-encoding the high quality video data according to the available bandwidth. In addition, when the available bandwidths of multiple receiving terminals watching the same broadcast program independently change, it is not sufficient to convert high-quality video data into a conversion encoder in real time according to the available bandwidth that each receiving terminal can allow. Follows.

In addition, the method using the SVC technology is difficult to process the high-quality video data in real time by layering and encoding the video data according to the available bandwidth of the receiving terminal in advance. In addition, there is a problem in that a lot of overhead for determining the layered stream according to the performance of the receiving terminal.

One embodiment of the present invention is the occupied bit of the frame constituting a group of pictures (GOP) in the compression domain to fit the network available bandwidth that allows a plurality of receiving terminals H.264 / AVC video stream encoded by the international video compression standard By removing frames according to the amount and layering them into video streams having multiple bit rates in real time, providing an apparatus and method for guaranteeing a seamless video streaming service and improving viewers' quality of experience (QoE). Its purpose is to.

The network adaptive variable stream layering apparatus according to an embodiment of the present invention analyzes a video stream compressed with H.264 / AVC and outputs NAL information of the video stream using the output NAL information. A map table generation unit for arranging frames included in the video stream in units of GOPs, a rate control unit for selectively removing or selecting the aligned frames with reference to the packet loss rate of the available bandwidth of each receiving terminal and the NAL information; And an output buffer for reconfiguring the selected frame to be decoded at each receiving terminal and transmitting the decoded frame to each receiving terminal.

In this case, the network adaptive variable stream layering apparatus further includes a first switching unit for determining whether the video stream is layered according to whether the available bandwidth allowed by each receiving terminal is changed, and the first switching unit is the available bandwidth. If there is no change, the video stream may be controlled to be transmitted to the receiving terminals as it is, and when the available bandwidth is reduced, the video stream may be transmitted to the stream analyzer to layer the video stream.

In this case, the video stream analyzer outputs information on the type of NAL, the order of the NALs, and the amount of NAL occupied bits, and the map table generator generates a frame included in the video stream according to the amount of occupied bits included in the NAL information. You can sort.

Here, the H.264 / AVC compressed video stream includes an I frame, which is an intra-frame compressed image, a P frame, which is a compressed image through forward prediction between frames, and a B, which is compressed by bidirectional prediction, in a frame constituting one GOP. It has a GOP structure composed of frames. In this case, the map table generation unit aligns I, P, and B frames according to the size of bits occupied by the NAL of the frames constituting the GOP. In this case, the rate controller may prevent the video stream from being broken by removing P frames and B frames other than the I frame.

In this case, the network adaptive variable stream layering apparatus further includes a GOP buffer for storing the frames included in the video stream in units of GOP, and the rate controller outputs from the GOP buffer by referring to the map table generator in a FIFO method. Among the frames to be removed, the frame to be removed may be controlled by the rate controller to remove the frames by one GOP, and the unremoved frames may be transferred to the output buffer.

According to an embodiment of the present invention, while sending video streams having the same bit rate to multiple receiving terminals, if the available bandwidth of the receiving terminals decreases, the compression domain is adapted to the available bandwidth of the receiving terminals without re-encoding the transmission video stream. By converting and transmitting the bit rate in the video stream, the bit rate of the video stream can be adjusted and layered and transmitted simultaneously to the corresponding receiving terminals.

According to an embodiment of the present invention, by selectively removing frames in a GOP of a video stream at high speed, a seamless video streaming service can be provided, and QoS for the video stream can be guaranteed.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings and accompanying drawings, but the present invention is not limited to or limited by the embodiments.

1 is a diagram illustrating an example of a video stream having a GOP structure according to an embodiment of the present invention.

Referring to FIG. 1, a video stream compressed with H.264 / AVC includes an I frame, which is an intra frame compressed image, a P frame, which is a compressed image through forward prediction between frames, in a frame constituting one GOP (Group Of Picture); It has a GOP structure composed of B frames, which are compressed images through bidirectional prediction.

2 is a diagram illustrating an example of a dependency relationship between frames constituting a GOP according to an embodiment of the present invention.

Referring to FIG. 2, interdependencies between I frames, P frames, and B frames configured in one GOP are shown. 'I frame', which constitutes GOP, compresses the screen independently in GOP, 'P frame' refers to previous I frame or P frame, and 'B frame' refers to I frame and P frame at the same time. Compressed video stream.

3 is a diagram illustrating an example of a bit string of a H.264 / AVC compressed video stream according to an embodiment of the present invention.

Referring to FIG. 3, a bit string of an H.264 / AVC video stream includes a start byte code of 4 bytes, a Nal header indicating a type of Nal of 1 byte, and one GOP. SPS (sequence parameter set) corresponding to video sequence information in the frame, PPS (picture parameter set) corresponding to frame information, slice header including type information of I, B, P frames and compressed information The branch is composed of slice data. Here, SPS and PPS Nal are divided into non-VCL (video coding layer) Nal, and the rest are classified into types of Nal units by VCL Nal.

4 is a block diagram illustrating a configuration of a network adaptive variable stream layering device according to an embodiment of the present invention.

Referring to FIG. 4, the network adaptive variable stream layering apparatus 400 may include a first switch 410, a map table generator 420, a rate controller 430, a stream analyzer 440, and a GOP buffer 450. ), A second switching unit 460, and an output buffer 470.

The first switching unit 410 determines whether the H.264 / AVC-compressed video stream is layered according to whether the available bandwidth allows each receiving terminal. For example, the first switching unit 410 controls to transmit the video stream as it is to each receiving terminal when the available bandwidth is unchanged, and when the available bandwidth is reduced, to stream the video stream, It may be delivered to the analysis unit 440. In addition, the first switching unit 410 may determine whether a packet lost from the rate control unit 430 is lost.

The stream analyzer 440 analyzes the video stream and outputs Nal information of the video stream. For example, referring to FIG. 3, the stream analyzer 440 detects a Nal unit boundary from the bit stream of the video stream with 4 bytes of start code information in the H.264 / AVC bit string, and 1-byte Nal header information. The type of Nal unit is determined by analyzing a 5 bit nal_unit_type field. In this case, in the case of VCL Nal, the stream analyzer 440 distinguishes the I, B, and P frame types from the frame_type field information of the slice header.

That is, the stream analyzer 440 transmits Nal information including the type of Nal, the order of Nal, and the amount of Nal occupied bits occupied by the Nal unit in the GOP to the map table generator 420, and transmits bits of the video stream. The heat is transferred to the GOP buffer 450 as it is.

The map table generator 420 arranges the frames included in the video stream using the output NAL information. That is, the map table generator 420 may arrange the frames included in the video stream according to the occupied bit amount included in the NAL information. For example, the map table generator 420 first sorts the occupied bit amount of the B frames constituting the GOP regardless of position, and the P frames are arranged in FIFO from the last frame of the GOP to the start I frame of the GOP in reverse. Create a mapped map table

5 is a diagram illustrating an example of arranging frames in a map table generator according to an exemplary embodiment of the present invention.

Referring to FIG. 5, when the order of frames constituting one GOP and the occupied bit amount are equal to 510, the map table generator 420 may arrange them as shown in 520 according to mutual dependence of I, B, and P frames. have.

The rate controller 430 may layer the video stream by selectively removing or selecting the aligned frames with reference to the packet loss rate of the available bandwidth of each receiving terminal and the NAL information.

Here, the GOP buffer 450 stores the frames included in the video stream in units of GOP. In this case, the rate controller 430 refers to the map table generator 420 in the FIFO (First In First Out) manner and selects a frame to be removed from the frames output from the GOP buffer 450 as the second switching unit ( 460 may be controlled to remove one GOP unit, and the unremoved frame may be transmitted to the output buffer 470. That is, the rate controller 430 may control the second switching unit 460 to remove the frame or select and transfer the selected frame to the output buffer 470. For example, the rate controller 430 may control the second switching unit 460 to transfer a frame allowed by the available bandwidth of each receiving terminal to the output buffer 470.

In this case, since the SPS and PPS Nal are information that should always be transmitted to the receiving terminal, the rate controller 430 may always deliver the SPS and PPS Nal to each output buffer 470.

In addition, the rate controller 430 prevents the video stream from being broken by removing P frames and B frames other than the I frame.

The output buffer 470 configures the selected frame to be decoded at each receiving terminal and transmits the selected frame to each receiving terminal. That is, the output buffer 470 arranges the SPS and PPS Nal into the higher output stream, and arranges the order of the frames so that the frames selectively input by the second switching unit 460 can be decoded by the respective receiving terminals. Can be configured.

6 is a flowchart illustrating a procedure of a network adaptive variable stream layering method according to an embodiment of the present invention.

Referring to FIG. 6, in the network adaptive variable stream layering method, a video stream compressed with H.264 / AVC is input to a first switching unit (610). The first switching unit may determine whether to layer the H.264 / AVC compressed video stream according to whether the available bandwidth allows each receiving terminal.

The network adaptive variable stream layering method analyzes the video stream in which the layering is determined and outputs NAL information of the video stream (620). Here, the NAL information may include the type of NAL, the order of the NALs, and the amount of bits occupied by the NAL.

The network adaptive variable stream layering method aligns the frames included in the video stream using the output NAL information (630). For example, the network adaptive variable stream layering method may arrange the frames included in the video stream according to the occupied bit amount included in the NAL information.

The network adaptive variable stream layering method selectively selects the aligned frames with reference to the packet loss rate of the available bandwidth of each receiving terminal and the NAL information (640). For example, the network adaptive variable stream layering method may select a frame allowed by the available bandwidth of each receiving terminal, and remove a frame to be removed in one GOP unit among frames stored in the GOP buffer.

The network adaptive variable stream layering method configures the selected frame to be decoded at each receiving terminal and transmits the selected frame to each receiving terminal (650).

In the network adaptive variable stream layering method illustrated in FIG. 6, the contents of the network adaptive variable stream layering apparatus described with reference to FIGS. 1 to 5 may be applied. Therefore, a detailed description of the network adaptive variable stream layering method is omitted here.

Further, embodiments of the present invention include a computer readable medium having program instructions for performing various computer implemented operations. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

While specific embodiments of the present invention have been described so far, various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims and the claims.

1 is a diagram illustrating an example of a video stream having a GOP structure according to an embodiment of the present invention.

2 is a diagram illustrating an example of a dependency relationship between frames constituting a GOP according to an embodiment of the present invention.

3 is a diagram illustrating an example of a bit string of a H.264 / AVC compressed video stream according to an embodiment of the present invention.

4 is a block diagram illustrating a configuration of a network adaptive variable stream layering device according to an embodiment of the present invention.

5 is a diagram illustrating an example of arranging frames in a map table generator according to an exemplary embodiment of the present invention.

6 is a flowchart illustrating a procedure of a network adaptive variable stream layering method according to an embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

400: network adaptive variable stream layering device

410: the first switching unit

420: map table generation unit

430: rate control unit

440: stream analysis unit

450: GOP buffer

460: second switching unit

470: output buffer

Claims (10)

A stream analyzer for analyzing the H.264 / AVC compressed video stream and outputting NAL information of the video stream; A map table generator for arranging frames included in the video stream using the output NAL information; A rate controller for selectively removing or selecting the aligned frames with reference to a packet loss rate of the available bandwidth of each receiving terminal and the NAL information; And An output buffer configured to decode the selected frame at each of the receiving terminals and to transmit the selected frame to each of the receiving terminals Network adaptive variable stream layering apparatus comprising a. The method of claim 1, A first switching unit determining whether the video stream is layered according to whether the available bandwidth allowed by each receiving terminal changes More, The first switching unit, When the available bandwidth does not change, the network adaptive variable stream layering apparatus for controlling to transmit the video stream as it is, and if the available bandwidth is reduced, and delivers the video stream to the stream analyzer. . The method of claim 1, The NAL information includes the type of NAL, the order of the NALs, the amount of bits occupied by the NAL, The map table generation unit, And align the frames included in the video stream according to the amount of occupied bits included in the NAL information. The method of claim 1, The H.264 / AVC compressed video stream, In a frame constituting one GOP, I have a GOP structure consisting of an I frame, which is an intra-frame compressed image, a P frame, which is a compressed image through forward prediction between frames, and a B frame, which is a compressed image through bidirectional prediction. The map table generation unit, A network adaptive variable stream layering device, which sorts the GOP configuration frames into FIFOs in the order of GOP occupancy bits of B frames, GOP configuration order of P frames, and I frame order from the last frame. The method of claim 4, wherein The rate control unit, And do not remove the I frame, thereby preventing the video stream from being broken. The method of claim 1, GOP buffer for storing the frames included in the video stream in units of GOP More, The rate control unit, Referencing the map table generator in a FIFO method to remove a frame to be removed from the GOP buffer to a GOP unit, and to transfer the unremoved frame to the output buffer by controlling a second switching unit Network adaptive variable stream layering device. Analyzing the video stream compressed with H.264 / AVC and outputting NAL information of the video stream; Arranging a frame included in the video stream using the output NAL information; Selectively selecting the aligned frames with reference to a packet loss rate of the available bandwidth of each receiving terminal and the NAL information; And Configuring the selected frame to be decoded at each receiving terminal and transmitting the decoded frame to each receiving terminal Network adaptive variable stream layering method comprising a. The method of claim 7, wherein The NAL information includes the type of NAL, the order of the NALs, the amount of bits occupied by the NAL, Arranging the frames included in the video stream, Sorting the frames included in the video stream according to the amount of bits occupied in the NAL information. Network adaptive variable stream layering method comprising a. The method of claim 7, wherein The H.264 / AVC compressed video stream, In a frame constituting one GOP, I have a GOP structure consisting of an I frame, which is an intra-frame compressed image, a P frame, which is a compressed image through forward prediction between frames, and a B frame, which is a compressed image through bidirectional prediction. Arranging the frames included in the video stream, Arranging the frames of the GOP unit in the order of B frames, P frames, and I frames from the last frame Network adaptive variable stream layering method comprising a. The method of claim 7, wherein Storing the frames included in the video stream in a GOP buffer in units of GOP More, Selectively selecting the aligned frame, Removing the frame to be removed in one GOP unit among frames output from the GOP buffer by referring to the NAL information in a FIFO method, and transferring the unremoved frame to an output buffer Network adaptive variable stream layering method comprising a.

Images