KR20060030879A - The multi-media streaming method and system of a network adaptation live broadcasting for packet filtering - Google Patents

Abstract

The present invention provides various transmissions of multimedia data generated in real time when streaming various multimedia data generated in real time from multiple multimedia sources to multiple clients on the Internet where QoS (Quality of Service) is not uniform and not guaranteed. Network adaptive through packet filtering based on the cumulative status of packets to be transmitted to adapt the streaming service to the network more economically without synchronization and reprocessing at multiple bit rates. The present invention relates to a live multimedia streaming system and method, and more particularly, to an encoder unit for receiving data generated in real time from multiple multimedia sources and encoding at an arbitrary transmission bit rate to enable streaming, and an encoder unit. A multimedia buffer unit for temporarily storing various multimedia packets to be classified by source and a multimedia buffer switch unit for controlling the order and method of access according to an arbitrary policy in accessing the multimedia packets stored in the multimedia buffer unit; And a packet filter unit for packet filtering to correct a packet transmission delay based on a cumulative state of packets to be generated according to the client network state, and a streaming engine unit for streaming the filtered packet to the client. .

    Live Broadcast, Multimedia Streaming, Multi-Media Buffer, Buffer Switch, Delay Compensation, Packet Filter

Description

The multi-media streaming method and system of a network adaptation live broadcasting for packet filtering

    1 is a block diagram showing the overall configuration of a network adaptive live broadcasting multimedia streaming system through packet filtering according to an embodiment of the present invention

    FIG. 2 is a flowchart illustrating an operation of a multimedia buffer switch unit of a network adaptive live broadcasting multimedia streaming system through packet filtering shown in FIG. 1.

    FIG. 3 is a flowchart illustrating an operation of a packet filter unit of a network adaptive live broadcasting multimedia streaming system through packet filtering shown in FIG. 1.

    4 is a flowchart illustrating an operation of a streaming engine unit of a network adaptive live broadcasting multimedia streaming system through packet filtering shown in FIG. 1.

Explanation of symbols on main parts of drawing

     10 (11 ~ 12): encoder part, 20 (21 ~ 25): multimedia buffer part, 30: multimedia buffer switch part, 40 (41 ~ 43): packet filter part, 50 (51 ~ 53): streaming engine part

    The present invention relates to a network adaptive live broadcast multimedia streaming system and method through packet filtering used for real-time video conferencing, tele-learning, etc. More specifically, the QoS (Quality of Service) is not uniform and guaranteed on the Internet When streaming various multimedia data generated in real time from multimedia sources to multiple clients, the network can be synchronized between multiple simultaneous users without generating or reprocessing the multimedia data generated in real time at various transmission bit rates. The present invention relates to a network adaptive live broadcast multimedia streaming system through packet filtering based on a cumulative state of packets to be transmitted to provide a streaming service adaptively.

   In general, since the Internet network is the best-effort network, the bandwidth is not uniform and is not guaranteed, and thus the available network bandwidth may deteriorate from time to time depending on network congestion or traffic volume.

   The live multimedia streaming service made under a network environment having such disadvantages has a problem that a video is interrupted and a transmission delay occurs due to a change in network conditions. Live multimedia streaming service methods have been proposed.

   However, the existing live multimedia streaming service methods are focused on the following two factors in order to have the characteristics of adaptation to the network. First, to get network status, network status information is obtained from the client, the transmission bit rate is calculated, or the packet loss rate is calculated.

   Second, the multimedia data generated from one multimedia source is regenerated and streamed at a transmission bit rate for each network state, or the multimedia data is generated and allocated for each network state while generating multimedia data at various preset bit rates.

   These conventional methods are applied to a system that needs to stream various multimedia data generated from multiple multimedia sources in real time from multiple multimedia sources to multiple clients such as video conferencing where synchronization between simultaneous users is important. If the number of receiving users increases, there is a problem that the delay occurs due to a sudden increase in load on devices such as the CPU in the process of generating or reprocessing data to be streamed at various transmission bit rates. In order to solve these problems, there is a problem in that the economic burden of increasing server equipment or improving performance is increased.

SUMMARY OF THE INVENTION The present invention has been proposed to solve the above-mentioned problems, and an object thereof is to provide a plurality of clients with various multimedia data generated in real time from multiple multimedia sources on the Internet where QoS (Quality of Service) is not uniform and not guaranteed. When streaming to a mobile station, multimedia data that is generated in real time can be adaptively provided to the network to provide a streaming service that can be synchronized to multiple simultaneous users without having to multi-generate or reprocess the data at various transmission bit rates. The present invention aims to provide a network adaptive live broadcasting multimedia streaming system and method through packet filtering based on cumulative state.

   Network adaptive live broadcasting multimedia streaming system through packet filtering of the present invention for achieving the above object is an encoder for encoding at any transmission bit rate to receive data generated in real time from a plurality of multimedia sources to enable streaming And a multimedia buffer unit for temporarily storing various multimedia packets generated by the encoder by type of each source, and controlling the order and method of access according to an arbitrary policy in accessing the multimedia packets stored in the multimedia buffer unit. A multimedia buffer switch unit, a packet filter unit for packet filtering to correct a packet transmission delay based on a cumulative state of packets to be generated according to a client network state, and a stream of filtered packets to a client. Ha It characterized by a portion composed streaming engine.

   In addition, in order to achieve the above object, the network adaptive live broadcast multimedia streaming method according to the present invention performed in the network adaptive live broadcast multimedia streaming system through the above-described packet filtering data generated in real time from a plurality of multimedia sources Encoding the data at an arbitrary transmission bit rate to enable streaming and storing the multimedia packets generated by the encoding step by type of each source and temporarily storing the multimedia packets, and accessing the temporarily stored multimedia packets. Controlling the order and method of access according to a certain policy, and performing packet filtering to correct a packet transmission delay based on a cumulative state of packets to be transmitted according to a client network state. Done Characterized by the kit made of an step of streaming to the client (streaming).

   In addition, in the network adaptive live broadcasting multimedia streaming system through packet filtering according to the present invention, the packet filtering step may be performed based on a cumulative state of packets to be transmitted when accumulation of packets to be transmitted depends on a client network state. The packet transmission delay is corrected in such a manner that only the packets necessary for multimedia reproduction are selected and transmitted from the client.

   Hereinafter, a configuration and operation of a network adaptive live broadcast multimedia streaming system through packet filtering according to the present invention will be described with reference to the accompanying drawings.

    1 is a block diagram of a network adaptive live broadcasting multimedia streaming system through packet filtering according to the present invention, which receives data generated in real time from a plurality of multimedia sources and encodes the data at an arbitrary transmission bit rate to enable streaming. The encoder unit 10, the multimedia buffer unit 20 which temporarily stores various multimedia packets generated by the encoder unit by type of source, and the multimedia packet stored in the multimedia buffer unit according to an arbitrary policy in accessing the multimedia packet unit. Multimedia buffer switch unit 30 for controlling the order and method of access, and packet filtering unit for packet filtering to correct the packet transmission delay based on the cumulative state of the packets to be generated according to the client network conditions 40, and stream the filtered packet to the client (str eaming) is composed of a streaming engine unit (50).

   In the above description, the encoder unit 10 may include a plurality of encoder units 11 to 12 as many as the number of multimedia sources, and each encoder unit 11 to 12 may receive multimedia data input from each real-time multimedia source. The video data of the frame structure, the screen motion video of the cell structure, and the voice are divided into multimedia types and encoded at a specific transmission bit rate to be transmitted to the multimedia buffer unit 20 in a packet form.

Here, the plurality of encoders 11 to 12 operate independently of each other, and can be simply increased or decreased according to the number of multimedia sources. The multimedia types can be arbitrarily configured.

    That is, the multimedia buffer unit 20 divides packet data transmitted from the encoder unit 10 into multimedia sources and multimedia types, and divides the packet data into a plurality of multimedia buffer units 21 to 25 to temporarily store the packet data.

    The packet data stored in the multimedia buffer units 21 to 25 are sequentially stored by a predetermined reference amount such as 10 seconds or 20 seconds while the previous data is deleted to always maintain the latest data.

    The multimedia buffer switch unit 30 grasps packet information stored in the plurality of multimedia buffer units 21 to 25 classified by multimedia source and multimedia type, and transmits the packet information to the packet filter unit 40, and the packet filter unit 40. The specific packet requested by the multi-media buffer unit 21 to 25 is transmitted to the packet filter unit 40, and in particular, controls the order and method of packet information transmission and packet transmission according to a predetermined policy. .

   Further, in this case, a policy setting such that multimedia packet data generated from a specific multimedia source or a specific type of multimedia packet data is processed first may be necessary.

  The packet filter unit 40 includes packet filters 51 to 53 as many as the number of clients to be streamed, and the packet filter requests generated from the streaming engine unit 50 according to the client network situation, that is, as many as the number of clients. When the plurality of streaming engine units 51 to 53 configured to receive the packet transmission requests generated according to the client network conditions are received by the packet filter units 41 to 43 that match each, the multimedia buffer is switched through the multimedia buffer switch unit 30. After grasping the packet information in the sections 21-25, the packet transmission delay is corrected based on the accumulated status of packets to be transmitted according to the client network status with respect to the packets in the multimedia buffer sections 21-25. Packet filtering, and requests the filtered specific packet to the multimedia buffer switch unit 30 so that the multimedia buffer unit 2 1 to 25) and transmits to the corresponding streaming engine 51-53.

   The plurality of packet filter units 41 to 43 operate independently of each other, and may be simply increased or decreased according to the number of clients or the number of streaming engine units 51 to 53.

   Meanwhile, even if the number of multimedia sources and the number of clients increase in combination, synchronization between clients in different network states is possible through packet filtering for correcting a packet transmission delay of the packet filter unit 40.

   The streaming engine unit 50 is composed of as many streaming engine units 51 to 53 as the number of clients. When each client network is in a state capable of being transmitted, the streaming engine unit 50 requests and receives a packet from the packet filter units 41 to 43 and receives the packet. Stream the packet to the client.

   Live streaming multimedia streaming service having the above configuration and function is adaptively streamed to each client network state when streaming multimedia data generated from various multimedia sources to multiple clients in real time. The packet transmission delay is corrected based on the accumulated status of packets to be generated, allowing synchronization between clients regardless of network conditions, and policy priority without multi-generation or reprocessing of multimedia data generated at various transmission rates. By ranking, packet transmission loss can be adjusted to make streaming more economical.

    2 is a multimedia buffer switch for controlling the order and method of accessing a multimedia packet stored in the multimedia buffer unit 20 according to an arbitrary policy in a network adaptive live broadcasting multimedia streaming system through packet filtering shown in FIG. 1. As a flowchart illustrating the operation of the unit 30, the operation of the multimedia buffer switch unit 30 will be described in detail with reference to FIGS. 1 and 2 as follows.

    The multimedia buffer switch unit 30 transmits the packet information transmission unit 300 for processing information about the packet stored in the multimedia buffer unit 20 and the packet stored in the multimedia buffer 20 to the packet filter unit 40. The transmitter 350 may be classified.

    The packet information transmitter 300 operates in the following procedure. Upon receiving (301) an information request about a packet to be transmitted stored in the multimedia buffer unit 20 from the packet filter unit 40, the packet filter unit 40 appropriately matches the policies according to the priorities of the predetermined policies. It is determined whether there are packets to be transmitted (302 to 304).

    In this case, arbitrarily set policies may be applied. For example, a policy corresponding to priority 0 may be configured to process packets in voice buffers 23 and 25, and a policy corresponding to priority 1 may include a screen motion buffer unit ( 22 may be set to process packets, and a policy corresponding to priority 2 may be set to process packets in the frame video buffers 21 and 24.

    Referring to the processes 302 to 306 according to the above example, it is determined that there are packets to be transmitted to the multimedia buffer units 23 and 25 corresponding to priority 0 (302). The packet information to be transmitted is obtained from the multimedia buffer units 23 and 25 (305). If there is no packet to be transmitted, the multimedia buffer unit 22 determines whether there are packets to be transmitted to the multimedia buffer unit 22 corresponding to priority 1 (303). If there is a packet to be transmitted, the multimedia buffer unit 22 retrieves information to be transmitted (305). If there is no packet to be transmitted, it is determined whether there are packets to be transmitted to the multimedia buffer units 21 and 24 corresponding to priority 2 (304). If there is a packet to be transmitted, the packet information to be transmitted is fetched from the multimedia buffer unit 21 or 24 (305). And 306. The

    Accordingly, the packet information obtained in the above steps 302-306 or the packet information indicating that there are no packets to be transmitted generated in the above steps 302-306 is transmitted 307 to the packet filter unit 40. This operation process is repeated repeatedly. Policies applied in the above process can be arbitrarily set and increased according to the needs of the system.

    On the other hand, the packet information transmission unit 350 operates in the following procedure. Based on the packet information transmitted from the packet information transmitter 300, the corresponding packet filter 41 to 43 receives a specific packet transmission request generated through packet filtering (351), and then the corresponding multimedia buffer. The packets 21 to 25 take the corresponding packet (352), and transmit the retrieved packet to the packet filter units 41 to 43 (353). This process of operation is repeated.

    Through the above process, the multimedia buffer switch unit 30 accesses the multimedia packets stored in the multimedia buffer units 21 to 25 classified by the multimedia source and the multimedia type. You control the order and method.

    FIG. 3 is a packet filtering method for correcting a packet transmission delay based on a cumulative state of packets to be transmitted according to a client network state in a network adaptive live broadcasting multimedia streaming system through packet filtering shown in FIG. 1. The operation of the packet filter unit 40 will now be described in detail with reference to FIGS. 1 and 3 as a flowchart showing the operation of the packet filter unit 40.

   When each packet filter unit 41 to 43 receives a packet transmission request from each streaming engine unit 51 to 53 (401), the packet information is obtained from the multimedia buffer switch unit 30 (402). It is determined whether there are packets to be transmitted to the multimedia buffer units 21 to 25 (403), and if there are no packets that have not yet been transmitted and accumulated in the multimedia buffer units 21 to 25, the multimedia buffer switch unit until there are packets to be transmitted. In step 30, the packet information is repeatedly retrieved (402), and if there are packets to be transmitted, it is determined whether the last packet is valid even if there is no previous packet (404).

    The criterion for determining whether the packet is valid is whether the packet is necessary for multimedia reproduction by the client.

    That is, if the last packet is a valid packet without the previous packets, the previous packets are skipped and only the last packet is taken from the multimedia buffer switch unit 30 (405). Otherwise, the oldest packet among the packets to be transmitted is skipped. 405 is taken from the multimedia buffer switch unit 30, and the received packet is transmitted to the corresponding streaming engine units 51 to 53 (407).

   The above processes 402 to 406 are called packet filtering, and when the packet is accumulated according to the client network state through the packet filtering process, the multimedia is transmitted from the client based on the accumulated state of the packet to be transmitted. By correcting the packet transmission delay by selecting only the packets necessary for reproduction, the transmission delay is always kept below a threshold value, thereby maintaining the synchronization of multimedia streaming services between clients.

    4 is a flowchart illustrating an operation of a streaming engine unit 50 streaming a filtered packet to a client in a network adaptive live broadcasting multimedia streaming system through packet filtering shown in FIG. 1. Referring to the operation of the streaming engine unit 50 in detail as follows.

    Each streaming engine unit 51 to 53 requests 501 a packet transmission to the packet filter units 41 to 43, receives a packet to be transmitted from the packet filter units 41 to 43, and then 502 it. In operation 503, it is determined whether there is free bandwidth with the network connected to the client.

    In this case, the free network bandwidth means that data can be transmitted to the client, which can be easily identified as writable in the network.

   Here, if there is no free network bandwidth (Bandwidth) repeatedly checks the network band with the free network band with the occurrence of the free packet band 41 to 43 to try to transmit the packet to the client (504).

   If it is not transmitted by determining whether all the packets have been transmitted (505), the process from identifying the free network bandweed (503) to transmitting all the packets (505) is repeated repeatedly, and transmits all the packets. When the packet is transmitted to the packet filter unit 41 to 43 again, the packet is repeatedly transmitted from the request 501 to the step of transmitting all the obtained packets 505.

   Through this process, the streaming engine units 51 to 53 stream multimedia packets to the client adaptively to the network in connection with a packet filtering process based on the cumulative state of packets to be transmitted.

   In the above process, when the network state connected to the client temporarily deteriorates to such an extent that there is no free bandweed or the packet transmission is not smooth, such a packet transmission delay may occur. As shown in FIG. The packet transmission delay correction is performed based on the cumulative state of the packets to be transmitted by the packet filters 41 to 43.

    Network adaptive live broadcasting multimedia streaming system and method through packet filtering according to an embodiment of the present invention is more economical when applied to a videoconferencing system in which synchronization between a plurality of users having a variety of network environment is important, real-time remote When applied to the lecture system, it is possible to more economically accommodate large users, and also economically applicable to fields such as real-time Internet panel broadcasting system.

As described above, the network adaptive live broadcasting multimedia streaming system and method through packet filtering according to the present invention provide a plurality of clients to generate various multimedia data generated in real time from a plurality of multimedia sources on the Internet where QoS (Quality of Service) is not guaranteed. Packet transmission delay that occurs depending on the network status is corrected based on the accumulated status of packets to be transmitted, and synchronization between clients is possible regardless of the network status. By allowing adjustment according to policy priorities, there is an advantage that the multimedia data can be streamed more economically without the process of multi-generation or reprocessing at various transmission bit rates.

Claims (6)

    An encoder unit which receives data generated in real time from a plurality of multimedia sources and encodes at an arbitrary transmission bit rate to enable streaming;     A multimedia buffer unit configured to temporarily store various multimedia packets generated by an encoder according to each type of source and temporarily store the multimedia packets;     A multimedia buffer switch unit controlling an order and a method of accessing the multimedia packet stored in the multimedia buffer unit according to an arbitrary policy;     A packet filter unit for packet filtering to correct a packet transmission delay based on a cumulative state of packets to be transmitted according to a client network state; A network adaptive live broadcasting multimedia streaming system through packet filtering, characterized in that the streaming engine unit for streaming the filtered packet to the client (streaming). The network adaptive live broadcasting multimedia streaming system according to claim 1, wherein packet filtering is performed to correct a packet transmission delay based on a cumulative state of packets to be transmitted according to a client network state. . The network adaptive live broadcasting multimedia streaming system according to claim 1, wherein the order and method of accessing the multimedia packet stored in the multimedia buffer unit are controlled according to an arbitrary policy. A network adaptive live broadcasting multimedia streaming method through packet filtering, comprising: receiving data generated in real time from a plurality of multimedia sources and encoding the data at an arbitrary transmission bit rate to enable streaming;   Temporarily storing the various multimedia packets generated in the encoding step according to types of each source and temporarily storing them; Controlling the order and method of access according to any policy in accessing temporarily stored multimedia packets; Packet filtering to correct a packet transmission delay based on a cumulative state of packets to be transmitted according to a client network state; Network adaptive live broadcast multimedia streaming method through packet filtering, characterized in that the step of streaming the filtered packet to the client (streaming). 5. The network adaptive live broadcasting according to claim 4, further comprising packet filtering for correcting a packet transmission delay based on a cumulative state of packets to be transmitted according to a client network state. Multimedia streaming method. 6. The method of claim 5, comprising controlling the order and method of access according to an arbitrary policy in accessing the temporarily stored multimedia packet.

Images