KR20160125157A - Live streaming system using http-based non-buffering video transmission method - Google Patents
Live streaming system using http-based non-buffering video transmission method Download PDFInfo
- Publication number
- KR20160125157A KR20160125157A KR1020150055984A KR20150055984A KR20160125157A KR 20160125157 A KR20160125157 A KR 20160125157A KR 1020150055984 A KR1020150055984 A KR 1020150055984A KR 20150055984 A KR20150055984 A KR 20150055984A KR 20160125157 A KR20160125157 A KR 20160125157A
- Authority
- KR
- South Korea
- Prior art keywords
- media
- http
- client device
- streaming
- real
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/23602—Multiplexing isochronously with the video sync, e.g. according to bit-parallel or bit-serial interface formats, as SDI
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/23605—Creation or processing of packetized elementary streams [PES]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/23614—Multiplexing of additional data and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2368—Multiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The present invention relates to a real-time streaming system using an HTTP-based unbuffered image transmission method, which multiplexes acquired image packets in a TS format in units of GOPs and transmits the multiplexed images to a client through a socket, The present invention aims to provide a system which does not require an additional storage resource of a segment file, shortens an end transfer delay time, and can use a memory at a minimum.
According to an aspect of the present invention, there is provided a media streaming system comprising: a media converter for multiplexing video packets obtained from a network camera according to a media data request signal received from a client device through a streaming providing unit on a GOP (Group Of Picture) basis; And a web server, which receives a media data request signal from the client device via a CGI (Common Gateway Interface), and transmits the TS data packet of the GOP unit multiplexed through the media converter to the client device via the socket Real-time streaming service; .
Description
The present invention relates to a real-time streaming system using an HTTP-based unbuffered image transmission method, and more particularly, to a streaming system based on an embedded network camera supporting an elementary stream (ES) of an H.264 codec. And to a system capable of real-time unbuffered image transmission.
Regarding video surveillance and management using an IP network, a number of such devices are registered and disclosed in addition to Korean Patent Laid-Open No. 10-2010-0038560 (hereinafter referred to as 'prior art document').
The above-mentioned prior art discloses a configuration in which an image received through a network is encoded and streamed. As in the preceding literature, many analog / digital CCTVs are installed and operated in public places for the purpose of surveillance security. Recently, video surveillance system using IP camera has become popular as low-cost system.
Recently, real-time media streaming technology such as broadcasting system and video surveillance system has become an essential technology. However, instead of adopting a stateful protocol requiring stateful and special software, an HTTP based streaming method, which is an Internet open standard protocol, is being studied.
A video and audio source of an HTTP-based streaming protocol is encoded in a media format that can be reproduced by a client, divided into media files having a predetermined playback time, and provided to a client, thereby using a method of transmitting an image.
The client requests media files that are sequentially partitioned and receives media files through a progressive download technique of the HTTP protocol. The divided media files are sequentially played through the playlist. The media to be reproduced has continuous data, and the entire media is continuously reproduced. The HTTP-based streaming method is capable of continuous playback using a divided file. However, according to the generation time of the first divided media file and the playback time of the divided media file, -to-end delay occurs.
That is, if the media file is a 10-second divided file, a delay of at least 10 seconds occurs. If the playback time of the divided media file is shortened in order to reduce the delay of the terminal transmission, a computation load may be generated on the client side and a delay may occur.
The present invention focuses on MPEG transport stream (MPEG-TS) file transmission to achieve real-time streaming over HTTP without compatibility problems with the devices used by clients. MPEG-2 TS is a data format for transmitting ES (Elementary Stream) packetized video and audio with multiplexed TS packets.
There are two streaming solutions that support the transmission of TS files over HTTP: Dynamic adaptive streaming over HTTP (MPEG-DASH) and HTTP live streaming (HLS).
In the process of implementing the HLS server, APPLE Inc. recommends that the playback time of each divided media file of multimedia be maintained in 10 seconds and at least 3 divided media files. If the HLS is implemented using three divided media files with a playback time of 10 seconds, the client will have an end-to-end delay of at least 30 seconds.
In order to solve this problem, it is possible to provide a streaming service using a media file having a short playback time. However, this method generates unnecessary network overhead, and there is a problem that a large number of operations must be performed in order to generate an image in a CCTV camera have.
Meanwhile, in order to implement real-time streaming based on HLS, a method of transmitting real-time media streaming based on cloud computing has been proposed. This method can provide real-time streaming without changing HLS's server structure, but the end-to-end delay caused by the characteristics of HLS has not been studied.
1 shows a HLS system structure implemented in a network camera in the conventional method. In order to support HLS based on HTTP, a web server is required. For example, GoAhead web server is built inside a network camera and acquires media from camera and provides it to users through HTTP. Existing web servers can be implemented in compliance with APPLE recommendations. The server shall list Uniform Resource Locators (URLs) for the three divided media files in the m3u8 file, which is a playlist file, as proposed in the standard. The following is an example of a URL to be displayed in a playlist.
http://example.com/test/segmented_video0.ts
When configuring a streaming system in accordance with APPLE recommendations, the server MUST generate the split media files in advance of the client request. This means that a streaming server requires a storage capable of storing divided media files, and a server for providing real-time media streaming must continuously generate new divided media files. When a media file having a long playback time is generated, the client can not play the media for more than the playback time. This creates an end-to-end delay between the server and the client.
SUMMARY OF THE INVENTION The present invention has been conceived in view of the above problems, and it is an object of the present invention to provide a method and apparatus for multiplexing an acquired video packet in TS format in GOP units and transmitting the multiplexed video to a client through a socket, And does not require additional storage resources of the segment file, shortens the end transfer delay time, and minimizes the use of the memory.
According to an aspect of the present invention, there is provided a real-time streaming system using an HTTP-based unbuffered image transmission method embedded in a network camera, A media converter for multiplexing image packets obtained from a network camera in units of GOP (Group Of Pictures); And a web server, which receives a media data request signal from the client device via a CGI (Common Gateway Interface), and transmits the TS data packet of the GOP unit multiplexed through the media converter to the client device via the socket Real-time streaming service; .
The media converter may further include: a multiplexing module for multiplexing ES video packets obtained from the network camera into TS video packets in units of GOP; And a storage module for storing the obtained ES video packet while the multiplexing module multiplexes the ES video packet into the TS video packet; And a control unit.
Further, the multiplexing module is activated based on a media data request signal received from the client device through a CGI (Common Gateway Interface).
The storage module is a buffer of a circular queue structure.
According to the present invention, the end-to-end delay time can be reduced, and memory can be minimized even in terms of resource management that must be considered in the embedded system.
1 shows an example of a conventional HLS system built in a network camera.
FIG. 2 is a schematic view illustrating a concept of a real-time streaming system using an HTTP-based unbuffered image transmission method embedded in a network camera according to the present invention.
3 illustrates an example of a multiplexing pseudo code in units of GOPs according to the present invention.
4 is a graph comparing file-based stream data traffic with packet-based stream data traffic according to the present invention.
5 is an example of an actual reproduction experiment according to the present invention.
Specific features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. It is to be noted that the detailed description of known functions and constructions related to the present invention is omitted when it is determined that the gist of the present invention may be unnecessarily blurred.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings.
The real-time streaming system using the HTTP-based unbuffered image transmission method according to the present invention will be described with reference to FIG. 2 to FIG.
FIG. 2 is a block diagram conceptually showing a real-time streaming system S using an HTTP-based unbuffered image transmission method built in a network camera according to the present invention. As shown in FIG. 2, the
The
Specifically, the
At this time, the
Here, the Presentation Time Stamp (PTS) for video display is in the header of the multiplexed TS video packet and starts from 0 second. The presentation time table (PTS) indicates the time information on which the TS video packet is to be reproduced
The storage module 120 stores the obtained ES video packet while the
At this time, the storage module 120 is a buffer of a circular queue structure, which can continuously acquire ES packets even during the conversion into a TS video packet of an ES video packet, thereby preventing a video interruption.
The streaming providing unit 200 is a web server that receives a media data request signal from the
As described above, the streaming providing unit 200 activates the multiplexing of the
Meanwhile, the
3 shows a multiplexing pseudo code in GOP units.
The
Thereafter, continuous multipreflexing is performed using the circular queue buffer in packet units. Then, the video multiplexed in the GOP unit is transmitted to the streaming providing unit 200, which is a web server, through the socket communication, and is transmitted to the
Hereinafter, the experimental results of the real-time streaming system S using the HTTP-based unbuffered image transmission method built in the network camera will be described.
The streaming system supports HTTP-based interaction with the client device, and the image provided from the network camera is set to have a frame rate of 256Kbps CBR, resolution 320x176, and 30.0 frames per second.
The streaming system provided a TS file with a duration of 30 seconds consisting of 900 frames and measured the data traffic between the streaming system and the client device.
As shown in FIG. 4, it shows experimental results of transmitting about 1219.9 kB data of a packet-based stream and about 1256.1 kB of file-based data for 30 seconds.
Although the size of the transmitted data was similar, the initial delay occurred because, in providing a file-based TS stream, the streaming system needed to generate the required number of segment files at the request of the client device.
On the other hand, in the method using the packet-based streaming system S according to the present invention, the initial delay is reduced to approximately one second, and the time interval between transmitted packets is converted to about 0.5 second in units of GOP (i.e., 15 frames) .
Figure 5 shows an actual reproduction experiment.
5 (a) shows the same time as the actual image. That is, the time shown in (a) of FIG. 5 is displayed on the client device in two ways using a network camera.
First, the client device of the packet-based streaming system (S) according to the present invention was performed on the Android 4.4.4 (SM-G850) platform. Playback experiments on Android devices are performed through an additional media player. J2 Interactive developed the MX Player as a network streaming player (Figure 5 (c)).
The result shows the video transmission in the transition process from the current time 26 to 27 seconds.
FIG. 5 (b) shows a video transmission result based on RTSP, which is a real-time video transmission protocol using a separate plug-in. The RTSP based live viewers show 26 seconds as shown on the screen and show a delay of about 1 second compared to the actual image.
However, the method using the packet-based streaming system S according to the present invention shows about 25 seconds and shows an end-to-end latency of about 2 seconds.
This result is the minimum latency in the HTTP based protocol real - time image transmission technique without using a separate plug - in program.
This result shows a delay of about 30 seconds or more when using HLS. Therefore, although RTSP based real-time streaming provides fast response time, the widespread use of this protocol has a problem of compatibility with current firewall and a problem of installing a separate protocol.
Therefore, compared with the HLS solution using the file-based streaming method, the method using the packet-based streaming system (S) according to the present invention greatly reduces the end-to-end delay time.
As a result, the present invention proposes an improved structure of a real-time transmission system in a real-time image transmission method via HTTP.
Compared to existing file-based streaming systems, the proposed design does not require additional storage resources of the segment file. It also generates continuous packet data and multiplexes ES based media converters implemented based on GOP units.
Therefore, the delay time of each file unit is greatly reduced in units of packets of GOP units. In addition, this packet-based approach relates to multiplexing technology that can be used both in HLS as well as in existing HTTP-based media communication solutions such as MPEG-DASH.
While the present invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It will be appreciated by those skilled in the art that numerous changes and modifications may be made without departing from the invention. Accordingly, all such appropriate modifications and changes, and equivalents thereof, should be regarded as within the scope of the present invention.
100: media converter 200: streaming provider
110: multiplexing module 120: storage module
210: Socket 10: Client device
Claims (4)
A media converter for multiplexing video packets obtained from the network camera on a GOP (Group Of Picture) basis according to a media data request signal received from a client device through a streaming providing unit; And
A web server, which receives a media data request signal from a client device via a CGI (Common Gateway Interface), and multiplexes the TS video packet of the GOP unit by the media converter to the client device Streaming provider; Real - time streaming system using HTTP - based unbuffered video transmission method.
Wherein the media converter comprises:
A multiplexing module for multiplexing ES video packets obtained from the network camera into TS video packets in units of GOP; And
A storage module for storing the obtained ES video packet while the multiplexing module multiplexes the ES video packet into the TS video packet; Based streaming system using an HTTP-based unbuffered image transmission method.
The multiplexing module comprises:
Based on the media data request signal received from the client device via a CGI (Common Gateway Interface).
Wherein the storage module comprises:
Wherein the buffer is a buffer of a circular queue structure, and the real-time streaming system using the HTTP-based unbuffered image transmission method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020150055984A KR101690153B1 (en) | 2015-04-21 | 2015-04-21 | Live streaming system using http-based non-buffering video transmission method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR1020150055984A KR101690153B1 (en) | 2015-04-21 | 2015-04-21 | Live streaming system using http-based non-buffering video transmission method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| KR20160125157A true KR20160125157A (en) | 2016-10-31 |
| KR101690153B1 KR101690153B1 (en) | 2016-12-28 |
Family
ID=57445915
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| KR1020150055984A KR101690153B1 (en) | 2015-04-21 | 2015-04-21 | Live streaming system using http-based non-buffering video transmission method |
Country Status (1)
| Country | Link |
|---|---|
| KR (1) | KR101690153B1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018105771A1 (en) * | 2016-12-07 | 2018-06-14 | 주식회사 이에스엠연구소 | Real-time video playback file transmission apparatus and method |
| CN109640038A (en) * | 2018-11-30 | 2019-04-16 | 武汉烽火众智智慧之星科技有限公司 | A kind of video monitoring platform access method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20000025323A (en) * | 1998-10-10 | 2000-05-06 | 김영환 | Apparatus for generating transport stream for use in mpeg-2 system |
| KR20040096718A (en) * | 2003-05-10 | 2004-11-17 | 삼성전자주식회사 | Multimedia data decoding apparatus, audio data receiving method and audio data structure therein |
| KR20100038560A (en) | 2008-10-06 | 2010-04-15 | 주식회사 씨앤비텍 | System and method for managing supervisory video using ip network |
| KR101299003B1 (en) * | 2013-02-19 | 2013-08-23 | 주식회사 투윈스컴 | Real-time image checking apparatus using smart phones |
-
2015
- 2015-04-21 KR KR1020150055984A patent/KR101690153B1/en active IP Right Grant
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20000025323A (en) * | 1998-10-10 | 2000-05-06 | 김영환 | Apparatus for generating transport stream for use in mpeg-2 system |
| KR20040096718A (en) * | 2003-05-10 | 2004-11-17 | 삼성전자주식회사 | Multimedia data decoding apparatus, audio data receiving method and audio data structure therein |
| KR20100038560A (en) | 2008-10-06 | 2010-04-15 | 주식회사 씨앤비텍 | System and method for managing supervisory video using ip network |
| KR101299003B1 (en) * | 2013-02-19 | 2013-08-23 | 주식회사 투윈스컴 | Real-time image checking apparatus using smart phones |
Non-Patent Citations (1)
| Title |
|---|
| Byoung Wook et al. "Implementation of HLS Protocol for an IP Camera", Advanced Science and Technology Letters, Vol. 46, Network and Communication 2014, pp.111-114.* * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018105771A1 (en) * | 2016-12-07 | 2018-06-14 | 주식회사 이에스엠연구소 | Real-time video playback file transmission apparatus and method |
| CN109640038A (en) * | 2018-11-30 | 2019-04-16 | 武汉烽火众智智慧之星科技有限公司 | A kind of video monitoring platform access method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| KR101690153B1 (en) | 2016-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11470405B2 (en) | Network video streaming with trick play based on separate trick play files | |
| JP6783293B2 (en) | Synchronizing multiple over-the-top streaming clients | |
| CN107409234B (en) | Streaming based on file format using DASH format based on LCT | |
| US9247317B2 (en) | Content streaming with client device trick play index | |
| RU2652099C2 (en) | Transmission device, transmission method, reception device and reception method | |
| Sodagar | The mpeg-dash standard for multimedia streaming over the internet | |
| US20140359678A1 (en) | Device video streaming with trick play based on separate trick play files | |
| US9813740B2 (en) | Method and apparatus for streaming multimedia data with access point positioning information | |
| US9462024B2 (en) | System and method of media content streaming with a multiplexed representation | |
| US20150281800A1 (en) | Broadcast content to http client conversion | |
| US20140297804A1 (en) | Control of multimedia content streaming through client-server interactions | |
| WO2014193996A2 (en) | Network video streaming with trick play based on separate trick play files | |
| US20140359681A1 (en) | Playback synchronization across playback devices | |
| US20160127798A1 (en) | Content supply device, content supply method, program, and content supply system | |
| KR20120114016A (en) | Method and apparatus for network adaptive streaming user data in a outer terminal | |
| KR20160110424A (en) | Robust live operation of dash | |
| CN106789976A (en) | The player method of media file, service end, client and system | |
| US20170331666A1 (en) | Real-time control interface for broadcast object streaming | |
| KR102137858B1 (en) | Transmission device, transmission method, reception device, reception method, and program | |
| KR101690153B1 (en) | Live streaming system using http-based non-buffering video transmission method | |
| Sodagar et al. | Reinventing multimedia delivery with MPEG-DASH | |
| KR101568317B1 (en) | System for supporting hls protocol in ip cameras and the method thereof | |
| US11856242B1 (en) | Synchronization of content during live video stream |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A201 | Request for examination | ||
| E902 | Notification of reason for refusal | ||
| E701 | Decision to grant or registration of patent right | ||
| GRNT | Written decision to grant | ||
| FPAY | Annual fee payment |
Payment date: 20191029 Year of fee payment: 4 |