CN105007465A - Cipher-text H264 video live-broadcasting and on-demand platform based on live555 frame and ffmpeg library - Google Patents
Cipher-text H264 video live-broadcasting and on-demand platform based on live555 frame and ffmpeg library Download PDFInfo
- Publication number
- CN105007465A CN105007465A CN201510430852.9A CN201510430852A CN105007465A CN 105007465 A CN105007465 A CN 105007465A CN 201510430852 A CN201510430852 A CN 201510430852A CN 105007465 A CN105007465 A CN 105007465A
- Authority
- CN
- China
- Prior art keywords
- ciphertext
- video
- live555
- server
- external storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005540 biological transmission Effects 0.000 claims abstract description 19
- 238000005538 encapsulation Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims 3
- 238000012856 packing Methods 0.000 claims 2
- 238000012163 sequencing technique Methods 0.000 claims 1
- 238000012544 monitoring process Methods 0.000 abstract description 9
- 238000000034 method Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004806 packaging method and process Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
本发明提供一种基于live555框架和ffmpeg库的密文H264视频直播和点播平台，包括：基于live555框架的流媒体服务器和外部存储服务器；所述流媒体服务器和外部存储服务器通过网络连接；所述流媒体服务器包括live555直播代理逻辑服务器和live555点播逻辑服务器；所述外部存储服务器上设有密文H264视频转码模块和ffmpeg库；live555点播逻辑服务器和外部存储服务器之间通过TCP协议通信，为此，在live555点播逻辑服务器上建有TCP客户端，在外部存储服务器上建有TCP服务器端。本发明实现了城市安防监控体系中密文视频流的直播和密文视频文件的点播，可以保证密文视频传输的实时性需求和保证用户的较佳体验度。
The present invention provides a kind of ciphertext H264 video live broadcast and on-demand platform based on live555 framework and ffmpeg storehouse, comprising: a streaming media server and an external storage server based on the live555 framework; the streaming media server and the external storage server are connected through a network; the Streaming media server comprises live555 live broadcast proxy logic server and live555 order logic server; Described external storage server is provided with ciphertext H264 video transcoding module and ffmpeg storehouse; Between live555 order logic server and external storage server, communicate by TCP protocol, for Therefore, a TCP client is built on the live555 VOD logic server, and a TCP server is built on the external storage server. The invention realizes the live broadcast of the ciphertext video stream and the on-demand broadcast of the ciphertext video files in the urban security monitoring system, and can ensure the real-time requirement of the ciphertext video transmission and ensure better user experience.
Description
技术领域 technical field
本发明涉及一种应用于城市安防监控体系的视频平台，尤其是一种密文H264视频直播和点播平台。 The invention relates to a video platform applied to a city security monitoring system, in particular to a ciphertext H264 video live broadcast and on-demand platform.
背景技术 Background technique
Live555流媒体框架是一个为流媒体提供解决方案的跨平台的C++开源项目，它实现了对标准流媒体的传输，并提供对标准流媒体传输协议如RTP/RTCP、RTSP、SIP等的支持，主要应用于对流媒体的音视频的处理，实现媒体的实时推送；ffmpeg是一套可以用来记录、转换数字音频、视频，并能将其转化为流的开源计算机程序。 The Live555 streaming media framework is a cross-platform C++ open source project that provides solutions for streaming media. It realizes the transmission of standard streaming media and provides support for standard streaming media transmission protocols such as RTP/RTCP, RTSP, SIP, etc. It is mainly used to process audio and video of streaming media and realize real-time push of media; ffmpeg is a set of open source computer programs that can be used to record, convert digital audio and video, and convert them into streams.
随着视频编解码技术和网络技术的快速发展，网络视频通信技术成为了人们生活的一部分，然而在开放网络中，视频信息很容易遭受人为的攻击，尤其是在一些重要的应用场景。在城市安防监控体系中，网络信息采集是公共安防系统的基础，目前面临的威胁有1.劫持前端信息采集设备2.盗取、篡改所采集的信息3.前端设备故障、信息采集质量诊断4.特殊应用场景对摄录器材的安全性有特别要求。现在市面上已经有部分对视频加密的产品，主要方法是将视频视为普通文本，对存储在本地的视频文件全部加密。这种方式未考虑数据格式不变性和数据可操作性，难以针对传输中的视频进行加密，因此应用受到极大的限制。在城市安防监控体系中，设计的加密方案是在视频被压缩编码成H.264格式后，保留视频帧的参数和其它重要信息，仅对帧的数据主体进行加密。这样可以既保证视频内容的安全性，又能通过普通视频传输协议进行视频的传输。 With the rapid development of video codec technology and network technology, network video communication technology has become a part of people's life. However, in an open network, video information is vulnerable to human attacks, especially in some important application scenarios. In the urban security monitoring system, network information collection is the foundation of the public security system. The current threats are 1. Hijacking the front-end information collection equipment 2. Stealing and tampering with the collected information 3. Front-end equipment failure and information collection quality diagnosis 4. .Special application scenarios have special requirements for the safety of video recording equipment. There are already some video encryption products on the market. The main method is to treat the video as ordinary text and encrypt all the video files stored locally. This method does not consider the data format invariance and data operability, and it is difficult to encrypt the video in transmission, so the application is greatly limited. In the urban security monitoring system, the designed encryption scheme is to retain the parameters and other important information of the video frame after the video is compressed and encoded into H.264 format, and only encrypt the data subject of the frame. In this way, the security of the video content can be ensured, and the video can be transmitted through a common video transmission protocol.
由于加密方式的选择，完全没有破坏加密视频的结构，对其进行流传输完全可以实现，然而不同于明文视频流推送，密文视频流推送主要面临两个问题，第一：要保证密文视频传输过程中的可靠性，关键帧一旦丢失，会影响密文播放器对于加密视频的解密；第二：对密文视频的封装，一旦视频的帧结构和字节改变，后期将无法解密播放。要实现密文视频的可用性，需要研制相应的密文视频流推送平台。 Due to the choice of the encryption method, the structure of the encrypted video is not destroyed at all, and its streaming transmission is completely achievable. However, unlike plaintext video streaming, ciphertext video streaming mainly faces two problems. The first is to ensure that the ciphertext video Reliability during the transmission process, once the key frame is lost, it will affect the decryption of the encrypted video by the ciphertext player; second: the encapsulation of the ciphertext video, once the frame structure and bytes of the video change, it will not be able to decrypt and play later. To realize the availability of ciphertext video, it is necessary to develop a corresponding ciphertext video streaming platform.
对于密文视频推送平台的建立，必须针对于上述存在的两个问题进行解决。 For the establishment of the ciphertext video push platform, the above two problems must be solved.
发明内容 Contents of the invention
本发明的目的在于提供一种基于live555和ffmpeg库的密文H264视频直播和点播平台，此方案结合了live555框架和ffmpeg开源库提出了一套应用于城市安防监控体系中针对加密视频流流媒体直播点播的平台，可以实现密文视频流的直播和密文视频文件的点播，可靠性高，可以保证密文视频传输的实时性需求和保证用户的较佳体验度。本发明采用的技术方案是： The purpose of the present invention is to provide a live broadcast and on-demand platform of ciphertext H264 video based on live555 and ffmpeg library. This solution combines the live555 framework and ffmpeg open source library to propose a set of applications for encrypted video stream streaming media in urban security monitoring systems. The live on-demand platform can realize the live broadcast of ciphertext video stream and the on-demand of ciphertext video files, with high reliability, which can ensure the real-time requirements of ciphertext video transmission and ensure the best experience of users. The technical scheme adopted in the present invention is:
一种基于live555框架和ffmpeg库的密文H264视频直播和点播平台，包括：基于live555框架的流媒体服务器和外部存储服务器；所述流媒体服务器和外部存储服务器通过网络连接； A kind of ciphertext H264 video live broadcast and on-demand platform based on live555 framework and ffmpeg library, comprising: a streaming server and an external storage server based on the live555 framework; the streaming server and the external storage server are connected through a network;
所述流媒体服务器包括live555直播代理逻辑服务器和live555点播逻辑服务器； Described streaming media server comprises live555 live acting proxy logical server and live555 order logical server;
所述外部存储服务器上设有基于ffmpeg库的密文H264视频转码模块和ffmpeg库； The external storage server is provided with a ciphertext H264 video transcoding module and a ffmpeg storehouse based on the ffmpeg storehouse;
live555点播逻辑服务器和外部存储服务器之间通过TCP协议通信，为此，在live555点播逻辑服务器上建有TCP客户端，在外部存储服务器上建有TCP服务器端。 The communication between the live555 on-demand logic server and the external storage server is through the TCP protocol. Therefore, a TCP client is built on the live555 on-demand logic server, and a TCP server is built on the external storage server.
进一步地， further,
所述live555直播代理逻辑服务器采用RTSP/RTP/RTCP协议簇，接收从加密摄像机传送来的密文H264视频流，并进行RTP打包，在用户视频播放器的直播请求下，将密文H264视频流通过RTSP/RTP/RTCP协议簇推送至用户视频播放器进行播放； The live555 live broadcast agent logic server adopts RTSP/RTP/RTCP protocol cluster, receives the ciphertext H264 video stream transmitted from the encrypted camera, and performs RTP packaging, and sends the ciphertext H264 video stream to the live broadcast request of the user video player Push to the user video player for playback through RTSP/RTP/RTCP protocol cluster;
在外部存储服务器上，所述密文H264视频转码模块，通过调用ffmpeg库，对密文H264视频文件进行关键帧和视频帧的解析，并对密文H264视频文件的帧进行MKV容器的封装，形成密文MKV视频文件并作保存；密文H264视频文件通过接收加密摄像机的密文H264视频流并保存获得或在外部存储服务器中预置密文H264视频文件获得； On the external storage server, the ciphertext H264 video transcoding module analyzes the key frames and video frames of the ciphertext H264 video file by calling the ffmpeg library, and encapsulates the frames of the ciphertext H264 video file in the MKV container , form a ciphertext MKV video file and save it; the ciphertext H264 video file is obtained by receiving the ciphertext H264 video stream of the encrypted camera and saving it or presetting the ciphertext H264 video file in an external storage server;
所述live555点播逻辑服务器采用RTSP/RTP/RTCP协议簇，在收到用户视频播放器的点播请求时，通过TCP协议从外部存储服务器上获取与密文H264视频文件对应的密文MKV视频文件，并将密文MKV视频文件进行RTP打包，然后通过RTSP/RTP/RTCP协议簇推送至用户视频播放器进行播放。 Described live555 on-demand logic server adopts RTSP/RTP/RTCP protocol cluster, when receiving the on-demand request of user's video player, obtains the ciphertext MKV video file corresponding to ciphertext H264 video file from external storage server by TCP protocol, The ciphertext MKV video file is RTP packaged, and then pushed to the user's video player for playback through the RTSP/RTP/RTCP protocol cluster.
更进一步地， go a step further,
所述密文H264视频转码模块进行如下操作，以形成密文MKV视频文件： Described ciphertext H264 video transcoding module carries out following operation, to form ciphertext MKV video file:
S201，首先，输入密文H264视频文件和音频文件； S201, first, input the ciphertext H264 video file and audio file;
S202，打开并初始化密文H264视频文件，调用编解码库libavcodec，赋值AVCodeContext参数；调用媒体格式库libavformat；产生密文H264视频码流； S202, open and initialize the ciphertext H264 video file, call the codec library libavcodec, assign the AVCodeContext parameter; call the media format library libavformat; generate the ciphertext H264 video stream;
S203，打开并初始化音频文件，调用编解码库libavcodec，赋值AVCodeContext参数；调用媒体格式库libavformat；产生音频码流； S203, open and initialize the audio file, call the codec library libavcodec, assign the AVCodeContext parameter; call the media format library libavformat; generate an audio code stream;
S204，比较视频码流中的数据流分包AVpacket和音频码流中的数据流分包AVpacket的时间戳，按照时间戳先后顺序决定读取视频码流还是音频码流中的AVpacket； S204, compare the time stamps of the AVpacket in the video code stream and the AVpacket in the audio code stream, and determine whether to read the AVpacket in the video code stream or the audio code stream according to the sequence of time stamps;
S205，按照时间戳先后依次读取视频码流中的数据流分包AVpacket或音频码流中的数据流分包AVpacket； S205, sequentially read the data stream sub-packet AVpacket in the video code stream or the data stream sub-packet AVpacket in the audio code stream according to the time stamp;
S206，将读取的数据流分包AVpacket封装入MKV容器； S206, subpackage the read data stream into AVpacket and encapsulate it into the MKV container;
S207，输出密文MKV视频文件。 S207, output the ciphertext MKV video file.
本发明的优点在于： The advantages of the present invention are:
1）本发明采用了实时流协议，很好的保证密文视频传输的实时性需求和保证用户的较佳体验度。 1) The present invention adopts a real-time streaming protocol, which can well ensure the real-time requirements of ciphertext video transmission and ensure a better user experience.
2）保证了密文视频文件封装过程中对密文视频流数据结构的保护。 2) It ensures the protection of the data structure of the ciphertext video stream during the encapsulation process of the ciphertext video file.
3）基于可靠传输层协议TCP协议，可以实现多路密文视频文件的点播，测试稳定性可靠。 3) Based on the reliable transport layer protocol TCP protocol, it can realize the on-demand broadcast of multi-channel ciphertext video files, and the test is stable and reliable.
附图说明 Description of drawings
图1为本发明的结构组成示意图。 Fig. 1 is a schematic diagram of the structure and composition of the present invention.
图2为本发明的密文H264视频转码模块工作流程图。 Fig. 2 is a working flowchart of the ciphertext H264 video transcoding module of the present invention.
具体实施方式 Detailed ways
下面结合具体附图和实施例对本发明作进一步说明。 The present invention will be further described below in conjunction with specific drawings and embodiments.
本发明中的密文视频包括密文视频流和密文视频文件。 The ciphertext video in the present invention includes a ciphertext video stream and a ciphertext video file.
如图1所示，一种基于live555框架和ffmpeg库的密文H264视频直播和点播平台，包括：基于live555框架的流媒体服务器和外部存储服务器；所述流媒体服务器和外部存储服务器通过网络连接。 As shown in Figure 1, a kind of ciphertext H264 video live broadcast and on-demand platform based on live555 framework and ffmpeg storehouse, comprise: the stream media server and external storage server based on live555 framework; Described stream media server and external storage server are connected by network .
所述流媒体服务器包括live555直播代理逻辑服务器和live555点播逻辑服务器；所述外部存储服务器上设有基于ffmpeg库的密文H264视频转码模块和ffmpeg库；live555点播逻辑服务器和外部存储服务器之间通过TCP协议通信，为此，在live555点播逻辑服务器上建有TCP客户端，在外部存储服务器上建有TCP服务器端。 Described streaming media server comprises live555 live broadcast proxy logic server and live555 order logic server; The ciphertext H264 video transcoding module based on ffmpeg storehouse and ffmpeg storehouse are provided on the described external storage server; Between live555 order order logic server and external storage server It communicates through TCP protocol. For this purpose, a TCP client is built on the live555 on-demand logical server, and a TCP server is built on the external storage server.
城市安防监控体系中的加密摄像机可提供密文H264视频流。 Encrypted cameras in the urban security monitoring system can provide ciphertext H264 video streams.
下面对本发明的几个主要组成部分做详细说明。 Several main components of the present invention will be described in detail below.
（一）基于live555框架的流媒体服务器。 (1) A streaming media server based on the live555 framework.
现在通用的流媒体服务器，主要是基于三种协议，RTSP/RTP/RTCP、RTMP、HTTP协议，这三种都是应用层协议。HTTP协议不属于实时流媒体协议，是顺序流传输协议，直接通过HTTP协议下载文件到本地播放，不能满足实时观看的需求；RTMP和RTSP/RTP/RTCP协议满足实时流传输的要求，目前主要用于视频的直播，也能满足点播视频的需求；RTMP目前主流是用于FLV的直播点播，协议性能方面能满足要求，但是，在城市安防监控体系中的加密摄像头是对编码后的H264裸流进行推送，在进行密文视频直播时，如果采用RTMP协议，通常需要将其封装为FLV格式，大大降低了直播的效率且提高了直播时延，RTSP/RTP/RTCP协议簇，可以直接进行密文H264裸流的推送，因此，流媒体服务器上的live555直播代理逻辑服务器和live555点播逻辑服务器都采用RTSP/RTP/RTCP协议簇。 The current general-purpose streaming media server is mainly based on three protocols, RTSP/RTP/RTCP, RTMP, and HTTP protocols, all of which are application layer protocols. The HTTP protocol is not a real-time streaming protocol, but a sequential streaming protocol. Downloading files directly through the HTTP protocol and playing them locally cannot meet the needs of real-time viewing; RTMP and RTSP/RTP/RTCP protocols meet the requirements of real-time streaming. Currently, they are mainly used It is suitable for the live broadcast of video, and can also meet the needs of video on demand; RTMP is currently used for live broadcast of FLV on demand, and the protocol performance can meet the requirements. However, the encrypted camera in the urban security monitoring system is to encode the H264 naked stream For push, if the RTMP protocol is used for ciphertext video live broadcast, it usually needs to be encapsulated into FLV format, which greatly reduces the efficiency of live broadcast and improves the delay of live broadcast. The RTSP/RTP/RTCP protocol cluster can directly perform encryption The push of H264 naked stream, therefore, the live555 live broadcast proxy logic server and live555 on-demand logic server on the streaming media server both use the RTSP/RTP/RTCP protocol cluster.
Live555开发库源代码主要包含：(1)UsageEnvironment 模块是 Live555 中最基本的基类之一，它的功能简单，主要是用于消息的输入输出和用户交互功能。(2)BasicUsageEnvrionment库。(3)GroupSock 模块主要是对套接字和网络接口的封装，用于实现视音频数据包的发送和接收功能。(4)LiveMedia 是 Live555 的核心模块。它声明了一个抽象基类 Medium，其他许多类包括 RTSPClient、MediaSource、MediaSink、MediaSession 和 RTCPInstance 都派生自该类。 The source code of the Live555 development library mainly includes: (1) The UsageEnvironment module is one of the most basic base classes in Live555. Its functions are simple and are mainly used for message input and output and user interaction functions. (2) BasicUsageEnvrionment library. (3) The GroupSock module is mainly to encapsulate the socket and network interface, and is used to realize the sending and receiving function of video and audio data packets. (4) LiveMedia is the core module of Live555. It declares an abstract base class Medium from which many other classes, including RTSPClient, MediaSource, MediaSink, MediaSession, and RTCPInstance derive.
实时传输协议RTP（Realtime Transport Protocol）：是针对Internet上多媒体数据流的一个传输协议, 由IETF(Internet工程任务组)作为RFC1889发布。RTP本身只保证实时数据的传输，并不能为按顺序传送数据包提供可靠的传送机制，也不提供流量控制或拥塞控制，它依靠RTCP提供这些服务。实时传输控制协议RTCP（Realtime Transport Control Protocol）：负责管理传输质量，在当前应用进程之间交换控制信息。RTSP（Real Time Streaming Protocol），实时流传输协议，是TCP/IP协议体系中的一个应用层协议。详见RFC2326。RTSP是一个非常类似于HTTP的应用层协议。它是一个多媒体播放控制协议，主要用来使用户在播放流媒体时可以像操作本地的影碟机一样进行控制，即可以对流媒体进行暂停/继续、后退和前进等控制。 Real-time Transport Protocol RTP (Realtime Transport Protocol): It is a transmission protocol for multimedia data streams on the Internet, published by IETF (Internet Engineering Task Force) as RFC1889. RTP itself only guarantees the transmission of real-time data, and does not provide a reliable delivery mechanism for in-order delivery of data packets, nor does it provide flow control or congestion control. It relies on RTCP to provide these services. Real-time Transport Control Protocol RTCP (Realtime Transport Control Protocol): Responsible for managing transmission quality and exchanging control information between current application processes. RTSP (Real Time Streaming Protocol), real-time streaming protocol, is an application layer protocol in the TCP/IP protocol system. See RFC2326 for details. RTSP is an application layer protocol very similar to HTTP. It is a multimedia playback control protocol, which is mainly used to enable users to control the streaming media like operating a local DVD player, that is, to control the streaming media such as pause/continue, rewind and forward.
（二）外部存储服务器。 (2) External storage server.
由于本平台针对城市安防监控体系应用，最终要迁移到嵌入式系统之中，由于嵌入式板子空间容量有限，必须将密文视频文件存储在外部存储设备中，因此必须设置该外部存储服务器。 Since this platform is aimed at the application of the urban security monitoring system, it will eventually be migrated to the embedded system. Due to the limited space capacity of the embedded board, the encrypted video files must be stored in an external storage device, so the external storage server must be set.
外部存储服务器和live555点播逻辑服务器通过TCP协议通信，TCP是一种面向连接的、可靠的、基于字节流的传输层通信协议，在流媒体服务器中（具体在live555点播逻辑服务器中）建立TCP客户端，在外部存储服务器中建立TCP服务器端，实现加密视频文件的稳定快速传输，同时加入多线程，保证多通路同时进行，以确保多用户同时进行密文视频文件的点播。 The external storage server and the live555 on-demand logic server communicate through the TCP protocol. TCP is a connection-oriented, reliable, and byte-stream-based transport layer communication protocol. TCP is established in the streaming media server (specifically in the live555 on-demand logic server). The client side establishes a TCP server in the external storage server to realize the stable and fast transmission of encrypted video files. At the same time, multi-threading is added to ensure that multiple channels are carried out at the same time, so as to ensure that multiple users can simultaneously play encrypted video files on demand.
在外部存储服务器上，所述密文H264视频转码模块，通过调用ffmpeg库(此处是指ffmpeg原始库），对密文H264视频文件进行关键帧和视频帧的解析，并对密文H264视频文件的帧进行MKV容器的封装，形成密文MKV视频文件并作保存；密文H264视频文件通过接收加密摄像机的密文H264视频流并保存获得或在外部存储服务器中预置密文H264视频文件获得。 On the external storage server, the ciphertext H264 video transcoding module, by calling the ffmpeg storehouse (referring to ffmpeg original storehouse here), carries out the analysis of the key frame and the video frame to the ciphertext H264 video file, and to the ciphertext H264 The frame of the video file is encapsulated in the MKV container to form a ciphertext MKV video file and save it; the ciphertext H264 video file is obtained by receiving the ciphertext H264 video stream from the encrypted camera and saving it, or the ciphertext H264 video is preset in the external storage server The file is obtained.
在加密摄像头中，推送和保存的都是基于H264的密文视频流，对于直播而言，没有对拖动进度条和快进快退的需求，所以密文播放器直接接受H264流就可以，但是对于点播而言，由于H264裸流的结构都是紧密的关键帧和视频帧数据，没有包含视频时长和视频参数信息，无法进行进度条拖拽和快进快退的功能，所以必须对密文H264视频流保存成的文件进行视频容器的封装。 In the encrypted camera, the ciphertext video stream based on H264 is pushed and saved. For live broadcast, there is no need to drag the progress bar and fast forward and rewind, so the ciphertext player can directly accept the H264 stream. But for VOD, since the structure of H264 naked stream is tight key frame and video frame data, it does not include video duration and video parameter information, and cannot perform progress bar dragging and fast forward and fast rewind functions, so it must be encrypted The file saved as the H264 video stream is encapsulated in the video container.
由于在城市安防监控体系中，需要加密摄像机产生加密过的视频流，而使用现有的编译好的ffmpeg工具直接使用，是不能正确解码的，会改变密文H264视频的关键帧的结构，推送出去的RTSP密文视频流，到达用户端无法解密。 Because in the urban security monitoring system, the encrypted video stream needs to be generated by the encrypted camera, and the existing compiled ffmpeg tool can not be decoded correctly, and the structure of the key frame of the ciphertext H264 video will be changed. The outgoing RTSP ciphertext video stream cannot be decrypted when it reaches the client.
因此密文H264视频转码模块必须调用ffmpeg原始库，对密文H264视频文件进行关键帧和视频帧的解析，然后进行密文视频帧的封装，在封装密文视频的基础上，不改变密文H264视频的结构，为后期视频解密提供保障。在各种视频封装容器中，MKV容器格式符合本系统要求，在不会改变加密视频的基础上对密文H264视频增加时间戳信息和视频参数信息。 Therefore, the ciphertext H264 video transcoding module must call the ffmpeg original library to analyze the key frame and video frame of the ciphertext H264 video file, and then encapsulate the ciphertext video frame. On the basis of encapsulating the ciphertext video, the encryption will not be changed. The structure of H264 video provides guarantee for later video decryption. Among various video packaging containers, the MKV container format meets the requirements of this system, and adds time stamp information and video parameter information to the ciphertext H264 video without changing the encrypted video.
ffmpeg原始库包括核心工具（libutils）、媒体格式（libavformat）、编解码（libavcodec）、设备（libavdevice）和后处理（libavfilter, libswscale, libpostproc）等库，分别负责提供公用的功能函数、实现多媒体文件的读包和写包、完成音视频的编解码、管理音视频设备的操作以及进行音视频后处理。 The ffmpeg original library includes core tools (libutils), media format (libavformat), codec (libavcodec), device (libavdevice) and post-processing (libavfilter, libswscale, libpostproc) and other libraries, which are responsible for providing common functions and realizing multimedia files. Read and write packets, complete audio and video codec, manage the operation of audio and video equipment, and perform audio and video post-processing.
密文H264视频转码模块的具体工作过程如图2所示： The specific working process of the ciphertext H264 video transcoding module is shown in Figure 2:
S201，首先，输入密文H264视频文件和音频文件；音频文件为额外添加的，格式为mp3格式； S201, first, input the ciphertext H264 video file and audio file; the audio file is additionally added, and the format is mp3 format;
S202，打开并初始化密文H264视频文件，调用编解码库libavcodec，赋值AVCodeContext参数；调用媒体格式库libavformat；产生密文H264视频码流； S202, open and initialize the ciphertext H264 video file, call the codec library libavcodec, assign the AVCodeContext parameter; call the media format library libavformat; generate the ciphertext H264 video stream;
S203，打开并初始化音频文件，调用编解码库libavcodec，赋值AVCodeContext参数；调用媒体格式库libavformat；产生音频码流； S203, open and initialize the audio file, call the codec library libavcodec, assign the AVCodeContext parameter; call the media format library libavformat; generate an audio code stream;
密文H264视频码流和音频码流中包含了一个个顺序的数据流分包AVpacket，在AVpacket包头中含有时间戳； The ciphertext H264 video code stream and audio code stream contain sequential data stream subpackages AVpacket, and the AVpacket header contains a timestamp;
S204，比较视频码流中的数据流分包AVpacket和音频码流中的数据流分包AVpacket的时间戳，按照时间戳先后顺序决定读取视频码流还是音频码流中的AVpacket； S204, compare the time stamps of the AVpacket in the video code stream and the AVpacket in the audio code stream, and determine whether to read the AVpacket in the video code stream or the audio code stream according to the sequence of time stamps;
S205，按照时间戳先后依次读取视频码流中的数据流分包AVpacket或音频码流中的数据流分包AVpacket； S205, sequentially read the data stream sub-packet AVpacket in the video code stream or the data stream sub-packet AVpacket in the audio code stream according to the time stamp;
S206，将读取的数据流分包AVpacket封装入MKV容器； S206, subpackage the read data stream into AVpacket and encapsulate it into the MKV container;
S207，输出密文MKV视频文件。 S207, output the ciphertext MKV video file.
本发明在外部存储服务器中加入了多线程TCP服务器，开启后，将会一直处于监听状态；在live555点播逻辑服务器中加入TCP客户端，为了是TCP传输和live555主线程循环分隔开，加入多线程机制。当播放器请求密文视频时，live555中的TCP客户端发出请求，外部存储中的TCP响应，进行密文MKV视频文件的可靠传输。 The present invention adds a multi-threaded TCP server to the external storage server. After it is turned on, it will always be in a listening state; a TCP client is added to the live555 on-demand logic server. In order to separate the TCP transmission from the live555 main thread cycle, multiple thread mechanism. When the player requests ciphertext video, the TCP client in live555 sends out the request, and the TCP in the external storage responds to perform reliable transmission of ciphertext MKV video files.
本发明的主要工作流程是：所述live555直播代理逻辑服务器采用RTSP/RTP/RTCP协议簇，接收从加密摄像机传送来的密文H264视频流，并进行RTP打包，在用户视频播放器的直播请求下，将密文H264视频流通过RTSP/RTP/RTCP协议簇推送至用户视频播放器进行播放； The main workflow of the present invention is: the live555 live broadcast agent logic server adopts RTSP/RTP/RTCP protocol cluster, receives the ciphertext H264 video stream transmitted from the encrypted camera, and performs RTP packaging, and sends the live broadcast request to the user's video player Next, the ciphertext H264 video stream is pushed to the user's video player through the RTSP/RTP/RTCP protocol cluster for playback;
在外部存储服务器上，所述密文H264视频转码模块，通过调用ffmpeg库，对密文H264视频文件进行关键帧和视频帧的解析，并对密文H264视频文件的帧进行MKV容器的封装，形成密文MKV视频文件并作保存；密文H264视频文件通过接收加密摄像机的密文H264视频流并保存获得或在外部存储服务器中预置密文H264视频文件获得； On the external storage server, the ciphertext H264 video transcoding module analyzes the key frames and video frames of the ciphertext H264 video file by calling the ffmpeg library, and encapsulates the frames of the ciphertext H264 video file in the MKV container , form the ciphertext MKV video file and save it; the ciphertext H264 video file is obtained by receiving the ciphertext H264 video stream of the encrypted camera and saving it or presetting the ciphertext H264 video file in the external storage server;
所述live555点播逻辑服务器采用RTSP/RTP/RTCP协议簇，在收到用户视频播放器的点播请求时，通过TCP协议从外部存储服务器上获取与密文H264视频文件对应的密文MKV视频文件，并将密文MKV视频文件进行RTP打包，然后通过RTSP/RTP/RTCP协议簇推送至用户视频播放器进行播放。 Described live555 on-demand logic server adopts RTSP/RTP/RTCP protocol cluster, when receiving the on-demand request of user's video player, obtains the ciphertext MKV video file corresponding to ciphertext H264 video file from external storage server by TCP protocol, The ciphertext MKV video file is RTP packaged, and then pushed to the user's video player for playback through the RTSP/RTP/RTCP protocol cluster.
本发明可形成一套完整的密文直播点播方案，基于live555开源框架的RTSP直播代理服务器和点播服务器；基于ffmpeg开源库的密文H264转码，并且有效的保护了密文视频关键帧的结构，实现了视频时间戳和视频信息的添加，实现了自动扫描文件夹中的H264密文视频和多路封装的功能；基于传输层TCP协议的密文视频传输，实现了密文视频存储和推送分开进行，TCP协议很好的保证了视频传输的丢包率和稳定性，同时加入的多线程实现了多用户多路点播密文视频的功能。 The present invention can form a complete set of ciphertext live broadcast on-demand program, RTSP live broadcast proxy server and on-demand server based on live555 open source framework; ciphertext H264 transcoding based on ffmpeg open source library, and effectively protects the structure of ciphertext video key frames , realize the addition of video time stamp and video information, realize the function of automatic scanning of H264 ciphertext video in the folder and multi-channel encapsulation; ciphertext video transmission based on the transport layer TCP protocol, realize ciphertext video storage and push Separately, the TCP protocol well guarantees the packet loss rate and stability of video transmission, and the multi-threading added at the same time realizes the function of multi-user multi-channel on-demand ciphertext video.
Claims (3)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510430852.9A CN105007465B (en) | 2015-07-21 | 2015-07-21 | Ciphertext H264 net casts and program request platform based on live555 frames and ffmpeg libraries |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510430852.9A CN105007465B (en) | 2015-07-21 | 2015-07-21 | Ciphertext H264 net casts and program request platform based on live555 frames and ffmpeg libraries |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105007465A true CN105007465A (en) | 2015-10-28 |
| CN105007465B CN105007465B (en) | 2018-06-08 |
Family
ID=54379961
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510430852.9A Active CN105007465B (en) | 2015-07-21 | 2015-07-21 | Ciphertext H264 net casts and program request platform based on live555 frames and ffmpeg libraries |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105007465B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105872581A (en) * | 2016-05-31 | 2016-08-17 | 深圳市佑嘉互动科技有限公司 | System and method for providing video direct broadcasting room services |
| CN105898392A (en) * | 2015-11-24 | 2016-08-24 | 乐视云计算有限公司 | Online live video player and method |
| CN107483974A (en) * | 2017-08-29 | 2017-12-15 | 深圳市茁壮网络股份有限公司 | A kind of service processing method and system |
| CN108900898A (en) * | 2018-08-21 | 2018-11-27 | 北京深瞐科技有限公司 | Video structural method, apparatus and system |
| CN109819236A (en) * | 2019-02-15 | 2019-05-28 | 西安勺子智能科技有限公司 | A kind of multi-user's VR live broadcast system based on unmanned plane binocular video |
| CN110072115A (en) * | 2019-04-17 | 2019-07-30 | 视联动力信息技术股份有限公司 | Data processing method, device and storage medium |
| CN114866836A (en) * | 2022-04-12 | 2022-08-05 | 北京小米移动软件有限公司 | Video processing method, apparatus, electronic device and storage medium |
| CN115002558A (en) * | 2022-05-18 | 2022-09-02 | 武汉斗鱼鱼乐网络科技有限公司 | Audio and video recording method in live broadcasting process and related equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101321280A (en) * | 2008-07-22 | 2008-12-10 | 国家广播电影电视总局广播科学研究院 | A real-time video transcoding streaming media server |
| CN101547345A (en) * | 2008-03-25 | 2009-09-30 | 华为技术有限公司 | Method and system for processing streaming media transcoding |
| CN102306103A (en) * | 2011-08-25 | 2012-01-04 | 华南理工大学 | Software development kit (SDK) module for Android real time streaming protocol (RTSP) player |
| CN102428696A (en) * | 2009-06-11 | 2012-04-25 | 维里逊专利及许可公司 | Enhanced simultaneous recording in multi-room digital video recorder |
| CN102647559A (en) * | 2012-04-26 | 2012-08-22 | 广州盈可视电子科技有限公司 | Pan-tilt tracing and recording method and device |
| CN103870574A (en) * | 2014-03-18 | 2014-06-18 | 江苏物联网研究发展中心 | Label manufacturing and indexing method based on H. 264 ciphertext cloud video storage |
| CN104735457A (en) * | 2015-03-27 | 2015-06-24 | 南京中新赛克科技有限责任公司 | Video encryption and decryption method based on H.264 code |
-
2015
- 2015-07-21 CN CN201510430852.9A patent/CN105007465B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101547345A (en) * | 2008-03-25 | 2009-09-30 | 华为技术有限公司 | Method and system for processing streaming media transcoding |
| CN101321280A (en) * | 2008-07-22 | 2008-12-10 | 国家广播电影电视总局广播科学研究院 | A real-time video transcoding streaming media server |
| CN102428696A (en) * | 2009-06-11 | 2012-04-25 | 维里逊专利及许可公司 | Enhanced simultaneous recording in multi-room digital video recorder |
| CN102306103A (en) * | 2011-08-25 | 2012-01-04 | 华南理工大学 | Software development kit (SDK) module for Android real time streaming protocol (RTSP) player |
| CN102647559A (en) * | 2012-04-26 | 2012-08-22 | 广州盈可视电子科技有限公司 | Pan-tilt tracing and recording method and device |
| CN103870574A (en) * | 2014-03-18 | 2014-06-18 | 江苏物联网研究发展中心 | Label manufacturing and indexing method based on H. 264 ciphertext cloud video storage |
| CN104735457A (en) * | 2015-03-27 | 2015-06-24 | 南京中新赛克科技有限责任公司 | Video encryption and decryption method based on H.264 code |
Non-Patent Citations (1)
| Title |
|---|
| 刘畅棂，包杰，王宁国: "基于 Live555 的网络视频监控系统设计与实现", 《现代电信科技》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105898392A (en) * | 2015-11-24 | 2016-08-24 | 乐视云计算有限公司 | Online live video player and method |
| CN105872581A (en) * | 2016-05-31 | 2016-08-17 | 深圳市佑嘉互动科技有限公司 | System and method for providing video direct broadcasting room services |
| CN107483974A (en) * | 2017-08-29 | 2017-12-15 | 深圳市茁壮网络股份有限公司 | A kind of service processing method and system |
| CN107483974B (en) * | 2017-08-29 | 2020-04-24 | 深圳市茁壮网络股份有限公司 | Service processing method and system |
| CN108900898A (en) * | 2018-08-21 | 2018-11-27 | 北京深瞐科技有限公司 | Video structural method, apparatus and system |
| CN109819236A (en) * | 2019-02-15 | 2019-05-28 | 西安勺子智能科技有限公司 | A kind of multi-user's VR live broadcast system based on unmanned plane binocular video |
| CN110072115A (en) * | 2019-04-17 | 2019-07-30 | 视联动力信息技术股份有限公司 | Data processing method, device and storage medium |
| CN114866836A (en) * | 2022-04-12 | 2022-08-05 | 北京小米移动软件有限公司 | Video processing method, apparatus, electronic device and storage medium |
| CN115002558A (en) * | 2022-05-18 | 2022-09-02 | 武汉斗鱼鱼乐网络科技有限公司 | Audio and video recording method in live broadcasting process and related equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105007465B (en) | 2018-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105007465B (en) | Ciphertext H264 net casts and program request platform based on live555 frames and ffmpeg libraries | |
| US10715844B2 (en) | Method and apparatus for transceiving data for multimedia transmission system | |
| CN102104762B (en) | Media recording method, equipment and system of IMS (Internet Management Specification) video conference | |
| KR102073739B1 (en) | Method of delivering media data based on packet with header minimizing delivery overhead | |
| TWI846795B (en) | Multiple decoder interface for streamed media data | |
| US9596522B2 (en) | Fragmented file structure for live media stream delivery | |
| CN103581699B (en) | Video playing method, streaming media server and streaming media playing system | |
| MX2014012361A (en) | Methods and systems for real-time transmuxing of streaming media content. | |
| US10477282B2 (en) | Method and system for monitoring video with single path of video and multiple paths of audio | |
| WO2008061416A1 (en) | A method and a system for supporting media data of various coding formats | |
| US7249264B2 (en) | Secure IP based streaming in a format independent manner | |
| US20110299586A1 (en) | Quality adjustment using a fragmented media stream | |
| TW201828709A (en) | Detecting and signaling new initialization segments during manifest-file-free media streaming | |
| EP3096524B1 (en) | Communication apparatus, communication data generation method, and communication data processing method | |
| CN105577645A (en) | Proxy-based HLS client device and its implementation method | |
| CN108632679B (en) | A kind of method that multi-medium data transmits and a kind of view networked terminals | |
| CN112953899A (en) | Streaming media distribution system and method supporting multi-audio video streaming protocol | |
| CN100442846C (en) | A mobile multi-point monitoring method using SMIL technology and mobile streaming media technology | |
| TW202448163A (en) | Signaling media timing information from a media application to a network element | |
| KR100820350B1 (en) | Integrated streaming server and streaming implementation method to support various file container formats | |
| CN108124183B (en) | Method for synchronously acquiring video and audio to perform one-to-many video and audio streaming | |
| CN117395466A (en) | Video transmission real-time monitoring method and system and electronic equipment | |
| TWI600319B (en) | A method for capturing video and audio simultaneous for one-to-many video streaming | |
| CN101399963B (en) | Real-time control method and system for media stream | |
| TW202524900A (en) | Tethered devices for webrtc in a cellular system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |