CN115209230A

CN115209230A - Method for realizing real-time video transmission based on RTMP protocol

Info

Publication number: CN115209230A
Application number: CN202110391578.4A
Authority: CN
Inventors: 吴振华; 孙志强; 吕英明; 吕�峥
Original assignee: Individual
Current assignee: Individual
Priority date: 2021-04-14
Filing date: 2021-04-14
Publication date: 2022-10-18

Abstract

The invention relates to a method for realizing real-time video transmission based on an RTMP protocol, which transmits video data to a streaming media server in a local area network environment for scheduling and using by a client. The method for realizing real-time video transmission comprises three stages of video frame data uploading, streaming media server storage and client streaming. In the video frame data uploading stage, the video data acquired by the network camera is uploaded to the streaming media server in the form of RTMP packets, so that the reliability of the transmission process is ensured; the streaming media server ensures the real-time property of transmission through an RTMP real-time module of Nginx; the client pull stream adopts the video stream transmitted by the FFmpeg decoding server to ensure the video accuracy. The method uses the local area network to maintain the overall operation Of the system, and improves the real-time performance Of the video transmission system by adjusting the interval Of GOPs (Group Of Pictures) in the video uploading and receiving.

Description

Method for realizing real-time video transmission based on RTMP protocol

Technical Field

The invention relates to a method for realizing real-time video transmission based on an RTMP protocol. Real-time video transmission is a core component of a video monitoring system and belongs to the field of communication. It is widely used in the fields of direct broadcast, traffic, medical treatment, military and the like.

Background

In the field of transport protocols, RTP is used as a network protocol for transmitting video streams in the industry, the protocol relies on UDP network protocol for transmission, and since UDP is connectionless and unreliable, packet loss, delay and jitter are easily caused during network transmission, and when the network condition is not good, the user experience is seriously affected. The RTMP protocol uses TCP as a protocol of a transport layer, can effectively ensure the transmission quality of video streams, and supports the dynamic transmission of sound and video from a server to a client. In addition, the FLV packaging format supported by the RTMP protocol can display clear pictures under the condition of low code rate, and is very suitable for watching videos under the condition of insufficient bandwidth. The RTMP protocol is adopted to transmit the video data, so that the functions of reliability and stability are achieved.

The method for realizing real-time video transmission based on the RTMP protocol collects video data through the network camera, transmits the video data to the Nginx streaming media server based on the RTMP protocol, provides about 3000 request connection numbers through the epoll concurrency technology of the Nginx, and improves the reliability and the real-time performance of video transmission.

Disclosure of Invention

The invention aims to provide a method for realizing real-time video transmission based on an RTMP protocol, which can effectively solve the problems of packet loss in the video data transmission process and picture blockage in the video playing process and has the characteristics of high efficiency, low delay and the like in video data transmission.

The invention is realized by adopting the following technical means:

1. a real-time video transmission implementation method based on RTMP protocol, the video transmission process is divided into three stages of video frame data uploading, streaming media server storage and client streaming:

1.1, acquiring high-definition video data through a remote network camera, converting the video data format into a YUV420 format, coding according to an H264 standard, and finally converting the video data format into an FLV format file meeting the transmission requirement of an RTMP protocol through FFmpeg software encapsulation context;

1.2 the streaming media server adopts a Nginx lightweight server, can provide a corresponding website for the uploaded video data to be stored or broadcasted, establishes an RTMP connection channel for transmission in the process of receiving or broadcasting, and realizes concurrent connection of a plurality of clients through the epoll technology of the streaming media server after the video data reaches the streaming media server;

1.3 the client of the video monitoring system is built through a Qt framework, the video data broadcasted by the streaming media server is received by adopting a mode of circularly starting threads, one channel is opened for transmission each time the threads are started, each thread is monitored, and resources are released immediately when the threads stop decoding videos;

1.4, the problem of picture distortion is solved by creating a FIFO buffer queue, firstly, a client receives video data transmitted by a streaming media server, stores the data into the FIFO buffer queue, a decoder reads the video data from the FIFO buffer queue for decoding, and the video data is continuously buffered in an idle space.

2. The method according to claim 1, wherein the RTMP protocol is used for real-time video transmission, and the method comprises: the video input source transmits data to the FFmpeg, the data is uploaded to the Nginx server in an RTMP data format after being encoded and packaged, and then the server forwards the data to the client. Before network connection of uplink data is established, a video push end and a server are subjected to handshake operation by calling a ngx _ rtmp _ handshake function and a ngx _ rtmp _ handshake _ recv function in the server, a Nginx-rtmp module of Nginx is selected to build a streaming media server, and then forwarding of video data streams is achieved by modifying the rtmp module in a configuration file.

3. The method according to claim 1, wherein the RTMP protocol is used for real-time video transmission, and the method comprises: the aforementioned Nginx server in 1.2 establishes a plurality of RTMP connections for data transmission, which include connections of a plurality of channels in one client and connections of a plurality of channels in a plurality of clients.

4. The method according to claim 1, wherein the RTMP protocol is used for real-time video transmission, and the method comprises: the FIFO buffer queue in 1.4 above buffers video data at the entry of the queue and decodes video at the exit of the queue.

5. The method according to claim 1, wherein the RTMP protocol is used for real-time video transmission, and the method comprises: the FIFO buffer queue in 1.4 above buffers the double key frames and sets reasonable GOP _ size for both pushing video data and fetching video data to adjust the key frame volume and the gap between GOP frames.

The invention discloses a method for realizing real-time video transmission based on an RTMP protocol, which has the following advantages:

1. even when a plurality of thread channels are opened, the system can realize zero packet loss in the video transmission process, and high-definition display can be continuously and stably performed at the client without distortion.

2. Through the local area network and the setting of the GOP key frame, the video monitoring system can keep low video delay all the time from uploading to receiving.

3. The server has the characteristic of well scheduling resources by using the Nginx lightweight server, and can provide good and stable connection for the client.

Drawings

FIG. 1 is a flow chart of the present invention for real-time video transmission

FIG. 2 is a real-time video transmission architecture diagram of the present invention

FIG. 3 is a flow chart of the present invention for forwarding streaming media

FIG. 4 is a diagram of the structure of the FIFO of the present invention

FIG. 5 is a GOP frame structure diagram of the present invention

FIG. 6 is a flow chart of a video stream push module of the present invention

FIG. 7 is a flow chart of a video data acquisition module of the present invention

Detailed Description

The invention is further described by the following description and the accompanying drawings:

fig. 1 is a flow chart of real-time video transmission, which is respectively a device side, a streaming media server side and a client side. Firstly, setting a network camera to start video data acquisition, coding according to an H.264 standard format, and packaging according to an FLV packaging strategy provided by FFmpeg; then transmitting the data packet to a streaming media server configured by Nginx through a local area network for broadcasting, and entering a waiting request state; the client side sends a request to the streaming media server to obtain the video, when the video reaches the client side, the video firstly enters a cache region to start a thread to be decoded, and finally the obtained real-time video is played through a playing module. And RTMP/TCP protocol is adopted in the video data uploading and video data acquisition processes.

Fig. 2 is an architecture diagram of real-time video transmission, which is mainly divided into four parts: the system comprises a camera video acquisition unit, a streaming media server, a network transmission unit and a client. The four parts are connected through a switch router to form a local area network, video data transmission is acquired by a camera at first, and RTMP connection is established through the local area network and transmitted to a streaming media server; the server mainly comprises two servers: the first is a streaming media server which is responsible for the forwarding function of video data, and the second is a database server which is responsible for the storage and query functions of user information and equipment information. If the number of users and the number of devices are not large, the database server and the streaming media server can be built in the same server; the video monitoring client is responsible for managing user equipment information, the plurality of clients are connected into the local area network, send requests to the streaming media server to acquire real-time video data, and display the video data to the clients for users to watch. Due to the adoption of Nginx as a streaming media server, 3000 RTMP connections can be supported for different clients.

Fig. 3 is a flow chart of streaming media forwarding, and the streaming media needs to go through three processes in the uploading process: establishing network connection, establishing network flow and transmitting video data. Before network connection is established for uplink data, handshaking operation is carried out between a video push end and a server by calling ngx _ rtmp _ handshake function and ngx _ rtmp _ handshake _ recv function in the server, rtmp network connection is established after handshaking is completed, nginx calls ngx _ rtmp _ recv function to analyze a message block, video data sent by the video push end is acquired by using ngx _ event _ t structure, and the data is stored in an in linked list structure of ngx _ rtmp _ stream _ t. In the continuous process of connection, the server initializes network connection by utilizing ngx _ rtmp _ cmd _ connect _ init function, and obtains amf information through ngx _ rtmp _ receive _ amf, wherein the amf information is format information of data exchange between the remote server and the flash. Finally, necessary parameters in the connection process are set through ngx _ rtmp _ cmd _ args, and the parameters configured by the video data sending end are notified through ngx _ rtmp _ notify _ connect, so that information establishment in the connection process is completed.

The network flow indicates a channel through which audio and video data is transmitted, and a plurality of channels may be included in one network connection. After the ngx _ rtmp _ recv function completes the analysis of the received data, the analyzed data is transferred to the ngx _ rtmp _ receive _ message function to establish a network flow. After receiving the information, the Nginx calls a processing function ngx _ rtmp _ protocol _ message _ handle and ngx _ rtmp _ amf _ message _ handle, wherein the former can carry out secondary processing on the information according to the type of the information, and the latter can process the corresponding amf control information. Finally, a network stream is initialized through a ngx _ rtmp _ cmd _ create _ stream _ init function, a ngx _ rtmp _ cmd _ create _ stream function is called to create a network stream, and after creation is completed, a client and a server can transmit information to realize a stream forwarding function.

FIG. 4 is a structure of a FIFO buffer queue, which has a first-in first-out characteristic, and solves the picture distortion problem by creating the FIFO buffer queue, when the video data starts to be transmitted, the video data continuously enters the FIFO buffer queue until the size of a frame of data is full, then the decoding thread decodes, the video data is still buffered in the FIFO, and a state of simultaneous buffer decoding is formed; the decoder reads the video data from the FIFO, and the FIFO continuously downloads the data from the streaming media server, so that when the network is unreliable, the FIFO buffer queue created by the client can make the user visually unnoticeable, and the playing process is distortion-free and smooth.

Fig. 5 is a structural diagram of a GOP frame, which shows the number of B frames between an I frame and a P frame. A GOP (Group of picture) frame in a video stream is a Group of consecutive pictures, is composed of one I frame and several B/P frames, is a basic unit for access by a video codec, and has a certain volume. The GOP frames are key frames of the videos, at least one key frame is needed to be used for each second of video, each frame of data is transmitted on the network after being marked with a time sequence tag when the videos are uploaded, and due to the fact that the data volume of the GOP frames is large, when the network environment is poor, the GOP frames can not be downloaded within the second level, and delay is affected. According to the invention, the volume of the key frame is adjusted by reducing the distance between the frame I and the frame P in the GOP at both sides of pushing the video data and acquiring the video data, and the loading process can be completed in a short time by caching the double key frames, so that the delay generated in the video transmission process is reduced.

Fig. 6 is a flow chart of a video stream pushing module, and video data acquisition and pushing are equivalent to a data input source of a system, which affects video quality and overall performance of a monitoring system. After the two processes are completed, the push end initiates a connection request according to the server address specified in the encapsulation head, a network connection is established through handshaking, and after the connection between the push end and the server end is established, the push end can transmit video data to the server through a network flow.

Fig. 7 is a flow chart of a video data acquisition module. The video frame data receiving module is the most important module in the video monitoring client and is also a bridge for connecting the server and the client in the video transmission process. The invention adopts a multithread mode to encapsulate the video data acquisition flow in the run () function of the thread, thus when n windows are played simultaneously, n threads are opened to acquire the video stream from the streaming media server and display the video stream to the corresponding interface of the client. During the receiving of the video frame data, a device list module of the client requests a URL (Uniform Resource Locator) of the device from the database server, and then requests the video data from the streaming media server through the URL of the device. In order to prevent the mosaic phenomenon of the monitoring video, the video data need to enter a cache when arriving at a client, the client reads the video data from a cache region, and the client performs decapsulation, decoding and transcoding operations on the data to obtain image data in an RGB format.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention and not to limit it; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art will understand that: modifications of the embodiments of the invention or equivalent substitutions for parts of the technical features are possible; without departing from the spirit of the present invention, it is intended to cover all aspects of the invention as defined by the appended claims.

Claims

1. A method for realizing real-time video transmission based on RTMP protocol, the video transmission process is divided into three stages of video frame data uploading, streaming media server storage and client terminal stream pulling:

1.1, acquiring high-definition video data through a remote network camera, converting the video data format into a YUV420 format, coding according to an H.264 standard, and finally converting the video data format into an FLV format file meeting the transmission requirement of an RTMP protocol through FFmpeg software encapsulation context;

2. The method of claim 1, wherein the RTMP protocol is applied to a real-time video transmission system, and the method comprises: the video input source transmits data to the FFmpeg, the data is uploaded to the Nginx server in an RTMP data format after being encoded and packaged, and then the server forwards the data to the client. Before network connection is established for uplink data, a video pushing end and a server are subjected to handshake operation by calling ngx _ rtm _ handshake function and ngx _ rtm _ handshake _ recv function in the server, a Nginx-rtmp module of Nginx is selected to build a streaming media server, and then forwarding of video data streams is achieved by modifying the rtmp module in a configuration file.