CN111225222A

CN111225222A - Video stream playing method, device and system based on screen data of RTMP (real time Messaging protocol)

Info

Publication number: CN111225222A
Application number: CN201811413863.6A
Authority: CN
Inventors: 梁晓静; 李珂; 黄威
Original assignee: Beijing Qihoo Technology Co Ltd
Current assignee: Beijing Qihoo Technology Co Ltd
Priority date: 2018-11-26
Filing date: 2018-11-26
Publication date: 2020-06-02

Abstract

The invention provides a video stream playing method, a device and a system of screen data based on a real-time message transfer protocol (RTMP). The system comprises: the stream pushing end is used for acquiring video stream data of a screen of the mobile equipment, converting the acquired video stream data into an RTMP format and pushing the converted video stream data to an RTMP stream server; the RTMP streaming server is used for receiving the video streaming data pushed by the pushing end and responding to a playing request of the pulling client to the video streaming data; and the stream pulling client is used for sending a playing request of the video stream data to the RTMP stream server and pulling the video stream data from the RTMP stream server for playing after receiving a response of the RTMP stream server to the playing request. The system has the advantages of low time delay and easiness in expansion, screen data of the mobile equipment can be played in real time at the cloud end, and the system is suitable for a scene of mobile test through a remote real-machine renting platform.

Description

Video stream playing method, device and system based on screen data of RTMP (real time Messaging protocol)

Technical Field

The invention relates to the technical field of remote real machine testing, in particular to a method, a device and a system for playing a video stream of screen data based on a real-time message transfer protocol (RTMP), a computer storage medium and computing equipment.

Background

The remote real machine renting platform is a platform for remotely controlling a mobile phone and has the functions of installation, screenshot, shell, file transmission and the like. In the mobile test, the remote real-machine renting platform can provide real-machine operation of a cloud end, so that tests such as a functional module, performance, safety, capacity, compatibility, usability, a hardware environment, a cross event and an interface of mobile-end app are provided for a user. In the test, the screen data of the mobile phone end is transmitted to the display client for real-time display, so that a user can know the test condition at any time.

In the prior art, when a remote real-machine renting platform is used for mobile testing, a source opening tool minicap is usually adopted to realize mobile phone screen information transmission. Because the use of the minicap tool on the mobile phone strictly depends on the Android version of the mobile phone, and the implementation mode of the minicap adopts the private API (Application programming interface) of the Android, the display mode of the screen data of the existing mobile phone end is limited. In addition, the existing mobile phone screen display mode has the problems of high time delay and the like, and the user experience is low.

Therefore, a technology for playing the screen data of the mobile phone end in real time at the cloud end with low time delay and easy expansion is needed.

Disclosure of Invention

In view of the above problems, the present invention has been made to provide a method, apparatus, system, computer storage medium and computing device for video stream playing of screen data based on real time message transfer protocol RTMP that overcome or at least partially solve the above problems.

According to an aspect of the embodiments of the present invention, there is provided a video stream playing system of screen data based on a real time messaging protocol RTMP, including:

the stream pushing end is used for acquiring video stream data of a screen of the mobile equipment, converting the acquired video stream data into an RTMP format and pushing the converted video stream data to an RTMP stream server;

the RTMP streaming server is used for receiving the video streaming data pushed by the pushing end and responding to a playing request of the pulling client to the video streaming data; and

the stream pulling client is used for sending a playing request of the video stream data to the RTMP stream server, and pulling the video stream data from the RTMP stream server for playing after receiving a response of the RTMP stream server to the playing request.

Optionally, the video stream playing system is applied to a scene that displays screen information of the mobile device when a mobile test is performed through a remote real machine rental platform.

Optionally, the RTMP streaming server includes a livego server;

the RTMP streaming server is further configured to:

and starting the livego server before receiving the video stream data pushed by the push streaming end.

Optionally, the data format of the acquired video stream data of the screen of the mobile device is h.264.

Optionally, the push flow end is further configured to:

and establishing communication with the mobile device by executing an adb forward command so as to acquire video stream data of the self screen acquired by the mobile device from the mobile device.

Optionally, the video stream data of the own screen collected by the mobile device is obtained by the mobile device by running a screen copy tool on the mobile device and calling a MediaCodec interface.

Optionally, the push flow end is further configured to:

and converting the acquired video stream data into an RTMP format by calling an instance srs _ h264_ raw _ publishing.c of the C library srs, and pushing the converted video stream data to the RTMP stream server.

Optionally, the push flow end is further configured to:

converting the acquired video stream data in the H.264 format into an RTMP format by calling a read _ h264_ frame function;

and pushing the converted video stream data to the RTMP stream server by calling srs _ h264_ write _ raw _ frames function.

Optionally, the playback protocol of the pull streaming client includes any one of the following:

RTMP、HTTP-FLV、HLS。

optionally, the pull client comprises a VLC media player and/or a livelayer component.

According to another aspect of the embodiments of the present invention, there is also provided a video stream playing method based on screen data of an RTMP, applied to a stream push terminal, including:

acquiring video stream data of a screen of a mobile device;

converting the acquired video stream data into an RTMP format, and pushing the converted video stream data to an RTMP stream server so that a stream pulling client pulls the video stream data from the RTMP stream server for playing.

Optionally, the method is applied to a scenario in which screen information of the mobile device is displayed when a mobile test is performed through a remote real machine rental platform.

Optionally, acquiring video stream data of a screen of the mobile device includes:

Optionally, the obtained video stream data is converted into an RTMP format by calling the C library srs instance srs _ h264_ raw _ publishing.c, and the converted video stream data is pushed to the RTMP stream server.

Optionally, the step of converting the obtained video stream data into an RTMP format by calling the C library srs instance srs _ h264_ raw _ publishing.c, and streaming the converted video stream data to the RTMP streaming server, further includes:

According to another aspect of the embodiments of the present invention, there is also provided a video stream playing method based on screen data of an RTMP, which is applied to an RTMP stream server, and includes:

receiving video stream data of a screen of the mobile equipment converted into an RTMP format and pushed by a push stream end;

responding to a playing request of the pull stream client to the video stream data, so that the pull stream client pulls the video stream data from the RTMP stream server for playing.

Optionally, the RTMP streaming server comprises a livego server;

before receiving video stream data of a screen of a mobile device converted into an RTMP format, to which a plug-end plug-flows, the method further comprises:

and starting the livego server.

According to another aspect of the embodiments of the present invention, there is also provided a video stream playing method based on screen data of an RTMP, applied to a pull stream client, including:

sending a play request to an RTMP streaming server for video streaming data pushed to a screen of a mobile device of the RTMP streaming server;

and after receiving the response of the RTMP streaming server to the playing request, pulling the video streaming data from the RTMP streaming server for playing.

Optionally, when the RTMP streaming server is a livego server, the protocol for pulling the video streaming data from the RTMP streaming server to play includes any one of:

RTMP、HTTP-FLV、HLS。

According to another aspect of the embodiments of the present invention, there is also provided a flow pushing end, including:

the data acquisition module is suitable for acquiring video stream data of a screen of the mobile equipment; and

and the conversion flow pushing module is suitable for converting the acquired video stream data into an RTMP format and pushing the converted video stream data to the RTMP stream server so that the stream pulling client can pull the video stream data from the RTMP stream server for playing.

Optionally, the plug flow terminal is applied to a scene that displays screen information of the mobile device when a mobile test is performed through a remote real machine rental platform.

Optionally, the data acquisition module is further adapted to:

Optionally, the conversion plug-flow module is further adapted to:

According to another aspect of the embodiments of the present invention, there is also provided an RTMP streaming server, including:

the plug flow receiving module is suitable for receiving video stream data of a screen of the mobile equipment converted into the RTMP format and pushed to by the plug flow end; and

and the pull stream response module is suitable for responding to a playing request of the pull stream client to the video stream data so that the pull stream client can pull the video stream data from the RTMP stream server for playing.

Optionally, the RTMP streaming server is applied to a scenario in which screen information of the mobile device is displayed when a mobile test is performed through a remote live machine rental platform.

Optionally, the RTMP streaming server is a livego server;

the RTMP streaming server further comprises:

and the server side starting module is suitable for starting the livego server side before receiving the video stream data of the screen of the mobile equipment converted into the RTMP format and pushed by the push streaming side.

According to another aspect of the embodiments of the present invention, there is also provided a pull streaming client, including:

a request sending module adapted to send a play request for video stream data of a screen of a mobile device pushed to an RTMP stream server to the RTMP stream server; and

and the pull stream playing module is suitable for pulling the video stream data from the RTMP stream server to play after receiving the response of the RTMP stream server to the playing request.

Optionally, the pull streaming client is applied to a scene displaying screen information of the mobile device when a mobile test is performed through a remote live machine rental platform.

Optionally, when the RTMP streaming server is a livego server, the playback protocol of the pull streaming client includes any one of the following:

RTMP、HTTP-FLV、HLS。

optionally, the pull stream playing module includes a VLC media player and/or a liveplayer component.

According to yet another aspect of the embodiments of the present invention, there is also provided a computer storage medium storing computer program code which, when run on a computing device, causes the computing device to execute any one of the video stream playing method applied to the RTMP-based screen data of the push streaming end, the video stream playing method applied to the RTMP-based screen data of the RTMP streaming server, or the video stream playing method applied to the RTMP-based screen data of the pull streaming client described above.

According to still another aspect of the embodiments of the present invention, there is also provided a computing device including:

a processor; and

a memory storing computer program code;

when executed by the processor, the computer program code causes the computing device to perform any of the video stream playing method applied to the RTMP-based screen data of the push streaming end, the video stream playing method applied to the RTMP-based screen data of the RTMP streaming server, or the video stream playing method applied to the RTMP-based screen data of the pull streaming client described above.

The video stream playing scheme of the screen data based on the RTMP provided by the embodiment of the invention mainly comprises a stream pushing end, an RTMP stream server and a stream pulling client end, wherein the video stream data of the screen of the mobile equipment is obtained through the stream pushing end, the video stream data is converted into an RTMP format and then is pushed to the RTMP stream server, the RTMP stream server receives the video stream data of the RTMP stream pushing and responds to a playing request of the stream pulling client end, and the stream pulling client end pulls the video stream data from the RTMP stream server for playing after receiving the response of the RTMP stream server to the playing request, so that the screen data of the mobile equipment is played in real time at a cloud end, and the video stream playing scheme is suitable for displaying a scene of screen information of a mobile phone end when a mobile test is carried out through a remote real-. The scheme of the embodiment of the invention adopts the RTMP protocol to carry out push stream and pull stream playing of the video stream data, and is particularly suitable for occasions of long-time playing because the RTMP stream has low delay and can be continuously played for a long time. Meanwhile, the video stream playing system provided by the embodiment of the invention has clear overall architecture, the components of the architecture have lower coupling, the RTMP stream pushing end, the stream server and the stream pulling end can be moved randomly without influencing the overall performance of the system, and the easy expansibility of the system is enhanced.

Further, the RTMP streaming server adopts a livego server. The livego server is developed based on golang, supports massive lightweight level coroutines, and has the advantages of supporting RTMP stream pushing and pulling, statistical information, http online stream state checking and the like. The live server is adopted to realize RTMP stream pushing and stream pulling to complete the playing and displaying of the video stream, so that the performance of the RTMP stream server for processing multi-process concurrency is improved, and a better display effect and better operation experience are provided for a user during stream pulling playing.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

fig. 1 is a schematic diagram illustrating an architecture of a video stream playing system based on RTMP screen data according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating an architecture of a video stream playing system based on RTMP screen data according to another embodiment of the present invention; and

FIG. 3 shows the image display effect of the cell phone screen based on the minicap tool;

fig. 4 illustrates a pull stream display effect of a cell phone screen of a video stream playing system based on RTMP screen data according to an embodiment of the present invention;

fig. 5 is a flowchart illustrating a video stream playing method applied to a push stream end based on RTMP screen data according to an embodiment of the present invention;

fig. 6 is a flowchart illustrating a video stream playing method based on RTMP screen data applied to an RTMP streaming server according to an embodiment of the present invention;

fig. 7 is a flowchart illustrating a video stream playing method of RTMP-based screen data applied to a pull streaming client according to an embodiment of the present invention;

fig. 8 is a flowchart illustrating a video stream playing method of RTMP-based screen data in combination with a push streaming client, an RTMP streaming server and a pull streaming client according to an embodiment of the present invention;

FIG. 9 shows a schematic structural diagram of a push flow end according to an embodiment of the invention;

fig. 10 is a schematic structural diagram of an RTMP streaming server according to an embodiment of the present invention; and

fig. 11 is a schematic structural diagram of a pull streaming client according to an embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

In the mobile test, the remote real-machine renting platform can provide real-machine operation of a cloud, and screen data of the mobile phone end is transmitted to the display client to be displayed in real time, so that a user can know the test condition at any time. An existing remote real machine renting platform generally adopts an open source tool minicap to realize transmission and display of mobile phone screen information, however, because the minicap tool is developed by NDK, the implementation mode adopts a hidden private API of Android, the use of the minicap tool must strictly depend on the Android version of the mobile phone and the minicap file corresponding to the Android version, and the minicap tool can be controlled by access restriction of Google to the private API, so that the universality of a screen copy scheme based on the minicap tool is greatly limited. Therefore, when the screen display mode of the mobile phone is improved, the cloud display is carried out in a video streaming mode.

The RTMP (Real Time Messaging Protocol) is a Protocol of an application layer proposed by Adobe Systems, which is based on TCP (Transmission Control Protocol), and is a Protocol family including RTMP basic Protocol and various variants such as RTMPT, RTMPs, RTMPE, and the like. RTMP is a network protocol designed for real-time data communication, provides a protocol capable of comprehensively and simply realizing to ensure the transmission of stream information, and is mainly used for audio-video and data communication between a Flash/AIR platform and a stream media/interaction server supporting the RTMP protocol. Software supporting the protocol includes Adobe Media Server, ultra Media Server, Red5, and the like. Based on the characteristics of good support of RTMP on adobe, suitability for long-time playing, low delay and the like, RTMP is widely applied to the fields of streaming media live broadcast (such as video live broadcast) and the like. The inventor considers the requirement of playing the screen data of the mobile equipment in real time at the cloud end in the mobile test, and completes the playing and displaying work of the video stream of the screen of the mobile equipment by using the RTMP protocol.

In order to solve the above technical problem, an embodiment of the present invention provides a video stream playing system based on screen data of an RTMP. Fig. 1 is a schematic diagram illustrating an architecture of a video stream playing system based on RTMP screen data according to an embodiment of the present invention. Referring to fig. 1, the system essentially comprises: a push streaming client 110, an RTMP streaming server 120, and a pull streaming client 130.

In the embodiment of the present invention, the stream pushing terminal 110 is configured to obtain video stream data of a screen of the mobile device, convert the obtained video stream data into an RTMP format, and push stream the converted video stream data to the RTMP stream server 120. The RTMP streaming server 120 is configured to receive the video stream data pushed by the stream pushing client 110, and respond to the request of the stream pulling client 130 to play the video stream data. The pull streaming client 130 is configured to send a play request for the video streaming data to the RTMP streaming server 120, and after receiving a response to the play request from the RTMP streaming server 120, pull the video streaming data from the RTMP streaming server 120 for playing.

In the system provided in this embodiment, the stream pushing end 110 obtains video stream data of the screen of the mobile device, converts the video stream data into an RTMP format, and then pushes the stream to the RTMP stream server 120, the RTMP stream server 120 receives the video stream data of the RTMP stream pushing and responds to the play request of the stream pulling client 130, and the stream pulling client 130 pulls the video stream data from the RTMP stream server 120 for playing after receiving the response of the RTMP stream server 120 to the play request, thereby realizing real-time playing of the screen data of the mobile device at the cloud end. The scheme of the embodiment of the invention adopts the RTMP protocol to carry out push stream and pull stream playing of the video stream data, and is particularly suitable for occasions of long-time playing because the RTMP stream has low delay and can be continuously played for a long time. Meanwhile, the components of the video stream playing system provided by the embodiment of the invention have lower coupling, and the stream pushing end 110, the RTMP stream server 120 and the stream pulling client 130 can be moved randomly without affecting the overall performance of the system, thereby enhancing the expansibility of the system.

Based on the above characteristics, the video stream playing system based on the screen data of the RTMP provided by the embodiment of the present invention can be well applied to a scene where the screen information of the mobile device is displayed when a mobile test is performed through the remote live machine rental platform, so as to provide a better live machine operation experience for the user.

In an actual application scenario, the above-mentioned mobile device may include an intelligent mobile terminal such as a mobile phone, a tablet computer, and a smart watch, on which an operating system may be loaded, and which has an internet access capability. Preferably, the Android system is loaded on the mobile device. The push stream end 110 may be a server or a PC (Personal Computer) which has functions of data transmission and data parsing. When the stream pushing end 110 is a server, the RTMP stream server 120 and the stream pushing end 110 may be integrated on one server or may be separately disposed. The pull streaming client 130 may be a display on which a software-like client for video streaming playing, such as a browser, a media player, etc., is loaded.

Livego servers are RTMP servers developed based on golang (The Go Programming Language, Go Programming Language). The golang language basically supports balanced use of the multi-core CPU, supports massive lightweight threads and improves concurrency of the threads. The method solves the problems that the current open source srs (Simple-RTMP-Server) can only start a plurality of srs monitoring different ports to improve concurrency when needing multi-core operation, the Nginx-RTMP needs secondary development to realize the forwarding of the message in a plurality of processes after starting the plurality of processes, and the development efficiency far exceeds C/C + +, at the language level. The live server developed based on the golang is very simple to install and use, can run across platforms, has high performance of processing multi-process concurrency, and supports common transmission protocols, file formats and encoding formats. The transmission protocol supported by the Livego server comprises the following steps: RTMP, AMF (action message Format), HLS (HTTP Live Streaming), HTTP-FLV (Flash Video); the supported container formats include: FLV, TS (Transport Stream); the supported encoding formats include: h.264, AAC (Advanced Audio Coding), MP 3. In addition, the livego server has the following advantages: RTMP push flow and pull flow are supported; statistical information support; and http online stream state viewing is supported.

In order to improve the performance of the RTMP streaming server, in an optional implementation manner of the embodiment of the present invention, the RTMP streaming server 120 may include a livego server.

Before the screen display application is launched, a live server is installed, for example, the live server may be installed on a cloud server side to serve as the RTMP streaming server 120. In one embodiment, the compiled binary file may be downloaded directly and then installed in the command line. In another embodiment, the source code may also be downloaded by: git clonehttps://github.com/gwuhaolin/livego.gitAnd then, compiling the source code file by executing the build in the live directory to obtain an executable file.

After installing the livego server, before receiving the video stream data pushed by the push stream terminal 110, the live stream server needs to be started in the RTMP stream server 120 (i.e., the livego server) to provide corresponding services for RTMP push stream and pull stream.

The following lists an example of a command line for building a livego server and starting the livego server in a Windows environment:

C:\Users\phonerenter>livego.exe

2018/09/27 17:12:58main.go:106:start livego,version 0.0.4

2018/09/27 17:12:58main.go:40:HLS Listen On:7002

2018/09/27 17:12:58main.go:58:RTMP Listen On:1935

2018/09/27 17:12:58main.go:75:HTTP-FLV Listen On:7001

in the above example, a live machine rental platform executes a live machine executable file for live machine server installation, starts 0.0.4 version live machine service end, and listens to HLS, RTMP and HTTP-FLV protocol transmission at ports 7002, 1935 and 7001, respectively.

In the embodiment, the live server is adopted to realize RTMP stream pushing and stream pulling to complete the playing and displaying of the video stream, so that the performance of the RTMP stream server for processing multi-process concurrency is improved, and a better display effect and better operation experience are provided for a user during stream pulling playing.

In an optional implementation manner of the embodiment of the present invention, the data format of the video stream data of the screen of the mobile device acquired by the stream pushing end 110 may be h.264.

H.264, also part ten of MPEG-4, is a digital Video coding standard developed by Joint Video Team (JVT) of VCEG (Video coding experts group) of ITU-T and MPEG (moving picture coding experts group) of ISO/IEC, and has the advantages of low code rate, high image quality, strong fault-tolerant capability and strong network adaptability. Because H.264 has a high data compression ratio, video data compressed by the H.264 standard requires less bandwidth in the network transmission process and is more economical.

In an optional implementation manner of the embodiment of the present invention, the stream pushing end 110 may further establish communication with the mobile device by executing an adb forward command, so as to acquire video stream data of the own screen captured by the mobile device from the mobile device.

Adb (Android Debug Bridge ) plays the effect of connecting the Bridge, can couple together PC and equipment, and then can manage the state of equipment or cell-phone simulator with the help of the Adb instrument. By establishing one forwarding through adb forward, all data sent by the device side server program can be received at the PC.

Further, the video stream data of the own screen captured by the mobile device mentioned above may be captured by the mobile device by running a screen copy tool on the mobile device itself and calling the MediaCodec interface. The screen copy tool mentioned herein may include a scrcpy tool and the like, and the present invention is not particularly limited.

MediaCodec is an API of Android itself, and MediaCodec class can access the underlying media codec, can encode/decode media, and is part of the Android low-level multimedia support infrastructure. Through the MediaCodec interface, the original video data collected by the screen copy tool can be encoded into video stream data in h.264 format.

In one embodiment, the mobile device and the push streaming peer 110 may be connected via USB or Wifi. The stream pushing end 110 executes the adb forward command, and establishes communication between the socket on the stream pushing end 110 and the screen copying tool process of the mobile device end, so that the corresponding screen data acquisition request is sent to the mobile device end, and video stream data of the screen of the mobile device end is received.

In an optional implementation manner of the embodiment of the present invention, the stream pushing end 110 may further convert the obtained video stream data into an RTMP format by calling the C library srs instance srs _ h264_ raw _ publishing.c, and push-stream the converted video stream data to the RTMP stream server 120.

srs locate clusters of internet live servers that are carrier grade, seeking better conceptual integrity and simplest implementation code. srs-librtmp provides a series of interface functions to send data to a server in a certain format or to read audio and video data from the server. The srs-librtmp supports release of H.264 bare code streams, directly calls an API (application programming interface), can send data to srs, and then releases the data to the RTMP server through the RTMP protocol. Specifically, srs provides an instance srs _ h264_ raw _ publishing.c to read and distribute recorded h.264 files to the RTMP server, see specificallyhttps:// github.com/winlinvip/simple-rtmp-server/tree/develop/trunk/research/librtmp/ srs_h264_raw_publish.c。

Further, the stream pushing terminal 110 may also convert the acquired video stream data in h.264 format into RTMP format by calling the read _ h264_ frame function, and push stream the converted video stream data to the RTMP stream server 120 by calling srs _ h264_ write _ raw _ frames function.

srs _ h264_ raw _ publishing.c reads H.264 bare stream (video stream data), and basically one packet in H.264annexb format is read. When converting a naked code stream of h.264 into an RTMP, a dts (Decode TimeStamp) and a pts (Presentation TimeStamp) are externally given, an SPS (Sequence Parameter Set) and a PPS (Picture Parameter Set) are sent in an RTMP packet, and the RTMP packet is 5 bytes RTMP header + h.264 data. Then, the video stream data is converted and pushed through srs _ h264_ write _ raw _ frames function for distribution.

In an optional implementation manner of the embodiment of the present invention, the pull streaming client 130 may adopt a playback protocol of any one of the following: RTMP, HTTP-FLV, HLS.

Based on the property that the livego server can support various common transmission protocols, the pull stream client 130 can adopt any one of the RTMP, HTTP-FLV and HLS playing protocols according to the actual playing requirement. For example, for a live broadcast occasion with high real-time requirement, an RTMP playing protocol can be adopted, and for an occasion without real-time requirement and with large distribution amount (such as on-demand), an HTTP-FLV or HLS playing protocol can be adopted.

The respective playing addresses of the RTMP, HTTP-FLV and HLS playing protocols are listed as follows:

·RTMP:rtmp://localhost:1935/live/movie

·FLV:http://127.0.0.1:7001/live/movie.flv

·HLS:http://127.0.0.1:7002/live/movie.m3u8

in an alternative implementation of the embodiment of the present invention, the pull stream client 130 may include a VLC (VideoLanClient) Media Player and/or a livelayer component.

VLC Media Player is a piece of Media Player that can play most formats without installing codec packs. It can play MPEG-1, MPEG-2, MPEG-4, DivX, DVD/VCD, Digital satellite Channels, Digital Terrestrial Television Channels (Digital Terrestrial Television Channels), and on-line movies under many operating platforms via broadband IPv4, IPv6 networks. The VLC Media Player can also support playing some video file part contents without downloading completion, and is therefore very suitable for the occasion of video stream playing.

The Livelayer component is a front-end Liverlayer component, and is embedded into clients such as a browser front-end web page and a WeChat applet to achieve pulling and playing of video stream data.

An example of code for the pull streaming client 130 to perform RTMP pull streaming is shown below:

the foregoing describes various implementations of various links of the embodiment shown in fig. 1, and the following describes in detail the architecture and display effect of the video stream playing system based on the RTMP screen data according to a specific embodiment.

Fig. 2 is a schematic diagram illustrating an architecture of a video stream playing system based on RTMP screen data according to another embodiment of the present invention. Referring to fig. 2, the system mainly includes: a push streaming client 210, an RTMP streaming server 220, and a pull streaming client 230. Wherein the push flow end 210 is further connected with the mobile device 240.

The system architecture and functions of each part of the video stream playing system based on the RTMP screen data according to the embodiment of the present invention are described with reference to fig. 2.

In the video stream playing system provided in this embodiment, the stream pushing end 210 is a personal computer with data transmission and data analysis functions. The RTMP streaming server 220 is a livego server, which is constructed by running a livego executable file in a windows environment and starting a livego server at the server. The pull stream client 230 is a display equipped with a browser, wherein a liveplayer component is embedded in a front web page of the browser, and an RTMP playing protocol is adopted to realize the pull and play of video stream data. The mobile device 240 is a mobile phone end with an Android system, and is connected to the stream pushing end 210 through a USB or Wifi.

Referring to fig. 2, the stream pushing terminal 210 establishes communication with the mobile device 240 by executing an adb forward command, so as to send a corresponding screen data acquisition request to the mobile device 240. According to the request, the mobile device 240 runs the screen copy tool on itself and calls the MediaCodec interface to collect screen data, so as to obtain video stream data of the h.264 format screen, and sends the video stream data of the h.264 format screen to the stream pushing terminal 210 through the adb connection. The push streaming end 210 receives the video stream data of the h.264 format screen, and completes the acquisition of the video stream data of the screen of the mobile device 240.

After the push streaming end 210 acquires the video stream data of the h.264 format screen, it calls the C library srs instance srs _ h264_ raw _ publish.c, converts the acquired video stream data of the h.264 format into the RTMP format through the read _ h264_ frame function, and pushes the converted video stream data to the RTMP streaming server 220 through the srs _ h264_ write _ raw _ frames function. Further, the RTMP streaming server (livego server) 220 receives the video stream data pushed by the push streaming end 210.

The pull streaming client 230 transmits a play request for the video streaming data to the RTMP streaming server (livego server) 220 according to the operation of the user. The RTMP streaming server (livego server) 220 monitors a playback request of the RTMP protocol, and then responds to the playback request. Further, the streaming client 230 pulls the video stream data from the RTMP streaming server (livego server) 220 to play after receiving a response to the play request from the RTMP streaming server (livego server) 220.

The embodiment of the invention realizes the real-time playing of the screen data of the mobile phone end at the cloud end by a simple structure, and solves the problems of low server performance, high display delay and poor expansibility in the prior art.

Fig. 3 and 4 respectively show an image display effect of a mobile phone screen based on a minicap tool and a pull stream display effect of a mobile phone screen of a video stream playing system based on RTMP screen data according to an embodiment of the present invention. As can be seen from comparing fig. 3 and fig. 4, the video stream playing system based on the RTMP screen data provided by the embodiment of the present invention can provide a high-definition screen display effect as in the minisap under the premise of low latency. Meanwhile, the pull stream client can also provide operation keys such as exit and return and a selection key with definition, so that the operation of a user is facilitated, and the use experience of the user is improved.

Based on the same inventive concept, the embodiment of the invention also provides a video stream playing method based on the screen data of the RTMP. Fig. 5 is a flowchart illustrating a video stream playing method applied to the RTMP-based screen data of the push stream terminal according to an embodiment of the present invention. Referring to fig. 1, the method may include at least the steps of:

step S502, video stream data of the screen of the mobile device is acquired.

Step S504, converting the acquired video stream data into an RTMP format, and pushing the converted video stream data to an RTMP stream server, so that the stream pulling client pulls the video stream data from the RTMP stream server for playing.

The embodiment of the invention realizes the real-time playing of the screen data of the mobile equipment at the cloud end, and is suitable for displaying the scene of the screen information of the mobile equipment when the mobile test is carried out through the remote real machine renting platform.

In a practical application scenario, the mobile device mentioned above may include a mobile phone, a tablet computer, a smart watch, and other smart mobile terminals.

In an alternative embodiment, the data format of the acquired video stream data of the screen of the mobile device may be h.264. Because h.264 has a high data compression ratio and image quality, the bandwidth required for network transmission can be reduced by using video data compressed by the h.264 standard.

In an alternative embodiment, step S502 may be implemented as:

and establishing communication with the mobile device by executing an adb forward command so as to acquire the video stream data of the self screen acquired by the mobile device from the mobile device.

All data sent by a device side server program can be received at a stream pushing side (such as a PC) through adb forward establishing a forwarding.

Further, the video stream data of the self screen collected by the mobile device is obtained by the mobile device by running a screen copy tool on the self and calling a MediaCodec interface to collect. The screen copy tool mentioned here may be, for example, a scrcpy tool or the like.

In an alternative embodiment, step S504 may be implemented as:

the obtained video stream data is converted into the RTMP format by calling the C library srs instance srs _ h264_ raw _ publish.c, and the converted video stream data is streamed to the RTMP streaming server.

srs orientation is operation levelThe Internet live broadcast server cluster can read the recorded H.264 file by simpler codes and release the file to the RTMP server, and the details can be seen inhttps://github.com/ossrs/srs。

Further, the obtained video stream data in h.264 format may be converted into RTMP format by calling the read _ h264_ frame function in srs _ h264_ raw _ publish.c, and the converted video stream data is streamed to the RTMP stream server by calling the srs _ h264_ write _ raw _ frames function in srs _ h264_ raw _ publish.c.

Accordingly, fig. 6 shows a flowchart of a video stream playing method based on RTMP screen data applied to an RTMP streaming server according to an embodiment of the present invention. Referring to fig. 6, the method may include at least the steps of:

step S602, receiving video stream data of the screen of the mobile device converted into the RTMP format, to which the push stream terminal pushes stream.

Step S604, responding to the playing request of the pull streaming client to the video streaming data, so that the pull streaming client pulls the video streaming data from the RTMP streaming server for playing.

In a preferred embodiment, the RTMP streaming server comprises a livego server. At this time, before performing step S602, the method may further include a step of starting the livego server.

The Livego server is an RTMP server developed based on golang and is provided with functions of supporting RTMP push flow and pull flow; statistical information support; and http online viewing of the streaming state is supported. The live server is used as the RTMP streaming server, so that the performance of the RTMP streaming server can be greatly improved.

Before screen display application is launched, a live server is installed, for example, the live server can be installed at a cloud server end and serves as an RTMP streaming server.

After the live server is installed, before receiving video stream data of a screen of the mobile device converted into an RTMP format from a push streaming end, the live server needs to be started in the RTMP streaming server (i.e., the live server) to provide corresponding services of RTMP push streaming and pull streaming. In particular, for listening to HLS, RTMP and HTTP-FLV protocol transmissions, respectively.

Accordingly, fig. 7 shows a flowchart of a video stream playing method for RTMP-based screen data applied to a pull streaming client according to an embodiment of the present invention. Referring to fig. 7, the method may include at least the steps of:

step S702, sending a play request for the video stream data of the screen of the mobile device pushed to the RTMP streaming server.

Step S704, after receiving the response from the RTMP streaming server to the playing request, pulls the video streaming data from the RTMP streaming server for playing.

In an alternative embodiment, when the RTMP streaming server is a livego server, based on the characteristics that the livego server can support multiple common transport protocols, the protocol for pulling the video streaming data from the RTMP streaming server to play may include any one of the following: RTMP, HTTP-FLV, HLS. The respective playing addresses of the RTMP, HTTP-FLV and HLS playing protocols are as follows:

·RTMP:rtmp://localhost:1935/live/movie

·FLV:http://127.0.0.1:7001/live/movie.flv

·HLS:http://127.0.0.1:7002/live/movie.m3u8

in an alternative embodiment, the pull client may include a VLC media player and/or a livelayer component.

The VLC media player can support to play some video file partial contents which are not downloaded completely, and is very suitable for the occasion of video stream playing. The Livelayer component is a front-end Liverlayer component, and can be embedded into clients such as a browser front-end web page and a WeChat applet to pull and play video stream data.

The following describes in detail the implementation process of the video stream playing method based on the RTMP screen data according to the present invention by using a specific embodiment. Fig. 8 is a flowchart illustrating a video stream playing method of RTMP-based screen data in combination with a push streaming client, an RTMP streaming server and a pull streaming client according to an embodiment of the present invention. In this embodiment, the stream side is a PC. The method is realized by adopting a livego server as an RTMP stream server, installing a live broadcast server as the RTMP stream server before the RTMP stream server is put into service, and starting a livego server. The pull stream client adopts a display loaded with a VLC media player.

The method may include at least the following steps S802 to S812.

Step S802, the stream pushing end establishes communication with the mobile equipment by executing an adb forward command, and acquires video stream data of an H.264 format of a self screen acquired by the mobile equipment from the mobile equipment.

In this embodiment, the mobile device is specifically a mobile phone equipped with an Android system. The mobile phone acquires the video stream data of the H.264 format of the screen of the mobile phone by running a screen copying tool on the mobile phone and calling a MediaCodec interface.

In step S804, the stream pushing end calls srs _ h264_ raw _ publish.c read _ h264_ frame function to convert the acquired h.264 format video stream data into RTMP format, and further calls srs _ h264_ write _ raw _ frames function to stream the converted video stream data to the RTMP stream server.

In step S806, the RTMP streaming server receives the video stream data pushed by the push streaming end.

Step S808, the stream pulling client sends a play request for the video stream data to the RTMP stream server.

Step S810, after receiving the play request, the RTMP streaming server sends a response signal to the play request to the streaming client.

In step S812, after receiving the response signal, the stream pulling client pulls the video stream data from the RTMP stream server, and plays the video stream data through the VLC media player.

The video stream playing method based on the RTMP screen data provided by the embodiment of the invention realizes the real-time playing of the screen data of the mobile equipment at the cloud, and has the advantages of low delay and long-time playing.

Based on the same inventive concept, the embodiment of the present invention further provides a stream pushing end, which is used to support the video stream playing method of the screen data based on the RTMP provided in any one of the above embodiments or a combination thereof. Fig. 9 shows a schematic structure diagram of the push flow end 910 according to an embodiment of the present invention. Referring to fig. 9, the plug end 910 may include at least: a data acquisition module 911 and a conversion plug-flow module 912.

The functions of the components or devices of the stream pushing end 910 and the connection relationship between the components will now be described:

a data acquisition module 911 adapted to acquire video stream data of a screen of the mobile device.

And the conversion plug-flow module 912 is connected with the data acquisition module 911 and is adapted to convert the acquired video stream data into an RTMP format and plug-flow the converted video stream data to an RTMP stream server, so that the stream pulling client pulls the video stream data from the RTMP stream server for playing.

In an alternative embodiment, the plug-flow end 910 may be applied to a scenario where screen information of a mobile device is displayed when a mobile test is performed through a remote live rental platform.

In an alternative embodiment, the data format of the video stream data of the screen of the mobile device acquired by the data acquisition module 911 is h.264.

In an alternative embodiment, the data acquisition module 911 is further adapted to:

Further, the video stream data of the self screen collected by the mobile device is obtained by the mobile device by running a screen copy tool on the self and calling a MediaCodec interface to collect.

In an alternative embodiment, the transition plug flow module 912 is further adapted to:

Further, the transition plug flow module 912 is further adapted to:

the converted video stream data is streamed to the RTMP streaming server by calling srs _ h264_ write _ raw _ frames function.

Correspondingly, the embodiment of the invention also provides an RTMP streaming server. Fig. 10 is a schematic structural diagram of an RTMP streaming server 920 according to an embodiment of the present invention. Referring to fig. 10, the RTMP streaming server 920 may include at least: a push flow receiving module 921 and a pull flow response module 922.

Now, the functions of the components or devices of the RTMP streaming server 920 and the connection relationship between the components are described:

the push stream receiving module 921 is adapted to receive video stream data of a screen of the mobile device converted into the RTMP format, to which the push stream terminal pushes stream.

The pull stream response module 922 is adapted to respond to a request of the pull stream client for playing the video stream data, so that the pull stream client pulls the video stream data from the RTMP stream server for playing.

In an alternative embodiment, the RTMP streaming server 920 may be applied to a scenario where screen information of the mobile device is displayed when a mobile test is performed through a remote live rental platform.

In an alternative embodiment, the RTMP streaming server 920 may be a livego server. At this time, the RTMP streaming server 920 may further include:

Correspondingly, the embodiment of the invention also provides a pull streaming client. Fig. 11 is a schematic structural diagram of a pull streaming client 930 according to an embodiment of the present invention. Referring to fig. 11, the pull client 930 may include at least: a request sending module 931 and a pull stream playing module 932.

The functions of the components or devices of the pull client 930 and the connection relationship between the components will now be described:

a request sending module 931 adapted to send a play request of the video stream data pushed to the screen of the mobile device of the RTMP streaming server to the RTMP streaming server.

The pull stream playing module 932 is adapted to pull the video stream data from the RTMP streaming server for playing after receiving the response from the RTMP streaming server to the playing request.

In an alternative embodiment, the pull streaming client 930 may be applied to a scenario where screen information of the mobile device is displayed when a mobile test is performed through a remote live rental platform.

In an alternative embodiment, when the RTMP streaming server is a livego server, the playback protocol of the pull streaming client 930 includes any one of the following:

RTMP、HTTP-FLV、HLS。

in an alternative embodiment, the pull stream play module 932 may include a VLC media player and/or a livelayer component.

Based on the same inventive concept, the embodiment of the invention also provides a computer storage medium. The computer storage medium stores computer program code which, when run on a computing device, causes the computing device to perform a video stream playing method applied to the RTMP-based screen data of the push streaming end, a video stream playing method applied to the RTMP-based screen data of the RTMP streaming server, or a video stream playing method applied to the RTMP-based screen data of the pull streaming client according to any one or a combination of the above embodiments.

Based on the same inventive concept, the embodiment of the invention also provides the computing equipment. The computing device may include:

a processor; and

a memory storing computer program code;

when executed by a processor, the computer program code causes the computing device to perform a video stream playing method applied to the RTMP-based screen data of the push streaming end, a video stream playing method applied to the RTMP-based screen data of the RTMP streaming server, or a video stream playing method applied to the RTMP-based screen data of the pull streaming client according to any one or a combination of the above embodiments.

According to any one or a combination of multiple optional embodiments, the embodiment of the present invention can achieve the following advantages:

It is clear to those skilled in the art that the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and for the sake of brevity, further description is omitted here.

In addition, the functional units in the embodiments of the present invention may be physically independent of each other, two or more functional units may be integrated together, or all the functional units may be integrated in one processing unit. The integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.

Those of ordinary skill in the art will understand that: the integrated functional units, if implemented in software and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computing device (e.g., a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention when the instructions are executed. And the aforementioned storage medium includes: u disk, removable hard disk, Read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disk, and other various media capable of storing program code.

Alternatively, all or part of the steps of implementing the foregoing method embodiments may be implemented by hardware (such as a computing device, e.g., a personal computer, a server, or a network device) associated with program instructions, which may be stored in a computer-readable storage medium, and when the program instructions are executed by a processor of the computing device, the computing device executes all or part of the steps of the method according to the embodiments of the present invention.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments can be modified or some or all of the technical features can be equivalently replaced within the spirit and principle of the present invention; such modifications or substitutions do not depart from the scope of the present invention.

According to an aspect of the embodiments of the present invention, a video stream playing system for screen data based on a real-time messaging protocol RTMP is provided, including:

A2. The video stream playing system according to a1, wherein the system is applied to a scene displaying screen information of the mobile device when a mobile test is performed through a remote real machine rental platform.

A3. The video stream playing system according to a1, wherein the RTMP streaming server includes a livego server;

the RTMP streaming server is further configured to:

A4. The video stream playing system according to a3, wherein the data format of the video stream data of the acquired screen of the mobile device is h.264.

A5. The video stream playing system according to a4, wherein the stream pushing end is further configured to:

A6. The video stream playing system according to a5, wherein the video stream data of the own screen captured by the mobile device is captured by the mobile device by running a screen copy tool on the mobile device and invoking a MediaCodec interface.

A7. The video stream playing system according to a4, wherein the stream pushing end is further configured to:

A8. The video stream playing system according to a7, wherein the stream pushing end is further configured to:

A9. The video stream playing system according to a3, wherein the playing protocol of the pull streaming client includes any one of the following:

RTMP、HTTP-FLV、HLS。

A10. the video stream playing system according to a3, wherein the pull stream client includes a VLC media player and/or a livelayer component.

According to another aspect of the embodiments of the present invention, there is also provided a method for playing a video stream based on screen data of an RTMP, applied to a stream push terminal, including:

acquiring video stream data of a screen of a mobile device;

B12. The method of B11, wherein the method is applied to a scenario where screen information of the mobile device is displayed when a mobile test is performed through a remote live rental platform.

B13. The method according to B11, wherein the data format of the video stream data of the acquired screen of the mobile device is h.264.

B14. The method of B13, wherein obtaining video stream data for a screen of a mobile device comprises:

B15. The method according to B14, wherein the video stream data of the own screen captured by the mobile device is captured by the mobile device by running a screen copy tool on itself and invoking MediaCodec interface.

B16. The method of B13, wherein the obtained video stream data is converted into RTMP format by calling C library srs instance srs _ h264_ raw _ publishing.c, and the converted video stream data is streamed to the RTMP stream server.

B17. The method of B16, wherein converting the obtained video stream data into the RTMP format by calling srs _ h264_ raw _ publishing.c instance of C library srs and streaming the converted video stream data to the RTMP streaming server, further comprises:

According to another aspect of the embodiments of the present invention, there is also provided a method for playing a video stream based on screen data of an RTMP, which is applied to an RTMP streaming server, and includes:

C19. The method of C18, wherein the method is applied to a scenario where screen information of the mobile device is displayed when a mobile test is performed through a remote live rental platform.

C20. The method of C18, wherein the RTMP streaming server comprises a livego server;

and starting the livego server.

According to another aspect of the embodiments of the present invention, there is also provided a video stream playing method based on screen data of RTMP, applied to a pull stream client, including:

D22. The method of D21, wherein the method is applied to a scenario where screen information of the mobile device is displayed when a mobile test is conducted through a remote live rental platform.

D23. The method according to D21, wherein, when the RTMP streaming server is a livego server, the protocol for pulling the video streaming data from the RTMP streaming server to play includes any one of:

RTMP、HTTP-FLV、HLS。

D24. the method of D21, wherein the pull client includes a VLC media player and/or a livelayer component.

According to another aspect of the embodiments of the present invention, there is also provided a plug flow end, including:

E26. The push stream peer according to E25, wherein the push stream peer is applied to a scenario displaying screen information of the mobile device when performing a mobile test through a remote live rental platform.

E27. The push streaming end according to E25, wherein the data format of the acquired video stream data of the screen of the mobile device is h.264.

E28. The push stream terminal of E27, wherein the data acquisition module is further adapted to:

E29. The push streaming terminal according to E28, wherein the video stream data of the own screen collected by the mobile device is obtained by the mobile device by running a screen copy tool on the mobile device and invoking a MediaCodec interface.

E30. The push flow end of E27, wherein the conversion push flow module is further adapted to:

E31. The push flow end of E30, wherein the conversion push flow module is further adapted to:

According to another aspect of the embodiments of the present invention, there is also provided f32. an RTMP streaming server, including:

F33. The RTMP streaming server according to F32, wherein the RTMP streaming server is applied to a scene in which screen information of the mobile device is displayed when a mobile test is performed through a remote live rental platform.

F34. The RTMP streaming server according to F32, wherein the RTMP streaming server is a livego server;

the RTMP streaming server further comprises:

According to another aspect of the embodiments of the present invention, there is also provided a stream pulling client, including:

G36. The pull client according to G35, wherein the pull client is applied to a scenario displaying screen information of the mobile device when a mobile test is performed through a remote live rental platform.

G37. The pull streaming client according to G35, wherein, when the RTMP streaming server is a livego server, a playing protocol of the pull streaming client includes any one of:

RTMP、HTTP-FLV、HLS。

G38. the pull client according to G35, wherein the pull playing module comprises a VLC media player and/or a livelayer component.

There is also provided, in accordance with yet another aspect of an embodiment of the present invention, a computer storage medium having stored thereon computer program code which, when run on a computing device, causes the computing device to execute a video stream playing method applied to RTMP-based screen data of a push streaming end in accordance with any one of B11-B17, a video stream playing method applied to RTMP-based screen data of an RTMP streaming server in accordance with any one of C18-C20, or a video stream playing method applied to RTMP-based screen data of a pull streaming client in accordance with any one of D21-D24.

There is also provided, in accordance with yet another aspect of an embodiment of the present invention, apparatus for computing, including:

a processor; and

a memory storing computer program code;

when executed by the processor, the computer program code causes the computing device to perform a video stream playing method applied to RTMP based screen data of a push streaming end according to any of B11-B17, a video stream playing method applied to RTMP based screen data of an RTMP streaming server according to any of C18-C20, or a video stream playing method applied to RTMP based screen data of a pull streaming client according to any of D21-D24.

Claims

1. A video stream playing system of screen data based on real time messaging protocol RTMP, comprising:

2. The video stream playing system according to claim 1, wherein the system is applied to a scene displaying screen information of the mobile device when a mobile test is performed through a remote live rental platform.

3. A video stream playing method based on screen data of RTMP is applied to a stream pushing end and comprises the following steps:

acquiring video stream data of a screen of a mobile device;

4. A video stream playing method based on screen data of RTMP is applied to an RTMP stream server and comprises the following steps:

5. A video stream playing method based on screen data of RTMP is applied to a pull stream client and comprises the following steps:

6. A push tip, comprising:

7. An RTMP streaming server, comprising:

8. A pull client, comprising:

9. A computer storage medium storing computer program code which, when run on a computing device, causes the computing device to execute the video stream playing method applied to the RTMP-based screen data of the push streaming end according to claim 3, the video stream playing method applied to the RTMP-based screen data of the RTMP streaming server according to claim 4 or the video stream playing method applied to the RTMP-based screen data of the pull streaming client according to claim 5.

10. A computing device, comprising:

a processor; and

a memory storing computer program code;

when the computer program code is run by the processor, causing the computing device to execute the video stream playing method applied to the RTMP based screen data of the push streaming end according to claim 3, the video stream playing method applied to the RTMP based screen data of the RTMP streaming server according to claim 4, or the video stream playing method applied to the RTMP based screen data of the pull streaming client according to claim 5.