CN115714874A

CN115714874A - Video stream screen display method, device and system, stream pulling equipment and storage medium

Info

Publication number: CN115714874A
Application number: CN202211348480.1A
Authority: CN
Inventors: 戴福昊; 李勇; 李君君
Original assignee: TONGHUI JIASHI (BEIJING) INFORMATION TECHNOLOGY CO LTD
Current assignee: TONGHUI JIASHI (BEIJING) INFORMATION TECHNOLOGY CO LTD
Priority date: 2022-10-31
Filing date: 2022-10-31
Publication date: 2023-02-24

Abstract

The embodiment of the invention discloses a VIDEO stream screen display method, a VIDEO stream screen display device, a VIDEO stream screen display system, a stream pulling device and a storage medium, wherein the stream pulling device acquires a VIDEO stream pushed by a stream pushing client from a stream media server by using a WebRTC protocol in a local browser, converts the VIDEO stream into a WebRTC VIDEO stream, and then performs screen-splitting real-time playing on the WebRTC VIDEO stream by using a plurality of VIDEO tags in HTML 5. The technical scheme of the invention is based on the ultra-low delay characteristic of the WebRTC protocol, combines the ultra-low delay characteristic with the traditional push-pull streaming media system, and realizes the technical purpose of playing the video stream with multiple screens within the delay of 35-70 ms in the scene of needing to transfer the push-pull video stream in the streaming media server. Therefore, time delay is reduced, and user experience in real-time multi-screen video playing is improved.

Description

Video stream screen display method, device and system, stream pulling equipment and storage medium

Technical Field

The present invention relates to the field of streaming media and network transmission technologies, and in particular, to a method, an apparatus, a system, a stream pulling device, and a storage medium for displaying a video stream on a screen.

Background

In recent years, with the advancement of technology, after the era of File (File) and page (Web), the era of "Flows/Streams" is now coming, and the rhythm of life of the public is becoming faster and faster among various information Streams, data Streams, and media Streams.

The most typical application for the initial transition to "Streaming" is Streaming media (Streaming media) technology, which refers to the distribution of audio, video, and other multimedia files over networks using Streaming technology. The streaming transmission is a process of encoding and compressing audio and video multimedia files into small compressed packets through a certain algorithm, continuously transmitting the compressed packets in real time through a specific network protocol by a streaming media server, and decompressing the compressed packets in real time by playing software after a user side receives the compressed packets to realize playing.

There are many application scenarios in the streaming media technology field, including: video interaction, remote education and medical treatment, live broadcast delivery, monitoring and the like. In some scenarios, for example: in the technical scheme of implementing the function in the prior art, after a device at a Streaming end (such as a monitoring camera, a mobile device, and the like) pushes acquired Video data to a Streaming Media server, one of the processing modes is that the Streaming Media server is required to convert a Video stream into a Video stream in a protocol format such as HLS (HTTP Live Streaming, i.e., adaptive code rate Streaming Media transmission protocol based on HTTP), MSE (Media Source Extensions, i.e., media Source Extensions), or FLV (Flash Video) that can be directly played in a web browser, and the web end then pulls and plays the converted Video stream, which has the following problems:

1. for the HLS protocol, it is a streaming media network transmission protocol proposed by Apple inc, and is mainly used for audio and video services of PCs and Apple terminals. The HLS does not generally encounter the condition of being shielded by a firewall because the HTTP protocol used for transmitting data is used, in addition, the HLS protocol realizes code rate self-adaption, and equipment with different bandwidths can be automatically switched to video playing which is most suitable for the code rate of the equipment. However, the HLS is played in segments in the form of video slices, and includes an index file of M3U8, a TS media slice file, and a key encryption string file. Due to the characteristic of fragment loading, the delay of playing a video is relatively long, for example, an M3U8 has 5 TS files, and the playing time of each TS file is 2 seconds, then the playing time of an M3U8 file is 10 seconds, that is, the progress of playing the M3U8 is at least 10 seconds ago, although apple corporation issues a new solution at WWDC 2019, which can reduce the delay to 1 to 2 seconds, this is still a big disadvantage for scenes with high real-time requirements.

2. For MSE, MSE is a new Web API supported by mainstream browsers, and provides functions for realizing plug-in-free and Web-based streaming media. However, if some browsers do not support MSE, creating compatible MSE in these browsers is time consuming and labor intensive, and also consumes a lot of computer resources and energy. In addition, the time delay is about 3 seconds when the video is played, and the time delay is slightly larger.

3. For the FLV protocol, the FLV is a streaming media format, which is widely supported due to its small size and relatively simple protocol, and has the advantages of high maturity, high concurrency and no pressure, but has the disadvantages of requiring the integration of SDK to play, mainly affected by GOP size and TCP weak network transmission backlog, and high playing delay, generally between 2 seconds and 3 seconds.

Therefore, although these protocol methods have their advantages, the problem of high video playing delay inevitably exists, and the high delay causes a series of problems caused by untimely video frame update, which directly reduces the user experience.

Based on the problems in the prior art when the video is live broadcast on the webpage side, it is urgently needed to provide a video stream split-screen display method capable of reducing the time delay so as to improve the user experience when the video is played in a real-time multi-screen mode.

Disclosure of Invention

In view of this, embodiments of the present invention provide a method, an apparatus, a device, a system, and a storage medium for displaying a VIDEO stream, where a stream pulling device obtains a VIDEO stream pushed by a stream pushing client on a streaming media server by using a WebRTC protocol in a local browser, converts the VIDEO stream into a WebRTC VIDEO stream, and then performs split-screen real-time playing on the WebRTC VIDEO stream by using a plurality of VIDEO tags in HTML 5. The technical scheme of the invention is based on the ultra-low delay characteristic of the WebRTC protocol, combines the ultra-low delay characteristic with the traditional push-pull streaming media system, and realizes the technical purpose of playing the video stream with multiple screens within the delay of 35-70 ms in the scene of needing to transfer the push-pull video stream in the streaming media server. Therefore, time delay is reduced, and user experience in real-time multi-screen video playing is improved.

In a first aspect, an embodiment of the present invention provides a video stream screen display method, which is applied to a stream pulling device, where the stream pulling device is installed with a browser supporting a WebRTC protocol for web real-time communication, and the method includes:

performing pull flow configuration locally in advance to generate a flow list; the stream list comprises one or more stream list items, and each stream list item comprises a pull stream address of a video stream corresponding to the stream list item in a stream media server;

responding to a trigger instruction for playing a video stream corresponding to a streaming list item in the streaming list, acquiring a pull stream address of the video stream corresponding to the streaming list item in a streaming media server, and pulling the video stream from the streaming media server based on the pull stream address of the video stream corresponding to the streaming list item in the streaming media server, wherein the video stream is an original RTSP video stream sent to the streaming media server by a stream pushing client or an RTSP video stream transcoded by the streaming media server;

decoding and unpacking the video stream to generate a WebRTC video stream conforming to a WebRTC protocol;

and performing split-screen display on the WebRTC Video stream in a split-screen interface generated according to a preset split-screen format and based on a Video tag of a local browser HTML 5.

Preferably, the performing pull flow configuration locally in advance to generate the flow list specifically includes:

acquiring stream configuration data for each video stream input by a user, and generating corresponding stream list items;

generating the stream list based on stream list items of each video stream;

wherein the flow configuration data comprises: video stream name, pull stream address.

Preferably, the flow configuration data further comprises: a stream play type; the stream play type includes: and continuously playing and clicking to play.

Preferably, the pulling the video stream from the streaming media server based on the stream pulling address of the video stream corresponding to the stream list item in the streaming media server specifically includes:

and establishing a WebRTC connection channel with the streaming media server based on a WebRTC protocol interface in the local browser, and pulling the video stream through the WebRTC connection channel.

Preferably, the performing screen splitting display on the WebRTC Video stream in a screen splitting interface generated according to a preset screen splitting format and based on a Video tag of a local browser HTML5 specifically includes:

and based on a Video tag of a local browser HTML5, dividing the split screen interface into a plurality of independent display windows according to a preset split screen format, wherein each display window corresponds to one Video and is used for playing a WebRTC Video stream.

Preferably, each display window includes an area responding to the addition and playing of the video stream, and is used for receiving a trigger instruction of a user for adding and playing the video stream corresponding to the stream list item in the stream list in the display window.

In a second aspect, an embodiment of the present invention further provides a video stream split-screen display apparatus, which is disposed in a streaming device, where the streaming device is installed with a browser supporting a WebRTC protocol for web real-time communication, and the apparatus includes:

the flow list generation module is set to carry out pull flow configuration locally in advance to generate a flow list; the stream list comprises one or more stream list items, and each stream list item comprises a pull stream address of a video stream corresponding to the stream list item in a stream media server;

the stream pulling module is configured to respond to a trigger instruction for playing a video stream corresponding to a stream list item in the stream list, acquire a stream pulling address of the video stream corresponding to the stream list item in a stream media server, and pull the video stream from the stream media server based on the stream pulling address of the video stream corresponding to the stream list item in the stream media server, wherein the RTSP video stream is an original RTSP video stream sent to the stream media server by a stream pushing client or an RTSP video stream transcoded by the stream media server;

the stream decoding module is used for decoding and unpacking the video stream to generate a WebRTC video stream conforming to a WebRTC protocol;

and the stream screen display module is used for displaying the WebRTC Video stream in a screen splitting interface generated according to a preset screen splitting format and based on a Video tag of a local browser HTML5 in a screen splitting manner.

In a third aspect, an embodiment of the present invention provides a stream pulling apparatus, including a memory and a processor, where the processor executes program instructions in the memory, so as to implement the method in the first aspect.

In a fourth aspect, an embodiment of the present invention provides a video stream split-screen display system, including: a push stream client, a stream media server and the pull stream device of the third aspect.

In a fifth aspect, an embodiment of the present invention provides a storage medium, where the storage medium is used to store a computer program, and the computer program is used to implement the method described in the first aspect.

Drawings

The above and other objects, features and advantages of the present invention will become more apparent from the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

FIG. 1 is a flow chart of a video stream split-screen display method according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a video stream split-screen display device according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a hardware structure of a stream pulling apparatus according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a video stream split-screen display system according to an embodiment of the present invention.

Detailed Description

The present invention will be described below based on examples, but the present invention is not limited to only these examples. In the following detailed description of the present invention, certain specific details are set forth. It will be apparent to one skilled in the art that the present invention may be practiced without these specific details. Well-known methods, procedures, components and circuits have not been described in detail so as not to obscure the present invention.

Further, those of ordinary skill in the art will appreciate that the drawings provided herein are for illustrative purposes and are not necessarily drawn to scale.

Meanwhile, it should be understood that, in the following description, a "circuit" refers to a conductive loop constituted by at least one element or sub-circuit through electrical or electromagnetic connection. When an element or circuit is referred to as being "connected to" another element or element/circuit is referred to as being "connected between" two nodes, it may be directly coupled or connected to the other element or intervening elements may be present, and the connection between the elements may be physical, logical, or a combination thereof. In contrast, when an element is referred to as being "directly coupled" or "directly connected" to another element, it is intended that there are no intervening elements present.

Unless the context clearly requires otherwise, throughout the description, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is, what is meant is "including, but not limited to".

In the description of the present invention, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present invention, "a plurality" means two or more unless otherwise specified.

The technical scheme adopted by the invention is that a Browser/Server (Browser/Server, B/S) architecture is adopted for deployment and is divided into a stream pulling device, a streaming media Server and a stream pushing client, wherein the stream pulling device can be a PC or a Mobile terminal and is provided with a Browser supporting a WebRTC (Web Real-Time Communication) protocol, such as a Google Chrome23, a Mozilla Firefox22 and an Opera18 Browser suitable for the PC, or a Google Chrome 28 (started by default from version 29), a Mozilla Firefox 24 and an Opera Mobile 12 Browser suitable for an Android client, and a video stream is directly played through a Browser webpage interface; the streaming media server is deployed through NodeServerProject based on node.js, supports RTSP, RTMP/RTMPE/RTMPT/RTMPTE and HTTP, and only supports video data in H.264 format and audio data in AAC format, wherein the node.js uses an event-driven and non-blocking I/O model, and is a JavaScript operating environment based on Chrome V8 engine; the stream pushing client can be an Android client or a network monitoring camera (such as a Haikang camera) with a stream pushing function, and for the Android client, a stream pushing program package (such as RTMP-RTSP-stream-client-java) is required to be installed for transmitting audio and video streams to the streaming media server in an RTMP or RTSP protocol.

In the technical scheme, the stream pulling equipment acquires the VIDEO stream pushed by the stream pushing client side on the streaming media server by using the WebRTC protocol in the local browser, converts the VIDEO stream into the WebRTC VIDEO stream, and then performs split-screen real-time playing on the WebRTC VIDEO stream by using a plurality of VIDEO tags in HTML 5. The WebRTC protocol is a set of API supporting a web browser to perform real-time audio and video, and is an HTML5 specification, and is an open source technology which can perform real-time communication work in a web page without installing any plug-in the browser, and the WebRTC protocol directly adds real-time media communication between the browser and equipment and is point-to-point (peer) communication, namely: the client side and the client side are directly connected without storing and forwarding related real-time information through a server, and the method is mainly applied to video conferences and microphone connection scenes at present. The method uses a UDP (user Datagram protocol) private protocol to push media streaming without creating discrete media segments, is oriented to no connection, and has no mechanism of waving confirmation and connection closing when TCP (Transmission control protocol) connection is disconnected, so that based on the two points, the WebRTC can achieve millisecond-level ultra-low delay which is far lower than the delay of CDN (content delivery network) distribution based on the RTMP protocol. The invention is based on the ultra-low delay of the WebRTC protocol, combines the ultra-low delay with the traditional push-pull streaming media system, and realizes the technical purpose of realizing multi-screen playing of the video stream within the delay of 35-70 ms under the scene that the push-pull video stream is required to be converted in the streaming media server.

Fig. 1 is a flowchart of a video stream split-screen display method according to an embodiment of the present invention, where the method is applied to a stream pulling device, and the stream pulling device is installed with a browser supporting a WebRTC protocol for web real-time communication, and specifically as shown in fig. 1, the video stream split-screen display method includes the following steps:

step S110: carrying out pull flow configuration locally in advance to generate a flow list; the stream list comprises one or more stream list items, and each stream list item comprises a pull stream address of a video stream corresponding to the stream list item in the stream media server.

The method comprises the steps of configuring relevant information of video streams to be played on the stream pulling equipment to form a stream list item for summarizing each video stream, so that when a certain video stream is required to be played, a user can select one of the video streams to be played and displayed in real time in an interface.

In some exemplary embodiments, a trigger menu item or a list item for performing pull flow configuration and stream list presentation may be set in an interface, and the pull flow configuration and stream list presentation may be initiated by receiving a trigger operation of a user.

Specifically, after receiving a trigger operation of a user for performing pull configuration, entering a pull configuration interface, where the interface includes information items for configuring video streams to be pulled from a streaming media server, and configuring each video stream by acquiring stream configuration data of each information item for each video stream, where the stream configuration data includes, but is not limited to: video stream name, pull stream address. In some exemplary embodiments, the video stream name may be any string, such as: my room or 123; the format of the pull stream address is: rtsp:// < ip > < port >/path, wherein ip is the address of the streaming media server, port is the port of the streaming media server, and path is the self-defined configuration parameter. In some exemplary embodiments, after obtaining the address and the port of the streaming media server in advance, the user customizes the parameter of path to distinguish different push streaming clients, and then sets a push streaming address at the push streaming client. If the stream pushing protocol adopts the RTMP protocol, the stream pushing address is an address with RTMP:// as a prefix, the stream pushing client pushes the video stream to the stream media server, and the stream media server transcodes the RTMP video stream into the RTSP video stream, and the stream pulling address where the transcoded RTSP video stream is located is converted into an address with RTSP:// as a prefix, for example: if the push flow address is rtmp://124.139.232.61, 1935/live/123, then the pull flow address is rtsp://124.139.232.61, 1935/live/123; if the stream pushing protocol adopts the RTSP protocol, the stream pushing address is the address with RTSP:// as the prefix, after the stream pushing client pushes the video stream to the stream media server, the stream media server is only used as a transfer center of the video stream and does not need to perform transcoding processing by the stream media server, and thus, the stream pushing address is the stream pulling address. Therefore, the user can know the pull address under the condition of knowing the push address in advance, and then can configure the pull address information item on the pull configuration interface.

In some exemplary embodiments, the stream configuration data further comprises: a stream play type, the stream play type including: and continuously playing and clicking to play. The continuous playing means that the streaming media server continuously obtains the video stream from the stream pushing client before the stream pulling device plays the video stream; the meaning of click playing is that the streaming media server starts to acquire the video stream from the push streaming client only when the pull streaming device plays the video stream.

After the configuration of all the information items of the video stream is completed by the pull stream configuration interface, a stream list item for the video stream is generated by clicking the save operation, and then, if other video streams need to be configured, the pull stream configuration interface is re-entered to configure the video streams. Then, the stream list items of the video streams are aggregated into a table, and a stream list is generated, in some exemplary embodiments, all the set stream list items can be viewed by triggering the stream list display menu item or list item set in the interface.

Step S120: responding to a trigger instruction for playing a video stream corresponding to a stream list item in the stream list, acquiring a stream pulling address of the video stream corresponding to the stream list item in a stream media server, and pulling the video stream from the stream media server based on the stream pulling address of the video stream corresponding to the stream list item in the stream media server, wherein the video stream is an original RTSP video stream sent to the stream media server by a stream pushing client or an RTSP video stream transcoded by the stream media server.

Specifically, the video stream configured in the step S110 of triggering and playing the stream device by the pull stream device may be triggered in a specific triggering manner according to specific item requirements, and in some exemplary embodiments, a menu item or a list item triggering a split-screen display interface may be added to an interface displaying a triggering menu item or a list item displayed in the pull stream configuration and the stream list, and after the menu item or the list item of the split-screen display interface is clicked, real-time playing and displaying of all or part of the configured video stream is triggered, or after the split-screen display interface is entered, in a display area specifically displaying one video stream, real-time playing of a certain video stream in the stream list item is triggered by clicking an add and play video stream button set in the display area.

After receiving a trigger instruction for playing a video stream corresponding to a stream list item in a stream list, the stream pulling device responds to the trigger instruction, and the method specifically operates as follows:

first, a pull address of a video stream corresponding to the stream list item in the streaming media server is obtained, and in a specific example, the pull address may be implemented by calling a relevant API function interface, where the pull address is a pull address in an RTSP protocol format input in a pull address information item in a pull configuration interface.

Then, pulling the video stream from the streaming media server based on the stream pulling address of the video stream corresponding to the stream list item in the streaming media server, where the video stream is an original RTSP video stream sent by the stream pushing client to the streaming media server, that is: the stream pushing client pushes the RTSP video stream to the streaming media server by the RTSP protocol, or the RTSP video stream which is pushed to the streaming media server by the RTMP protocol and then is transcoded by the streaming media server by the stream pushing client.

Specifically, when the video stream is pulled from the streaming media server based on the stream pulling address in the streaming media server of the video stream corresponding to the stream list item, a WebRTC connection channel is established with the streaming media server based on a WebRTC protocol interface in the local browser, and the video stream is pulled through the WebRTC connection channel. Specifically, the RTCPeerConnection interface of the WebRTC protocol needs to be called to establish a WebRTC connection channel between a local (streaming device) and a remote (streaming server), and the interface provides an implementation of a method for creating, maintaining, monitoring, and closing a connection. In specific implementation, local related parameter initialization is required to be performed first, then a local RTCPeerConnection object is created, and then: 1. exchange of ICE, namely: webRTC uses ICE (interactive connectivity facility) framework protocol, and local and remote networks need to exchange network information with each other before establishing connection, and inform each other of addresses; 2. exchanging SDP, i.e.: the WebRTC uses SDP (session description protocol) to exchange local and remote media information, such as resolution format, mainly to exchange offers (Offer) and answers (Answer) of both parties; 3. and remotely adding the video stream to be transmitted and transmitting the video stream to the local.

In some exemplary embodiments, the specific implementation code is as follows:

step S130: and decoding and unpacking the video stream to generate a WebRTC video stream conforming to a WebRTC protocol.

Since the WebRTC protocol only supports video data in the h.264 format at present, the obtained video stream needs to be decoded and unpacked before being played, so as to generate a WebRTC video stream conforming to the WebRTC protocol, that is: h.264 video stream.

Step S140: and performing split-screen display on the WebRTC Video stream in a split-screen interface generated according to a preset split-screen format and based on a Video tag of a local browser HTML 5.

The "preset split screen format" in this step refers to a format definition that when a video stream is split-screen displayed, the split screen of the display interface needs to be divided into rows and columns, for example: the screen division format comprises a 2 × 2 screen division format, a 4 × 4 screen division format, a 7 × 7 screen division format and the like, if the screen division format is the 2 × 2 screen division format, one display interface can display 4 videos, if the screen division format is the 4 × 4 screen division format, one display interface can display 16 videos, if the screen division format is the 7 × 7 screen division format, one display interface can display 49 videos, and other screen division formats can be defined according to actual needs. In some exemplary embodiments, a default split screen format may be set to a 4 x 4 split screen format, and menu items for selecting different split screen formats may be set in the split screen interface.

And if the split-screen format is determined, performing split-screen display on the WebRTC Video stream in a split-screen interface generated based on a Video tag of a local browser HTML 5. Specifically, the split screen interface is divided into a plurality of independent display windows according to a preset split screen format based on a Video tag of a local browser HTML5, wherein each display window corresponds to one Video and is used for playing one WebRTC Video stream. Each display window comprises an area for responding to the addition and playing of the video stream, and is used for receiving a trigger instruction of a user for adding and playing the video stream corresponding to the stream list item in the stream list in the display window.

In some exemplary embodiments, in an interface of a menu item or a list item provided with a split-screen display interface, a split-screen interface may appear after clicking the menu item or the list item of the split-screen display interface. After entering the split-screen interface, the user defaults not to play any video, clicks an area for adding a video stream in one of the display windows, then selects one of the stream list items in the popped stream list, and after the steps S120-S130 and the step are sequentially executed, the video stream corresponding to the stream list item is added and played in the corresponding display window. If other video streams need to be added, the operation is repeated. Wherein, each display window can repeatedly play the same video stream, and can also play different video streams. Therefore, the purpose of playing and displaying a plurality of video streams on one display interface in real time is achieved.

As can be seen from the above steps, the streaming device in the embodiment of the present invention obtains the VIDEO stream pushed by the streaming client on the streaming media server by using the WebRTC protocol in the local browser, converts the VIDEO stream into the WebRTC VIDEO stream, and then performs split-screen real-time playing on the WebRTC VIDEO stream by using a plurality of VIDEO tags in the HTML 5. The technical scheme of the invention is based on the ultra-low delay characteristic of the WebRTC protocol, combines the ultra-low delay characteristic with the traditional push-pull streaming media system, and realizes the technical purpose of playing the video stream with multiple screens within the delay of 35-70 ms in the scene of needing to transfer the push-pull video stream in the streaming media server. Therefore, time delay is reduced, and user experience in real-time video playing is improved.

Fig. 2 is a schematic structural diagram of a video streaming split-screen display device according to an embodiment of the present invention, which is disposed in a streaming device, where the streaming device is installed with a browser supporting a WebRTC protocol for web real-time communication, and includes the following modules:

a flow list generation module 210 configured to perform local pull flow configuration in advance to generate a flow list; the stream list comprises one or more stream list items, and each stream list item comprises a pull stream address of a video stream corresponding to the stream list item in a stream media server;

the stream pulling module 220 is configured to, in response to a trigger instruction for playing a video stream corresponding to a stream list item in the stream list, obtain a stream pulling address of the video stream corresponding to the stream list item in the stream media server, and pull the video stream from the stream media server based on the stream pulling address of the video stream corresponding to the stream list item in the stream media server, where the RTSP video stream is an original RTSP video stream sent to the stream media server by a stream pushing client, or an RTSP video stream transcoded by the stream media server;

a stream decoding module 230 configured to decode and unpack the video stream to generate a WebRTC video stream conforming to a WebRTC protocol;

and the stream screen display module 240 is configured to display the WebRTC Video stream in a split screen interface generated according to a preset split screen format and based on a Video tag of a local browser HTML 5.

Fig. 3 is a schematic diagram of a hardware structure of a stream pulling apparatus according to an embodiment of the present invention. As shown in fig. 3, the screen projection server includes: a memory 310 and a processor 320, wherein the memory 310 and the processor 320 are in communication; illustratively, the memory 310 and the processor 320 communicate via a communication bus 330, the memory 310 being used for storing computer programs, and the processor 320 executing the computer programs for implementing the methods shown in one of the embodiments described above.

Optionally, the streaming device may further comprise a transmitter and/or a receiver.

Alternatively, the Processor may be a Central Processing Unit (CPU), and may also be implemented by other general processors, a PLC (Programmable Logic Controller), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), or an ASIC (Application Specific Integrated Circuit). A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.

An embodiment of the present invention provides a video stream split-screen display system, as shown in fig. 4, where the video stream split-screen display system 4 includes: push streaming client 410, streaming media server 420 and pull streaming device 430 as described above

An embodiment of the present invention provides a storage medium, where the storage medium is used to store a computer program, and the computer program is used to implement the video stream screen display method described in any method embodiment above.

The embodiment of the present invention provides a chip, where the chip is used to support a receiving device (e.g., a terminal device, a network device, etc.) to implement the functions shown in the embodiment of the present invention, and the chip is specifically used in a chip system, where the chip system may be formed by a chip, and may also include a chip and other discrete devices. When the chip in the receiving device implementing the method includes a processing unit, the chip may further include a communication unit, and when the chip includes a communication unit, the communication unit may be, for example, an input/output interface, a pin, a circuit, or the like. The processing unit executes all or part of the actions executed by each processing module in the embodiment of the invention, and the communication unit can execute corresponding receiving or sending actions. In another specific embodiment, the processing module of the receiving device in the embodiment of the present invention may be a processing unit of a chip, and the receiving module or the sending module of the control device is a communication unit of the chip.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, apparatus (device) or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may employ a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations of methods, apparatus (devices) and computer program products according to embodiments of the application. It will be understood that each flow in the flow diagrams can be implemented by computer program instructions.

These computer program instructions may be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows.

These computer program instructions may also be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows.

Another embodiment of the invention is directed to a non-transitory storage medium storing a computer-readable program for causing a computer to perform some or all of the above-described method embodiments.

That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be accomplished by specifying the relevant hardware through a program, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made to the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video stream screen display method is applied to stream pulling equipment, the stream pulling equipment is provided with a browser supporting a WebRTC protocol of web real-time communication, and the method is characterized by comprising the following steps:

responding to a trigger instruction for playing a video stream corresponding to a stream list item in the stream list, acquiring a stream pulling address of the video stream corresponding to the stream list item in a stream media server, and pulling the video stream from the stream media server based on the stream pulling address of the video stream corresponding to the stream list item in the stream media server, wherein the video stream is an original RTSP video stream sent to the stream media server by a stream pushing client or an RTSP video stream transcoded by the stream media server;

2. The method according to claim 1, wherein the performing pull flow configuration locally in advance to generate the flow list specifically includes:

generating the stream list based on stream list items of each video stream;

3. The method of claim 2, wherein the flow configuration data further comprises: a stream play type; the stream play type includes: and continuously playing and clicking to play.

4. The method according to claim 1, wherein pulling the video stream from a streaming media server based on a stream pulling address of the video stream corresponding to the stream list item in the streaming media server specifically includes:

5. The method according to claim 1, wherein the performing split-screen display on the WebRTC Video stream in a split-screen interface generated according to a preset split-screen format and based on a Video tag of a local browser HTML5 specifically includes:

6. The method of claim 5, wherein each display window comprises an area for responding to the addition and playing of the video stream, and is used for receiving a trigger instruction of a user for adding and playing the video stream corresponding to the stream list item in the stream list in the display window.

7. The utility model provides a video stream screen display device, sets up in pulling a class equipment, pull a class equipment and install the browser that supports webpage real-time communication WebRTC agreement, its characterized in that, the device includes:

the flow list generating module is set to perform pull flow configuration locally in advance to generate a flow list; the stream list comprises one or more stream list items, and each stream list item comprises a pull stream address of a video stream corresponding to the stream list item in a stream media server;

and the stream screen display module is used for displaying the WebRTC Video stream in a split screen interface generated according to a preset split screen format and based on the Video tag of the local browser HTML5 in a split screen mode.

8. A pull stream device comprising a memory and a processor executing program instructions in the memory for implementing the method of any one of claims 1 to 6.

9. A video stream on-screen display system, comprising: a push streaming client, a streaming server and a pull streaming device according to claim 8.

10. A storage medium for storing a computer program for implementing the method of any one of claims 1 to 6.