CN115209189B - Video stream transmission method, system, server and storage medium - Google Patents

Video stream transmission method, system, server and storage medium Download PDF

Info

Publication number
CN115209189B
CN115209189B CN202211119798.2A CN202211119798A CN115209189B CN 115209189 B CN115209189 B CN 115209189B CN 202211119798 A CN202211119798 A CN 202211119798A CN 115209189 B CN115209189 B CN 115209189B
Authority
CN
China
Prior art keywords
target
video stream
receiving end
video
sending
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211119798.2A
Other languages
Chinese (zh)
Other versions
CN115209189A (en
Inventor
王志钢
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Maile Information Technology Co ltd
Original Assignee
Guangzhou Maile Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Maile Information Technology Co ltd filed Critical Guangzhou Maile Information Technology Co ltd
Priority to CN202211119798.2A priority Critical patent/CN115209189B/en
Publication of CN115209189A publication Critical patent/CN115209189A/en
Application granted granted Critical
Publication of CN115209189B publication Critical patent/CN115209189B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H04N21/26216—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the channel capacity, e.g. network bandwidth

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention discloses a video stream transmission method, a video stream transmission system, a video stream transmission server and a storage medium. After the receiving end is connected with the display equipment or uninstalled, receiving equipment information of the display equipment which is currently connected with the receiving end and fed back by the receiving end, determining the limited bandwidth for sending the video stream to the receiving end based on the equipment information of the display equipment which is currently connected with the receiving end, and forwarding the target video stream from the sending end to the receiving end. When the number of the display devices of the receiving end changes, the device information of the display devices currently accessed by the receiving end is updated, the SDP protocol is further updated, the limited bandwidth of the video stream sent to the receiving end is updated, and the new target video is subscribed again from the sending end, so that the sum of the code rates of the target video streams uploaded by the sending ends is matched with the limited bandwidth, the problem of video quality reduction or bandwidth resource waste of the receiving end caused by the mismatching of the code rates of the video streams and the device bandwidth of the receiving end is avoided, and the video quality and the bandwidth utilization rate of the receiving end are improved.

Description

Video stream transmission method, system, server and storage medium
Technical Field
The present invention relates to video data transmission technologies, and in particular, to a video streaming method, a video streaming system, a server, and a storage medium.
Background
In a real-time video transmission scenario (e.g., a video conference, a video teaching scenario, and a live scenario), a video is encoded by a sending end, uploaded to a server, and then sent by the server to a receiving end. The receiving end receives the video pictures forwarded from the server, thereby realizing the real-time video function.
Before the server sends the video stream to the receiving end, the server may set a maximum bit rate of the video stream sent to the receiving end according to a bandwidth of the receiving end, which is called as a limited bandwidth. In the subsequent video stream transmission process, even if the code rate of the video stream sent to the receiving end by the server is greater than the limited bandwidth, the receiving end can only receive the video stream with the limited bandwidth. Therefore, in order to avoid the waste of bandwidth resources and improve the transmission stability, the bitrate of the video stream sent by the server to the receiving end does not exceed the limit bandwidth.
In general, the number of display devices configured at the receiving end is constant, and the limited bandwidth is also constant. However, in some cases, when a user needs to connect more display devices to present different video images or needs to reduce the number of connected display devices, if the video stream is still transmitted with the original limited bandwidth, the bit rate of the video stream is much smaller than the device bandwidth of the receiving end, which causes the video quality of the receiving end to be reduced, or the bit rate of the video stream is much larger than the device bandwidth of the receiving end, which causes the waste of bandwidth resources.
Disclosure of Invention
The invention provides a video stream transmission method, a video stream transmission system, a server and a storage medium, which are used for solving the problems of video quality reduction or bandwidth resource waste of a receiving end caused by mismatching of a code rate of a video stream and equipment bandwidth of the receiving end and improving the video quality and the bandwidth utilization rate of the receiving end.
In a first aspect, an embodiment of the present invention provides a video streaming method, which is applied to a server, and the method includes:
after a receiving end is accessed to display equipment or unloads the display equipment, receiving equipment information of the display equipment currently accessed by the receiving end, which is fed back by the receiving end;
determining the limited bandwidth for sending the video stream to the receiving end based on the equipment information of the display equipment currently accessed by the receiving end;
and forwarding the target video stream from a sending end to the receiving end, wherein the sending end can generate a plurality of video streams with different code rates, and the sum of the code rates of the target video streams is matched with the limited bandwidth.
In a second aspect, an embodiment of the present invention further provides a video streaming apparatus, which is applied to a server, and includes:
the device information receiving module is used for receiving the device information of the display device currently accessed by the receiving terminal, which is fed back by the receiving terminal, after the receiving terminal is accessed to the display device or unloads the display device;
the bandwidth limiting determination module is used for determining the bandwidth limiting of the video stream sent to the receiving end based on the equipment information of the display equipment currently accessed by the receiving end;
and the video stream forwarding module is used for forwarding a target video stream from a sending end to the receiving end, the sending end can generate a plurality of video streams with different code rates, and the sum of the code rates of the target video streams is matched with the limited bandwidth.
In a third aspect, an embodiment of the present invention further provides a video streaming system, including a sending end, a server, and a receiving end, where the server is configured to execute the video streaming method provided in the first aspect of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a server, including:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a video streaming method as provided by the first aspect of the invention.
In a fifth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the video streaming method according to the first aspect of the present invention.
The invention provides a video stream transmission method, which is applied to a server, and is used for receiving equipment information of display equipment currently accessed by a receiving end, which is fed back by the receiving end after the receiving end is accessed to the display equipment or unloads the display equipment, determining the limited bandwidth for sending a video stream to the receiving end based on the equipment information of the display equipment currently accessed by the receiving end, forwarding a target video stream from the sending end to the receiving end, wherein the sending end can generate a plurality of video streams with different code rates, and the sum of the code rates of all the target video streams is matched with the limited bandwidth. When the number of the display devices of the receiving end changes, the device information of the display devices currently accessed by the receiving end is updated, the SDP protocol is further updated, the limited bandwidth of the video stream sent to the receiving end is updated, and the new target video is subscribed again from the sending end, so that the sum of the code rates of the target video streams uploaded by the sending ends is matched with the limited bandwidth, the problem of video quality reduction or bandwidth resource waste of the receiving end caused by the mismatching of the code rates of the video streams and the device bandwidth of the receiving end is avoided, and the video quality and the bandwidth utilization rate of the receiving end are improved.
It should be understood that the statements in this section are not intended to identify key or critical features of the embodiments of the present invention, nor are they intended to limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1A is a flowchart of a video streaming method provided in the present application;
fig. 1B is a schematic structural diagram of a video streaming system provided in the present application;
fig. 1C is a schematic structural diagram of another video streaming system provided in the present application;
fig. 1D is a schematic structural diagram of another video streaming system provided in the present application;
fig. 1E is a flow chart of video streaming according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a video streaming apparatus provided in the present application;
fig. 3 is a schematic structural diagram of a server according to the present application.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
Fig. 1A is a flowchart of a video streaming method provided by the present application, where the method is applied to a server, and this embodiment may be applicable to a situation where, in a video streaming process, the number of display devices connected to a receiving end changes, which results in a mismatch between a video streaming bitrate and a bandwidth of the receiving end, and the method may be executed by a video streaming apparatus provided in an embodiment of the present invention, where the apparatus may be implemented in a software and/or hardware manner and is configured in the server in the embodiment of the present invention, as shown in fig. 1A, the method specifically includes the following steps:
s101, after the receiving end is connected with the display equipment or uninstalled, equipment information of the display equipment currently connected with the receiving end and fed back by the receiving end is received.
The embodiment of the present invention may be applicable to a multi-user video call scene as an example, for example, a multi-user video conference scene, and may also be applicable to a one-to-one video call scene, which is not limited herein.
The sending end may be a terminal device having a video capture device (e.g., a camera), such as an intelligent mobile terminal, a personal computer, and the like, and the embodiment of the present invention is not limited herein. The sending terminal is used for collecting videos, encoding the videos to form video streams and uploading the video streams to the server. In the embodiment of the invention, in order to deal with the fluctuation of the communication bandwidth between the sending end and the server, the sending end has the capability of dynamic coding and can generate a plurality of video streams with different code rates by coding. For example, when the communication bandwidth between the sender and the server decreases, the server may send an instruction to the sender to subscribe to a video stream with a lower bitrate than the currently transmitted video stream. And the sending end generates a video stream with a code rate lower than that of the currently transmitted video stream according to the instruction and uploads the video stream to the server. When the communication bandwidth between the sending end and the server is recovered to be normal, the server can send an instruction to the sending end to subscribe the video stream with the code rate higher than that of the currently transmitted video stream. And the sending end generates a video stream with a code rate higher than that of the currently transmitted video stream according to the instruction and uploads the video stream to the server. In this embodiment of the present invention, one or more sending ends may be used, and this embodiment of the present invention is not limited herein.
The server has the function of selecting and forwarding the video stream, and can select the corresponding video stream according to the video subscribed by the receiving end. Illustratively, the server may be a media forwarder (SFU). The media repeater forwards the video stream of the sending end to the receiving end. Because the server only receives and forwards the video stream, the processing such as encoding, decoding and the like is not needed, the delay of the real-time video is greatly reduced, and the real-time performance is improved.
The receiving end is used for receiving the video stream forwarded by the server, and the receiving end can be an intelligent mobile terminal, a personal computer and the like. In the embodiment of the present invention, the receiving end may be connected to one or more display devices, for example, the display device may be a display. After receiving the video streams uploaded by the transmitting ends, the receiving end needs to decode and mix the video streams, so that a plurality of video images are displayed on one display device. Illustratively, the display device at the receiving end may display video images captured by all terminal devices in the form of a squared figure. The display device at the receiving end has a maximum transmission bit rate (i.e., device bandwidth) representing the maximum bit rate of the video stream that the display device is capable of receiving. Therefore, the sum of the bit rates of the video streams subscribed by the receiving end to the transmitting end cannot exceed the maximum transmission bit rate of the display device. In order to avoid the waste of bandwidth resources and improve the transmission stability, the server sets the maximum bit rate of the video stream sent to the receiving end according to the maximum transmission bit rate of the receiving end, which is called as the limit bandwidth, and the bit rate of the video stream sent to the receiving end by the server does not exceed the limit bandwidth.
It should be noted that, in the above embodiments, the sending end and the receiving end are distinguished by initiating and receiving a video stream, and in other embodiments of the present invention, the sending end may simultaneously serve as the receiving end, and the receiving end may also simultaneously serve as the sending end. For example, in a video conference scene, the terminal device of each participant may collect a video image of the participant, upload the video image to the server, and receive video images of other participants forwarded by the server.
In a scene of multi-person video call, when a user at a receiving end needs to connect more display devices to present views from different sending ends or to reduce the number of connected display devices, in the scene of one-to-one video call, when the user at the receiving end needs to connect more display devices to present views from the sending end alone or to reduce the number of connected display devices, at this time, since the number of display devices changes, the device bandwidth at the receiving end also changes, if the video stream is transmitted with the original limited bandwidth, the bit rate of the video stream is far smaller than the device bandwidth at the receiving end, which causes the video quality at the receiving end to decrease, or the bit rate of the video stream is far larger than the device bandwidth at the receiving end, which causes the waste of bandwidth resources.
In the embodiment of the invention, after the receiving end is accessed to the display equipment or unloads the display equipment, the equipment information of the display equipment currently accessed by the receiving end is received and fed back by the receiving end. For example, the connection and disconnection in the embodiment of the present invention may be hot plugging of the display device and the terminal device through a joint. When the display equipment and the terminal equipment are plugged through the connector, the connection is called as access; when the connector of the display device is unplugged from the interface of the terminal device, the connection is called uninstalling. After the receiving end accesses the display device or unloads the display device, the receiving end feeds back the device information of the currently accessed display device to the server, and the server receives the device information of the currently accessed display device fed back by the receiving end, so that a Session Description Protocol (SDP) is updated. The session description protocol is used for media negotiation between a sending end and a receiving end, the problem that bandwidths among all members participating in a session are not equal is solved, if the terminals participating in the session support high-quality conversation without carrying out the protocol, and for compatibility, conversation formats with common quality are used, so that bandwidth resources are wasted.
S102, determining the limited bandwidth for sending the video stream to the receiving end based on the equipment information of the display equipment currently accessed by the receiving end.
In the embodiment of the invention, after receiving the equipment information of the display equipment currently accessed by the receiving end, the server determines the limited bandwidth for sending the video stream to the receiving end based on the equipment information of the display equipment currently accessed by the receiving end. For example, the device information may include a maximum transmission bit rate of each display device currently accessed by the receiving end, and the limited bandwidth is a sum of the maximum transmission bit rates of the display devices currently accessed by the receiving end.
After the limited bandwidth for sending the video stream to the receiving end is updated based on the device information of the display device currently accessed by the receiving end, the receiving end may update subscription information for subscribing the video stream to the sending end, and the server forwards the subscription information to the sending end. And the sending end uploads the target video stream with the code rate matched with the updated limited bandwidth according to the new subscription information.
When the number of the display devices of the receiving end changes, the device information of the display devices currently accessed by the receiving end is updated, the SDP protocol is further updated, the limited bandwidth of the video stream sent to the receiving end is updated, and the new target video is subscribed again from the sending end, so that the sum of the code rates of the target video streams uploaded by the sending ends is matched with the limited bandwidth, the problem of video quality reduction or bandwidth resource waste of the receiving end caused by the mismatching of the code rates of the video streams and the device bandwidth of the receiving end is avoided, and the video quality and the bandwidth utilization rate of the receiving end are improved.
S103, forwarding the target video streams from the sending end to the receiving end, wherein the sum of the code rates of all the target video streams is matched with the limited bandwidth.
And after the sending end uploads the target video stream with the code rate matched with the updated limited bandwidth according to the new subscription information, the server sends the target video stream to the receiving end, and the sum of the code rates of the target video streams uploaded by the sending ends is matched with the limited bandwidth. The code rate of the target video stream is matched with the limited bandwidth for a one-to-one video call scene, and the sum of the code rates of the target video streams uploaded by each sending end is matched with the limited bandwidth for a multi-user video call scene. Illustratively, in one embodiment of the present invention, the sum of the coding rates of the target video streams uploaded by each transmitting end is as large as possible within the limited bandwidth, for example, equal to or slightly smaller than the limited bandwidth. In another embodiment of the present invention, in order to ensure that the sum of the code rates of the video streams sent by the server to the receiving end is equal to the limited bandwidth, and utilize the limited bandwidth to the maximum extent, and ensure that the receiving end displays the best video quality, the sum of the code rates of the target video streams sent by the sending end may be greater than the limited bandwidth, and the server may perform frame dropping processing on at least one of the video streams uploaded by the sending end within the range of the maximum frame dropping rate, so that the sum of the code rates of the target video streams sent by the server to the receiving end is equal to the limited bandwidth. The frame loss rate is the ratio of the number of frames of the video stream after frame loss to the number of frames before frame loss, and the maximum frame loss rate is the frame loss rate with the lowest image quality for ensuring that the video can be normally watched.
The embodiment of the invention provides a video stream transmission method, which is applied to a server, and is used for receiving equipment information of display equipment currently accessed by a receiving end, which is fed back by the receiving end after the receiving end is accessed to the display equipment or uninstalled, determining the limited bandwidth for sending a video stream to the receiving end based on the equipment information of the display equipment currently accessed by the receiving end, forwarding a target video stream from the sending end to the receiving end, wherein the sending end can generate a plurality of video streams with different code rates, and the sum of the code rates of all the target video streams is matched with the limited bandwidth. When the number of the display devices of the receiving end changes, the device information of the display devices currently accessed by the receiving end is updated, the SDP protocol is further updated, the limited bandwidth of the video stream sent to the receiving end is updated, and the new target video is subscribed again from the sending end, so that the sum of the code rates of the target video streams uploaded by the sending ends is matched with the limited bandwidth, the problem of video quality reduction or bandwidth resource waste of the receiving end caused by the mismatching of the code rates of the video streams and the device bandwidth of the receiving end is avoided, and the video quality and the bandwidth utilization rate of the receiving end are improved.
The above is the core idea of the present invention, and the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention. Based on the embodiments of the present invention, those skilled in the art can obtain all other embodiments without making creative efforts, which all belong to the protection scope of the embodiments of the present invention.
In the embodiment of the present invention, a scheme of the present invention is exemplarily described by taking a multi-user real-time video conference as an example, fig. 1B is a schematic structural diagram of a video streaming system provided by the present invention, and fig. 1C is a schematic structural diagram of another video streaming system provided by the present invention, as shown in fig. 1B, the present invention takes a terminal device of a user a as a receiving end (with respect to other users, the terminal device of the user a is a transmitting end), and with respect to the terminal device of a, the terminal devices of other users B-I are transmitting ends (respectively, transmitting ends B-I). And the terminal equipment of all users encodes the acquired video according to the subscription of the receiving end to form a video stream matched with the code rate of the video stream subscribed by the receiving end, and uploads the video stream to the server through an independent uploading channel. And the receiving end (the terminal equipment of the user A) downloads the video streams uploaded by the terminal equipment of other users from the server through an independent channel. That is, there are an upload channel and 8 download channels between each terminal device and the server, and the 8 download channels are respectively used for downloading the video streams sent to the server by the terminal devices of the users B-I.
As shown in fig. 1B, the terminal device of user a is connected to a display device which displays video images of users a-I in the form of a squared figure. Assuming that the maximum transmission bit rate of the display device is 4Mbps, the limited bandwidth for sending the video stream to the receiving end by the server is 4Mbps, that is, the sum of the code rates of the video stream transmitted by the server to the receiving end cannot exceed 4Mbps. Because the display equipment at the receiving end displays each video image in the nine-square grid, the resolution and the code rate of the video stream of each video image can be considered to be equal, the code rate of the video stream uploaded by the terminal equipment of the user B-I is divided into two parts to limit the bandwidth, and the code rate of the video stream uploaded by the terminal equipment of the user B-I is 0.5Mbps. It should be noted that, in other embodiments of the present invention, the display device at the receiving end may also display each video image in other forms, for example, the picture size of the video image of the user C is larger than that of the video images of other users, and in order to make the resolutions of the video images the same, the bitrate requirement corresponding to the video image of the user C is larger, but the sum of the bitrates of the video streams uploaded by the transmitting ends B (terminal devices of the users B-I) still needs not to be larger than the limit bandwidth.
Based on the structure shown in fig. 1B, when a user a at the receiving end needs a display device to separately display a video image at a certain transmitting end, the user needs to access a new display device. As shown in fig. 1C, the user accesses a new display device for displaying the video image of the presenter B of the conference alone. Since the newly added display device displays the video image of the speaker B alone, in order to ensure the video quality, the code rate of the video stream uploaded by the terminal device of the user B needs to be matched with the maximum transmission bit rate of the newly added display device. Illustratively, if the maximum transmission bit rate of the new display device is 4Mbps, that is, the code rate of the video stream uploaded by the terminal device of the user B needs to be 4Mbps, the video quality can be guaranteed, however, the server sends the video stream uploaded by the terminal device of the user B to the receiving end, and also sends the video streams uploaded by the terminal devices of other users, obviously, the original limited bandwidth of 4Mbps cannot meet the transmission requirement, and cannot guarantee the video quality of the video image of the speaker B of the conference to be displayed alone.
In the embodiment of the invention, after a receiving end (terminal equipment of a user A) accesses a new display device, the receiving end feeds back the device information of the currently accessed display device to a server. And the server receives the equipment information of the display equipment currently accessed by the receiving end, which is fed back by the receiving end, so as to update the SDP protocol. For example, after receiving the device information of the display device currently accessed by the receiving end, the server determines, based on the device information of the display device currently accessed by the receiving end, a limited bandwidth for transmitting the video stream to the receiving end. For example, the device information may include maximum transmission bit rates of two display devices currently accessed by the receiving end, and the limited bandwidth is the sum of the maximum transmission bit rates of the display devices currently accessed by the receiving end. Illustratively, in the embodiment of the present invention, the maximum transmission bit rates of the two display devices are both 4Mbps, and the updated limited bandwidth is 8Mbps, so that the terminal device of the user B can upload a video stream with a higher code rate, and the video quality of a video image of the speaker B of the conference is ensured to be displayed separately.
Illustratively, in some embodiments of the present invention, after the bandwidth limit is updated, the server sends a first instruction including the bandwidth limit to a sending end, where the sending end is configured to determine a target video stream to be uploaded according to the first instruction, where a sum of code rates of the target video streams matches the bandwidth limit. In the embodiment of the present invention, matching the sum of the code rates of the target video streams with the limited bandwidth indicates that the sum of the code rates of the target video streams is slightly less than or equal to the limited bandwidth, or the sum of the code rates of the target video streams is less than or equal to the limited bandwidth after the server performs the code rate reduction processing on at least one of the target video streams uploaded by the sending end within the maximum code rate reduction range. For example, the embodiment of the present invention takes an example that the sum of the bit rates of each target video stream after the server performs the bit rate reduction processing on at least one target video stream uploaded by the sending end within the maximum bit rate reduction range is smaller than or equal to the limited bandwidth, and the sum of the bit rates of each target video stream matches the limited bandwidth. The maximum code rate reduction range is the code rate range which can still meet the requirement of normal watching of human eyes after the code rate reduction processing is carried out on the video stream. For example, the rate reduction processing may include frame dropping processing and delayed transmission processing.
The server receives the target video streams sent by the sending ends, judges whether the sum of the code rates of the target video streams sent by the sending ends is larger than the limit bandwidth or not, reduces the code rate of at least one target video stream within the maximum code rate reduction range if the sum of the code rates of the target video streams sent by the target sending ends is larger than the maximum transmission bit rate of the target display equipment, enables the sum of the code rates of the target video streams to be smaller than or equal to the limit bandwidth, and sends the target video streams to the receiving end. And if the sum of the code rates of the target video streams sent by the target sending end is less than or equal to the maximum transmission bit rate of the target display equipment, directly forwarding the target video streams sent by the target sending end to the target receiving end.
Illustratively, the process of frame loss processing is as follows:
1. and acquiring each video frame to be transmitted in the target video stream to be frame-lost.
For a target video stream to be subjected to frame loss processing, obtaining video frames to be transmitted in the target video stream, exemplarily, in an embodiment of the present invention, the video frames to be transmitted may be cached in a transmission queue, and a server obtains the video frames to be transmitted in the target video from the transmission queue.
2. A discardable frame and a discardable frame drop priority are determined from the video frames.
Illustratively, in the embodiment of the present invention, a sending end is based on Scalable Video Coding (SVC), and a coded Video stream may be coded normally after a certain Video frame is discarded according to a certain rule. In the frame loss process, it is necessary to reduce the playing effect of the original video stream as much as possible, for example, to avoid the phenomenon of screen splash or picture skipping of the decoded picture as much as possible. The scalable video coding technique divides video frames into video coding key frames and non-key frames, the video coding key frames are video frames with complete picture data or video frames that can be decoded only by the frame data, and the non-key frames need to be decoded by the video coding key frames or other non-key frames, so in the embodiment of the present invention, the non-key frames are regarded as discardable frames. Therefore, the frame dropping process needs to avoid dropping the video coding key frame as much as possible to avoid the phenomenon of screen splash or picture skipping of the decoded picture. In addition, the non-key frames are set with different discarding priorities according to the influence on the decoded video quality, and the video frames with higher discarding priorities have smaller influence on the decoded video quality after discarding.
In the embodiment of the present invention, it can be determined whether a video frame to be transmitted is a discardable frame and the discardable priority of the discardable frame through the identifier. Illustratively, the identification of the video coding key frame is T0, and the identification of the non-key frames is T1-Tn according to the drop priority from low to high.
3. And sequentially discarding the discardable frames within the range of the maximum frame loss rate according to the discardable frames' discarding priority until the sum of the code rates of all target video streams after the frames are discarded is equal to the limited bandwidth.
Illustratively, the discardable frames are discarded in sequence from high to low according to the discarding priority of the discardable frames, the discardable frames with high discarding priority (that is, the video frames with small influence on the quality of the decoded video are discarded in priority) are discarded preferentially, the frame discarding rate after frame discarding is ensured to be within the range of the maximum frame discarding rate, and the fluency of the video received by a receiving end is ensured until the sum of the bit rates of all target video streams after frame discarding is equal to the limit bandwidth.
Illustratively, the delay process is to increase the transmission interval time between each frame of image, for example, if the bitrate of the target video stream is 150kb per second, it would take 1 second to originally transmit 150kb of video, and now adjusting the transmission interval time between each frame of image to delay it for 1.5 seconds is equivalent to reducing the bitrate to 100kbps.
In some embodiments of the present invention, if a previously accessed display device displays views of video streams corresponding to each terminal in a grid form, a user wants to display one of the views on a newly accessed display device in a full screen, and in order to ensure the video quality of a video image displayed by the newly accessed display device, a video stream corresponding to the video image displayed by the newly accessed display device may be newly subscribed, and the specific process is as follows:
1. and receiving the video stream information of the video stream to be displayed of the newly accessed target display equipment from the receiving end.
For example, the video stream information of the video stream generally includes address information, for example, a physical address, of a transmitting end that transmits the video stream, and after determining a video stream to be displayed by a newly-accessed target display device, the server acquires the video stream information of the video stream.
2. And determining a target sending end for sending the video stream to be displayed according to the video stream information.
As mentioned above, the video stream information of a video stream usually includes address information of a transmitting end that transmits the video stream, and by means of the address information, a target transmitting end that transmits the video stream to be displayed can be determined.
3. And sending a second instruction comprising the maximum transmission bit rate of the target display equipment to a target sending end, wherein the target sending end is used for determining a target video stream to be uploaded according to the second instruction, and the target video stream of each target sending end is a video stream of which the sum of the code rates after the code rate reduction processing is smaller than or equal to the maximum transmission bit rate of the target display equipment.
And after determining a target sending end for sending the video stream to be displayed, the server sends a second instruction comprising the maximum transmission bit rate of the target display equipment to the target sending end. And the target sending ends are used for determining target video streams to be uploaded according to the second instruction, wherein the target video streams of the target sending ends are video streams of which the sum of the code rates after the code rate reduction processing is less than or equal to the maximum transmission bit rate of the target display equipment, so that the video quality of the video images displayed by the newly accessed display equipment is ensured. Of course, in another embodiment of the present invention, the sum of the bit rates of the target video streams at each target transmitting end may also be slightly smaller than or equal to the maximum transmission bit rate of the target display device. It should be noted that, as shown in fig. 1C, in the embodiment of the present invention, an example is that a newly added target display device displays one video image alone, in other embodiments of the present invention, a newly added target display device may also display a plurality of video images, and fig. 1D is a schematic structural diagram of another video streaming system provided in the present application, as shown in fig. 1D, in this embodiment, a newly added target display device displays two video images of a user B and a user C. The code rate of the target video stream corresponding to each video image may be determined according to the display size of the video image, for example, if the display sizes of the two video images are the same, the code rates of the two target video streams are the same, and if the display size of the video image of the user B is larger than the display size of the video image of the user C, the code rate of the target video stream corresponding to the video image of the user B is larger than the code rate of the target video stream corresponding to the video image of the user C.
4. And receiving the target video stream of the target sending end, and judging whether the sum of the code rates of the target video stream sent by the target sending end is greater than the maximum transmission bit rate of the target display equipment.
And the target sending terminal generates a new target video stream according to the new subscription and uploads the new target video stream to the server. The server receives a target video stream of a target sending end. For example, if the sum of the bit rates of the target video streams at each target sending end is not greater than the maximum transmission bit rate of the target display device, it is not necessary to perform bit rate reduction processing on the target video streams. And if the sum of the code rates of the target video streams of all the target sending ends is greater than the maximum transmission bit rate of the target display equipment, performing code rate reduction processing on the target video streams to ensure that the sum of the code rates of the target video streams sent by the target sending ends is less than or equal to the maximum transmission bit rate of the target display equipment, and forwarding the target video streams to the receiving end. The process of the rate reduction processing may refer to the frame loss processing and the delay processing in the foregoing embodiments, and the details of the embodiment of the present invention are not repeated again.
In another embodiment of the present invention, the user a at the receiving end originally accesses two display devices, as shown in fig. 1C, the user a now does not need to separately display the video image of the speaker B, and needs to uninstall the display device that displays the video image of the speaker B, and the uninstalled structure is as shown in fig. 1B. Illustratively, the maximum transmission bit rate of both display devices is 4Mbps, and the limited bandwidth before offloading is 8Mbps. After one of the display devices is unloaded, because the code rate of the video stream corresponding to the video image of the original speaker B is large, the original limited bandwidth of 8Mbps is obviously much greater than the maximum transmission bit rate of the existing single display device, and if the video stream is still transmitted according to the original code rate, the waste of bandwidth resources is caused.
In the embodiment of the invention, after one display device is unloaded, the receiving end feeds back the device information of the currently accessed display device to the server. And the server receives the equipment information of the display equipment currently accessed by the receiving end, which is fed back by the receiving end, so as to update the SDP protocol. For example, after receiving the device information of the display device currently accessed by the receiving end, the server determines, based on the device information of the display device currently accessed by the receiving end, a limited bandwidth for transmitting the video stream to the receiving end. For example, the device information may include maximum transmission bit rates of two display devices currently accessed by the receiving end, and the limited bandwidth is the sum of the maximum transmission bit rates of the display devices currently accessed by the receiving end. Illustratively, in the embodiment of the present invention, the maximum transmission bit rates of the two display devices are both 4Mbps, and the updated limited bandwidth is 4Mbps, so that the terminal device of the user B can upload a video stream with a lower code rate, thereby avoiding waste of bandwidth resources.
Illustratively, in some embodiments of the present invention, after the bandwidth limit is updated, the server sends a first instruction including the bandwidth limit to a sending end, where the sending end is configured to determine a target video stream to be uploaded according to the first instruction, where a sum of code rates of the target video streams matches the bandwidth limit. In the embodiment of the present invention, matching the sum of the code rates of the target video streams with the limited bandwidth indicates that the sum of the code rates of the target video streams is slightly less than or equal to the limited bandwidth, or the sum of the code rates of the target video streams is equal to the limited bandwidth after the server performs the code rate reduction processing on at least one of the target video streams uploaded by the sending end within the maximum code rate reduction range.
In some embodiments of the present invention, in order to avoid bandwidth waste caused by a large bitrate of a video stream corresponding to a video image displayed by an offloaded display device, a video stream corresponding to a video image displayed by an offloaded display device may be re-subscribed, and the specific process is as follows:
1. determining a first bitrate of the first video stream displayed by the unloaded target display device and a second bitrate of the first video stream displayed by the un-unloaded display device.
Illustratively, the offloaded target display device displays a first video stream (e.g., a video stream corresponding to video images of user B displayed separately in fig. 1C) with a bitrate of 4Mbps. The first video stream (corresponding to the video image of user B shown in the nine-square grid of fig. 1C) displayed by the un-unloaded display device has a bitrate of 0.5Mbps.
2. And if the first code rate is greater than the second code rate, sending a third instruction to a target sending end of the first video stream, and determining the video stream to be uploaded according to the code rate uploaded by the target sending end according to the third instruction, wherein the code rate of the video stream to be uploaded is smaller than that of the first video stream.
If the first code rate is greater than the second code rate, a third instruction is sent to a target sending end (such as a sending end of a user B in fig. 1B) of the first video stream. And the target sending end determines the video stream to be uploaded according to the uploading code rate of the third instruction, wherein the video stream to be uploaded is a video stream with the code rate smaller than that of the first video stream. Illustratively, the target sending end determines that the code rate of the video stream to be uploaded is slightly smaller than or equal to the second code rate of the second video stream, so as to avoid waste of bandwidth resources.
3. And receiving the video stream uploaded by the target sending terminal and forwarding the video stream to the receiving terminal.
And the target sending end uploads the video stream to be uploaded to the server. And the server receives the video stream uploaded by the target sending terminal.
In some embodiments of the present invention, in order to ensure the video quality displayed by the receiving end, the target sending end determines that the code rate of the video stream to be uploaded can be greater than the second code rate, and then performs a code rate reduction process on the video stream so that the code rate of the video stream is equal to the second code rate, which includes the following specific processes:
1. and judging whether the code rate of the video stream uploaded by the target sending end is greater than the second code rate.
2. And if the code rate of the video stream uploaded by the target sending end is greater than the second code rate, reducing the code rate of the video stream uploaded by the target sending end within the maximum code rate reduction range, so that the code rate of the video stream uploaded by the target sending end is less than or equal to the second code rate, and forwarding the video stream uploaded by the target sending end to the receiving end.
And if the code rate of the video stream uploaded by the target sending end is greater than the second code rate, performing code rate reduction processing on the video stream uploaded by the target sending end within the maximum code rate reduction range to enable the code rate of the video stream to be equal to the second code rate, and forwarding the video stream uploaded by the target sending end to the receiving end. Specifically, the process of processing the rate reduction rate of the video stream has been described in detail in the foregoing embodiments, and the embodiments of the present invention are not described herein again. And if the code rate of the video stream uploaded by the target sending end is less than or equal to the second code rate, directly forwarding the video stream uploaded by the target sending end to the receiving end.
Fig. 1E is a video streaming flow chart according to an embodiment of the present invention, and as shown in fig. 1E, when a receiving end accesses or unloads a display device, a media forwarder SFU acquires device information of the receiving end, so as to update a session description protocol SDP, and then update a limited bandwidth. Judging whether the sum of the code rates of the video streams sent by each sending end is greater than the limited bandwidth or not, if the sum of the code rates of the video streams sent by each sending end is greater than the limited bandwidth, informing one or more sending ends to dynamically reduce the code rate of the video streams or subscribing the video streams with lower resolution (namely reducing the code rate); and if the sum of the code rates of the video streams sent by the sending ends is less than or equal to the limited bandwidth, directly forwarding the video streams to the receiving end. Further, after one or more sending terminals are informed to dynamically reduce the code rate of the video stream, or subscribe the video stream with lower resolution (namely, reduce the code rate), whether the sum of the code rates of the new video stream is larger than the limited bandwidth is judged again, if the code rate of the new video stream is still larger than the limited bandwidth, the code rate reduction processing is carried out on one or more video streams through a smoothing module, and the code rate reduction processing can be frame loss processing or transmission delay of the video stream is increased, so that the sum of the code rates of the video streams is smaller than or equal to the limited bandwidth; and if the code rate of the new video stream is less than or equal to the limited bandwidth, directly forwarding the video stream to a receiving end. When the number of the display devices of the receiving end changes, the device information of the display devices currently accessed by the receiving end is updated, the SDP protocol is further updated, the limited bandwidth of the video stream sent to the receiving end is updated, one or more sending ends are informed to dynamically reduce the code rate of the video stream or subscribe the video stream with lower resolution, if the sum of the code rates of the new video stream is still larger than the limited bandwidth, the frame loss or delayed transmission processing is further carried out on the video stream, so that the sum of the code rates of the video stream is smaller than or equal to the limited bandwidth, the problem of video quality reduction or bandwidth resource waste of the receiving end caused by the mismatch of the code rate of the video stream and the device bandwidth of the receiving end is avoided, the video quality and the bandwidth utilization rate of the receiving end are improved, and meanwhile, the video stream is processed by frame loss or delayed transmission, the resolution of the video is ensured to the maximum extent, and the user experience is improved.
Example two
Fig. 2 is a schematic structural diagram of a video streaming apparatus provided in the present application, where the video streaming apparatus is applied to a server, and as shown in fig. 2, the apparatus includes:
the device information receiving module 201 is configured to receive, after a receiving end accesses a display device or unloads the display device, device information of the display device currently accessed by the receiving end, which is fed back by the receiving end;
a bandwidth limit determining module 202, configured to determine, based on device information of a display device currently accessed by the receiving end, a bandwidth limit for sending a video stream to the receiving end;
a video stream forwarding module 203, configured to forward a target video stream from a sending end to the receiving end, where the sending end may generate a plurality of video streams with different code rates, and a sum of the code rates of the target video streams matches the limited bandwidth.
In some embodiments of the present invention, the device information includes a maximum transmission bit rate of each display device currently accessed by the receiving end, and the bandwidth limitation determining module 202 includes:
and the bandwidth limit determining submodule is used for calculating the sum of the maximum transmission bit rates of all display equipment currently accessed by the receiving end and sending the sum as the bandwidth limit of the video stream to the receiving end.
In some embodiments of the invention, the video stream forwarding module 203 comprises:
a first instruction sending submodule, configured to send a first instruction including the bandwidth limitation to the sending end, where the sending end is configured to determine a target video stream to be uploaded according to the first instruction, where the target video stream is a video stream in which a sum of code rates after the code rate reduction processing is equal to the bandwidth limitation;
a first judging unit, configured to receive the target video streams sent by the sending ends, and judge whether a sum of code rates of the target video streams sent by the sending ends is greater than the limited bandwidth;
a first bit rate reduction processing unit, configured to reduce, if the sum of the target video streams is greater than the limited bandwidth, the bit rate of at least one target video stream within a maximum bit rate reduction range, so that the sum of the bit rates of the target video streams is less than or equal to the limited bandwidth, and forward the target video streams to the receiving end;
and the video forwarding unit is used for forwarding the target video stream to the receiving end if the sum of the code rates of the target video streams is less than or equal to the limited bandwidth.
In some embodiments of the present invention, the first rate reduction processing unit is configured to perform frame loss processing on the at least one target video stream within a maximum frame loss rate range, or increase a transmission delay of the at least one target video stream within a maximum delay range.
In some embodiments of the present invention, if the receiving end is connected to a display device, the video stream forwarding module 203 further includes:
the video stream information acquisition submodule is used for receiving the video stream information of the video stream to be displayed of the newly accessed target display equipment from the receiving end;
the target sending terminal determining submodule is used for determining a target sending terminal for sending the video stream to be displayed according to the video stream information;
a second instruction sending submodule, configured to send a second instruction including a maximum transmission bit rate of the target display device to the target sending end, where the target sending end is configured to determine a target video stream to be uploaded according to the second instruction, and a target video stream of each target sending end is a video stream whose sum of code rates after the code rate reduction processing is smaller than or equal to the maximum transmission bit rate of the target display device;
the judgment submodule is used for receiving the target video stream of the target sending end and judging whether the sum of the code rates of the target video stream sent by the target sending end is greater than the maximum transmission bit rate of the target display equipment or not;
the code rate reduction submodule is used for reducing the code rate of at least one target video stream within the range of the maximum code rate reduction if the sum of the code rates of the target video streams sent by the target sending end is greater than the maximum transmission bit rate of the target display equipment, so that the sum of the code rates of the target video streams sent by the target sending end is smaller than or equal to the maximum transmission bit rate of the target display equipment, and the target video streams are transmitted to the receiving end;
and the first video stream forwarding sub-module is used for forwarding the target video stream sent by the target sending end to a target receiving end if the sum of the code rates of the target video stream sent by the target sending end is less than or equal to the maximum transmission bit rate of the target display equipment.
In some embodiments of the present invention, if the receiving end unloads the display device, the video stream forwarding module 203 further includes:
the code rate determining submodule is used for determining a first code rate of the first video stream displayed by the unloaded target display equipment and a second code rate of the first video stream displayed by the un-unloaded display equipment;
a third instruction sending submodule, configured to send a third instruction to a target sending end of the first video stream if the first code rate is greater than the second code rate, where the target sending end determines a video stream to be uploaded according to a code rate uploaded by the third instruction, where the video stream to be uploaded is a video stream whose code rate is smaller than that of the first video stream;
and the second video stream forwarding submodule is used for receiving the video stream uploaded by the target sending terminal and forwarding the video stream to the receiving terminal.
In some embodiments of the invention, the second video stream forwarding sub-module comprises:
a second judging unit, configured to judge whether a code rate of the video stream uploaded by the target sending end is greater than the second code rate;
a second code rate reduction processing unit, configured to reduce the code rate of the video stream uploaded by the target sending end within a maximum code rate reduction range if the code rate of the video stream uploaded by the target sending end is greater than the second code rate, so that the code rate of the video stream uploaded by the target sending end is less than or equal to the second code rate, and forward the video stream uploaded by the target sending end to the receiving end;
and the video stream forwarding unit is used for forwarding the video stream uploaded by the target sending end to the receiving end if the code rate of the video stream uploaded by the target sending end is less than or equal to the second code rate.
The video streaming device can execute the video streaming method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects for executing the video streaming method.
EXAMPLE III
The third embodiment of the invention provides a video streaming transmission system, which comprises a sending end, a server and a receiving end. The transmitting end has the capability of generating multiple video streams, each of which may have different transmission bandwidths and resolutions. The sending terminal is used for collecting videos, encoding the videos, generating a plurality of video streams and uploading the video streams to the server. The server has the function of selecting and forwarding the video stream, and can select the corresponding video stream according to the video subscribed by the receiving end. The server is adapted to perform a video streaming method as provided in any of the above embodiments of the invention. The receiving end is used for receiving the video stream forwarded by the server. For example, the structure of the video streaming system has been described in detail in the foregoing embodiment and fig. 1B to 1D, and the embodiment of the present invention is not described herein again.
Example four
Fig. 3 is a schematic structural diagram of a server provided in the present application, which may be used to implement the video streaming method provided in any of the foregoing embodiments of the present invention. As shown in FIG. 3, the server is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 3, the server 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the server 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
A number of components in the server 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the server 10 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 11 performs the various methods and processes described above, for example, a video streaming method.
In some embodiments, the video streaming method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed on the server 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the video streaming method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the video streaming method by any other suitable means (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here may be implemented on a server having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (9)

1. A video streaming method applied to a server, the method comprising:
after a receiving end is accessed to display equipment or unloaded to the display equipment, receiving equipment information of the currently accessed display equipment of the receiving end, which is fed back by the receiving end, wherein the currently accessed display equipment is the display equipment which establishes communication with the receiving end after being accessed or unloaded;
determining the limited bandwidth for sending the video stream to the receiving end based on the equipment information of the display equipment currently accessed by the receiving end;
forwarding the target video stream from each sending end to the receiving end, wherein the sending end can generate a plurality of video streams with different code rates, and the sum of the code rates of the target video streams is matched with the limited bandwidth;
if the receiving end unloads the display device, the target video stream from the transmitting end is forwarded to the receiving end, and the method further comprises the following steps:
determining a first code rate of a first video stream displayed by an uninstalled target display device and a second code rate of the first video stream displayed by an uninstalled display device;
if the first code rate is greater than the second code rate, sending a third instruction to a target sending end of the first video stream, and determining a video stream to be uploaded according to the third instruction by the target sending end, wherein the video stream to be uploaded is a video stream with a code rate smaller than the second code rate of the first video stream;
and receiving the video stream uploaded by the target sending terminal and forwarding the video stream to the receiving terminal.
2. The video streaming method according to claim 1, wherein the device information includes a maximum transmission bit rate of each display device currently accessed by the receiving end, and determining a limited bandwidth for transmitting the video stream to the receiving end based on the device information of the display device currently accessed by the receiving end comprises:
and calculating the sum of the maximum transmission bit rates of all display equipment currently accessed by the receiving end to serve as the limited bandwidth for sending the video stream to the receiving end.
3. The video streaming method according to claim 1, wherein forwarding the target video stream from the transmitting end to the receiving end comprises:
sending a first instruction including the limited bandwidth to the sending end, wherein the sending end is used for determining a target video stream to be uploaded according to the first instruction;
receiving the target video streams sent by the sending ends, and judging whether the sum of the code rates of the target video streams sent by the sending ends is larger than the limited bandwidth or not;
if the sum of the target video streams is larger than the limited bandwidth, reducing the code rate of at least one target video stream within the maximum code rate reduction range, so that the sum of the code rates of the target video streams is smaller than or equal to the limited bandwidth, and forwarding the target video streams to the receiving end;
and if the sum of the code rates of the target video streams is less than or equal to the limited bandwidth, forwarding the target video streams to the receiving end.
4. The video streaming method according to claim 3, wherein reducing the bitrate of at least one target video stream within a maximum bitrate reduction range comprises:
and performing frame loss processing on at least one target video stream within the range of the maximum frame loss rate, or increasing the transmission delay of at least one target video stream within the range of the maximum delay.
5. The video streaming method according to any one of claims 1 to 4, wherein if the receiving end accesses a display device, forwarding a target video stream from a transmitting end to the receiving end, further comprising:
receiving video stream information of a video stream to be displayed of newly accessed target display equipment from the receiving end;
determining a target sending end for sending the video stream to be displayed according to the video stream information;
sending a second instruction including the maximum transmission bit rate of the target display equipment to the target sending end, wherein the target sending end is used for determining a target video stream to be uploaded according to the second instruction, and the target video stream of each target sending end is a video stream of which the sum of the code rates after the code rate reduction processing is smaller than or equal to the maximum transmission bit rate of the target display equipment;
receiving a target video stream of the target sending end, and judging whether the sum of code rates of the target video stream sent by the target sending end is greater than the maximum transmission bit rate of the target display equipment or not;
if the sum of the code rates of the target video streams sent by the target sending end is larger than the maximum transmission bit rate of the target display equipment, reducing the code rate of at least one target video stream within the range of the maximum code rate reduction so that the sum of the code rates of the target video streams sent by the target sending end is smaller than or equal to the maximum transmission bit rate of the target display equipment, and forwarding the target video streams to the receiving end;
and if the sum of the code rates of the target video streams sent by the target sending end is less than or equal to the maximum transmission bit rate of the target display equipment, forwarding the target video streams sent by the target sending end to a target receiving end.
6. The video streaming method according to claim 5, wherein receiving the video stream uploaded by the target sending end and forwarding the video stream to the receiving end comprises:
judging whether the code rate of the video stream uploaded by the target sending end is greater than the second code rate or not;
if the code rate of the video stream uploaded by the target sending end is greater than the second code rate, reducing the code rate of the video stream uploaded by the target sending end within the maximum code rate reduction range, so that the code rate of the video stream uploaded by the target sending end is smaller than or equal to the second code rate, and forwarding the video stream uploaded by the target sending end to the receiving end;
and if the code rate of the video stream uploaded by the target sending end is less than or equal to the second code rate, forwarding the video stream uploaded by the target sending end to the receiving end.
7. A video streaming system comprising a transmitting end, a server and a receiving end, the server being configured to perform the video streaming method according to any one of claims 1 to 6.
8. A server, comprising:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the video streaming method of any of claims 1-6.
9. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the video streaming method according to any one of claims 1-6.
CN202211119798.2A 2022-09-15 2022-09-15 Video stream transmission method, system, server and storage medium Active CN115209189B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211119798.2A CN115209189B (en) 2022-09-15 2022-09-15 Video stream transmission method, system, server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211119798.2A CN115209189B (en) 2022-09-15 2022-09-15 Video stream transmission method, system, server and storage medium

Publications (2)

Publication Number Publication Date
CN115209189A CN115209189A (en) 2022-10-18
CN115209189B true CN115209189B (en) 2023-03-10

Family

ID=83572863

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211119798.2A Active CN115209189B (en) 2022-09-15 2022-09-15 Video stream transmission method, system, server and storage medium

Country Status (1)

Country Link
CN (1) CN115209189B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116634194B (en) * 2023-05-10 2024-05-24 北京国际云转播科技有限公司 Video live broadcast method, video live broadcast device, storage medium and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102422258A (en) * 2009-05-06 2012-04-18 汤姆森许可贸易公司 Methods and systems for delivering multimedia content optimized in accordance with presentation device capabilities
CN103686221A (en) * 2013-12-31 2014-03-26 北京大洋润博科技有限公司 Audio/video smooth broadcasting method adaptive to public wireless network
CN105940680A (en) * 2013-09-11 2016-09-14 爱立信股份有限公司 Streaming policy management system and method
CN112312137A (en) * 2020-10-27 2021-02-02 唐桥科技(杭州)有限公司 Video transmission method and device, electronic equipment and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7921212B2 (en) * 2008-10-14 2011-04-05 At&T Intellectual Property I, L.P. Methods and apparatus to allocate bandwidth between video and non-video services in access networks
KR20210064116A (en) * 2018-09-28 2021-06-02 인텔 코포레이션 Transmission Control Video Coding

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102422258A (en) * 2009-05-06 2012-04-18 汤姆森许可贸易公司 Methods and systems for delivering multimedia content optimized in accordance with presentation device capabilities
CN105940680A (en) * 2013-09-11 2016-09-14 爱立信股份有限公司 Streaming policy management system and method
CN103686221A (en) * 2013-12-31 2014-03-26 北京大洋润博科技有限公司 Audio/video smooth broadcasting method adaptive to public wireless network
CN112312137A (en) * 2020-10-27 2021-02-02 唐桥科技(杭州)有限公司 Video transmission method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN115209189A (en) 2022-10-18

Similar Documents

Publication Publication Date Title
EP2863632B1 (en) System and method for real-time adaptation of a conferencing system to current conditions of a conference session
US7898950B2 (en) Techniques to perform rate matching for multimedia conference calls
US10397298B2 (en) Method and systems for optimizing bandwidth utilization in a multi-participant full mesh peer-to-peer video session
WO2020220902A1 (en) Method and apparatus for distributing transmission parameters of video resources
US8760490B2 (en) Techniques for a rate-adaptive video conference bridge
JP6442507B2 (en) How to allocate available network bandwidth during ongoing traffic sessions performed by devices on the network, corresponding devices
KR20180031547A (en) Method and apparatus for adaptively providing multiple bit rate stream media in server
CN114600468B (en) Combiner system, receiver device, computer-implemented method and computer-readable medium for combining video streams in a composite video stream with metadata
US9743043B2 (en) Method and system for handling content in videoconferencing
CN111031389A (en) Video processing method, electronic device and storage medium
CN115209189B (en) Video stream transmission method, system, server and storage medium
US8749612B1 (en) Reduced bandwidth usage in video conferencing
CN115529300A (en) System and method for automatically adjusting key frame quantization parameter and frame rate
US9232192B2 (en) Method and system for video conference snapshot presence
CN115767149A (en) Video data transmission method and device
CN115988269A (en) Video playing method, device, system, electronic equipment and storage medium
CN113259730B (en) Code rate adjustment method and device for live broadcast
US11290680B1 (en) High-fidelity freeze-frame for precision video communication applications
US11770431B2 (en) Network-adaptive live media encoding system
KR20170071251A (en) Multi-point control unit for providing conference service
CN113727183B (en) Live push method, apparatus, device, storage medium and computer program product
US20220303596A1 (en) System and method for dynamic bitrate switching of media streams in a media broadcast production
CN116996709A (en) Video streaming transmission method, system, server and storage medium
CN116489133A (en) Video playing method and device, electronic equipment and storage medium
CN118354133A (en) Video on-screen transcoding method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant