CN108400956A

CN108400956A - Distribution method, the device and system of video data stream

Info

Publication number: CN108400956A
Application number: CN201710067446.XA
Authority: CN
Inventors: 胡飞阳; 聂绪平; 钟敬辉
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2017-02-07
Filing date: 2017-02-07
Publication date: 2018-08-14

Abstract

The invention discloses a kind of distribution method of video data stream, device and system.Wherein, which includes：Server and at least two terminals, wherein at least two terminals are used for server upload media stream；Server matches corresponding Media Stream to be shown, and Media Stream to be shown is returned to corresponding terminal respectively respectively for receiving the Media Stream for carrying out self terminal at least two terminals.During the present invention is solved due to media controller or each terminal video stream of server forwarding, the low technical problem of media flow transmission efficiency in caused video conference.

Description

Video data stream distribution method, device and system

Technical Field

The present invention relates to the field of communication technology application, and in particular, to a method, an apparatus, and a system for allocating video data streams.

Background

With the development of communication technology, various conveniences are provided for people, such as shopping, eating, paying fees, and the like, particularly in the office field, from the telephone facsimile era, between companies and individuals, and between people and people, to the video era, even if two places are separated, even if they are located in different countries, two or more users can take a video conference through a network as long as the time is agreed.

The video stream transmission modes of the video conference used at present mainly include: two transfer modes, namely a server transfer mode and a Media Control Unit (MCU for short), are provided, taking a video conference with three-terminal access as an example:

in a first mode, using a server-forwarding video conference mode, fig. 1a is a schematic diagram of three video terminals sending and receiving video streams to and from a forwarding server, and as shown in fig. 1a, three video terminals A, B, C enter a meeting and each send a captured video stream A, B, C to the forwarding server. The forwarding server forwards B, C video streams to the terminal A according to the participants of the video conference; forwarding A, C the video stream to terminal B; the video stream of A, B is forwarded to terminal C. Fig. 1b shows a final display of the video terminal, and fig. 1b is a schematic diagram of a display of any one of the three video terminals.

In a second mode, using a video conference mode of MCU relay, fig. 1c is a schematic diagram of three video terminals transmitting and receiving video streams to and from a relay server, and the three video terminals A, B, C enter a meeting and respectively transmit a collected video stream A, B, C to the relay server, as shown in fig. 1 c. The forwarding server forwards both video streams A, B, C to the media control unit. The MCU decodes the video stream A, B, C into the yuv image format, switches the resolution of the yuv image format, mixes the video stream A, B, C into a yuv image, and re-encodes the mixed yuv image into a media stream. And finally, transmitting the media stream subjected to MUC screen mixing to a terminal A, a terminal B and a terminal C.

The display of the video terminal in the first or second mode will be as shown in fig. 1 b.

In the existing implementation process, a video conference mode transferred by the MCU is used, videos are synthesized and then sent to a terminal, in the actual use process, the operations of video decoding, screen mixing and encoding are added, and the network delay of the videos is increased. Meanwhile, the resource overhead of the server is increased, an additional MCU or coding and decoding equipment is needed, and the video conference cost is increased.

In the process of using the server forwarding mode, each terminal only uploads one path of media stream, but the specifications (code stream size and resolution level) of the media stream uploaded by each terminal are the same in the uploading process, and if each path of receiving stream receives the stream with the main picture specification, the receiving bandwidth is too large; if each receiving stream receives a stream with a non-main picture specification, the main picture is too fuzzy, and the user experience of participants is influenced.

In view of the above-mentioned problem in the prior art that the video conference display efficiency is low in the process of forwarding the video stream of each terminal by the media controller or the server, no effective solution has been proposed at present.

Disclosure of Invention

The embodiment of the invention provides a video data stream distribution method, a device and a system, which are used for at least solving the technical problem of low media stream transmission efficiency in a video conference caused by the process of forwarding video streams of all terminals by a media controller or a server.

According to an aspect of the embodiments of the present invention, there is provided a video data stream distribution system, including: the system comprises a server and at least two terminals, wherein the at least two terminals are used for uploading media streams to the server; and the server is used for receiving the media streams from the terminals, respectively matching the corresponding media streams to be displayed for at least two terminals, and respectively returning the media streams to be displayed to the corresponding terminals.

Optionally, the at least two terminals include: the system comprises an image acquisition device and a media stream processing device, wherein the image acquisition device is used for acquiring an image of the current environment of the terminal and/or an image displayed by a display device in the terminal; and the media stream processing device is connected with the image acquisition device and is used for encoding the image according to a preset encoding format to obtain at least two types of media streams, wherein the at least two types of media streams are used for distributing the media streams corresponding to the resolution and the code stream by the server according to the display capability and the bandwidth transmission state of different terminals.

Further, optionally, the image capturing device includes: the device is externally connected with a camera shooting and collecting device and/or embedded with window intercepting software.

Optionally, the server includes: the system comprises a first receiving unit, a first analyzing unit and a first distributing unit, wherein the first receiving unit is used for receiving a first media stream obtaining request sent by a first terminal and a second media stream obtaining request sent by a second terminal in at least two terminals; the first analysis unit is connected with the first receiving unit and used for acquiring the equipment type and the bandwidth transmission state of the first terminal and the equipment type and the bandwidth transmission state of the second terminal according to the received first media stream acquisition request and the second media stream acquisition request; and the first distribution unit is connected with the first analysis unit and used for respectively distributing the corresponding media streams to be displayed to the first terminal and the second terminal according to the equipment type and the bandwidth transmission state of the first terminal and the equipment type and the bandwidth transmission state of the second terminal.

Optionally, the server includes: the second receiving unit is used for receiving a first media stream acquisition request sent by a first terminal of the at least two terminals and a second media stream acquisition request sent by a second terminal; the second analysis unit is connected with the second receiving unit and used for acquiring the display types of the media streams required by the first terminal and the second terminal according to the first media stream acquisition request and the second media stream acquisition request; and the second distribution unit is connected with the second analysis unit and used for distributing the corresponding media streams to be displayed to the first terminal and the second terminal according to the display type.

Optionally, the media stream includes: one or a combination of at least two of audio, video, images or text.

According to another aspect of the embodiments of the present invention, there is also provided a method for allocating video data streams, including: uploading the multiple media streams to a server, wherein the uploading the multiple media streams to the server comprises: acquiring an image through an image acquisition device to generate a media stream; coding the media stream according to a preset coding format to obtain a plurality of paths of media streams; uploading the multi-path media stream to a server; the multi-path media stream is used for distributing media streams corresponding to resolution and code streams by the server according to the display capability and bandwidth transmission state of different terminals; sending a media stream acquisition request to a server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; receiving a media stream to be displayed distributed by a server; and displaying the media stream to be displayed.

According to another aspect of the embodiments of the present invention, there is provided another video data stream allocation method, including: uploading a plurality of paths of media streams to a server, wherein the plurality of paths of media streams are used for distributing media streams corresponding to resolution and code streams by the server according to the display capacities and bandwidth transmission states of different terminals; sending a media stream acquisition request to a server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; receiving a media stream to be displayed distributed by a server; and displaying the media stream to be displayed.

Optionally, uploading the multiple media streams to the server includes: acquiring an image through an image acquisition device to generate a media stream; coding the media stream according to a preset coding format to obtain a plurality of paths of media streams; and uploading the multi-path media stream to a server.

Optionally, before sending the media stream acquisition request to the server, the method further includes: and encapsulating the equipment type and/or the bandwidth transmission state to obtain a media stream acquisition request.

Optionally, before sending the media stream acquisition request to the server, the method further includes: packaging the media stream type for displaying on the main picture to obtain a media stream acquisition request, wherein the media stream type comprises: the terminal identification and the current display resolution in the participating video.

According to another aspect of the embodiments of the present invention, there is also provided a method for allocating video data streams, including: receiving a plurality of paths of media streams uploaded by each terminal; receiving a media stream acquisition request sent by each terminal, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; and distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal.

According to another aspect of the embodiments of the present invention, there is provided another video data stream allocation method, including: receiving a plurality of paths of media streams uploaded by each terminal; receiving a media stream acquisition request sent by each terminal, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: terminal identification and current display resolution.

According to another aspect of the embodiments of the present invention, there is also provided a method for allocating video data streams, including: receiving a plurality of paths of media streams uploaded by each terminal; receiving a media stream acquisition request sent by each terminal, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: a terminal identifier and a current display resolution; and sending the media stream to be displayed carrying the equipment type and/or the bandwidth transmission state corresponding to each terminal, or the media stream to be displayed appointed by each terminal acquired according to the media stream acquisition request to each terminal.

According to still another aspect of the embodiments of the present invention, there is also provided a video data stream distribution apparatus, including: the first uploading module is used for uploading the multi-path media stream to the server, wherein the uploading the multi-path media stream to the server comprises: acquiring an image through an image acquisition device to generate a media stream; coding the media stream according to a preset coding format to obtain a plurality of paths of media streams; uploading the multi-path media stream to a server; the multi-path media stream is used for distributing media streams corresponding to resolution and code streams by the server according to the display capability and bandwidth transmission state of different terminals; the first sending module is used for sending a media stream obtaining request to the server, wherein the media stream obtaining request is a request for obtaining media streams of other terminals in the video; the first receiving module is used for receiving the media stream to be displayed distributed by the server; the first display module is used for displaying the media stream to be displayed.

According to still another aspect of the embodiments of the present invention, there is provided another video data stream distribution apparatus, including: the second uploading module is used for uploading the multi-path media streams to the server, wherein the multi-path media streams are used for distributing the media streams corresponding to the resolution and the code streams by the server according to the display capabilities and the bandwidth transmission states of different terminals; the second sending module is used for sending a media stream acquisition request to the server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; the second receiving module is used for receiving the media stream to be displayed distributed by the server; and the second display module is used for displaying the media stream to be displayed.

According to an aspect of another embodiment of the present invention, there is provided another video data stream distribution apparatus, including: the third receiving module is used for receiving the multi-path media streams uploaded by each terminal; a fourth receiving module, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals; and the first distribution module is used for distributing the corresponding media stream to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal.

According to an aspect of another embodiment of the present invention, there is provided a video data stream distribution apparatus, including: the fifth receiving module is used for receiving the multi-path media streams uploaded by each terminal; a sixth receiving module, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals; the second distribution module is used for distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: terminal identification and current display resolution.

According to an aspect of another embodiment of the present invention, there is also provided another video data stream distribution apparatus, including: the seventh receiving module is used for receiving the multi-path media streams uploaded by each terminal; an eighth receiving module, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals; the third distribution module is used for distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: a terminal identifier and a current display resolution; and the third sending module is used for sending the media stream to be displayed carrying the equipment type and/or the bandwidth transmission state corresponding to each terminal, or the media stream to be displayed appointed by each terminal, which is obtained according to the media stream obtaining request, to each terminal.

In the embodiment of the invention, the media stream is uploaded to the server through the server and at least two terminals; the server is used for receiving the media streams from the terminals, respectively matching the corresponding to-be-displayed media streams for at least two terminals, and respectively returning the to-be-displayed media streams to the corresponding terminals, so that the purpose of distributing the corresponding media streams according to the terminal requirements is achieved, the technical effect of distributing the corresponding media streams according to the terminal display capacity is achieved, and the technical problem of low media stream transmission efficiency in a video conference caused by the fact that a media controller or the server forwards the video streams of the terminals is solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1a is a schematic diagram of three video terminals transceiving video streams to a forwarding server;

FIG. 1b is a schematic diagram of a display screen of any one of three video terminals;

FIG. 1c is a schematic diagram of three video terminals transmitting and receiving video streams to a forwarding server mixed by an MCU;

FIG. 2 is a schematic diagram of a video data stream distribution system according to an embodiment of the present application;

FIG. 3 is an interactive schematic diagram of a video data stream distribution system according to an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a video data stream distribution system according to an embodiment of the present application;

FIG. 5 is a flowchart of a method for allocating video data streams according to the second embodiment of the present application;

FIG. 6 is a flowchart of a method for allocating video data streams according to a third embodiment of the present application;

FIG. 7 is a flowchart of a method for allocating video data streams according to the fourth embodiment of the present application;

fig. 8 is a flowchart of a method for allocating video data streams according to an embodiment of the present application;

fig. 9 is a flowchart of a method for allocating video data streams according to a sixth embodiment of the present application;

FIG. 10 is a schematic structural diagram of an apparatus for distributing video data streams according to an embodiment of the present application

Fig. 11 is a schematic structural diagram of an apparatus for distributing video data streams according to an embodiment eight of the present application;

FIG. 12 is a schematic structural diagram of an apparatus for distributing video data streams according to an embodiment nine of the present application;

fig. 13 is a schematic structural diagram of an apparatus for distributing video data streams according to an embodiment of the present application;

fig. 14 is a schematic structural diagram of an apparatus for distributing video data streams according to an eleventh embodiment of the present application.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, some terms or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

media stream: multimedia data stream collected by the terminal side. The multimedia data stream may include: any one or combination of at least two of images, text or sound, such as an audio-visual signal.

Example 1

In accordance with an embodiment of the present application, there is provided an embodiment of a video data stream distribution system, it should be noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than that illustrated herein.

The distribution system of video data streams provided in the embodiments of the present application may be applicable to a video conference, where the data streams distributed in the distribution system are embodied as media streams, and the distribution system includes: the server and at least two terminals, upload the multi-channel media stream of their own terminal to the server separately by at least two terminals, in the course of the video conference, the server obtains the media stream that the request distributes the correspondent media stream according to the media stream of each terminal, thus has avoided appearing in the prior art because each terminal uploads the media stream of the same specification to cause, when the server distributes each media stream, because of the influence of terminal type and transmission bandwidth, distribute the media stream of the same specification to the problem that the transmission pressure of the bandwidth that the terminal locates is big when different terminals cause, wherein, the above-mentioned specification can include: codestream size and resolution level.

Fig. 2 is a schematic structural diagram of a video data stream distribution system according to an embodiment of the present application, and as shown in fig. 2, the video data stream distribution system includes: a server 21 and at least two terminals 22, wherein,

at least two terminals 22 for uploading media streams to the server 21; the server 21 is connected to the at least two terminals 22, and is configured to receive the media streams from the terminals, match corresponding to-be-displayed media streams for the at least two terminals 22, and return the to-be-displayed media streams to the corresponding terminals respectively.

It should be noted that, in the embodiment of the present application, the at least two terminals 22 may be a notebook Computer, a Personal Computer (PC) smart phone, a smart television, a tablet Computer, a handheld business, and an intelligent wearable device, such as a Virtual Reality device (VR) or an Augmented Reality device (AR). The server 21 is a server in which the media control unit MCU is not embedded.

Specifically, the distribution system of video data streams provided in the embodiments of the present application is different from the prior art in that, firstly, in terms of the server structure, the server in the embodiments of the present application does not embed an MCU or does not need to externally connect a device carrying an MCU, and a terminal can obtain a media stream requested by a media stream to display when each terminal locally displays according to a preset setting, that is, operations of video decoding, screen mixing and encoding that need to be performed in an MCU in the prior art are omitted, and these operations increase the network delay of a video. Meanwhile, the resource overhead of the server is increased, and an additional MCU or coding and decoding equipment is needed, so that the video conference cost is increased.

Secondly, in media stream transmission, at each terminal side, the collected video data is generated into media streams of various levels, for example, 4k standard (2048 resolution), 1080P resolution or 720P resolution, and classified coding compression is performed according to different resolutions to obtain media streams of different resolution levels, or, in order to ensure normal transmission of bandwidth, the video streams are coded and compressed according to different sizes of code streams to obtain media streams of different sizes of code streams.

For example, in the case of uploading in two paths on the terminal, one of the media streams may be set to have a resolution of 1280 × 720 and a bit rate of 2000 kbps; setting the resolution of the other path of media stream to be 160 × 90 and the code rate to be 56 kbps;

under the condition of uploading in three paths, setting the resolution of the first path of media stream to be 1280 x 720 and the code rate to be 2000 kbps; the resolution of the second path of media stream is set to 640 x 360, and the code rate is 512 kbps; the third media stream sets the resolution to 160 x 90 and the bitrate to 56 kbps.

Finally, the multi-path media streams are uploaded to a server, in the process of the video conference, the server allocates the corresponding media streams according to the indication in the media stream acquisition request sent by the terminal, or allocates the corresponding media streams according to the terminal identification to be displayed and the transmission state of the current bandwidth carried in the media stream acquisition request, the server allocates the corresponding media streams according to the terminal identification and the transmission state of the current bandwidth, the overhead on bandwidth resources can be saved in the media stream transmission and the terminal display, or the waste of the media streams in the display process is avoided, for example, the media streams with 1080P or 2048P image quality are displayed on the terminal capable of displaying 720P image quality, because the terminal can only display the image with 720P effect, the terminal intelligently compresses the resolution to the effect of 720P by compressing the media streams with high code rate, the waste of the bandwidth resources can be caused in the acquisition process of the media streams, it is also a waste in display.

The different levels of the multiple media streams are merely illustrative, and are not limited to the implementation of the distribution system for video data streams provided in the embodiments of the present application.

The technical problem to be solved by the video data stream distribution system provided by the embodiment of the application belongs to the field of video application, can be suitable for a video conference, and solves the problems that in the video application, due to too many people participating in video sharing, if a media stream with uniform code stream and resolution ratio is downloaded according to the prior art, the CPU calculation pressure of a video conference terminal is too high, the memory consumption is high, the network bandwidth occupies a lot, and the network delay is serious.

The video data stream distribution system provided by the embodiment of the application reduces the calculation pressure of a terminal CPU, reduces the memory consumption and saves the network bandwidth, because the corresponding media stream is adapted through the server, the bandwidth is saved in comparison with the bandwidth obtained in the prior art, and because the corresponding media stream is adapted to the terminal, the resources inside the terminal are saved in the terminal playing process, for example, the storage pressure and the calculation pressure (the calculation pressure in the decoding process) are reduced; in addition, the frequency mixing process of the MCU or the server carrying the MCU is omitted in the embodiment of the application, so that the problem of network delay is solved.

In the distribution system of the video data stream provided by the embodiment of the application, the system comprises a server and at least two terminals, wherein the at least two terminals are used for uploading the media stream to the server; the server is used for receiving the media streams from the terminals, respectively matching the corresponding to-be-displayed media streams for at least two terminals, and respectively returning the to-be-displayed media streams to the corresponding terminals, so that the purpose of distributing the corresponding media streams according to the terminal requirements is achieved, the technical effect of distributing the corresponding media streams according to the terminal display capacity is achieved, and the technical problem of low media stream transmission efficiency in a video conference caused by the fact that a media controller or the server forwards the video streams of the terminals is solved.

Optionally, the at least two terminals 22 include:

the system comprises an image acquisition device and a media stream processing device, wherein the image acquisition device is used for acquiring an image of the current environment of the terminal and/or an image displayed by a display device in the terminal; and the media stream processing device is connected with the image acquisition device and is used for encoding the image according to a preset encoding format to obtain at least two types of media streams, wherein the at least two types of media streams are used for distributing the media streams corresponding to the resolution and the code stream by the server according to the display capability and the bandwidth transmission state of different terminals.

The distribution system of video data streams provided in this embodiment of the present application may take at least two terminals as an example for description, where it is to be noted that each terminal at least includes the image capturing device and a media stream output device, and based on the above hierarchical encoding and compression of the media streams, an image of an environment where the current terminal is located is captured by the image capturing device, and/or an image displayed by a display device in the terminal is encoded by the media stream processing device according to a preset encoding format, so as to obtain at least two types of media streams.

It should be noted that the image displayed by the display device may be an application window for displaying various types of text, for example: word documents, PPT, excel tables, auto CAD, VISO, and the like.

The distribution system of the video data streams provided by the embodiment of the application is suitable for a video conference, and can shoot the current environment where a certain terminal is located in a conventional sense in the process of obtaining the media streams, obtain the media streams according to the shot pictures, and also obtain the application window currently displayed by the terminal through a device embedded with window capturing software, so that the text image spoken by the current speaker is displayed when the video conference personally speaks.

Optionally, the server 21 includes: the system comprises a first receiving unit, a first analyzing unit and a first distributing unit, wherein the first receiving unit is used for receiving a first media stream obtaining request sent by a first terminal and a second media stream obtaining request sent by a second terminal in at least two terminals; the first analysis unit is connected with the first receiving unit and used for acquiring the equipment type and the bandwidth transmission state of the first terminal and the equipment type and the bandwidth transmission state of the second terminal according to the received first media stream acquisition request and the second media stream acquisition request; and the first distribution unit is connected with the first analysis unit and used for respectively distributing the corresponding media streams to be displayed to the first terminal and the second terminal according to the equipment type and the bandwidth transmission state of the first terminal and the equipment type and the bandwidth transmission state of the second terminal.

Optionally, the server 21 includes: the second receiving unit is used for receiving a first media stream acquisition request sent by a first terminal of the at least two terminals and a second media stream acquisition request sent by a second terminal; the second analysis unit is connected with the second receiving unit and used for acquiring the display types of the media streams required by the first terminal and the second terminal according to the first media stream acquisition request and the second media stream acquisition request; and the second distribution unit is connected with the second analysis unit and used for distributing the corresponding media streams to be displayed to the first terminal and the second terminal according to the display type.

Fig. 3 is an interaction schematic diagram of a video data stream distribution system according to an embodiment of the present application, where, on the server 21 side, as shown in fig. 3, when distributing a media stream, two cases are divided:

in the first situation, distributing corresponding media streams to be displayed to each terminal according to the respective equipment type and bandwidth transmission state of each terminal; taking two terminals as an example, the terminal can be a mobile terminal,

s1, in the video conference process, the first terminal sends a first media stream acquisition request to the server; the second terminal sends a second media stream acquisition request to the server;

s2, the server analyzes according to the first media stream acquisition request and the second media stream acquisition request to respectively obtain the equipment type and the bandwidth transmission state of the first terminal and the equipment type and the bandwidth transmission state of the second terminal;

and S3, the server allocates corresponding media streams to be displayed according to the device type and the bandwidth transmission state of the first terminal and the device type and the bandwidth transmission state of the second terminal.

It should be noted that, in step S2, the device type and the bandwidth transmission state of the first terminal and the device type and the bandwidth transmission state of the second terminal obtained by the server may be obtained when a communication connection is established, or may be carried in respective media stream acquisition requests sent by the terminals, where the present application is not limited to implement the distribution system of the video conference data stream provided in the embodiment of the present application.

In case two, distributing corresponding media streams to be displayed according to the media streams required by the appointed terminals; still by way of example in terms of the two terminals described above,

s2, the server obtains the display type of the media stream needed by the first terminal and the second terminal according to the first media stream obtaining request and the second media stream obtaining request;

and S3, the server allocates corresponding media streams to be displayed to the first terminal and the second terminal respectively according to the display types of the media streams required by the first terminal and the second terminal.

The foregoing example takes two terminals as an example for description, and three or even more terminals are implemented in the same manner in the video data stream distribution system provided in the embodiment of the present application, and are not described herein again.

To sum up, in the distribution system of video data streams provided in the embodiment of the present application, fig. 4 is a schematic structural diagram of the distribution system of video data streams provided in an embodiment of the present application, and as shown in fig. 4, after each terminal camera acquires video data, the camera data is sent to two or more streams. And setting the resolution, code rate and frame rate of each path of media stream according to the requirement. The configuration shown in fig. 4 will be described with three terminals as an example.

In practical use, the media stream of the main picture a in fig. 1b needs to be set to have higher resolution and bitrate. When setting the media streams of the change B and the picture C in fig. 1B, only relatively low resolution and code rate are needed. The aspect ratio of the resolution of the multi-path stream needs to be kept consistent, and better user experience is achieved when the main picture and the small picture of the video are switched.

Therefore, when the terminal uploads, two paths of media streams or more paths of media streams can be uploaded at the same time, so that the receiving terminal can conveniently select; when the user changes the main picture, the media stream required to be displayed is switched in real time to complete the function.

When two paths of streams are uploaded at present, the resolution of one path of media stream is set to 1280 × 720, and the code rate is 2000 kbps; the other media stream sets the resolution to 160 x 90 and the bitrate to 56 kbps.

The three-way flow uploading time can be set as follows: setting the resolution of the first path of media stream to be 1280 x 720 and the code rate to be 2000 kbps; the resolution of the second path of media stream is set to 640 x 360, and the code rate is 512 kbps; the third media stream sets the resolution to 160 x 90 and the bitrate to 56 kbps.

After the two or three streams of the terminal a are uploaded to the forwarding server, the terminal B and the terminal C may select which one of the streams uploaded by the terminal a is used as the currently displayed media stream. And those media streams not displayed, terminal B and terminal C do not have to be downloaded. According to the actual requirement, if the media stream is displayed on the main picture, the media stream with high resolution and code rate is selected, so that better user experience is achieved; if the media stream is not displayed on the main picture, the media stream with low resolution and code rate is selected, so that the network bandwidth is reduced, and the expenses of a CPU (central processing unit) and a memory of the terminal are reduced.

The embodiment provided by the application uploads a plurality of paths of videos on the same picture by adopting a method of uploading two or more paths of streams simultaneously, and grades the resolution and definition of the videos. Each receiving terminal selects the stream to be received according to the self requirement, and the resolution and code rate of the picture B and the picture C are reduced while the high resolution and the definition of the main picture are met. And finally, better user experience is achieved, and meanwhile, network bandwidth, CPU and memory cost are occupied as little as possible.

Example 2

In the above operating environment, the present application provides a method for distributing video data streams as shown in fig. 5. On the terminal side, fig. 5 is a flowchart of a video data stream allocation method according to the second embodiment of the present application.

Step S502, uploading multi-path media streams to a server, wherein the multi-path media streams are used for distributing media streams corresponding to resolution and code streams by the server according to the display capability and bandwidth transmission state of different terminals;

in the foregoing step S502 of the present application, based on the operating environment provided in the first embodiment, the method for allocating video conference data streams provided in the first embodiment of the present application may be applied to a terminal side, where the terminal uploads multiple media streams to a server.

Specifically, based on the first embodiment, in this embodiment, multiple media streams uploaded by the terminal are video streams obtained by the terminal through an image acquisition device, the same video stream is compressed according to different pixel and/or code stream size standards in a manner of a preset encoding format, multiple media streams corresponding to different pixel and code stream size standards are obtained, the obtained multiple media streams are determined to be multiple media streams, and the multiple media streams are uploaded to the server.

Step S504, sending a media stream acquisition request to a server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals;

in the above step S504, in the video conference process, that is, before the video conference starts, or in the process of interacting with other terminals after the video conference, for example, in order to communicate with other terminals, the terminal sends a media stream acquisition request to the server, so that the server allocates corresponding media streams according to the media stream acquisition request.

Step S506, receiving a media stream to be displayed distributed by a server;

in step S506, the terminal receives a to-be-displayed media stream returned by the server based on the media stream acquisition request sent to the server in step S504.

Step S508, displaying the media stream to be displayed.

In the above step S508, based on the to-be-displayed media stream returned by the server received in the above step S506, the terminal displays the to-be-displayed media stream.

Specifically, after the terminal sends the media stream acquisition request to the server, the media stream to be displayed returned by the server may be the media stream to be displayed of at least one terminal, that is, assuming that the current terminal is terminal a, after the terminal a sends the media stream acquisition request to the server, the media stream to be displayed returned by the server may include the media stream to be displayed of the terminal participating in the current video conference, such as terminal B, terminal C, or terminal D, so as to display the picture shown in fig. 1B on the terminal a side.

With reference to steps S502 to S508, in the terminal provided in this embodiment, in a video conference scene, the execution steps of each terminal are the same as those of steps S502 to S508, and are not described herein again.

In the method for allocating video data streams provided by the embodiment of the application, multiple paths of media streams are uploaded to a server, wherein the multiple paths of media streams are used for allocating media streams corresponding to resolution and code streams by the server according to the display capabilities and bandwidth transmission states of different terminals; sending a media stream acquisition request to a server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; receiving a media stream to be displayed distributed by a server; the media stream to be displayed is displayed, the purpose of distributing the corresponding media stream according to the terminal requirements is achieved, the technical effect of distributing the corresponding media stream according to the terminal display capacity is achieved, and the technical problem that the transmission efficiency of the media stream in the video conference is low due to the fact that a media controller or a server forwards the video stream of each terminal is solved.

Optionally, the uploading the multiple media streams to the server in step S502 includes:

step1, collecting images through an image collecting device to generate a media stream;

step2, encoding the media stream according to a preset encoding format to obtain a plurality of paths of media streams;

step3, uploading the multi-path media stream to the server.

Specifically, based on the process of respectively encoding media streams by the terminal in the first embodiment, the collected video data is generated into media streams of various levels, for example, 4k standard (2048 resolution), 1080P resolution, or 720P resolution, and classified encoding compression is performed according to different resolutions to obtain media streams of different resolution levels, or, to ensure normal transmission of bandwidth, the video streams are encoded and compressed according to different sizes of code streams to obtain media streams of different sizes of code streams.

It should be noted that, the above example is only for explaining a process of uploading multiple media streams to a server, so as to implement the allocation method of video data streams provided in the embodiment of the present application, and is not limited specifically.

Optionally, before sending the media stream acquisition request to the server in step S504, the method for allocating video data streams provided in the embodiment of the present application further includes:

step S503, the device type and/or the bandwidth transmission state are encapsulated to obtain the media stream acquisition request.

step S503', encapsulating the media stream type for displaying on the main screen to obtain a media stream acquisition request, wherein the media stream type includes: the terminal identification and the current display resolution in the participating video conference.

Specifically, the step S503 and the step S503' are divided into two different cases of the media stream acquisition request of the terminal, which respectively correspond to two cases of the server distributing the media stream in the first embodiment:

the first condition is as follows: packaging the device type and/or the bandwidth transmission state to obtain a media stream acquisition request;

for example, the device type of the current terminal is any one of a laptop, a Personal Computer (PC) smart phone, a smart television, a tablet PC, a handheld business and an intelligent wearable device, such as a Virtual Reality device (VR) or an Augmented Reality device (AR); and/or testing the bandwidth transmission state (link state) between the current terminal and the server through communication interaction with the server, if the current bandwidth transmission state is lower than a preset transmission standard, feeding back the current test result to the server in the media stream acquisition request, and judging the type of media streams to be displayed which are allocated to the current terminal by the server.

Case two: packaging the media stream type for displaying on the main picture to obtain a media stream acquisition request;

for example, the current terminal directly indicates the type of the media stream displayed by the main screen of the current terminal to the server and the display media streams of other display windows, that is, assuming that the current terminal a indicates the current main screen to the server as the media stream of the terminal B to be displayed, and because the link transmission rate between the current terminal a and the server is slow, the bandwidth for downloading the media stream of the terminal B is low, the current terminal a indicates the server to acquire the media stream of the terminal B that meets the current display of the terminal a, and the above criteria are encapsulated to obtain the media stream acquisition request.

Example 3

In the above operating environment, the present application provides a method for allocating video data streams as shown in fig. 6. On the terminal side, fig. 6 is a flowchart of a video data stream allocation method according to the third embodiment of the present application.

Step S602, uploading multiple media streams to a server, where uploading multiple media streams to the server includes: acquiring an image through an image acquisition device to generate a media stream; coding the media stream according to a preset coding format to obtain a plurality of paths of media streams; uploading the multi-path media stream to a server; the multi-path media stream is used for distributing media streams corresponding to resolution and code streams by the server according to the display capability and bandwidth transmission state of different terminals;

step S604, sending a media stream acquisition request to a server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals;

step S606, receiving the media stream to be displayed distributed by the server;

step S608, displaying the media stream to be displayed.

In the method for distributing video data streams provided in the embodiment of the present application, uploading multiple media streams to a server, where uploading multiple media streams to the server includes: acquiring an image through an image acquisition device to generate a media stream; coding the media stream according to a preset coding format to obtain a plurality of paths of media streams; uploading the multi-path media stream to a server; the multi-path media stream is used for distributing media streams corresponding to resolution and code streams by the server according to the display capability and bandwidth transmission state of different terminals; sending a media stream acquisition request to a server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; receiving a media stream to be displayed distributed by a server; the media stream to be displayed is displayed, the purpose of distributing the corresponding media stream according to the terminal requirements is achieved, the technical effect of distributing the corresponding media stream according to the terminal display capacity is achieved, and the technical problem that the transmission efficiency of the media stream in the video conference is low due to the fact that a media controller or a server forwards the video stream of each terminal is solved.

Example 4

Under the above operating environment, the present application provides a method for distributing video data streams as shown in fig. 7. On the server side, fig. 7 is a flowchart of a video data stream distribution method according to the fourth embodiment of the present application.

Step S702, receiving a plurality of paths of media streams uploaded by each terminal;

in the foregoing step S702 of the present application, based on the operating environment provided in the first embodiment, the method for allocating video data streams provided in the first embodiment of the present application may be applied to a server side, where the server receives multiple media streams uploaded by each terminal.

Specifically, as mentioned in embodiment 2, each terminal uploads the media stream to the server in a multi-path uploading manner, so that the media stream uploaded by one terminal is connected to media streams of the same media content and different types on the server side.

Step S704, receiving a media stream acquisition request sent by each terminal, wherein the media stream acquisition request is a request for acquiring media streams of other terminals;

in step S704, in the video conference process, the server receives a media stream acquisition request sent by each terminal.

Step S706, according to the media stream acquisition request and the multiple media streams uploaded by each terminal, allocating corresponding media streams to be displayed to each terminal.

In step S706, based on the media stream acquisition request received in step S704, the server obtains the requirement of each terminal by analyzing the media stream acquisition request sent by each terminal, and then allocates a corresponding to-be-displayed media stream to each terminal according to the requirement.

In the method for distributing video data streams provided by the embodiment of the application, multiple paths of media streams uploaded by each terminal are received; in the video conference process, receiving a media stream acquisition request sent by each terminal, wherein the media stream acquisition request is a request for acquiring media streams of other terminals participating in the video conference; and distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request. The purpose of distributing the corresponding media streams according to the terminal requirements is achieved, the technical effect of distributing the corresponding media streams according to the terminal display capacity is achieved, and the technical problem that the transmission efficiency of the media streams in the video conference is low due to the fact that the media controller or the server forwards the video streams of the terminals is solved.

Example 5

Under the above operating environment, the present application provides a method for allocating video data streams as shown in fig. 8. On the server side, fig. 8 is a flowchart of a method for allocating video data streams according to the fifth embodiment of the present application.

Step S802, receiving a plurality of paths of media streams uploaded by each terminal;

step S804, receiving a media stream acquisition request sent by each terminal, wherein the media stream acquisition request is a request for acquiring media streams of other terminals;

step 806, distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal;

the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps:

analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state;

or,

analyzing the media stream acquisition request, and acquiring the media stream types used by each terminal for displaying in the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: terminal identification and current display resolution.

Specifically, the step S806 of allocating the corresponding to-be-displayed media stream to each terminal according to the media stream acquisition request includes:

step1, analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal;

step2, allocating corresponding media streams to be displayed to each terminal according to the device type and/or bandwidth transmission state.

Or,

step 1', analyzing the media stream obtaining request, obtaining the media stream type used by each terminal in the main picture display;

step 2', distributing corresponding media stream to be displayed to each terminal according to the media stream type, wherein the media stream type comprises: the terminal identification and the current display resolution in the participating video conference.

Specifically, step1 and step2 in step S806 in the present embodiment correspond to step S503 and step S503' in step S806 in the second embodiment, in the step S503 corresponding to the terminal in the second embodiment, on the server side; in step S503 ' corresponding to the terminal in the second embodiment, step1 ' and step2 ' in step S806 in this embodiment correspond.

That is, in the first embodiment, when the server distributes the media stream, the first case and the second case are divided.

Here, this embodiment is different from embodiment 4, in which the server allocates different media streams to the terminals according to the media stream acquisition requests sent by the terminals and according to different instructions in the media stream acquisition requests.

In the method for distributing video data streams provided by the embodiment of the application, multiple paths of media streams uploaded by each terminal are received; receiving a media stream acquisition request sent by each terminal, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: terminal identification and current display resolution. The purpose of distributing the corresponding media streams according to the terminal requirements is achieved, the technical effect of distributing the corresponding media streams according to the terminal display capacity is achieved, and the technical problem that the transmission efficiency of the media streams in the video conference is low due to the fact that the media controller or the server forwards the video streams of the terminals is solved.

Example 6

Under the above operating environment, the present application provides a method for distributing video data streams as shown in fig. 9. On the server side, fig. 9 is a flowchart of a video data stream allocation method according to a sixth embodiment of the present application.

Step S902, receiving a plurality of paths of media streams uploaded by each terminal;

step S904, receiving a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals;

step S906, distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: a terminal identifier and a current display resolution;

step S908 is to send the to-be-displayed media stream carrying the device type and/or bandwidth transmission status corresponding to each terminal, or the to-be-displayed media stream designated by each terminal and acquired according to the media stream acquisition request, to each terminal.

In this embodiment, different from embodiment 5, on the server side, after allocating a to-be-displayed media stream to each terminal, the server returns the to-be-displayed media stream to each terminal.

In the method for distributing video data streams provided by the embodiment of the application, multiple paths of media streams uploaded by each terminal are received; receiving a media stream acquisition request sent by each terminal, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: a terminal identifier and a current display resolution; and sending the media stream to be displayed carrying the equipment type and/or the bandwidth transmission state corresponding to each terminal, or the media stream to be displayed appointed by each terminal acquired according to the media stream acquisition request to each terminal. The purpose of distributing the corresponding media streams according to the terminal requirements is achieved, the technical effect of distributing the corresponding media streams according to the terminal display capacity is achieved, and the technical problem that the transmission efficiency of the media streams in the video conference is low due to the fact that the media controller or the server forwards the video streams of the terminals is solved.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

Through the above description of the embodiments, those skilled in the art can clearly understand that the distribution method of the video conference data stream according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

Example 7

According to an embodiment of the present invention, an embodiment of an apparatus for implementing the foregoing method embodiment is also provided.

Fig. 10 is a schematic structural diagram of an apparatus for distributing video data streams according to an embodiment of the present application.

As shown in fig. 10, the video data stream distribution apparatus may include: a second upload module 1002, a second transmit module 1004, a second receive module 1006, and a second display module 1008.

The second uploading module 1002 is configured to upload multiple media streams to the server, where the multiple media streams are used for the server to allocate media streams corresponding to resolutions and code streams according to display capabilities and bandwidth transmission states of different terminals; a second sending module 1004, configured to send a media stream obtaining request to the server, where the media stream obtaining request is a request for obtaining media streams of other terminals; a second receiving module 1006, configured to receive a to-be-displayed media stream distributed by a server; and a second display module 1008, configured to display the media stream to be displayed.

In the embodiment of the application, the second uploading module is used for uploading a plurality of paths of media streams to the server, wherein the plurality of paths of media streams are used for allocating media streams corresponding to resolution and code streams by the server according to the display capabilities and bandwidth transmission states of different terminals; the second sending module is used for sending a media stream acquisition request to the server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; the second receiving module is used for receiving the media stream to be displayed distributed by the server; the second display module is used for displaying the media stream to be displayed, and the purpose of executing corresponding data processing according to the node structure of the event is achieved, so that the technical effect of improving the processing efficiency of the network platform event is achieved, and the technical problem of low media stream transmission efficiency in a video conference caused by the fact that a media controller or a server forwards video streams of all terminals is solved.

It should be noted here that the second uploading module 1002, the second sending module 1004, the second receiving module 1006, and the second displaying module 1008 correspond to steps S502 to S508 in the second embodiment, and the four modules are the same as the corresponding steps in the implementation example and application scenarios, but are not limited to the disclosure in the second embodiment. It should be noted that the above four modules as a part of the apparatus may operate in the terminal provided in the second embodiment, and may be implemented by software or hardware.

Example 8

Fig. 11 is a schematic structural diagram of an apparatus for distributing video data streams according to an embodiment eight of the present application.

As shown in fig. 11, the video data stream distribution apparatus may include: a first upload module 1102, a first transmit module 1104, a first receive module 1106, and a first display module 1108.

The first uploading module 1102 is configured to upload multiple media streams to a server, where uploading multiple media streams to the server includes: acquiring an image through an image acquisition device to generate a media stream; coding the media stream according to a preset coding format to obtain a plurality of paths of media streams; uploading the multi-path media stream to a server; the multi-path media stream is used for distributing media streams corresponding to resolution and code streams by the server according to the display capability and bandwidth transmission state of different terminals; a first sending module 1104, configured to send a media stream acquisition request to a server, where the media stream acquisition request is a request for acquiring media streams of other terminals in a video; a first receiving module 1106, configured to receive a media stream to be displayed, which is distributed by a server; the first display module 1108 is configured to display a media stream to be displayed.

In this embodiment of the present application, the first uploading module is configured to upload multiple media streams to a server, where uploading multiple media streams to the server includes: acquiring an image through an image acquisition device to generate a media stream; coding the media stream according to a preset coding format to obtain a plurality of paths of media streams; uploading the multi-path media stream to a server; the multi-path media stream is used for distributing media streams corresponding to resolution and code streams by the server according to the display capability and bandwidth transmission state of different terminals; the first sending module is used for sending a media stream obtaining request to the server, wherein the media stream obtaining request is a request for obtaining media streams of other terminals in the video; the first receiving module is used for receiving the media stream to be displayed distributed by the server; the first display module is used for displaying the media stream to be displayed, and the purpose of executing corresponding data processing according to the node structure of the event is achieved, so that the technical effect of improving the processing efficiency of the network platform event is achieved, and the technical problem of low media stream transmission efficiency in a video conference caused by the fact that a media controller or a server forwards video streams of all terminals is solved.

It should be noted here that the first uploading module 1102, the first sending module 1104, the first receiving module 1106 and the first displaying module 1108 correspond to steps S602 to S608 in the third embodiment, and the four modules are the same as the corresponding steps in the implementation example and application scenarios, but are not limited to the disclosure in the third embodiment. It should be noted that the above four modules as a part of the apparatus may operate in the terminal provided in the third embodiment, and may be implemented by software or hardware.

Example 9

According to an embodiment of the present invention, an apparatus embodiment for implementing the foregoing method embodiment is further provided, and the apparatus provided by the foregoing embodiment of the present application may be run on a server.

Fig. 12 is a schematic structural diagram of an apparatus for distributing video data streams according to an embodiment nine of the present application.

As shown in fig. 12, the video data stream distribution apparatus may include: a third receiving module 1202, a fourth receiving module 1204 and a first distributing module 1206.

The third receiving module 1202 is configured to receive multiple media streams uploaded by each terminal; a fourth receiving module 1204, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals; the first allocating module 1206 is configured to allocate, according to the media stream acquisition request and the multiple media streams uploaded by each terminal, a corresponding media stream to be displayed to each terminal.

In the embodiment of the application, the third receiving module is used for receiving the multiple paths of media streams uploaded by each terminal; a fourth receiving module, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals; the first distribution module is used for distributing corresponding to-be-displayed media streams to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal, and the purpose of executing corresponding data processing according to the node structure of the event is achieved, so that the technical effect of improving the processing efficiency of the network platform event is achieved, and the technical problem of low media stream transmission efficiency in a video conference caused by the fact that a media controller or a server forwards the video streams of each terminal is solved.

It should be noted here that the third receiving module 1202, the fourth receiving module 1204 and the first distributing module 1206 correspond to the steps S702 to S706 in the fourth embodiment, and the three modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in the fourth embodiment. It should be noted that the three modules described above as a part of the apparatus may run in the server provided in the fourth embodiment, and may be implemented by software or hardware.

Example 10

Fig. 13 is a schematic structural diagram of an apparatus for distributing video data streams according to an embodiment of the present application.

As shown in fig. 13, the video data stream distribution apparatus may include: a fifth receiving module 1302, a sixth receiving module 1304, and a second allocating module 1306.

The fifth receiving module 1302 is configured to receive multiple media streams uploaded by each terminal; a sixth receiving module 1304, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals; a second allocating module 1306, configured to allocate, according to the media stream acquisition request and the multiple media streams uploaded by each terminal, a corresponding media stream to be displayed to each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: terminal identification and current display resolution.

In the embodiment of the application, the fifth receiving module is configured to receive multiple paths of media streams uploaded by each terminal; a sixth receiving module, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals; the second distribution module is used for distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: the terminal identification and the current display resolution achieve the purpose of executing corresponding data processing according to the node structure of the event, thereby realizing the technical effect of improving the processing efficiency of the network platform event, and further solving the technical problem of low media stream transmission efficiency in the video conference caused by the media controller or the server forwarding the video stream of each terminal.

It should be noted here that the fifth receiving module 1302, the sixth receiving module 1304 and the second allocating module 1306 correspond to steps S802 to S806 in the fifth embodiment, and the three modules are the same as the corresponding steps in the example and the application scenario, but are not limited to the disclosure in the fifth embodiment. It should be noted that the three modules described above as a part of the apparatus may run in the server provided in the fifth embodiment, and may be implemented by software or hardware.

Example 11

As shown in fig. 14, the video data stream distribution apparatus may include: a seventh receiving module 1402, an eighth receiving module 1404, a third allocating module 1406, and a third transmitting module 1408.

The seventh receiving module 1402 is configured to receive multiple media streams uploaded by each terminal; an eighth receiving module 1404, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals; a third distributing module 1406, configured to distribute, according to the media stream acquisition request and the multiple media streams uploaded by each terminal, a corresponding media stream to be displayed to each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: a terminal identifier and a current display resolution; a third sending module 1408, configured to send the to-be-displayed media stream carrying the device type and/or the bandwidth transmission state corresponding to each terminal, or the to-be-displayed media stream specified by each terminal and acquired according to the media stream acquisition request to each terminal.

In the embodiment of the application, the seventh receiving module is configured to receive multiple paths of media streams uploaded by each terminal; an eighth receiving module, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals; the third distribution module is used for distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; the method for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises the following steps: analyzing the media stream acquisition request, and acquiring the equipment type and/or bandwidth transmission state of each terminal; distributing corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing corresponding media streams to be displayed to each terminal according to the media stream types, wherein the media stream types comprise: a terminal identifier and a current display resolution; the third sending module is configured to send the to-be-displayed media streams carrying the device types and/or bandwidth transmission states corresponding to the terminals, or the to-be-displayed media streams designated by the terminals and acquired according to the media stream acquisition request to the terminals, so that the purpose of executing corresponding data processing according to the node structure of the event is achieved, thereby achieving the technical effect of improving the processing efficiency of the network platform event, and further solving the technical problem of low media stream transmission efficiency in a video conference caused by the media controller or the server forwarding the video streams of the terminals.

It should be noted here that the seventh receiving module 1402, the eighth receiving module 1404, the third allocating module 1406, and the third sending module 1408 correspond to steps S902 to S908 in the sixth embodiment, and the four modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in the sixth embodiment. It should be noted that the four modules described above as a part of the apparatus may run in the server provided in the sixth embodiment, and may be implemented by software or hardware.

Example 12

The embodiment of the invention also provides a storage medium. Optionally, in this embodiment, the storage medium may be configured to store a program code executed by the method for allocating a video data stream provided in the first embodiment.

Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: uploading the multi-path media stream to a server; sending a media stream acquisition request to a server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals; receiving a media stream to be displayed distributed by a server; and displaying the media stream to be displayed.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: uploading the multi-path media stream to the server comprises: acquiring an image through an image acquisition device to generate a media stream; the method comprises the steps of coding a media stream according to a preset coding format to obtain a plurality of paths of media streams, wherein the plurality of paths of media streams are used for distributing media streams corresponding to resolution and code streams by a server according to display capabilities and bandwidth transmission states of different terminals; and uploading the multi-path media stream to a server.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: before sending a media stream acquisition request to a server, encapsulating the device type and/or the bandwidth transmission state to obtain the media stream acquisition request.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: before sending a media stream acquisition request to a server, encapsulating a media stream type for displaying on a main picture to obtain the media stream acquisition request, wherein the media stream type comprises: the terminal identification and the current display resolution in the participating video conference.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. A system for distributing video data streams, comprising: a server and at least two terminals, wherein,

the at least two terminals are used for uploading media streams to the server;

and the server is used for receiving the media streams from the terminals, respectively matching the corresponding media streams to be displayed for the at least two terminals, and respectively returning the media streams to be displayed to the corresponding terminals.

2. A system for distributing video data streams according to claim 1, wherein said at least two terminals comprise: an image acquisition device and a media stream processing device, wherein,

the image acquisition device is used for acquiring an image of the current environment of the terminal and/or an image displayed by a display device in the terminal;

the media stream processing device is connected with the image acquisition device and is used for encoding the image according to a preset encoding format to obtain at least two types of media streams, wherein the at least two types of media streams are used for the server to allocate media streams corresponding to resolution and code streams according to the display capabilities and bandwidth transmission states of different terminals.

3. A system for distributing video data streams according to claim 2, wherein said image acquisition means comprise: the device is externally connected with a camera shooting and collecting device and/or embedded with window intercepting software.

4. A system for distributing video data streams according to claim 2, wherein said server comprises: a first receiving unit, a first parsing unit and a first allocating unit, wherein,

the first receiving unit is configured to receive a first media stream acquisition request sent by a first terminal and a second media stream acquisition request sent by a second terminal of the at least two terminals;

the first parsing unit is connected to the first receiving unit, and configured to obtain a device type and a bandwidth transmission state of the first terminal and a device type and a bandwidth transmission state of the second terminal according to the received first media stream obtaining request and the second media stream obtaining request;

the first allocation unit is connected to the first parsing unit, and configured to allocate the corresponding to-be-displayed media streams to the first terminal and the second terminal respectively according to the device type and the bandwidth transmission state of the first terminal and the device type and the bandwidth transmission state of the second terminal.

5. A system for distributing video data streams according to claim 2, wherein said server comprises: a second receiving unit, a second parsing unit, and a second allocating unit, wherein,

the second receiving unit is configured to receive a first media stream acquisition request sent by a first terminal of the at least two terminals and a second media stream acquisition request sent by a second terminal;

the second parsing unit is connected to the second receiving unit, and configured to obtain, according to the first media stream obtaining request and the second media stream obtaining request, display types of media streams required by the first terminal and the second terminal;

the second allocating unit is connected to the second parsing unit and configured to allocate corresponding to-be-displayed media streams to the first terminal and the second terminal according to the display type.

6. A distribution system of video data streams according to claim 1, characterized in that said media streams comprise: one or a combination of at least two of audio, video, images or text.

7. A method for distributing video data streams, comprising:

uploading the multiple media streams to a server, wherein the uploading the multiple media streams to the server comprises: acquiring an image through an image acquisition device to generate a media stream; coding the media stream according to a preset coding format to obtain the multi-path media stream; uploading the multi-path media stream to the server; the multi-path media stream is used for the server to distribute media streams corresponding to resolution and code streams according to the display capability and bandwidth transmission state of different terminals;

sending a media stream acquisition request to the server, wherein the media stream acquisition request is a request for acquiring media streams of other terminals;

receiving a media stream to be displayed distributed by the server;

and displaying the media stream to be displayed.

8. A method for distributing video data streams, comprising:

uploading a plurality of paths of media streams to a server, wherein the plurality of paths of media streams are used for distributing media streams corresponding to resolution and code streams by the server according to the display capacities and bandwidth transmission states of different terminals;

receiving a media stream to be displayed distributed by the server;

and displaying the media stream to be displayed.

9. The method for distributing video data streams according to claim 8, wherein said uploading multiple media streams to the server comprises:

acquiring an image through an image acquisition device to generate a media stream;

coding the media stream according to a preset coding format to obtain the multi-path media stream;

and uploading the multi-path media stream to the server.

10. Method for distributing video data streams according to claim 8 or 9, characterized in that before said sending of a media stream acquisition request to said server, said method further comprises:

and encapsulating the equipment type and/or the bandwidth transmission state to obtain the media stream acquisition request.

11. Method for distributing video data streams according to claim 8 or 9, characterized in that before said sending of a media stream acquisition request to said server, said method further comprises:

encapsulating the media stream type for displaying on the main picture to obtain the media stream acquisition request, wherein the media stream type comprises: and participating in the terminal identification and the current display resolution in the video.

12. A method for distributing video data streams, comprising:

receiving a plurality of paths of media streams uploaded by each terminal;

receiving a media stream acquisition request sent by each terminal, wherein the media stream acquisition request is a request for acquiring media streams of other terminals;

and distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal.

13. A method for distributing video data streams, comprising:

receiving a plurality of paths of media streams uploaded by each terminal;

distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal;

wherein, the allocating the corresponding media stream to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises: analyzing the media stream acquisition request to acquire the equipment type and/or bandwidth transmission state of each terminal; distributing the corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state;

or,

analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying in a main picture; distributing the corresponding media stream to be displayed to each terminal according to the media stream type, wherein the media stream type comprises: terminal identification and current display resolution.

14. A method for distributing video data streams, comprising:

receiving a plurality of paths of media streams uploaded by each terminal;

distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; wherein, the allocating the corresponding media stream to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises: analyzing the media stream acquisition request to acquire the equipment type and/or bandwidth transmission state of each terminal; distributing the corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing the corresponding media stream to be displayed to each terminal according to the media stream type, wherein the media stream type comprises: a terminal identifier and a current display resolution;

and sending the media stream to be displayed carrying the equipment type and/or bandwidth transmission state corresponding to each terminal, or the media stream to be displayed appointed by each terminal acquired according to the media stream acquisition request to each terminal.

15. An apparatus for distributing video data streams, comprising:

the first uploading module is configured to upload a plurality of media streams to a server, where uploading the plurality of media streams to the server includes: acquiring an image through an image acquisition device to generate a media stream; coding the media stream according to a preset coding format to obtain the multi-path media stream; uploading the multi-path media stream to the server; the multi-path media stream is used for the server to distribute media streams corresponding to resolution and code streams according to the display capability and bandwidth transmission state of different terminals;

the first sending module is used for sending a media stream obtaining request to the server, wherein the media stream obtaining request is a request for obtaining media streams of other terminals in a video;

the first receiving module is used for receiving the media stream to be displayed distributed by the server;

and the first display module is used for displaying the media stream to be displayed.

16. An apparatus for distributing video data streams, comprising:

the second uploading module is used for uploading a plurality of paths of media streams to the server, wherein the plurality of paths of media streams are used for allocating media streams corresponding to resolution and code streams by the server according to the display capabilities and bandwidth transmission states of different terminals;

a second sending module, configured to send a media stream acquisition request to the server, where the media stream acquisition request is a request for acquiring media streams of other terminals;

the second receiving module is used for receiving the media stream to be displayed distributed by the server;

and the second display module is used for displaying the media stream to be displayed.

17. An apparatus for distributing video data streams, comprising:

the third receiving module is used for receiving the multi-path media streams uploaded by each terminal;

a fourth receiving module, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals;

and the first distribution module is used for distributing the corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal.

18. An apparatus for distributing video data streams, comprising:

the fifth receiving module is used for receiving the multi-path media streams uploaded by each terminal;

a sixth receiving module, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals;

the second distribution module is used for distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal;

or,

19. An apparatus for distributing video data streams, comprising:

the seventh receiving module is used for receiving the multi-path media streams uploaded by each terminal;

an eighth receiving module, configured to receive a media stream acquisition request sent by each terminal, where the media stream acquisition request is a request for acquiring media streams of other terminals;

the third distribution module is used for distributing corresponding media streams to be displayed to each terminal according to the media stream acquisition request and the multi-path media streams uploaded by each terminal; wherein, the allocating the corresponding media stream to be displayed to each terminal according to the media stream acquisition request and the multiple media streams uploaded by each terminal comprises: analyzing the media stream acquisition request to acquire the equipment type and/or bandwidth transmission state of each terminal; distributing the corresponding media streams to be displayed to each terminal according to the equipment type and/or the bandwidth transmission state; or, analyzing the media stream acquisition request to acquire the media stream types used by each terminal for displaying on the main picture; distributing the corresponding media stream to be displayed to each terminal according to the media stream type, wherein the media stream type comprises: a terminal identifier and a current display resolution;

and a third sending module, configured to send the to-be-displayed media stream carrying the device type and/or bandwidth transmission state corresponding to each terminal, or the to-be-displayed media stream specified by each terminal, which is obtained according to the media stream obtaining request, to each terminal.