CN117640597A

CN117640597A - Media stream transmission method and device

Info

Publication number: CN117640597A
Application number: CN202210975050.6A
Authority: CN
Inventors: 廖涛
Original assignee: Huawei Cloud Computing Technologies Co Ltd
Current assignee: Huawei Cloud Computing Technologies Co Ltd
Priority date: 2022-08-15
Filing date: 2022-08-15
Publication date: 2024-03-01

Abstract

The invention discloses a media stream transmission method and device. The method comprises the following steps: the method comprises the steps that a first media server obtains a first media stream of a first channel, a first client accesses the first channel, and the first channel is a channel of a main conference; the first media server acquires a second media stream of a second channel, wherein the second media stream is obtained based on the processing of the first media stream, and the second channel is any conference-dividing channel; the first media server sends a second media stream to a second client, and the second client accesses a second channel. In the method, the number of clients which need to process the media stream in each channel can be reduced through the isolation of different channels. Thus, the complexity of media streaming does not appear to grow rapidly with the number of clients, thereby reducing the complexity of media streaming.

Description

Media stream transmission method and device

Technical Field

The present disclosure relates to the field of communications technologies, and in particular, to a method and an apparatus for transmitting a media stream.

Background

With the continued development of communication technology, a variety of technologies have emerged that can be used to enable multimedia information interaction between devices. For example, conference technology, means that multimedia information collected by multiple devices can be shared remotely. The conference may now be implemented by a conference system. Current conference systems include servers and clients. The clients are divided into clients with unidirectional rights and clients with bidirectional rights. A client of unidirectional authority can only receive a media stream unidirectionally from a server. The client of the bidirectional authority can receive the media stream from the server and can send the media stream to the server. The number of clients involved may reach thousands of clients when a large conference is held.

As clients increase, more media streams need to be forwarded. The server often needs to transmit a large number of media streams. The scaled media stream may result in a particularly complex transmission of the media stream. For example, in a meeting, searching for a certain client requires searching from millions of clients; for another example, the server needs to analyze the transmission paths of the media streams of millions of clients at the same time. Therefore, how to reduce the complexity of media streaming when a large conference is held is a urgent problem to be solved.

Disclosure of Invention

The application provides a media stream transmission method and device, which are used for reducing the complexity of media stream transmission.

In a first aspect, the present application provides a media streaming method, which may be applied to a first media server, where the first media server may be a media streaming device provided in the third aspect of the present application, and the method includes:

a first media server acquires a first media stream of a first channel, wherein the first client accesses the first channel, and the first channel is a channel of a main conference; the first media server acquires a second media stream of a second channel, wherein the second media stream is processed based on the first media stream, and the second channel is any conference-dividing channel; the first media server sends the second media stream to a second client, and the second client accesses the second channel.

In the above manner, the first media server obtains both the first media stream of the first channel and the second media stream of the second channel, where the second media stream is obtained based on the processing of the first media stream. Thereby separating the channels of the conference into the channels of the main conference and the channels of the sub conference, and dispersing the channels of the conference into a plurality of channels. And, the clients of each channel can be accessed through the first channel of the main conference and the second channel of the sub conference respectively. Taking the first media stream of the first channel and the second media stream of the second channel as examples, clients can be dispersed to different channels, the number of clients involved in a single channel can be reduced, and then the media stream processing capacity of the single channel is greatly reduced. Thus, the complexity of media streaming does not appear to grow rapidly with the number of clients, thereby reducing the complexity of media streaming.

In one possible design, the first media server obtaining a second media stream for a second channel includes: the first media server sends the first media stream to an intermediate device; the first media server receives the second media stream sent by the intermediate device.

In the above manner, the first media server can obtain the second media stream by sending the first media stream to the intermediate device, so that the intermediate device can undertake the task of processing the first media stream into the second media stream, and the first media server can concentrate on the transmission of the media stream, thereby reducing the complexity of the transmission of the media stream.

In one possible design, the method further comprises:

the first media server receives first access information sent by a first scheduling server, wherein the first access information is used for indicating the first client to access the first channel, and the first access information is determined according to authority information of the first client.

In the above manner, taking the first client as an example, the channel accessed by the first client can be determined according to the authority information through the first access information, so that different clients can access corresponding channels according to the authority information, the client media stream transmission management is more efficient, and the complexity of media stream transmission is further reduced.

In one possible design, after the authority information of the first client is changed, the first media server receives second access information sent by the first scheduling server, where the second access information is used to instruct to switch the access channel of the first client to the third channel, and the third channel is any channel of a sub-conference.

In the above manner, taking the first client as an example, through the second access information, after the authority information of the first client is changed, the channel accessed by the first client can be switched according to the authority information, so that the flexibility of the media stream transmission management of the client is higher, and the complexity of the media stream transmission is reduced when the channel accessed by the client is switched.

It should be noted that, the case where the authority information of the first client is changed may be as follows:

in one possible scenario, the first media stream is sent by the first client, the first client accessing the first channel with bidirectional rights, the first client having rights to send the media stream of the first channel and having rights to receive the media stream of the first channel, the method further comprising:

if the first client side is changed to access the third channel by the unidirectional authority, the client side with the unidirectional authority accessing the third channel has the authority of receiving the media stream of the first channel, and the first media server acquires a fourth media stream from the first channel; the first media server sends the fourth media stream to the intermediate device; the first media server receives a third media stream of the third channel sent by the intermediate device, wherein the third media stream is processed based on the fourth media stream; the first media server sends the third media stream to the first client.

In the above manner, when the first client changes to access the third channel with the unidirectional authority, the first client does not have the authority to transmit the media stream of the third channel. The first media server may obtain a fourth media stream of the first channel and obtain a third media stream by sending the fourth media stream to the intermediate device. Therefore, under the condition of ensuring the flexibility of updating the access authority of the first client, the intermediate device can still bear the work of converting the fourth media stream into the third media stream, and the first media server can concentrate on the transmission of the media stream, so that the complexity of the transmission of the media stream is reduced.

It should be noted that, the authority information of the second client may also be changed, and the changing situation may be as follows:

in one possible scenario, the second client accesses the client of the second channel with unidirectional rights, the client of the unidirectional rights accessing the second channel having rights to receive the media stream of the second channel, the method further comprising: if the second client changes to the bidirectional authority to access the first channel, the first media server obtains a fifth media stream of the first channel from the second client, and/or sends a sixth media stream of the first channel to the second client, and the client with the bidirectional authority to access the first channel has the authority to send the media stream of the first channel and the authority to receive the media stream of the first channel.

In the above manner, when the second client changes to the client with bidirectional authority to access the first channel, the second client has authority to transmit the media stream of the first channel and has authority to receive the media stream of the first channel. Thus, the first media server can still obtain the fifth media stream of the first channel from the second client and/or send the sixth media stream of the first channel to the second client. Therefore, under the condition of ensuring the flexibility of updating the access authority of the second client, the first media server can still complete the media stream transmission with the second client under the condition of not increasing the complexity of the media stream transmission.

In one possible design, the method further comprises:

the first media server obtains a third media stream of the third channel.

In the method, after the first media server switches the channel accessed by the client, the first media server can acquire the third media stream of the third channel, the acquisition of the media stream of the third channel is not influenced, the corresponding media stream transmission capability is reserved, and the complexity of media stream transmission is not increased.

In one possible design, the method further comprises: the first media server sends the media stream of the first channel to a second media server.

In the above manner, taking the first channel as an example, the media servers may also forward the media stream of the first channel. Therefore, no matter which media server of the channel is accessed, the media stream transmission among the media servers of the same channel is not affected, and only the media stream in the channel is considered, so that the complexity of the media stream transmission can be reduced.

In a second aspect, the present application provides a media streaming system. The system comprises: the system comprises a first media server, a first client and a second client; the first media server is configured to obtain a first media stream of a first channel, where the first client accesses the first channel, and the first channel is a channel of a main conference; the first media server is configured to obtain a second media stream of a second channel, where the second media stream is obtained based on the processing of the first media stream, and the second channel is any channel of a conference; the second client is configured to receive the second media stream sent by the first media server, where the second client accesses the second channel.

In one possible design, the system further includes an intermediary device; the first media server is specifically configured to: transmitting the first media stream to the intermediate device; the intermediate device is configured to obtain the second media stream based on the first media stream processing, and send the second media stream to the first media server.

In one possible design, the system further includes an intermediary device; the first client is further configured to send the first media stream to the intermediate device; the intermediate device is configured to obtain the second media stream based on the first media stream, and send the first media stream and the second media stream to the first media server.

In one possible design, the first media server is further configured to:

and receiving first access information sent by a first scheduling server, wherein the first access information is used for indicating the first client to be accessed to the first channel, and the first access information is determined according to the authority information of the first client.

In one possible design, the first media server is further configured to:

and after the authority information of the first client is changed, receiving second access information sent by the first scheduling server, wherein the second access information is used for indicating that the access channel of the first client is switched to a third channel, and the third channel is any channel of a conference.

In one possible scenario, the first media stream is sent by the first client, the first client accessing the first channel with bidirectional rights, the first client having rights to send the media stream of the first channel and having rights to receive the media stream of the first channel, the first media server further being configured to:

if the first client side is changed to access the third channel by the unidirectional authority, the client side with the unidirectional authority accessing the third channel has the authority of receiving the media stream of the first channel, and a fourth media stream from the first channel is obtained; transmitting the fourth media stream to the intermediate device; receiving a third media stream of the third channel sent by the intermediate device, wherein the third media stream is processed based on the fourth media stream; and sending the third media stream to the first client.

It should be noted that, in the system, the manner in which the client accesses the media server may be a token manner. The client access media server based on the token mode, and the situation that the client switches access channels can be specifically as follows:

in one possible scenario, the first client is further configured to send a first request to the first media server, where the first request is for the first client to request access to the first channel with bidirectional rights, where a client with bidirectional rights access to the first channel has rights to send a media stream of the first channel, and has rights to receive a media stream of the first channel; the first media server is further configured to send a first token to the first client in response to the first request, where the first token is a token that the first client accesses the first channel with a bidirectional authority.

In the above manner, taking the first client as an example, the server can manage the first client accessed by the first channel and the authority of accessing the channel through the token, so that the isolation between the media stream of the first channel and the media streams of other channels is ensured, and the stability of the transmission complexity of the media stream is ensured.

In one possible design, the system further includes an intermediary device; the first client is further configured to send a second request to the first media server, where the second request is used for the first client to request to access the third channel with unidirectional rights, and the client with unidirectional rights accessing the third channel has rights to receive a media stream of the third channel; the first media server is further configured to set the first token to be invalid in response to the second request, and send a second token to the first client, where the second token is a token that the first client accesses the third channel with unidirectional authority.

In the above manner, taking the first client as an example, the first client can obtain the second token by sending the second request to the first media server, and change from the bidirectional authority to the unidirectional authority to access the first channel to the unidirectional authority to access the third channel. Therefore, the first client can flexibly and rapidly update the access channel and the authority through the second token.

In one possible design, the first media server is further configured to send the fourth media stream to the intermediate device; the intermediate device is configured to obtain a third media stream of the third channel according to the fourth media stream, and send the third media stream to the first media server; the first media server sends the third media stream to the first client.

In the above manner, when the first client is changed to the client accessing the third channel with the unidirectional authority, the first client does not have the authority to transmit the media stream of the third channel. The intermediate device can obtain a third media stream through the obtained fourth media stream. Under the condition that the flexibility of updating the access authority of the first client is ensured, the intermediate equipment can still bear the work of converting the fourth media stream into the third media stream, and the first media server can concentrate on the transmission of the media stream, so that the complexity of the transmission of the media stream is reduced.

In one possible design, the system further includes a second media server; the second client is further configured to send a third request to the second media server, where the third request is used for the second client to request to access the second channel with unidirectional rights, and the client with unidirectional rights accessing the second channel has rights to receive a media stream of the second channel; the second media server is further configured to send a third token to the second client in response to the third request, where the third token is a token that the second client accesses the second channel with unidirectional authority.

In the above manner, taking the second client as an example, the server can manage the second client accessed by the second channel and the authority of accessing the channel through the token, so that the isolation between the media stream of the second channel and the media streams of other channels is ensured, and the stability of the transmission complexity of the media stream is ensured.

In one possible design, the system further includes an intermediary device; the second client is further configured to send a fourth request to the second media server, where the fourth request is used for the second client to request to access the first channel with bidirectional rights, where a client with bidirectional rights accessing the first channel has rights to send a media stream of the first channel, and has rights to receive the media stream of the first channel; the second media server is further configured to set the third token to be invalid in response to the fourth request, and send a fourth token to the second client, where the fourth token is a token that the second client accesses the second channel with bidirectional rights.

In the above manner, taking the second client as an example, the second client can obtain the fourth token by sending the fourth request to the second media server, and change from the unidirectional authority to the bidirectional authority to access the second channel to access the first channel. Therefore, the second client can flexibly and rapidly update the access channel and the authority through the second token.

In one possible design, the second media server is further configured to obtain a fifth media stream of the first channel from the second client, and/or send a sixth media stream of the first channel to the second client.

In the above manner, after the second client changes to the client accessing the first channel with the bidirectional authority, the second client can not only receive but also send the media stream of the first channel. Therefore, the second media server may acquire the fifth media stream of the first channel from the second client and/or send the sixth media stream of the first channel to the second client, so that the media stream transmission with the second client can still be completed. Therefore, under the condition of ensuring the flexibility of updating the access authority of the second client, the second media server can still complete the media stream transmission with the second client under the condition of not increasing the complexity of the media stream transmission.

In a third aspect, the present application provides a media streaming apparatus. The device comprises:

the system comprises an acquisition module, a first client and a second client, wherein the acquisition module is used for acquiring a first media stream of a first channel, the first client is accessed to the first channel, and the first channel is a channel of a main conference; acquiring a second media stream of a second channel, wherein the second media stream is obtained based on the first media stream, and the second channel is any conference-dividing channel; and the sending module is used for sending the second media stream to a second client, and the second client accesses the second channel.

In one possible design, the sending module is specifically configured to: transmitting the first media stream to an intermediate device; the acquisition module is specifically configured to: and receiving the second media stream sent by the intermediate equipment.

In one possible design, the acquisition module is further configured to:

and acquiring a third media stream of the third channel.

In one possible design, the transmitting module is further configured to:

and sending the media stream of the first channel to a second media server.

In one possible design, the first access information is determined according to authority information of the first client and location information of the first client.

In a fourth aspect, there is provided an electronic device comprising: one or more processors; one or more memories; wherein the one or more memories store one or more computer instructions that, when executed by the one or more processors, cause the electronic device to perform any of the possible methods of the first aspect described above.

In a fifth aspect, there is provided a computer readable storage medium comprising computer instructions which, when run on a computer, cause the computer to perform any one of the possible methods of the first aspect described above.

In a sixth aspect, the present application provides a chip comprising a processor coupled to a memory for reading and executing a software program stored in the memory to implement a possible method according to any one of the first aspects described above.

In a seventh aspect, the present application provides a computer program product which, when read and executed by a computer, causes the computer to perform any one of the possible methods of the first aspect described above.

The advantages of the third aspect to the seventh aspect are described above with reference to the advantages of the first aspect, and the description is not repeated.

Drawings

Fig. 1A to fig. 1B are schematic architecture diagrams of a system architecture to which a media stream transmission method according to an embodiment of the present application can be applied;

fig. 2 is a schematic block diagram of each device in a system architecture to which a media streaming method according to an embodiment of the present application may be applied;

fig. 3 is a schematic diagram of a client access conference flow provided in an embodiment of the present application;

fig. 4 is a schematic diagram of a media stream transmission flow provided in an embodiment of the present application;

fig. 5 is a schematic diagram of a client modification authority flow provided in an embodiment of the present application;

fig. 6 is a schematic diagram of another media streaming transmission flow provided in an embodiment of the present application;

fig. 7 is a schematic diagram of a media streaming device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include, for example, "one or more" such forms of expression, unless the context clearly indicates to the contrary. It should also be understood that in embodiments of the present application, "one or more" refers to one or more than two (including two); "and/or", describes an association relationship of the association object, indicating that three relationships may exist; for example, a and/or B may represent: a alone, a and B together, and B alone, wherein A, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

The term "plurality" in the embodiments of the present application means two or more, and for this reason, "plurality" may be also understood as "at least two" in the embodiments of the present application. "at least one" may be understood as one or more, for example as one, two or more. For example, including at least one means including one, two or more, and not limiting what is included. For example, at least one of A, B and C is included, then A, B, C, A and B, A and C, B and C, or A and B and C may be included. Likewise, the understanding of the description of "at least one" and the like is similar. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/", unless otherwise specified, generally indicates that the associated object is an "or" relationship.

Unless stated to the contrary, the embodiments of the present application refer to ordinal terms such as "first," "second," etc., for distinguishing between multiple objects and not for defining a sequence, timing, priority, or importance of the multiple objects.

For ease of understanding, the terms involved in the embodiments of the present application are explained as part of the summary of the embodiments of the present application.

Fig. 1A is a schematic diagram of a system architecture to which a media streaming method according to an embodiment of the present application can be applied. The system architecture may include servers (dispatch server 10 and media server 20), intermediate devices 30, and clients 40 (clients).

Alternatively, the server may be a physical server, a virtual server running on a computer device, or the like. The number of servers may be one or more. For example, the server may be a single server, or may be a server node in a server cluster. The servers can be distributed or centralized. For example, the servers may be deployed in a centralized manner, or may be deployed in a distributed manner according to different functions. In the architecture shown in fig. 1A, servers that deploy different functions in a distributed manner are illustrated as an example. The server shown in fig. 1A includes: a scheduling server 10 and a media server 20. Wherein, the scheduling server 10 may be used to control the channel corresponding to the conference and the authority of accessing the channel by the client 40, the media server 20 is used to control the channel accessed by the client 40, and the media streaming, and the media server 20 may implement the media streaming function by deploying the media streaming routing unit (switch forward unit, SFU) therein.

Note that, the conference technology is a technology in which media streams collected by a plurality of devices are shared. The device can access the channel corresponding to the conference in the modes of conference software, telephone and the like, and the device after accessing the same channel can realize the media stream sharing of the channel. In this embodiment of the present application, a main conference and at least one sub-conference may be created in the server, where each of the main conference and each sub-conference has a corresponding channel, and the channels may also be created in the server. If the architecture shown in fig. 1A is employed, a conference may be created in the dispatch server 10. A channel corresponding to the conference may be created in the media server 20 and a media stream of the channel corresponding to the conference may be transmitted. For example, a first media server may obtain a first media stream of a first channel from a first client.

Wherein the media stream of any conference corresponding channel can be obtained based on the media stream processing of another other conference corresponding channel. For example, the media streams of any sub-conference corresponding channel may be processed based on the media streams of the main conference corresponding channel. Thus, the media stream of the sub-conference corresponding channel may be consistent with the content of the media stream of the main conference corresponding channel, although there may be some delay with respect to the media stream of the main conference corresponding channel. The media stream processing of the channel may be performed by the intermediate device 30.

Optionally, the intermediate device 30 is a router, a modem, a Media Gateway (MGW), or a broadband remote access server (broadband remote access server, BRAS), etc. The number of intermediate devices 30 may be one or more. The intermediate device 30 may be configured to obtain a media stream for one channel and process the media stream for another channel based on the media stream for that channel. For example, the intermediate device 30 may obtain a first media stream of a first channel from a first media server, process a second media stream of a second channel based on the first media stream, and send the second media stream to the first media server. The first channel can be a channel of a main conference, and the second channel can be a channel of a sub-conference; the second channel may be a channel of a main conference, and the first channel may be a channel of a sub-conference; the first channel and the second channel may both be conference channels. In the architecture shown in fig. 1A, only one intermediate device 30 may be disposed, or a plurality of intermediate devices 30 may be disposed, which is not limited herein.

Alternatively, the client 40 is a personal computer (e.g., desktop or notebook computer, etc.), a tablet computer, a mobile phone, a server with a display function, a wearable device, a smart screen, or a special conference terminal (conference terminal includes, for example, a camera and a microphone), or the like. The implementation manner of the server, the intermediate device 30 and the client 40 is not limited in the embodiment of the present application. The client 40 may be configured to access a channel corresponding to a conference (main conference or sub conference), and may be configured to send a request to the media scheduler 10 and obtain access information, which may be a token. Client 40 may also send or receive media streams for channels corresponding to the accessed conference. It should be noted that, the client 40 may access the channels corresponding to the conference through different rights. The client 40 may access the channel with unidirectional rights or with bidirectional rights. The client 40 having unidirectional rights to access the channel may be understood as having the rights to receive the media stream of the accessed channel; alternatively, it may be understood that the rights to transmit the accessed channel media stream are provided. If not specifically stated, the client 40 having unidirectional rights to access the channel may default to having rights to receive the media stream of the accessed channel. The client 40 having the bidirectional rights to access the conference may be understood as having both the rights to receive the accessed channel media stream and the rights to transmit the accessed channel media stream.

Fig. 1A only briefly describes the functions of the respective devices. The function of each device in the architecture shown in FIG. 1A is described in more detail below in conjunction with FIG. 1B. The method can be concretely as follows:

the scheduling server 10 is used to create a conference, and the implementation of creating a conference may be to create a conference process. The conference created in the scheduling server 10 may be a main conference or a sub conference, and one or more sub conferences may be used. The conference created in the schedule server 10 is in a many-to-many relationship with the schedule server 10, one conference may be created on a plurality of schedule servers 10, and a plurality of conferences may be created on one schedule server 10. As shown in fig. 1B, a primary conference and a first secondary conference are created in the first scheduling server 10, and a second secondary conference is created in the second scheduling server 10. Each conference created in the scheduling server 10 uniquely corresponds to a channel created on the media server 20. Channels created in the media server 20 are also in a many-to-many relationship with the media server 20, one channel may be created on multiple media servers 20, and multiple channels may be created on one media server 20.

The scheduling server 10 may be configured to determine access information for the client 40 to access a channel corresponding to the conference, where the access information can indicate that the client accesses the channel; for example, the first access information is used to instruct the first client to access the first frequency channel. The access information of the client 40 may be determined by the rights information of the client to access the channel. The rights of the client 40 to access the corresponding channel of the conference (main conference or sub conference) can be classified into one-way rights and two-way rights. In an alternative implementation, the unidirectional authority client 40 accesses the sub-conference corresponding channel, and the bidirectional authority client 40 accesses the main conference corresponding channel. Thus, the media streams sent by the clients 40 with the bidirectional authorities are transmitted to the channels corresponding to the main conference first and then to the channels corresponding to the sub-conferences. In addition, the number of the clients 40 with the bidirectional authority is generally fixed and the occupied number is relatively small in the clients 40 participating in the conference, so that the channel of the main conference can be dedicated to media stream transmission of the clients 40 with the bidirectional authority, thereby being dedicated to acquiring the source of the media stream and controlling the complexity of media stream processing in the channel corresponding to the main conference. The scheduling server 10 may enable the client 40 to access a channel corresponding to the conference by sending access information to the client 40.

The scheduling server 10 may also be used to send access information to the media server 20 and the client 40, which may be sent by the conference process. The access information indicates the channel to which the client has access, and the channel rights to access.

It should be noted that the form of the access information is not limited in this application. For example, the access information may be a token, which is a credential of the client 40 to access the corresponding channel of the conference, and can identify the rights of the client 40 to access the channel. After the client 40 obtains the token, it can access the channel corresponding to the conference in the media server 20 according to the authority corresponding to the token, so as to perform media streaming of the channel with the media server 20. For example, when the first client 40 accesses the first channel with the bidirectional authority, the first token is obtained, and the first channel is the channel of the main conference. The first client 40 is able to send and acquire the media stream of the first channel.

The media server 20 is used to create channels, and the implementation of creating channels may be to create a channel process, one channel may be created on multiple media servers 20, and one media server 20 may also create multiple channels. For example, the first media server 20 creates a first channel and a second channel, and the second media server 20 creates a second channel and a third channel.

The media server 20 is configured to receive access information, which may be received by a channel process.

For example, the first media server receives first access information sent by a first scheduling server. The first access information indicates that the first client accesses a first channel, and the first access information may be determined according to authority information of the first client, where the first channel is a channel of the primary conference. The authority information of the first client may be preset or may be specified by the scheduling server 10.

For example, if the rights information of the first client is bidirectional rights, the scheduling server 10 determines that the first access information indicates that the first client accesses the first channel; if the entitlement information for the first client is one-way entitlement, then the dispatch server 10 determines that the first access information indicates that the first client accesses a frequency channel for the conference, e.g., the first access information indicates that the first client accesses a second frequency channel.

In a possible scenario, if the authority information of the first client is one-way authority and the number of clients that have accessed the first channel is less than a set threshold, the first access information indicates that the first client accesses the first channel; and when the authority information of the first client is one-way authority and the number of the clients accessed to the first channel is greater than or equal to a set threshold value, the first access information indicates the first client to access any channel except the first channel. The first client may also preset a priority order of the access channels, for example, the priority order is: a first channel, a second channel, a third channel, a fourth channel, etc. For any channel, if the number of clients of the channel is greater than or equal to a set threshold, detecting whether the number of clients of the channel with the next channel priority is smaller than the set threshold, and if so, the first access information indicates that the first client accesses the channel.

In another possible scenario, the first access information may be determined according to the rights information of the first client and the location information of the first client, e.g. a channel on the media server with the latest distance may be preferentially accessed. For example, the rights information of the first client is one-way rights, and the media server closest to the first client is determined to be the first media server according to the location information of the first client, then the channel on the first media server is accessed. For example, if the channels on the first media server include a first channel, a second channel, and a third channel, then one of the first channel, the second channel, and the third channel is accessed, and which channel on the first media server is accessed may be selected in the manner of the priority.

The media server 20 is also used for transmitting the media stream of the channel with the client 40, the intermediate device 30, which access the channel, and for transmitting the media stream of the channel with other media servers 20. For example, the media server 20 may receive a first media stream of a first channel sent by a first client 40, or may send a second media stream of a second channel to a second client 40.

The media server 20 is further configured to obtain access information sent by the client, where the access information may be the same as the access information sent by the first scheduling server. For example, the first media server may receive the first access information sent by the first scheduling server, where the first access information is used to instruct the first client to access the first channel, and the first media server may also receive the first access information sent by the client.

The access information may be in the form of a token, e.g. a first token. After obtaining the token, the media server 20 may determine the source and legitimacy of the media stream based on the received token in the media stream. For example, after obtaining the first token, the media server 20 can determine that the media stream is the media stream of the first channel when the received media stream includes the first token. Also, since the first token is able to identify that the media stream is from the first client 40 and that the first client 40 has the right to access the first channel bi-directionally, the first media stream of the first channel received from the first client is legitimate, allowing further processing.

The media server 20 may also be configured to send a media stream of one channel to the intermediate device 30 and receive a media stream of another channel sent by the intermediate device. For example, the media server 20 may receive a second media stream of a second channel transmitted by the intermediate device 30 by transmitting a first media stream of a first channel to the intermediate device 30, the second media stream being processed based on the first media stream. The media server 20 only needs to consider the media streaming of the clients 40 involved in a single channel when transmitting the media stream. For example, when the first media server 20 parses the client identifier from the first media stream of the first channel and sends the first media stream to the client 40 corresponding to the client identifier, the first media server 20 only needs to search for the client 40 corresponding to the client identifier from the clients 40 accessing the first channel.

It should be noted that, media streams of the same channel may also be transmitted between media servers 20 that create the channel; for example, as shown in fig. 1B, both the first media server 20 and the second media server 20 create a first channel, and the first media server 20 may send a media stream of the first channel to the second media server 20.

The intermediate device 30 is used to access a frequency channel and may access a plurality of frequency channels. The process of accessing the channel by the intermediate device 30 may be regarded as the process of accessing the channel by the client 40. The specific procedure of the intermediate device 30 accessing each channel may refer to the specific procedure of the client 40 accessing the channel. The intermediate device 30 is arranged to receive a media stream and to transmit a media stream. The intermediate device 30 may receive the media stream from the client 40 or the media server 20, or may send the media stream to the client 40 or the media server 20. The intermediate device 30 is further configured to process the media stream of one channel to obtain the media stream of another channel. For example, the intermediate device 30 receives a first media stream of a first channel, may process a second media stream of a second channel, and transmits the second media stream.

For example, the intermediate device 30 may obtain the first media stream of the first channel sent by the first media server 20, process the first media stream to obtain the second media stream of the second channel, and send the second media stream to the first media server 20, so that the first media server 20 can obtain the second media stream of the second channel.

The client 40 is used to access a corresponding channel of a conference (main conference or sub-conference). The client 40 may access the channel with unidirectional rights and may also access the channel with bidirectional rights. The manner in which the client 40 accesses the channel is not limited herein, and in one possible implementation, the manner in which the client 40 accesses the channel may be: the client 40 sends a request to the scheduling server 10 and obtains a token for the accessed channel. For example, the first client 40 sends a first request to the first scheduling server 10, where the first request is for the first client 40 to request access to a first channel corresponding to the main conference with bidirectional rights; the first client 40 obtains a first token, which is a token of the first client 40 accessing the first channel. After the first client 40 obtains the token for the first frequency channel, it accesses the first frequency channel.

In a possible scenario, the client 40 may preset a channel requesting access to a default conference, such as the first channel of the primary conference. The scheduling server 10 may not have the client 40 access the channel of the default meeting but re-access other channels than the default channel, such as the second channel corresponding to the sub-meeting. Then after client 40 sends a request to dispatch server 10, dispatch server 10 may return a redirect message indicating the channel of client 40 to access other conferences. For example, the second client 40 sends a second request to the first dispatch server 10, and the first dispatch server 10 sends a first redirect message to the second client 40. The second request is for the second client 40 to request access to a first channel corresponding to the primary conference with unidirectional rights, and the first redirect message is for instructing the second client 40 to access a second channel corresponding to the first sub conference with unidirectional rights. The second client 40 sends a third request to the first scheduling server 10 and the first scheduling server 10 sends a second token to the second client 40 and the second media server 20. The third request is for the second client 40 to request access to the second channel corresponding to the first conference with unidirectional authority, and the second token is a token for the second client 40 to access the second channel.

It should be noted that, after the client 40 accesses the channel, the rights of the access channel may be changed, so that the request may be sent to the scheduling server 10 again, and the token returned by the media server 20 may be obtained. For example, after the first client 40 obtains the first token, the first scheduling server 10 sends a second redirection message to the first client 40, where the second redirection message is used to instruct the first client 40 to access the second channel with unidirectional rights. The first client 40 sends a fourth request to the first media server 20 for the first client 40 to access the second channel with unidirectional rights and the third token is a token for the second client 40 to access the second channel.

The client 40 may also be configured to transfer media streams of channels corresponding to the accessed conference with the media server 20 based on the rights of the accessed channels. Since the clients 40 are divided into a unidirectional authority client 40 and a bidirectional authority client 40, the situation in which media streams are transferred between the clients 40 of different access authorities and the media server 20 may be different. For example, when the first client 40 accesses the first channel with the bidirectional authority, it can receive the media stream of the first channel and send the media stream of the first channel; the second client 40 is capable of receiving the media stream of the second channel while accessing the second channel with unidirectional rights. In addition, the rights of the channel accessed by the client 40 may be updated, where the rights of the channel accessed refer to the right of the latest channel accessed by the client 40. For example, after the first client 40 accesses the first frequency channel with the bidirectional authority, it may change to access the second frequency channel with the unidirectional authority. When the first client 40 changes to access the second channel with the unidirectional right, it is able to receive the media stream of the second channel, but it is unable to transmit the media stream of the second channel.

Modules may be deployed in each device in the system architecture shown in fig. 1A to implement corresponding functions, where the modules may be specifically shown in fig. 2.

The scheduling server 10 may include a conference control module 101, a channel access module 102, and a first signaling module 103. Wherein the conference control module 101 is used to create a conference (main conference or sub conference) and to allocate the media server 20 to the clients.

The channel access module 102 is configured to obtain a request for accessing a channel of a client. The channel access module 102 may be accessed by clients by providing an interface.

The first signaling module 103 is configured to send signaling. For example, the first signaling module 103 may send a media channel management message to the media server 20, may also send access information, such as a token, to the client, and a bridge channel management command to the intermediate device 30.

The media server 20 may comprise a second signaling module 201, a channel control module 202 and a first media streaming module 203.

The second signaling module 201 is configured to obtain a media channel management message from the scheduling server 10, and obtain access information, such as a token. The media channel management message is used by the media server 20 to create or delete channels.

The channel control module 202 is used to create or delete channels. The channel control module 202 may also be configured to control a channel accessed by the client according to the access information of the client, and perform media negotiation with the client.

The first media stream transmission module 203 is configured to obtain a media stream of the media server 20, and send the media stream to the media server 20. The first media stream transmission module 203 is further configured to receive a media stream sent by a client with bidirectional authority, and send the media stream to the client with bidirectional authority and the client with unidirectional authority. The first media streaming module 203 is further configured to send a media stream to the intermediate device 30, and receive the media stream sent by the intermediate device 30.

The intermediate device 30 may comprise a third signaling module 301, a bridge channel module 302 and a second media streaming module 303.

The third signaling transmission module 301 is configured to obtain a bridge channel management command from the scheduling server 10, where the bridge channel management command is used to instruct to create or delete a bridge channel.

The bridge channel module 302 is configured to create or delete a bridge channel according to the bridge channel management command. The bridge channel module 302 is also used for media negotiation with the media server 20.

The second media streaming module 303 is configured to receive the media stream from the media server 20 and send the media stream to the media server 20.

It should be noted that, each module may be implemented in a software manner or in a hardware manner. Each module is deployed in a device and is realized in a software mode. The modules may be implemented in part by software and in part by hardware, and are not limited thereto.

The multimedia information processing method according to the embodiment of the present application may be applied to the scenario shown in fig. 1B described above. The following describes a media streaming method according to an embodiment of the present application, taking an example that the method is applied to the scenario shown in fig. 1B. The client involved in the media stream transmission method according to the embodiment of the present application includes: client a, client B, client C and client D. In one possible scenario, client a and client B are clients with bidirectional rights, and client C and client D are clients with unidirectional rights; the first scheduling server is the closest scheduling server to the client a and the client C, and the second scheduling server is the closest scheduling server to the client B and the client D. The first media server is the closest media server to client a and client D, and the second media server is the closest media server to client B and client C. In this way, the client a and the client C can access the conference through the first scheduling server, and the client B and the client D can access the conference through the second scheduling server. The client A and the client D can transmit the media stream of the conference through the first media server, and the client B and the client C can transmit the media stream of the conference through the second media server. A method for transmitting a media stream provided in the present application will be described in detail below by taking this case as an example.

It should be understood that the above scenario is only one example scenario, and is not limiting of the present application. For example, the scheduling server through which the client accesses the conference and the media server to which the media stream is transferred are not limited. For example, the client a may also access the conference through the second scheduling server, and the client a may also transmit the media stream through the second media server. In some cases, there may be more or fewer clients, media servers, scheduling servers, etc. The flow of steps involved in other non-enumerated cases may be described with reference to the above-described cases.

The media stream transmission method provided by the application can comprise four flows: the method comprises the steps of accessing a channel by a client, transmitting media streams, changing channel access rights by the client and transmitting the media streams after changing the rights. The steps in the four processes are not limited to the order of execution among the steps. For example, the step of the client accessing the channel associated with the client C in the channel access procedure may be performed before the step of the client a in the media stream transmission procedure associated with the media stream, and may be performed later.

As shown in fig. 3, a flowchart of the steps for accessing a conference for a client is provided.

Step 301: the client a sends a first request to a first scheduling server.

The first request is for client a to request access to a first channel of the primary conference with bi-directional rights.

Step 302: the first scheduling server transmits first access information to the client a and the first media server in response to the first request.

The first access information is used to indicate that the first client is to be accessed to the first frequency channel. The first access information may be a first token. In the following description, access information of the clients B to D is also described by taking a token as an example. The first token is a token for the client A to access the first channel with the bidirectional authority. The client with bidirectional authority to access the first channel has the authority to send the media stream of the first channel and has the authority to receive the media stream of the first channel. In this way, after the client a accesses the first channel with the bidirectional rights, it can send and receive the media stream of the first channel. When a client A sends a media stream of a first channel to a first media server, the media stream of the first channel comprises a first token; the first media server can also determine whether the received media stream of the first channel is the media stream sent by the client a based on the first token. When client a receives a media stream of a first channel from a first media server, the media stream of the first channel includes a first token; the client a can also determine whether the received media stream is a media stream of the first channel based on the first token.

Step 303: the client B sends a second request to a second scheduling server.

The second request is for client B to request access to the first channel of the primary conference with bi-directional rights.

Step 304: the second scheduling server sends a second token to client B and the second media server in response to the second request.

The second token is a token for the client B to access the first channel with two-way rights. Thus, the client B can obtain the bidirectional right to access the first channel, and can transmit and receive the media stream of the first channel.

Step 305: the client C sends a third request to the first scheduling server.

The third request is for client C to request access to the primary conference with unidirectional rights. In the step flow shown in fig. 3, the conference that the client C accesses correspondingly may be set as the first sub-conference or the main conference in the first scheduling server. When the channel which the client C requests to access by default is a first channel corresponding to the main conference, the first scheduling server can adjust the channel which the client C accesses the conference to be a second channel corresponding to the first sub-conference. It is thus possible to cause client C to redirect to the first sub-conference and access the first sub-conference with unidirectional rights by performing steps 306-308.

Step 306: the first dispatch server sends a first redirect message to client C in response to the third request.

The first redirect message is used to instruct client C to access the second frequency channel with unidirectional rights.

Wherein the second channel is the channel of the first conference.

Step 307: the client C sends a fourth request to the first scheduling server in response to the first redirect message.

The fourth request is for client C to request access to the second channel with unidirectional rights.

Step 308: the first scheduling server sends a third token to the client C and the second media server.

The third token is a token for the client C to access the second channel with one-way rights. In this way, the client C can access the second channel with unidirectional authority, and receive the media stream of the second channel. The client with unidirectional authority to access the second channel has the authority to receive the media stream of the second channel.

Steps 305-308 above describe the procedure of client C accessing the second channel with unidirectional rights by redirecting. In some embodiments, steps 306-307 are optional steps, and the client C may directly access the second channel with unidirectional rights without redirection. For example, the third request may also be directly used for the client C to request to access the second channel with unidirectional rights, and the second scheduling server directly sends the third token to the client C without redirection, where the third token is a token that the client C accesses the second channel. In this way, the client C can directly access the second channel, access the second channel with unidirectional rights, and receive the media stream of the second channel.

Step 309: the client D sends a fifth request to the second scheduling server.

The fifth request is for client D to request access to the first channel with unidirectional rights. In the step flow shown in fig. 3, an access channel corresponding to the client D may be set as the second channel in the second scheduling server. And when the access channel corresponding to the client D is set as a second channel in the second scheduling server and the client D requests to access the first channel, the second scheduling server can adjust the access channel of the client D as the second channel. It is possible to cause client D to redirect to the second channel and access the second channel with unidirectional rights by performing steps 310-312.

Step 310: the second scheduling server sends a second redirect message to the client D in response to the fifth request.

The second redirection message is used to instruct client D to access the second frequency channel with unidirectional rights.

Step 311: client D sends a sixth request to the second scheduling server in response to the second redirect message.

The sixth request is for client D to request access to the second channel with unidirectional rights.

Step 312: the second scheduling server sends a fourth token to the client D and the first media server.

The fourth token is a token for the client D to access the second channel. In this way, the client D can directly access the second channel, access the second channel with unidirectional rights, and receive the media stream of the second channel.

Steps 309-312 describe the procedure of client D accessing the second channel with unidirectional rights by redirecting. In some embodiments, steps 310 to 311 are optional steps, and the client D may directly access the second channel with unidirectional rights without redirection. For example, the fifth request may also be directly used for the client D to request to access the second channel with unidirectional rights, and the second scheduling server directly sends the fourth token to the client D without redirection, where the fourth token is a token that the client D accesses the second channel. In this way, the client D can directly access the second channel, access the second channel with unidirectional rights, and receive the media stream of the second channel.

It should be noted that, the steps of accessing the channels by the different clients are not limited in execution sequence. For example, the step 301 of accessing the first channel by the client a may be performed before or after any of the steps 309-312 of accessing the second channel by the client D.

In addition, the step of sending the request from each client to the scheduling server, and the step of sending the token from the scheduling server to the client and the media server are also optional steps. For example, the scheduling server and the media server may not be separately provided, and the scheduling server and the media server may be integrated. Moreover, the client may not access the corresponding channel through interaction with the scheduling server, for example, the client and the media server may each obtain the first token through running the configuration file.

As shown in fig. 4, a flowchart of the steps of media streaming is shown.

Step 401: the client a sends a first media stream of a first channel to a first media server.

Step 402: the first media server sends a first media stream to the second media server.

Step 403: the second media server sends the first media stream to client B.

Step 404: the first scheduling server sends a first indication message to the intermediate device.

The first indication message instructs the intermediary device to create a first bridge channel. The first bridge channel may be understood as a virtual client, and has functions of receiving, processing and transmitting. The bridge channel may be a software module running in the intermediate device or may be hardware. For example, a plurality of chips may be provided in the intermediate device, and the chips may have functions of receiving, processing, and transmitting, and after the intermediate device receives the first indication message, the functions of the chips are enabled.

Step 405: the intermediary device creates a first bridge channel.

Steps 404 to 405 are optional steps, and the intermediate device may also be used for media stream transmission between different channels without setting a separate bridge channel; for example, the intermediate device may store media streams from different channels in different buffers, and process the media streams of different channels separately through the different buffers.

Step 406: the first media server sends a first media stream to the intermediary device.

Step 407: the intermediate device obtains a second media stream of the second channel based on the first media stream processing.

The specific implementation of step 407 may be as follows:

the intermediate device modifies the first token in the first media stream into a second token to obtain a second media stream.

Step 408: the intermediary device sends the second media stream to the first media server.

Steps 406-408 may be performed by a first bridge channel of the intermediary device. Steps 406 to 408 are optional steps, and in another implementation, the intermediate device may further receive the first media stream sent by the first client, process the first media stream to obtain a second media stream, and send the first media stream and the second media stream to the first media server.

Step 409: the first media server sends a second media stream to the second media server.

Step 410: the second media server sends a second media stream to the client C.

Step 411: the first media server sends a second media stream to the client D.

In the mode, the client can be scattered to access a plurality of channels by setting the sub-conference. And the media stream content of the main conference and the sub conference are consistent, and the media stream of the client can be transmitted in a scattered way through each channel through the intermediate equipment. And, the media streams of the clients in each channel can be logically isolated, and each channel only needs to process the media streams of the clients in the channel separately.

The above flow only uses two channels as an illustration of the media stream transmission process on different channels. The number of sub-conferences is not limited in the present application, and there may be a plurality of sub-conferences, and the number of channels is not limited. With the increase of the number of clients and the number of channels, the complexity of media stream processing by the media stream transmission method provided by the application is reduced more and more obviously. For example, according to the identifiers (such as job numbers) corresponding to the clients, the media streams are sequentially forwarded according to the order obtained by the bubbling sequencing, the number of times required to be compared by the bubbling sequencing is n (n-1)/2, and n is the total number of the clients participating in the sequencing. When the total number of clients is 2000, if all clients are in one channel according to the prior art, the comparison is 1999000 times. If the media streaming method provided by the application is used, it is assumed that there are 1 main conference and 9 sub-conferences, and there are 10 corresponding channels, and each channel is 200 clients. The number of comparisons per channel is 19900 and the total number of comparisons for 10 channels is 199000, which is only 1/10 of the prior art method, obviously greatly reducing the complexity of media stream processing.

The step flows shown in fig. 3-4 describe a specific process for each client to access a channel and transmit a media stream. From the foregoing description, it is clear that clients may access channels with unidirectional rights, and may access channels with bidirectional rights. However, during the conference opening process, the access rights of the clients may change. For example, a client may change from a unidirectional entitlement access channel to a bidirectional entitlement access channel, and a client may also change from a bidirectional entitlement access channel to a unidirectional entitlement access channel.

As shown in fig. 5, a flowchart of the steps for changing channel access rights for a client is provided. Fig. 5 illustrates the change of the rights of the access channels of the client B and the client C, that is, the bidirectional rights of the client B are changed to the unidirectional rights, and the unidirectional rights of the client C are changed to the bidirectional rights.

Step 501: the first dispatch server sends a notification message to the second dispatch server.

The notification message is used for indicating the second scheduling server to change the bidirectional permission of the client B to the unidirectional permission.

It should be noted that, the case corresponding to step 501 is a case where the first scheduling server determines the permission change of the client B. Step 501 is an optional step, and in some cases, the second scheduling server may directly determine the rights change of client B, i.e. step 502 or step 503 is directly performed without performing step 501.

Step 502: the second dispatch server sends a third redirect message to client B.

The third redirect message is used to instruct client B to access a third channel with unidirectional rights.

The third channel is the channel of the second conference. It should be noted that, step 502 is an optional step, and the second scheduling server may not notify the client B of the change of the authority of the client B to access the channel. For example, the client B may execute step 503 after sending the media stream to the second media server more than or equal to the set number of times. The client B may also autonomously request the right to change the access channel, i.e. step 503 is directly performed without performing step 502.

Step 503: the client B sends a seventh request to the second scheduling server.

The seventh request is for client B to request access to the third channel with unidirectional rights. It should be noted that, in the present application, the channel to which the client B redirects is not limited, but only a third channel corresponding to the second conference is taken as an example and described. In some cases, the seventh request may also be for client B to request access to the first channel or the second channel with unidirectional rights.

Step 504: the second scheduling server sends a fifth token to client B and the second media server.

The fifth token is a token for the client B to access the third channel with one-way authority.

Step 505: the first scheduling server sends a fourth redirect message to the client C.

The fourth redirection message is used to instruct the client C to access the first frequency channel with bidirectional rights.

It should be noted that, step 505 is an optional step, and the first scheduling server may not notify the client C of the change of the authority of the client C to access the channel. For example, the client C may also send a probing packet to the first media server according to a preset period, and if the sending is successful, step 506 is performed.

Step 506: the client C sends an eighth request to the first scheduling server.

The eighth request is for client C to request access to the first channel with bidirectional rights.

Step 507: the first scheduling server sends a sixth token to the client C and the second media server.

The sixth token is a token for the client C to access the first channel with the bidirectional right.

As shown in fig. 6, a flowchart of the steps of media streaming after the client changes the access rights of the conference is provided.

Step 601: the client C sends a third media stream of the first channel to the second media server.

Step 601 is an optional step. For example, the third media stream may be sent by client a to the second media server.

Step 602: the second scheduling server sends a third indication message to the intermediate device.

The third indication message instructs the intermediate device to create a second bridge channel, similar to the first bridge channel, and reference may be made in particular to the relevant description of the first bridge channel.

Step 603: the intermediary device creates a second bridge channel.

Steps 602-603 are optional steps, and other optional implementations may refer to the relevant description of the first bridge channel.

Step 604: the second media server sends a third media stream to the intermediary device.

Step 605: and the intermediate device obtains a fourth media stream of the third channel according to the third media stream of the first channel.

Specific implementations of step 605 may refer to the relevant description of step 407.

Step 606: the intermediary device sends a fourth media stream to the second media server.

Step 607: the second media server sends a fourth media stream to client B.

As shown in fig. 7, the present application provides a media streaming apparatus. The device comprises:

an obtaining module 701, configured to obtain a first media stream of a first channel, where the first client accesses the first channel, and the first channel is a channel of a main conference; acquiring a second media stream of a second channel, wherein the second media stream is obtained based on the first media stream, and the second channel is any conference-dividing channel; a sending module 702, configured to send the second media stream to a second client, where the second client accesses the second channel.

In one possible design, the sending module 702 is specifically configured to: transmitting the first media stream to an intermediate device; the acquiring module 701 is specifically configured to: and receiving the second media stream sent by the intermediate equipment.

In one possible design, the obtaining module 701 is further configured to:

and acquiring a third media stream of the third channel.

In one possible design, the sending module 702 is further configured to:

and sending the media stream of the first channel to a second media server.

The embodiment of the application also provides an electronic device, which may have a structure as shown in fig. 8, and may be a computer device or a chip system capable of supporting the computer device to implement the method.

An electronic device as shown in fig. 8 may comprise at least one processor 801, the at least one processor 801 being configured to couple to a memory, read and execute instructions in the memory to implement any steps of the media streaming method provided by embodiments of the present application. Optionally, the electronic device may further comprise a communication interface 802 for supporting the electronic device for signaling or data reception or transmission. A communication interface 802 in the electronic device may be used to enable interaction with other electronic devices. The processor 801 may be used to implement the electronic device to perform the steps in the methods shown in fig. 3-7.

Optionally, the electronic device may further comprise a memory 803 in which computer instructions are stored, the memory 803 may be coupled to the processor 801 and/or the communication interface 802 for supporting the processor 801 to call the computer instructions in the memory 803 to implement any of the steps of the methods as shown in fig. 3-7; in addition, the memory 803 may also be used to store data related to embodiments of the methods of the present application, for example, to store data, instructions necessary to support interaction by the communication interface 802, and/or to store configuration information necessary for the electronic device to perform the methods described by embodiments of the present application.

Embodiments of the present application also provide a computer readable storage medium, where computer instructions are stored, where the computer instructions, when executed by a computer, may cause the computer to perform the method involved in any one of the possible designs of the method embodiments and the method embodiments described above. In the embodiment of the present application, the computer readable storage medium is not limited, and may be, for example, RAM (random-access memory), ROM (read-only memory), or the like.

The present application also provides a chip, which may include a processor coupled to a memory for reading and executing a software program stored in the memory for performing the method involved in any one of the possible implementations of the method embodiments, the method embodiments described above, wherein "coupled" means that the two components are directly or indirectly combined with each other, which combination may be fixed or movable.

The present application also provides a computer program product which, when read and executed by a computer, causes the computer to perform the method as referred to in any one of the possible implementations of the method embodiments described above.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present invention are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, optical fiber), or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy Disk, a hard Disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.

The steps of a method or algorithm described in the embodiments of the present application may be embodied directly in hardware, in a software element executed by a processor, or in a combination of the two. The software elements may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. In an example, a storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may reside in a terminal device. In the alternative, the processor and the storage medium may reside in different components in a terminal device.

These computer instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Claims

1. A method of media streaming, comprising:

a first media server acquires a first media stream of a first channel, wherein the first client accesses the first channel, and the first channel is a channel of a main conference;

the first media server acquires a second media stream of a second channel, wherein the second media stream is processed based on the first media stream, and the second channel is any conference-dividing channel;

the first media server sends the second media stream to a second client, and the second client accesses the second channel.

2. The method of claim 1, wherein,

the first media server obtaining a second media stream of a second channel, comprising:

the first media server sends the first media stream to an intermediate device;

the first media server receives the second media stream sent by the intermediate device.

3. The method of claim 1 or 2, wherein the method further comprises:

4. A method as claimed in claim 3, wherein the method further comprises:

and after the authority information of the first client is changed, the first media server receives second access information sent by the first scheduling server, wherein the second access information is used for indicating that the access channel of the first client is switched to a third channel, and the third channel is any channel for sub-conference.

5. A method as claimed in claim 3, wherein the method further comprises:

the first media server obtains a third media stream of the third channel.

6. A method according to any of claims 3 to 5, wherein the first access information is determined from rights information of the first client and location information of the first client.

7. The method of any one of claims 1 to 6, wherein the method further comprises:

the first media server sends the media stream of the first channel to a second media server.

8. A media streaming system, the system comprising: the system comprises a first media server, a first client and a second client;

The first media server is configured to obtain a first media stream of a first channel, where the first client accesses the first channel, and the first channel is a channel of a main conference;

the first media server is configured to obtain a second media stream of a second channel, where the second media stream is obtained based on the processing of the first media stream, and the second channel is any channel of a conference;

the second client is configured to receive the second media stream sent by the first media server, where the second client accesses the second channel.

9. The system of claim 8, wherein the system further comprises an intermediary device;

the first media server is specifically configured to:

transmitting the first media stream to the intermediate device;

the intermediate device is configured to obtain the second media stream based on the first media stream processing, and send the second media stream to the first media server.

10. The system of claim 8, wherein the system further comprises an intermediary device;

the first client is further configured to send the first media stream to the intermediate device;

the intermediate device is configured to obtain the second media stream based on the first media stream, and send the first media stream and the second media stream to the first media server.

11. A media streaming apparatus, comprising:

the system comprises an acquisition module, a first client and a second client, wherein the acquisition module is used for acquiring a first media stream of a first channel, the first client is accessed to the first channel, and the first channel is a channel of a main conference; acquiring a second media stream of a second channel, wherein the second media stream is obtained based on the first media stream, and the second channel is any conference-dividing channel;

and the sending module is used for sending the second media stream to a second client, and the second client accesses the second channel.

12. The apparatus of claim 11, wherein the sending module is specifically configured to:

transmitting the first media stream to an intermediate device;

the acquisition module is specifically configured to:

and receiving the second media stream sent by the intermediate equipment.

13. The apparatus of claim 11 or 12, wherein the acquisition module is further to:

14. The apparatus of claim 13, wherein the acquisition module is further to:

15. The apparatus of claim 13, wherein the acquisition module is further to:

and acquiring a third media stream of the third channel.

16. The apparatus according to any of claims 13 to 15, wherein the first access information is determined based on rights information of the first client and location information of the first client.

17. The apparatus of any of claims 11 to 16, wherein the transmitting module is further configured to:

and sending the media stream of the first channel to a second media server.

18. An electronic device, the electronic device comprising: one or more processors; one or more memories; wherein the one or more memories store one or more computer instructions that, when executed by the one or more processors, cause the electronic device to perform the method of any of claims 1-7.

19. A computer readable storage medium comprising computer instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 7.