CN110072128B

CN110072128B - Real-time pushing method of media stream and server

Info

Publication number: CN110072128B
Application number: CN201910324891.9A
Authority: CN
Inventors: 姜红旗; 辛振涛; 姜红艳; 申素辉
Original assignee: Beijing Kaiguang Information Technology Co ltd
Current assignee: Beijing Kaiguang Information Technology Co ltd
Priority date: 2019-04-22
Filing date: 2019-04-22
Publication date: 2021-01-15
Anticipated expiration: 2039-04-22
Also published as: CN110072128A

Abstract

The invention discloses a real-time pushing method of media streams and a server, wherein the method comprises the following steps: receiving a media segment request sent by a client, wherein the media segment request carries at least one command, and the command comprises: starting a push command; starting a push task according to a push starting command, wherein when the push starting command carries at least one of a first type parameter indicating a target media stream to be transmitted, a second type parameter indicating the characteristics of candidate media units to be transmitted and a segmentation strategy, the target media stream to be transmitted is determined according to the first type parameter, a push task identifier is distributed for the push task, an initial media segment is generated according to the second type parameter, a new media segment is continuously generated according to the segmentation strategy, and the push task identifier and the generated media segment are sent to a client. The method realizes the segmented pushing of the media stream as required, simplifies the pushing process of the server side, and improves the transmission efficiency of the media stream and the support for large-scale self-adaptive pushing.

Description

Real-time pushing method of media stream and server

Technical Field

The present invention relates to the field of digital information transmission technologies, and in particular, to a real-time media stream pushing method and a server.

Background

With the rapid development of the internet, especially the mobile internet, the real-time transmission of multimedia data such as audio, video, and image through the internet becomes a basic requirement for many applications, and in order to meet the requirement, various streaming media real-time transmission technologies are proposed, which can be divided into two categories according to the different origins of data transmission: one is a stream pulling mode, the basic principle is that a client actively requests real-time data from a server, and the technical scheme of the stream pulling mode comprises the following steps: hls (http Live Streaming) proposed by apple, Smooth Streaming proposed by microsoft, hds (http Dynamic Streaming) proposed by Adobe, dash (Dynamic Adaptive Streaming over http) proposed by MPEG organization; the other type is a stream pushing mode, the basic principle is that a server actively pushes a media stream generated in real time to a client, and the technical scheme adopting the stream pushing mode comprises the following steps: Real-Time Transport Protocol (RTP)/RTSP), Real-Time Messaging Protocol (RTMP), and HTTP-FLV Protocol.

Compared with a stream pulling mode, a stream pushing mode does not need to wait for a request of a client, and can quickly send media stream data generated in real time to the client to realize low delay, so that the pushing-based mode is generally adopted in application occasions with higher real-time requirements, such as interactive video live broadcast, video monitoring and the like. However, the existing various stream push schemes have the following problems:

problem 1, lack of segment push of media data. In existing streaming push schemes, once media data is generated, the media data is immediately sent to a transmission buffer, but if the data amount of a single media unit is small, frequent single push will increase network transmission overhead and reduce transmission efficiency.

Problem 2, the push control is completely done by the server, and it is difficult to support large-scale adaptive push. In various existing streaming push schemes, the push process is completely controlled by the server, and the client only passively receives pushed data. The server needs to maintain a push process for each client, so as to monitor the transmission condition of the media stream to each client in real time, and handle various demand changes of the client and various connection anomalies on the internet, such as network congestion, packet loss, connection interruption, and the like. Due to the large processing overhead of a single push process, it is difficult for the server to support large-scale adaptive media stream pushing.

Disclosure of Invention

The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.

Therefore, a first objective of the present invention is to provide a real-time pushing method for a media stream, which can implement segmented pushing of the media stream as needed, simplify the pushing process at the server end, improve the transmission efficiency of the media stream, and support large-scale adaptive pushing.

The second objective of the present invention is to provide a real-time push server for media streams.

A third object of the invention is to propose a computer device.

A fourth object of the invention is to propose a non-transitory computer-readable storage medium.

A fifth object of the invention is to propose a computer program product.

In order to achieve the above object, a first embodiment of the present invention provides a method for pushing a media stream in real time, where the media stream is a sequence of media units generated in real time, where each media unit is associated with a generation time and/or a sequence number indicating a generation order, and the method includes the following steps:

receiving a media segment request sent by a client, wherein the media segment request carries at least one command, and the command comprises: starting a push command;

and starting a push task according to the push starting command, wherein when the push starting command carries at least one of a first type parameter indicating a target media stream to be transmitted, a second type parameter indicating the characteristics of candidate media units to be transmitted and a segmentation strategy, the target media stream to be transmitted is determined according to the first type parameter, a push task identifier is distributed for the push task, an initial media segment is generated according to the second type parameter, a new media segment is continuously generated according to the segmentation strategy, the push task identifier and the generated media segment are sent to the client, and the candidate media units in the target media stream are packaged by the media segment.

The real-time pushing method of the media stream of the embodiment of the invention can realize the segmented pushing of the media stream as required, and the segmented pushing can aggregate small media units into media segments, thereby reducing the pushing times and improving the transmission efficiency of the media stream; secondly, the pushing process of the server is simplified, the server only needs to generate media segments according to the strategy specified by the client and send the media segments to the client, a complex adaptive algorithm is not needed to deal with various problems in the transmission process, the state needing to be maintained and the processing cost of the server are reduced, and the large-scale media stream pushing is favorably supported; finally, each client can adjust the pushing process of the server at any time according to the transmission condition of the media stream, so that the pushing process can adapt to the network condition and the change of the client requirement more quickly.

In addition, the real-time receiving method of the media stream according to the above embodiment of the present invention may further have the following additional technical features:

in an embodiment of the present invention, when the push start command does not carry the first type of parameter, the target media stream is specified by default;

when the push starting command does not carry the second type of parameters, the initial media segment encapsulates a default specified media unit;

and when the pushing starting command does not carry the segmentation strategy, the segmentation strategy of the pushing task is specified by default.

In an embodiment of the present invention, the default designated media unit is a media unit in which sequence number intervals of all and latest media units in the target media stream are smaller than a first preset value, or a media unit in which generation time intervals of all and latest media units in the target media stream are smaller than a second preset value.

In one embodiment of the present invention, when the second type of parameter carried by the start push command includes a start sequence number and the start sequence number is valid, the sequence number of the candidate media unit is subsequent to the start sequence number.

In one embodiment of the present invention, when the second type of parameter carried by the start push command includes a start time and the start time is valid, the generation time of the candidate media unit is after the start time.

In an embodiment of the present invention, each media unit is associated with a priority, and when the second type of parameter carried by the push initiation command includes a unit priority range, the priority of the candidate media unit is within the unit priority range.

In an embodiment of the present invention, the segment policy is pushed at equal sequence number intervals, and parameters carried by the segment policy include: assigning sequence number intervals, and generating new media segments according to the segmentation strategy, comprising the following steps:

s801, determining a push start sequence number when an initial media segment is generated;

s802, when a new media unit is generated, judging whether the interval between the sequence number of the new media unit and the push starting sequence number is equal to the specified sequence number interval, if so, packaging all candidate media units with the sequence numbers behind the push starting sequence number into a new media segment, and updating the push starting sequence number;

s803, S802 is repeatedly executed.

In an embodiment of the present invention, the segmentation policy is equal time interval pushing, and parameters carried by the segmentation policy include: specifying a time interval, wherein the generating of the new media segment according to the segmentation policy comprises:

and from the time of generating the initial media segment, packaging the candidate media units generated in the time segment into a new media segment every time a time segment with the duration of the specified time interval passes.

In an embodiment of the present invention, a plurality of media units with consecutive sequence numbers form a unit set, the segmentation policy is pushed by the unit set, and parameters carried by the segmentation policy include: specifying the number of the unit sets, and generating a new media segment according to the segmentation strategy comprises the following steps:

when a new media unit is generated, judging whether the new media unit is the last media unit of a unit set, if so, judging whether the number of the unit sets to be pushed currently is equal to the number of the specified unit sets, and if so, packaging the candidate media units in the unit sets to be pushed currently into a new media segment.

In an embodiment of the present invention, a deadline is set for each started push task, and when the deadline is reached, the push task is terminated, wherein the terminating the push task includes: the generation of new media segments is stopped.

In one embodiment of the invention, the command further comprises: continuing to push the command, wherein the continuing to push the command carries at least one command parameter: pushing a task identifier, the method further comprising:

and resetting an deadline time for the pushing task corresponding to the pushing task identifier carried by the continuous pushing command according to the continuous pushing command, wherein the new deadline time for the pushing task is the sum of the current system time and a preset pushing duration time.

In one embodiment of the invention, the command further comprises: terminating a push command, wherein the terminating push command carries at least one command parameter: pushing a task identifier, the method further comprising:

and terminating the push task corresponding to the push task identifier carried by the push termination command according to the push termination command, wherein the generation of the new media segment is stopped.

In one embodiment of the invention, the command further comprises: restarting a push command, wherein the restart push command carries a push task identifier, and the restart push command does not carry or carries at least one of the following command parameters: a first class of parameters indicative of a target media stream to be transmitted, a second class of parameters indicative of a candidate media unit characteristic to be transmitted, and a segmentation policy, the method further comprising:

restarting a push task corresponding to the push task identifier carried by the restart push command according to the restart push command, wherein a current media segment generation and sending process of the push task is stopped at first, a target media stream to be transmitted is determined according to the first type of parameters carried by the restart push command, an initial media segment is generated, a new media segment is generated continuously according to the segmentation strategy carried by the restart push command, the generated media segment is sent to a client, and the media segment encapsulates candidate media units determined according to the second type of parameters carried by the restart push command in the target media stream.

In an embodiment of the present invention, the media segment further encapsulates media stream index information, where the media stream index information includes a plurality of media stream description information belonging to the same content, and the media stream description information includes a media stream identifier and a media stream bit rate.

In one embodiment of the invention, the command further comprises: a media segment pull command, wherein the media segment pull command does not carry or carries at least one command parameter, and the command parameter comprises: a first class of parameters indicative of a target media stream to be transmitted and a second class of parameters indicative of a candidate media unit characteristic to be transmitted, the method further comprising:

and generating a media segment according to the media segment pulling command and sending the generated media segment to a client, wherein the target media stream to be transmitted is determined according to the first type of parameters, the candidate media unit to be transmitted is determined according to the second type of parameters, and the candidate media unit to be transmitted is packaged into the media segment.

In order to achieve the above object, a second aspect of the present invention provides a real-time push server for a media stream, wherein the media stream is a sequence of media units generated in real time, and each media unit is associated with a generation time and/or a sequence number indicating a generation sequence, the server includes:

the client interface component is used for receiving a media segment request sent by a client and returning a corresponding media segment, wherein the media segment request carries at least one command, and the command comprises: starting a push command;

a media segment generating component, configured to start a push task according to a push starting command in the media segment request, where when the push starting command carries at least one of a first type parameter indicating a target media stream to be transmitted, a second type parameter indicating a candidate media unit feature to be transmitted, and a segmentation policy, the media segment generating component determines the target media stream to be transmitted according to the first type parameter, allocates a push task identifier to the push task, generates an initial media segment according to the second type parameter, continuously generates a new media segment according to the segmentation policy, and sends the push task identifier and the generated media segment to the client through the client interface component, where the media segment encapsulates the candidate media unit in the target media stream.

The real-time pushing server of the media stream can realize the segmented pushing of the media stream as required, and the segmented pushing can aggregate small media units into media segments, thereby reducing the pushing times and improving the transmission efficiency of the media stream; secondly, the pushing process of the server is simplified, the server only needs to generate media segments according to the strategy specified by the client and send the media segments to the client, a complex adaptive algorithm is not needed to deal with various problems in the transmission process, the state needing to be maintained and the processing cost of the server are reduced, and the large-scale media stream pushing is favorably supported; finally, each client can adjust the pushing process of the server at any time according to the transmission condition of the media stream, so that the pushing process can adapt to the network condition and the change of the client requirement more quickly.

In an embodiment of the present invention, the segment policy is pushed at equal sequence number intervals, and parameters carried by the segment policy include: specifying a sequence number interval, the media segment generation component further configured to perform the steps of:

s803, S802 is repeatedly executed.

In an embodiment of the present invention, the segmentation policy is equal time interval pushing, and parameters carried by the segmentation policy include: specifying a time interval, the media segment generation component further to:

In an embodiment of the present invention, a plurality of media units with consecutive sequence numbers form a unit set, the segmentation policy is pushed by the unit set, and parameters carried by the segmentation policy include: the number of the unit sets is specified, and the media segment generation component is further configured to:

In one embodiment of the present invention, the media segment generation component is further configured to:

setting a deadline for each started push task, and terminating the push task when the deadline is reached, wherein the terminating the push task comprises: the generation of new media segments is stopped.

In one embodiment of the invention, the command further comprises: continuing to push the command, wherein the continuing to push the command carries at least one command parameter: the task identifier is pushed, and the media segment generation component is further configured to:

In one embodiment of the invention, the command further comprises: terminating a push command, wherein the terminating push command carries at least one command parameter: the task identifier is pushed, and the media segment generation component is further configured to:

In one embodiment of the invention, the command further comprises: restarting a push command, wherein the restart push command carries a push task identifier, and the restart push command does not carry or carries at least one of the following command parameters: a first type of parameter indicating a target media stream to be transmitted, a second type of parameter indicating a candidate media unit feature to be transmitted, and a segmentation policy, the media segment generation component being further configured to:

In one embodiment of the invention, the command further comprises: a media segment pull command, wherein the media segment pull command does not carry or carries at least one command parameter, and the command parameter comprises: a first type of parameter indicating a target media stream to be transmitted and a second type of parameter indicating a candidate media unit feature to be transmitted, the media segment generation component further configured to:

In order to achieve the above object, a third embodiment of the present invention provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the real-time pushing method for media streams as described in the above embodiments.

To achieve the above object, a fourth aspect of the present invention provides a non-transitory computer-readable storage medium, where the program is executed by a processor to implement the real-time pushing method for media streams as described in the above embodiments.

To achieve the above object, a fifth aspect of the present invention provides a computer program product, wherein when the instructions in the computer program product are executed by a processor, the method for pushing a media stream in real time as described in the above embodiments is performed.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

fig. 1 is a flowchart of a real-time pushing method of a media stream according to an embodiment of the present invention;

fig. 2 is a schematic diagram of a real-time pushing process of a media stream according to an embodiment of the present invention;

FIG. 3 is a flow diagram of generating new media segments when the segmentation policy is equal sequence number interval push, according to one embodiment of the invention;

FIG. 4 is a diagram illustrating a push process with a segment policy at equal time intervals according to an embodiment of the present invention;

FIG. 5 is a diagram illustrating a push process in which a segmentation policy is pushing a set of units, according to an embodiment of the present invention;

FIG. 6 is a diagram illustrating a process of actively terminating a push task at a server according to an embodiment of the present invention;

FIG. 7 is a diagram illustrating a process of terminating a push task by a server according to a client request according to an embodiment of the present invention;

FIG. 8 is a diagram illustrating a process by which a server adjusts a segmentation policy according to a restart push command, according to an embodiment of the present invention;

FIG. 9 is a diagram illustrating a process of a server switching a target media stream according to a restart push command according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of a real-time push server for media streaming according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.

In the internet, it is often necessary to transmit various real-time generated audio, video or data streams from one network node to another, the network nodes including both various terminals such as PCs, mobile phones, tablets and various application servers such as video servers and audio servers, and the transmitted audio, video or data streams are collectively referred to as media streams. The delivery process of a media stream can be described in a general client-server model: the media stream generated in real time is delivered by the server to the client. The server and the client refer to logical functional entities, where the server is a functional entity that transmits a media stream, and the client is a functional entity that receives a media stream. The server and client may reside on any network node.

Each transmitted media stream is a sequence of media units generated in real time. Different media streams, their corresponding media units can be selected by themselves. When the media stream is a byte stream generated in real time, a byte can be selected as a media unit; when the media stream is an audio stream or a video stream obtained through real-time sampling, an original audio frame or a video frame can be selected as a media unit; when the media stream is an audio stream or a video stream which is sampled and encoded in real time, an encoded audio frame, an encoded video frame or an Access Unit (Access Unit) can be selected as a media Unit; when the media stream is an audio stream or a video stream that is sampled, encoded and encapsulated in real time, the encapsulated transport packets (e.g., RTP packets, PES/PS/TS packets, etc.) may be selected as media units; when the media stream is an audio or video stream that is sampled, encoded, encapsulated, and pre-segmented in real-time, a segmented media segment (e.g., a TS format segment used in HLS protocol, fMP4 format segment used in DASH protocol) may be selected as a media unit.

Each media unit may be associated with a generation time, which is typically a timestamp. Each media unit may also be associated with a sequence number that may be used to indicate the order in which the media units were generated. When sequence numbers are used to indicate the order in which the media units are generated, the meaning of the sequence numbers need to be defined in terms of the specific media unit. When the media unit is a byte, the serial number of the media unit is a byte serial number; when the media units are audio frames and video frames, the serial numbers of the media units are frame serial numbers; when the media unit is a transmission packet, the sequence number of the media unit is a packet sequence number; when the Media unit is a stream segment, the Sequence number of the Media unit is a segment Sequence number (e.g., Media Sequence of each TS segment in HLS).

For a media stream, a Sequence Number indicating the generation Sequence and a generation Time may be associated with a media unit, for example, when the real-Time media stream is an RTP packet stream, the RTP header has both a packet Sequence Number (Sequence Number) field for indicating the Sequence of the RTP packets and a Time-stamp field for indicating the generation Time of the media data encapsulated in the RTP. In this case, multiple consecutive RTP packets may correspond to the same generation time, but their sequence numbers are unique.

The method of the embodiments of the present invention may be implemented for any kind of real-time media stream. In the following embodiments, the RTP real-time media stream or the MPEG2-TS real-time media stream is selected to illustrate the implementation method of the embodiments of the present invention. For an RTP real-time stream, a media unit is an RTP packet, a packet sequence number (sequence number) of the RTP is selected as a sequence number of the media unit, the packet sequence number of the RTP packet is a 16-bit field, the maximum value is 65535, for the RTP packets which are continuously generated, the sequence number is circularly counted, if the current packet sequence number is Seq, the sequence number of the next packet is (Seq + 1)% 65536, therefore, the sequence number is limited by the bit length in realization, the situation that the sequence number size cannot reflect the sequence order can occur possibly, at the moment, whether the sequence number circularly counts or not can be judged according to the generation time of the media unit, and the sequence relation and the interval of the sequence numbers of the two media units can be accurately judged. For MPEG2-TS real-time streams, in a manner similar to HLS/DASH, a TS stream generated or received in real-time may be divided into TS segments of short duration, such as about 1 second, each TS segment may include a plurality of media frames, and then the segments are sequenced in the generation order as media units, and the timestamp of the first media frame contained in each segment indicates the generation time of the segment.

In a traditional real-time streaming media protocol such as RTP, RTMP or HTTP-FLV, a server push method is adopted: once a new media unit exists on the server, the new media unit is actively sent to the client, without involving the segmentation encapsulation of the media stream, but in various HTTP adaptive streams (such as HLS, smooth stream, MPEG-DASH), a pre-segmentation manner is adopted, and the media stream generates media segments according to a fixed time length and issues a manifest file, and then waits for the pull of the client. Different from the above manners, in the embodiment of the present invention, the server segments and pushes the media stream in real time according to the segmentation policy in the request of the client, and the client can adjust the segmentation policy of the media stream at any time.

The following describes a real-time pushing method of a media stream and a server according to an embodiment of the present invention with reference to the drawings, and first, a real-time pushing method of a media stream according to an embodiment of the present invention will be described with reference to the drawings.

Fig. 1 is a flowchart of a real-time pushing method of a media stream according to an embodiment of the present invention.

As shown in fig. 1, the method for pushing a media stream in real time is a sequence of media units generated in real time, wherein each media unit is associated with a generation time and/or a sequence number indicating a generation sequence, and the method includes:

in step S101, a media segment request sent by a client is received, where the media segment request carries at least one command, and the command includes one of: a push command is initiated.

Specifically, the media segment request may be submitted using any network Transmission Protocol, such as the common HyperText Transfer Protocol (HTTP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), quic (quick UDP Internet connection) Protocol, and so on. The server may reserve a designated protocol port to listen and receive media segment requests sent by the client, such as using 80 ports to receive media segment requests sent based on the HTTP protocol. When the client side submits the media segment request by adopting the HTTP protocol, the media segment request can be submitted to the server by adopting an HTTP-GET method or an HTTP-POST method.

At least one command is carried in the media segment request, and each command may carry no or at least one command parameter. The commands may be of various types, including one of: a push command is initiated and new commands can be defined as needed for further implementation. Different commands carry different command parameters. When the command is a push starting command, the command parameters that can be carried include: a first type of parameter indicative of a target media stream to be transmitted, a second type of parameter indicative of a candidate media unit characteristic to be transmitted, and a segmentation policy. Of course, the enlightenment push command may also not carry any command parameters. In a specific embodiment, the command parameters that can be the first type of parameters include: the media stream identifier and the media stream name may be command parameters of a second type of parameters, including: the starting sequence number, the starting time, the priority range, and the segmentation policy may be one of the following policies: equal sequence number interval pushing, equal time interval pushing and unit set pushing. Of course, new command parameters may also be defined for initiating a push command as required for further implementation.

The client needs to adopt a certain packaging method to package the command and the command parameters corresponding to the command into a byte stream or a character stream, and sends the byte stream or the character stream to the server. For example, when HTTP-GET is used to send a media segment request, if the media segment request only carries one command, the command and the command parameters carried by the command can be encapsulated into a character string, and then the character string is encapsulated into a URL for transmission.

An example of a media segment request with HTTP-GET is as follows:

GET“http://www.xxx-server.com/msreqcmd＝PushStart”

GET“http://www.xxx-server.com/msreqcmd＝PushStart&streamID＝601”

GET“http://www.xxx-server.com/msreqcmd＝PushStart&seqBegin＝1008”

GET“http://www.xxx-server.com/msreqcmd＝PushStart&timeBegin＝31000”

GET“http://www.xxx-server.com/msreqcmd＝PushStart&priorityRange＝1-2”

GET“http://www.xxx-server.com/msreqcmd＝PushStart&streamID＝602&seqBegin＝2010”

GET“http://www.xxx-server.com/msreqcmd＝PushStart&segStrategy＝(ESI:3)”

GET“http://www.xxx-server.com/msreqcmd＝PushStart&streamID＝601&timeBegin＝32000&segStrategy＝(ETI:2000)”

in the URL, the parameter names cmd, streamID, seqBegin, timeBegin, priority range, and segstream respectively represent the command type, media stream id, start sequence number, start time, priority range, and segment policy, and all command parameters of a command should be immediately following the command type.

The server may receive the media segment request of the client by using the Web server, extract the command type from the URL of the request, and then extract the corresponding command parameter, for example, when the URL contains cmd ═ PushStart, it indicates that the media segment request carries a push start command, and then classify the subsequent command parameters: if the command parameter is the media stream identifier, the command parameter is a first type parameter; if the parameter is the starting sequence number, the starting time and the priority range, the parameter is the second type parameter.

When a media segment request carries a plurality of commands, the request can also be sent in an HTTP GET mode or an HTTP POST mode. First, each command and the corresponding command parameters are packaged into a character string, and then the character strings are cascaded by a character '&' and attached to the URL part of the HTTP GET REQUEST, or each character string is individually packaged as a line to the BODY (REQUEST BODY) part of the HTTP POST REQUEST.

In step S102, a push task is started according to a push start command, wherein when the push start command carries at least one of a first type parameter indicating a target media stream to be transmitted, a second type parameter indicating a candidate media unit characteristic to be transmitted, and a segmentation policy, the target media stream to be transmitted is determined according to the first type parameter, a push task identifier is allocated to the push task, an initial media segment is generated, a new media segment is continuously generated according to the segmentation policy, the push task identifier and the generated media segment are sent to a client, and the media segment encapsulates the candidate media unit in the target media stream.

In particular, each initiate push command is for one media stream on the current server. If the push command is initiated to carry a first type of parameter, such as a media stream name or a media stream identifier, the first type of parameter may be directly utilized to determine the target media stream. If the command parameters do not include any first type parameters, the server will designate a default media stream as the target media stream to be delivered. For example, the server may determine one media stream of the same type as the historical media stream as the target media stream to be transmitted according to the historical media stream pushed to the client.

The server can simultaneously receive push starting commands of a plurality of clients and start a plurality of push tasks, and in order to distinguish different push tasks, the server distributes a unique push task identifier for each push task and returns the push task identifier to the clients. The client may make adjustments to a particular push task in subsequent requests.

Each push task corresponds to a continuous generation and transmission process of a media segment. The server first generates initial media segments and then generates new media segments according to the segmentation strategy. If no segmentation policy is specified in the request, the server may preset a default segmentation policy for each media stream to generate new media segments, and alternative segmentation policies may refer to later parts of this description of the implementation. The generation process of the media segment can be generated in two ways, one is to establish an independent push process/thread for each push task, the other is to establish a uniform push process/thread for multiple push tasks, and the process/thread can sequentially generate the media segment for each push task in a turn manner.

Each generated media segment encapsulates a candidate media unit indicated by the second type of parameter. The server may encapsulate one or more media units into media segments using a custom encapsulation protocol, for example, a simple encapsulation protocol is as follows: the media segment is composed of a segment head and a segment payload, the segment payload is formed by cascading a plurality of media units, and the segment head indicates the initial position and the length of each media unit, so that the analysis of the client side is facilitated. If the media unit does not include the generation time and/or the sequence number thereof, the generation time and/or the sequence number corresponding to the media unit also needs to be encapsulated in the media segment, which is convenient for the receiving end to receive and analyze. The push task identifier can be directly encapsulated into the segment header of the initial media segment, so that the push task identifier does not need to be separately returned to the client, and the client can analyze the push task identifier from the received initial media segment.

Whether the initial media segment or the new media segment, is sent to the client once generated. When the server receives the media segment request over a bi-directional connection (e.g., an HTTP connection, a TCP connection, or a QUIC connection), the server may send the media segment directly using the bi-directional network connection. When the server receives the media segment request through a unidirectional connection (such as UDP connection), the server can obtain the IP address and the sending port number of the client through receiving the UDP packet, and the sending port number is used as the default receiving port number of the client for receiving the media segment, and then the media segment is sent to the client.

When there are multiple push tasks, the server needs to establish a push task information table for maintaining context information related to the push tasks, including the following table entries: push task identification, client identification, target media stream, candidate media unit characteristic parameters, segmentation policy, current network connection identification (e.g., Socket), and push process/thread identification, among others. When the server receives a new client request, if the client request carries a push task identifier, the current state information of the push task can be acquired by inquiring a push task information table, and further processing is carried out.

Fig. 2 shows a schematic diagram of a push procedure in this embodiment, where a client initiates a media segment request MS _ REQ, where the media segment request MS _ REQ carries a push start command (cmd) and parameters carried in the push start command include a segmentation policy (segStrategy: ESI:3), and after receiving the request, a server first generates an initial media segment MS1, where a push service identifier (PTID) is encapsulated, and sends MS1 to the client, and then generates a subsequent new media segment in real time according to the segmentation policy (MS2, MS3, MS4, … …), and sends the media segment to the client.

With the above method, the server may start the push procedure upon request of the client and generate media segments according to the segmentation policy specified in the request. By combining small media units into a media segment, the method can reduce the pushing times and improve the pushing efficiency. Secondly, in the method, how to segment is determined by the client, the server only needs to generate the media segments according to the segmentation strategy specified by the client and send the media segments to the client, and the abnormal conditions encountered in various pushing processes do not need to be processed, so that the pushing processing process of the server is fully simplified.

The above is a detailed explanation of embodiment 1, and embodiment 2 will be described in detail below, and in the following embodiment, a description will be made of how the server generates the initial media segment.

Further, in an embodiment of the present invention, when the push command is started without carrying any second type parameter, the initial media segment encapsulates a default specified media unit, where the default specified media unit is a media unit whose sequence number interval between all and the latest media unit in the media stream is smaller than a first preset value, or a media unit whose generation time interval between all and the latest media unit in the media stream is smaller than a second preset value.

In order to reduce the transmission delay, the initial media segment typically contains the newly generated media unit, i.e. the newly generated media unit is pushed out immediately, on the other hand, the size of the initial media segment is related to the receiving buffer of the client, and the client stores the initial media segment into the receiving buffer after receiving the initial media segment, so that the length of the initial media segment should not exceed the size of the receiving buffer, and therefore, the server and the client should agree on the number of media units contained in the initial media segment (determined by a first preset value) or the duration of the initial media segment (determined by a second preset value) before pushing.

The following implementation takes MPEG2-TS real-time stream as an example, where a media unit is a TS segment, and each TS segment is associated with a sequence number. As shown in fig. 2, assuming that the sequence number of the newly generated TS fragment is 21, and the first preset value agreed by the client and the server is 2, 2 media units (sequence numbers 20 and 21) are encapsulated in the initial media segment MS 1. Of course, we can also use the second preset value to limit the duration of the initial media segment, for example, assuming that the duration of each TS segment is 1 second, and the second preset value is set to 3 seconds, it means that the duration of the initial media segment does not exceed 3 seconds, i.e. at most includes 3 media units.

In the following, a detailed description will be given of embodiment 3, and in the following embodiment, a description will be given of how the second type of parameter in the start push command affects the generation of the media segment.

The second-type parameter carried in the push starting command is used for indicating the characteristic requirement which should be met by the candidate media unit to be sent, and when a plurality of second-type parameters exist, the candidate media unit to be sent should meet the characteristic requirement of each second-type parameter. In the invention, three optional second-class parameters are provided, and the characteristic requirements corresponding to each parameter are as follows:

1) starting sequence number: when the starting sequence number is valid, the sequence number of the candidate media unit is behind the starting sequence number;

2) starting time: when the start time is valid, the generation time of the candidate media unit is after the start time;

3) priority range: the priority of the candidate media unit is within the priority range.

The validity and invalidity of the second type parameter described above means whether the value of the parameter is within a specified range. Taking the starting sequence number as an example, the value of the starting sequence number cannot exceed the sequence number of the current newest media unit, on the other hand, to ensure real-time performance, the value of the starting sequence number cannot be earlier than the sequence number of a certain existing media unit, and the starting sequence number in the above range is valid. If a certain second type parameter is invalid, it is equivalent to not carrying the second type parameter. When all the second type parameters are invalid, the candidate media unit to be transmitted is the default designated media unit.

In the second category of parameters mentioned above, some parameters, such as start sequence number and start time, only affect the generation of the initial media segment because: as long as the sequence numbers or generation times of the candidate media units in the initial media segment meet the requirements, the generated media segments are encapsulated with newly generated media units, and the sequence numbers and generation times of the media units also meet the requirements necessarily. While other second type parameters, such as priority ranges, affect not only the generation of the initial media segment, but also the generation of all subsequent media segments.

The following implementation process takes an RTP stream with priority as an example, where a media unit is an RTP packet, each RTP packet carries a packet sequence number, and a priority is associated according to payload content carried by the RTP packet. The priority of each media unit can be defined according to specific situations, for example, when the RTP packet is a multimedia data stream, the RTP packet can be divided into three priorities:

priority 3: the RTP package encapsulates audio information;

priority 2: the key video information is encapsulated in the RTP packet;

priority 1: non-key video information is encapsulated in the RTP packet;

assuming that the packet sequence number of the newly generated RTP packet is 1020, the priority associated with each RTP packet is as shown in table 1, starting with the RTP packet having the packet sequence number of 1000.

TABLE 1

When the server receives the following media segment requests:

1)GET“http://www.xxx-server.com/msreqcmd＝PushStart&seqBegin＝1008”

the media segment request carries only one second type parameter: the start sequence number according to which the sequence number of the candidate media unit can only follow the start sequence number, and therefore the media units encapsulated in the initial media segment include 12 RTP packets with sequence numbers from 1009 to 1020. The start sequence number only affects the generation of the initial media segment and does not affect the subsequent new media segments generated according to the default segmentation strategy.

2)GET“http://www.xxx-server.com/msreqcmd＝PushStart&timeBegin＝31000”

The media segment request carries only one second type parameter: the start time according to which the generation time of the candidate media units can only be after the start time, so the media units encapsulated in the initial media segment comprise 16 RTP packets with sequence numbers from 1005 to 1020. The start time only affects the generation of the initial media segment and does not affect the subsequent new media segments generated according to the default segmentation strategy.

3)GET“http://www.xxx-server.com/msreqcmd＝PushStart&seqBegin＝1008&priorityRange＝3”

The media segment request carries only two second type parameters: the sequence number of the candidate media unit can only be 3 according to the indication of the priority range after the start sequence number, and the sequence number and the priority of the candidate media unit should satisfy the indication of the two second-type parameters at the same time, so the media unit encapsulated in the initial media segment includes 3 RTP packets with sequence numbers 1012, 1016 and 1020. The initial sequence number only affects the generation of the initial media segment, and does not affect the new media segment generated subsequently according to the default segmentation strategy, however, the priority range also affects the generation of the subsequent media segment, that is, only the RTP packet with the priority of 3 can be encapsulated in the new media segment generated according to the default segmentation strategy.

Other second type parameters may also be defined at the time of patent implementation, such as the duration of the initial media segment, the maximum order bias of the initial media segment, the maximum time bias of the initial media segment, and so on. It can be seen from the foregoing embodiment that the client can control the server to push the media units with specific characteristics through the second type of parameters carried in the media segment request, so that the pushing process of the server can better adapt to the requirements of the client.

Embodiment 4, in the following embodiments, a description will be given of how a server generates new media segments according to a segmentation policy.

Further, in an embodiment of the present invention, the segment policy carried in the start push command is Equal Sequence number Interval push (ESI, Equal Sequence Interval), and parameters carried by the segment policy include: a method for generating a new media segment according to the segmentation policy by specifying sequence number intervals, as shown in fig. 3, includes the following steps:

s802, when a new media unit is generated, judging whether the interval between the sequence number of the new media unit and the push starting sequence number is larger than or equal to the interval of the appointed sequence number, if so, packaging all candidate media units with the sequence numbers behind the push starting sequence number into a new media segment, and updating the push starting sequence number;

s803, S802 is repeatedly executed.

Specifically, when generating an initial media segment, the server may set the sequence number of the media unit that is newly generated at that time as the initial push sequence number; when a new media unit is generated, the sequence number of the new media unit is checked, once the interval between the sequence number and the initial push sequence number is equal to the specified sequence number interval, a new media segment is generated, and then the sequence number of the new media unit is set as a new initial push sequence number.

Fig. 2 is a schematic diagram of a push process in which the segment policy is equal sequence number intervals, wherein a parameter is carried in the start push command: segStrategy ESI:3, the parameter indicates that the segmentation policy is equal sequence number interval push (ESI), the specified sequence number interval is 3, when the server generates the initial media segment MS1, the push start sequence number is 21, and when the sequence numbers of the newly generated media units are 24, 27, and 30, new media segments MS2, MS3, and MS4 are generated, respectively.

As can be seen from the above embodiments, the client may control the number of media units in each pushed media segment through a segmentation policy, and adapt to the dynamic change of the network by increasing or decreasing the number of media units in the media segment.

Embodiment 5, in the following embodiments, how the server generates new media segments according to a segmentation policy will be further explained.

Further, in an embodiment of the present invention, the segmentation policy carried in the start push command is Equal Time Interval push (ETI, Equal Time Interval), and the parameters carried by the segmentation policy include: and if the time interval is specified, generating a new media segment according to the segmentation strategy, wherein the generation process comprises the following steps: and from the time of generating the initial media segment, packaging the candidate media units generated in the time segment into a new media segment every time a time segment with the duration of a specified time interval passes.

Fig. 4 shows a schematic diagram of a push process with equal time intervals for the segment policy, in which a parameter is carried in the start push command: segStrategy-ETI: 2000, the parameter indicates that the segmentation policy is equal time interval push (ETI), the specified time interval is 2000 milliseconds, the time when the server generates the initial media segment MS1 is t0, the server starts a timer, when the time reaches t0+2000, encapsulates all candidate media units generated within the time period (t0, t0+ 2000) into a new media segment MS2, when the time reaches t0+4000, encapsulates all candidate media units generated within the time period (t0+2000, t0+ 4000) into a new media segment MS3, and so on.

As can be seen from the above embodiments, the client may control the duration of each media segment pushed through the segmentation policy, and adapt to the dynamic change of the network by increasing or decreasing the duration of the media segment.

Embodiment 6, in the following embodiments, how the server generates new media segments according to a segmentation policy will be further explained.

Further, in an embodiment of the present invention, a Unit Set is composed of a plurality of Media units with consecutive sequence numbers, the segmentation policy carried in the start push command is a Unit Set push (MUS), and the parameters carried in the segmentation policy include: and if the number of the unit sets is specified, generating a new media segment according to the segmentation strategy, wherein the method comprises the following steps:

when a new media unit is generated, judging whether the new media unit is the last media unit of a unit set, if so, judging whether the number of the unit sets to be pushed currently is equal to the number of the specified unit sets, and if so, packaging candidate media units in the unit sets to be pushed currently into a new media segment.

In particular, in many media streams, when several sequentially numbered media units have a common characteristic, there may be dependencies between these media units, which are preferably transmitted as a whole if it is meaningless to transmit only one of the media units. For example, when the media stream is a video RTP packet stream, an image frame may be encapsulated into a plurality of RTP packets, the timestamps of the RTP packets are the same, but the sequence numbers of the RTP packets are different, and at this time, the RTP packets with the same timestamp form a unit set. As another example, when selecting video frames as media units, consecutive video frames form a Group of Pictures (GOP), each Group of Pictures forming an independently decodable set, and the GOP may also be considered as a unit set.

Fig. 5 is a schematic diagram of a push process in which a segmentation policy is unit set push, wherein a parameter is carried in a push start command: segStrategy ═ MUS: 2, the parameter indicates that the segmentation strategy is Unit set push (MUS), and specifies 2 Unit set numbers. In this example, all media units that are generated at the same time are considered a set of units, e.g., the two media units U22 and U23 generated at time t1 constitute a set of units, where U23 is the last unit of the set of units; a media unit U24 generated at time t2 also constitutes a set of units, where U24 is the last unit of the set of units. When U24 is generated, the media unit sets to be sent have already reached the requirement of the number of the specified unit sets, at this time, the 3 media units of the two unit sets are packaged into a new media segment and sent to the client.

It can be seen from the above embodiments that the client can make the server perform pushing according to the unit set through the segmentation policy, so that the media units with dependency relationship can be sent as a whole, and the efficiency of delivery is improved.

In the above embodiments 4, 5, and 6, three segmentation policies and specific implementation methods thereof are respectively given, in practical applications, the server may select one of the segmentation policies as a default according to characteristics of a media stream, and when a media segment request of a client does not indicate a segmentation policy, the server performs segmentation according to the default specified segmentation policy, and of course, a new segmentation policy may be defined according to implementation needs.

It should be noted that the push start command carried in the media segment request may or may not carry command parameters, at this time, the server may determine the target media stream according to a preset default specification for the first type of parameters, generate an initial media segment according to a preset default specification for the second type of parameters, and generate a new media segment according to a preset default specified segmentation policy. That is, when the push command is started without carrying command parameters, the target media stream is specified by default, the initial media segment is obtained by encapsulating the media unit specified by default, and the segmentation policy is also specified by default.

Embodiment 7, in the following embodiments, how the server terminates the push task will be explained.

Further, in one embodiment of the present invention, an expiration time is set for each initiated push task, and when the expiration time is reached, the push task is terminated. The terminating the push task comprises: the generation of new media segments is stopped.

Specifically, after the push task identifier is allocated to each started push task, an expiration time may be set for the push task, where the expiration time is the current system time plus a preset push duration. The preset push duration may be a fixed duration (e.g., 60s or 120s), during which the client does not need to repeatedly submit the request, the server actively pushes the media segment to the client, and when the time reaches the deadline, the push task is terminated.

Further, in an embodiment of the present invention, the command carried by the media segment request further includes a type of: continuing to push the command, wherein the continuing to push the command carries at least one command parameter: the task identifier is pushed, and the method of the invention further comprises the following steps: and resetting the deadline of one pushing task for the pushing task corresponding to the pushing task identifier carried by the continuous pushing command according to the continuous pushing command, wherein the new deadline of the pushing task is the sum of the current system time and a preset pushing duration.

Specifically, to prevent the server from terminating the push task when the deadline arrives, the client needs to actively send a continue push command to the server to maintain the push task. In particular implementation, the client needs to know when to send the continue push command to the server, which can be implemented by the following method:

1) after receiving a push starting command, the server packages preset push duration time into the media segment and sends the media segment to the client, and the client self-infers how long the push task will last and sends a push continuing command before the deadline;

2) and the server packages the residual push duration into each media segment and sends the media segment to the client, and the client determines whether to send a push continuation command to the server or not according to the residual push duration. Once the server receives the continue push command, its expiration time is updated.

Fig. 6 is a schematic diagram of a process for actively terminating a push task at a server side. From the above embodiments, even if any abnormality occurs in the client, the server can automatically end the push process, and release the server resources and the transmission bandwidth occupied by the push task. The client can prove that the client is still in a normal receiving state to the server by periodically submitting a push continuing command so as to maintain the push task of the server.

Embodiment 8, in the following embodiments, how the server terminates the push task will be further explained.

Further, in an embodiment of the present invention, the command carried by the media segment request further includes a type of: terminating the push command, wherein the terminating push command carries at least one command parameter: the task identifier is pushed, and the method of the invention further comprises the following steps: terminating the push task corresponding to the push task identifier carried by the push termination command according to the push termination command, wherein the terminating of the push task corresponding to the push task identifier carried by the push termination command comprises: the generation of new media segments is stopped.

In particular, this embodiment presents a method for a client to actively terminate a push task. The client may send a terminate push command to the server to terminate the transfer task when needed. In this way, it can be guaranteed that the server quickly terminates unneeded push tasks.

Further, the client actively terminating the push task may also be used in combination with the server actively terminating the push task in embodiment 7, so as to fully ensure that the server can rapidly and reliably terminate the unnecessary push task, and fig. 7 shows a schematic diagram of a push process when the two methods are used in combination.

Embodiment 9, in the following embodiments, how the server adjusts the push task according to the command of the client to realize the adaptive push will be described.

Further, in an embodiment of the present invention, the command carried by the media segment request further includes a type of: restarting the push command, wherein the restart push command carries the push task identifier, and the restart push command may not carry or carry other command parameters, including at least one of the following parameters: a first type of parameter indicating a target media stream to be transmitted, a second type of parameter indicating a candidate media unit characteristic to be transmitted, and a segmentation policy, the method of the invention further comprising: restarting a push task corresponding to a push task identifier carried by a push restarting command according to the push restarting command, wherein the current media segment generation and sending process of the push task is stopped at first, a target media stream to be transmitted is determined according to a first type of parameter carried by the push restarting command, an initial media segment is generated, a new media segment is continuously generated according to a segment strategy carried by the push restarting command, the generated media segment is sent to a client, and the media segment encapsulates candidate media units determined according to a second type of parameter carried by the push restarting command in the target media stream.

Specifically, the restart push command actually includes two processing steps: step 1) terminating the current push process (including media segment generation and transmission) of the push task corresponding to the push task identifier carried in the restart push command; and step 2) restarting the push task according to the command parameters carried by the push restarting command. The latter step is similar to the process of initiating a push command, except that the task identity does not have to be reassigned to the push task.

When the client receives the pushed media segment, the client may monitor the receiving process, determine the transmission conditions of the current network, such as transmission bandwidth and transmission delay, and then adjust the current pushing process by sending a push restart command, which includes:

1) adjusting feature requirements of candidate media units

The client can judge the transmission bandwidth of the current network through the receiving of the media segments, and if the transmission bandwidth is smaller than the code rate of the media stream, the priority range of the candidate media units can be adjusted, so that the restarted push task only sends the media units with high priority.

The restart push command may carry explicit parameters of the second type to modify the feature requirements of the candidate media unit. When the restart push command only carries the push task identifier and does not carry any other command parameters, the server side considers that the target media stream and the segmentation strategy of the push task are still unchanged, and the original second-class parameters are invalid, at the moment, the server encapsulates the default specified candidate units into initial media segments, and generates new media segments according to the original segmentation strategy.

2) Adjusting segmentation policies

The client can also adjust the segmentation strategy according to the self delay requirement and reliability requirement, for example, when the client needs smaller transmission delay, the sequence number interval or time interval generated by the media segment can be shortened; when the client needs higher reliability, the sequence number interval or time interval of media segment generation can be increased.

Fig. 8 shows a schematic diagram of a server adjusting the fragmentation policy according to a restart push command. In one embodiment, the client sends a start push command "cmd ═ PushStart" to the server, where the start push command carries a parameter "segStrategy ═ ESI:3 ", i.e. the segmentation policy is equal sequence number interval, the designated sequence number interval is 3, the server first returns the initial media segment MS1, where the push task identifier PTID is indicated as 101, and then opens a new media segment generation process according to the segmentation policy. When the client finishes receiving the media segment MS3, it finds that the network transmission bandwidth is sufficient, and at this time, to reduce the transmission delay, the client sends a restart push command "cmd ═ PushRestart" to the server, which carries the push task identifier "PTID ═ 101", the start sequence number "seqBegin ═ 27", and the segmentation policy "segStrategy ═ ESI: 2', after receiving the push restarting command, the server first terminates the existing media segment generation process and transmission process corresponding to the push task, then generates the initial media segment MS4, and then starts the new media segment generation process according to the new segmentation strategy to generate the subsequent new media segment.

3) Switching target media streams

The client can judge the transmission bandwidth of the current network according to the receiving condition of the media segments, and if the transmission bandwidth of the current network cannot support the code rate of the current target media stream, the client can request the server to push a low-code-rate media stream belonging to the content.

Obviously, the server needs to inform the client which media streams currently belong to the same content. In particular implementations, the server may employ a variety of methods to advertise this information to the client.

Further, in an embodiment of the present invention, the media segment further encapsulates media stream index information, where the media stream index information includes multiple media stream description information belonging to the same content, and the media stream description information includes a media stream identifier and a media stream bit rate.

For example, when a plurality of real-time media streams belonging to the same content exist in the server, the real-time media streams can be distinguished by different media stream identifications, and media stream index information of the content is established; this media stream index information actually includes the mapping relationship between the media stream identification and the media stream bit rate. As shown in table 2, the media stream index information corresponding to the same content (e.g. a live concert live) is displayed: the content comprises three real-time media streams, wherein a media stream 1 (marked as 1000, with a code rate of 8Mbps) is a high definition stream, a media stream 2 (marked as 1001, with a code rate of 2Mbps) is a standard definition stream, and a media stream 3 (marked as 1002, with a code rate of 500Kbps) is a mobile standard definition stream.

TABLE 2

Media stream identification	Media stream bitrate
		601	8000Kbps
602	2000Kbps
		603	500Kbps

When a client requests to push a real-time stream with a media stream identifier 601, a server finds that a media stream 2 and a media stream 3 from the same content source exist by querying a media stream index table, and at this time, the server can package the media stream index information into a media segment together with other media units as a control message.

The client can select the target media stream requested to be pushed according to the index information of the media stream and the network transmission condition. Generally, since the media stream index information remains unchanged for a long period of time, it is not necessary to encapsulate the media stream index information in each media segment, and the media stream index information may be encapsulated in media segments selected at intervals or in the first few media segments sent to the client.

Fig. 9 shows a schematic diagram of a server switching a target media stream according to a restart push command. In one embodiment, the client sends a start push command "cmd ═ PushStart" to the server, which carries the media stream identifier "SID ═ 601" and the segmentation policy parameter "segStrategy ═ ESI: 3', the server starts pushing the 8Mbps media stream Streama: firstly, returning to an initial media segment MS1, where a push task identifier PTID is 101 and media stream index information as shown in table 2 are encapsulated, and then starting a media segment generation process according to a segmentation policy to sequentially generate a media segment MS2 and a media segment MS 3. When the client receives MS3, it finds that the actual transmission bandwidth is only 5Mbps, and cannot support the current media stream bitrate of 8Mbps, so the client sends a restart push command "cmd ═ PushRestart" to the server, which carries the push task identifier "PTID ═ 101", the media stream identifier "SID ═ 602" and the segmentation policy "segStrategy ═ ESI: and 3' after the server receives the push restarting command, the server firstly terminates the generation process and the transmission process of the existing 8Mbps media stream StreamA corresponding to the push task, then starts to push the 2Mbps media stream StreamB to generate an initial media segment MS4, and then starts a new media segment generation process according to a new segmentation strategy to generate a subsequent new media segment.

For simplicity, although the code rates of StreamA and StreamB are different in fig. 9, the sequence numbers of media segments generated at the same time are kept the same. Certainly, the serial numbers of StreamA and StreamB may not be consistent, and the client may guarantee the accuracy of time switching by carrying the second type of parameter (such as the start time) in the start push command, thereby guaranteeing the smooth switching of the code streams with different code rates.

It can be seen from the above embodiments that the server can adjust the push process in real time according to the request of the client, thereby implementing adaptive push. Because all the self-adaptive pushing control is completed by the client and the server does not need a complex processing process, the method can support the large-scale self-adaptive pushing of the server.

Embodiment 10, in the following embodiments, how the server supports both push and pull delivery modes will be described.

Further, in an embodiment of the present invention, the command carried by the media segment request further includes a type of: a media segment pull command, the media segment pull command may not carry or carry at least one command parameter, the command parameter comprising: a first type of parameter indicative of a target media stream to be transmitted and a second type of parameter indicative of a characteristic of a candidate media unit to be transmitted, the method further comprising:

and generating a media segment according to the media segment pulling command and sending the generated media segment to the client, wherein a target media stream to be transmitted is determined according to the first type of parameters, candidate media units to be transmitted are determined according to the second type of parameters, and the candidate media units to be transmitted are packaged into the media segment.

Specifically, when receiving the media segment pull command, the server generates only one media segment and sends the media segment to the client. The first type of parameter and the second type of parameter carried by the media segment pull command are consistent with the parameters carried by the start push command, and the second type of parameter may also include: start time, start sequence number, priority range, etc. When receiving the media segment request in the HTTP-GET manner, an example of the media segment request carrying the media segment pull command is as follows:

GET“http://www.xxx-server.com/msreqcmd＝MsPull&streamID＝601”

GET“http://www.xxx-server.com/msreqcmd＝MsPull&seqBegin＝1010”

GET“http://www.xxx-server.com/msreqcmd＝MsPull&timeBegin＝32000”

wherein "cmd ═ MsPull" is used to indicate that the command type is a media segment pull command, streamID, seqBegin, timeBegin respectively represent command parameters: a media stream identification, a start sequence number, a start time.

When the server receives the media segment request, it directly generates a media segment, and the processing procedure of generating the media segment is basically consistent with the processing procedure of generating the initial media segment after receiving the push starting command.

In this embodiment, the server supports both the push start command and the media segment pull command, and can support both the pull of the media stream and the push on demand, thereby better satisfying the transmission requirements of the client.

Next, a real-time push server for a media stream according to an embodiment of the present invention is described with reference to the drawings.

Fig. 10 is a schematic structural diagram of a real-time push server of a media stream according to an embodiment of the present invention, where the media stream is a sequence of media units generated in real time, where each media unit is associated with a generation time and/or a sequence number indicating a generation sequence.

As shown in fig. 10, the real-time push server 10 for media stream includes: a client interface component 100 and a media segment generation component 200.

The client interface component 100 is configured to receive a media segment request sent by a client and return a corresponding media segment, where the media segment request carries at least one command, and the command includes: a push command is initiated. Wherein, the push starting command may not carry or carry at least one command parameter, and the command parameter includes: a first type of parameter indicative of a target media stream to be transmitted, a second type of parameter indicative of a candidate media unit characteristic to be transmitted, and a segmentation policy.

The media segment generating component 200 is configured to start a push task according to a push starting command in the media segment request, where when the push starting command carries at least one of a first type parameter indicating a target media stream to be transmitted, a second type parameter indicating a candidate media unit feature to be transmitted, and a segmentation policy, the media segment generating component determines the target media stream to be transmitted according to the first type parameter, allocates a push task identifier to the push task, generates an initial media segment according to the second type parameter, then continuously generates a new media segment according to the segmentation policy, and sends the push task identifier and the generated media segment to the client through the client interface component 100, where the media segment encapsulates the candidate media unit in the target media stream. When the start-up push command does not carry any command parameters, the target media stream, the initial media segment and the new media segment may be determined by default specifications.

The server 10 of the embodiment of the present invention generates media segments in real time according to the request of the client, and returns the media segments to the client, so as to implement segmented pushing of the media stream as required, simplify the pushing process of the server, and improve the transmission efficiency of the media stream and support for large-scale adaptive pushing.

Specifically, the client interface component 100 is configured to receive a media segment request of a client, and send a generated media segment to the client; the start push command carried in the media segment request may carry 0, 1 or more command parameters; the command parameters include the following categories: a first type of parameter, a second type of parameter and a segmentation strategy; the first type of parameter is used for indicating a target media stream to be transmitted; the second type of parameter is used to indicate characteristics of the candidate media units to be transmitted. The client interface component may employ any specified protocol to receive media segment requests, for example, when the HTTP protocol is employed, the client interface component may be a Web server that may receive any media segment requests employing the HTTP protocol and return the media segments via HTTP responses; when using the TCP protocol, the client interface component is a TCP server and provides a fixed service port for receiving client requests.

The media segment generation component 200 is configured to start a push task according to a start push command in the media segment request, where the push task is responsible for generating the media segment. The media segment generation component 200 obtains a push starting command and command parameters thereof from the client interface component 100, determines a target media stream to be transmitted according to the first type of parameters, allocates a unique push task identifier for the push task, generates an initial media segment, delivers the push task identifier and the initial media segment to the client interface component for transmission, then continuously generates a new media segment according to a segmentation strategy, and delivers the new media segment to the client interface component for transmission.

In a possible implementation manner of the present invention, when the push start command does not carry the first type of parameter, the target media stream is specified by default; when the push starting command does not carry the second type of parameters, the initial media segment encapsulates the default specified media unit; and when the push starting command does not carry the segmentation strategy, the segmentation strategy of the push task is specified by default.

In a possible implementation manner of the present invention, the default designated media unit is a media unit in which sequence number intervals of all and latest media units in the target media stream are smaller than a first preset value, or a media unit in which generation time intervals of all and latest media units in the target media stream are smaller than a second preset value.

In a possible implementation manner of the present invention, when the second type of parameter carried by the push start command includes a start sequence number and the start sequence number is valid, the sequence number of the candidate media unit is subsequent to the start sequence number.

In one possible implementation of the present invention, when the second type of parameter carried by the start push command includes a start time and the start time is valid, the generation time of the candidate media unit is after the start time.

In a possible implementation manner of the present invention, each media unit is associated with a priority, and when the second type of parameter carried by the push initiation command includes a unit priority range, the priority of the candidate media unit is within the unit priority range.

In a possible implementation manner of the present invention, the segmentation policy is equal sequence number interval pushing, and parameters carried by the segmentation policy include: specifying a sequence number interval, the media segment generation component further configured to perform the steps of:

s802, when a new media unit is generated, judging whether the interval between the sequence number of the new media unit and the push starting sequence number is equal to the interval of the appointed sequence number, if so, packaging all candidate media units with the sequence numbers behind the push starting sequence number into a new media segment, and updating the push starting sequence number;

s803, S802 is repeatedly executed.

In a possible implementation manner of the present invention, the segmentation policy is equal time interval pushing, and parameters carried by the segmentation policy include: a specified time interval, a media segment generation component further to:

In a possible implementation manner of the present invention, a unit set is composed of a plurality of media units with consecutive sequence numbers, a segmentation policy is unit set push, and parameters carried by the segmentation policy include: the number of the unit sets is specified, and the media segment generation component is also used for:

In a possible implementation manner of the present invention, the media segment generating component is further configured to:

In a possible implementation manner of the present invention, the command further includes: continuing to push the command, wherein the continuing to push the command carries at least one command parameter: the pushing task identifier and the media segment generating component are further used for:

and resetting an deadline for the push task corresponding to the push task identifier carried by the continuous push command according to the continuous push command, wherein the new deadline for the push task is the sum of the current system time and a preset push duration.

In a possible implementation manner of the present invention, the command further includes: terminating the push command, wherein the terminating push command carries at least one command parameter: the pushing task identifier and the media segment generating component are further used for:

In a possible implementation manner of the present invention, the command further includes: restarting a push command, wherein the restart push command carries a push task identifier, and the restart push command does not carry or carries at least one of the following command parameters: a first type of parameter indicating a target media stream to be transmitted, a second type of parameter indicating a candidate media unit feature to be transmitted, and a segmentation policy, the media segment generation component further configured to:

restarting a push task corresponding to a push task identifier carried by a push restarting command according to the push restarting command, wherein the current media segment generation and sending process of the push task is stopped at first, a target media stream to be transmitted is determined according to a first type of parameter carried by the push restarting command, an initial media segment is generated, a new media segment is generated continuously according to a segmentation strategy carried by the push restarting command, the generated media segment is sent to a client, and a candidate media unit determined according to a second type of parameter carried by the push restarting command in the target media stream is encapsulated in the media segment.

In a possible implementation manner of the present invention, the media segment further encapsulates media stream index information, where the media stream index information includes a plurality of media stream description information belonging to the same content, and the media stream description information includes a media stream identifier and a media stream bit rate.

In a possible implementation manner of the present invention, the command further includes: the media segment pulling command does not carry or carries at least one command parameter, and the command parameter comprises: a first type of parameter indicating a target media stream to be transmitted and a second type of parameter indicating a candidate media unit feature to be transmitted, the media segment generating component further configured to:

It should be noted that the explanation of the foregoing embodiment of the real-time pushing method for media streams is also applicable to the real-time pushing server for media streams of this embodiment, and the implementation principle is similar, and is not described herein again.

In order to implement the foregoing embodiments, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where when the processor executes the computer program, the real-time pushing method of a media stream as described in the foregoing embodiments is implemented.

In order to implement the foregoing embodiment, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, where the program is executed by a processor to implement the real-time pushing method for a media stream described in the foregoing embodiment.

In order to implement the foregoing embodiments, the present invention further provides a computer program product, where instructions in the computer program product, when executed by a processor, perform the real-time pushing method for media streams as described in the foregoing embodiments.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.

Claims

1. A method for pushing a media stream in real time, wherein the media stream is a sequence of media units generated in real time, and each media unit is associated with a generation time and/or a sequence number indicating a generation sequence, the method comprising the following steps:

and starting a push task according to the push starting command, wherein when the push starting command carries at least one of a first type parameter indicating a target media stream to be transmitted, a second type parameter indicating the characteristics of candidate media units to be transmitted and a segmentation strategy indicating a generation method of a new media segment, the target media stream to be transmitted is determined according to the first type parameter, a push task identifier is distributed for the push task, an initial media segment is generated according to the second type parameter, the new media segment is continuously generated according to the segmentation strategy, the push task identifier and the generated media segment are sent to the client, and the media segment encapsulates the candidate media units in the target media stream.

2. The real-time pushing method of media stream according to claim 1, further comprising:

when the push starting command does not carry the first type of parameters, the target media stream is specified by default;

3. The real-time pushing method of media stream according to claim 2, wherein the default designated media unit is a media unit whose sequence number interval between all and the latest media unit in the target media stream is smaller than a first preset value, or a media unit whose generation time interval between all and the latest media unit in the target media stream is smaller than a second preset value.

4. The method of claim 1, wherein when the second type of parameter carried by the start push command includes a start sequence number and the start sequence number is valid, the sequence number of the candidate media unit is subsequent to the start sequence number.

5. The method of claim 1, wherein when the second type of parameter carried by the start push command includes a start time and the start time is valid, the generation time of the candidate media unit is after the start time.

6. A method for pushing media streams in real time as claimed in claim 1, wherein each media unit is associated with a priority, and when the second type of parameter carried by the push initiation command includes a unit priority range, the priority of the candidate media unit is within the unit priority range.

7. The real-time pushing method of media stream according to claim 1, wherein the segmentation policy is equal sequence number interval pushing, and the parameters carried by the segmentation policy include: the method for generating the new media segment indicated by the segmentation strategy by specifying the sequence number interval comprises the following steps:

s803, S802 is repeatedly executed.

8. The real-time pushing method of media stream according to claim 1, wherein the segmentation policy is equal time interval pushing, and parameters carried by the segmentation policy include: a method for generating a new media segment indicated by a segmentation strategy at a specified time interval comprises the following steps:

9. The method as claimed in claim 1, wherein a plurality of media units with consecutive sequence numbers form a unit set, the segmentation policy is unit set push, and the parameters carried by the segmentation policy include: the number of the unit sets is specified, and the method for generating the new media segment indicated by the segmentation strategy comprises the following steps:

10. The real-time pushing method of media stream according to claim 1, wherein an expiration time is set for each initiated pushing task, and when the expiration time is reached, the pushing task is terminated, wherein the terminating the pushing task comprises: the generation of new media segments is stopped.

11. The real-time pushing method of media stream according to claim 10, wherein the command further comprises: continuing to push the command, wherein the continuing to push the command carries at least one command parameter: pushing a task identifier, the method further comprising:

12. The real-time pushing method of media stream according to claim 1, wherein the command further comprises: terminating a push command, wherein the terminating push command carries at least one command parameter: pushing a task identifier, the method further comprising:

13. The real-time pushing method of media stream according to claim 1, wherein the command further comprises: restarting a push command, wherein the restart push command carries a push task identifier, and the restart push command does not carry or carries at least one of the following command parameters: a first class of parameters indicative of a target media stream to be transmitted, a second class of parameters indicative of a candidate media unit characteristic to be transmitted, and a segmentation policy, the method further comprising:

14. The real-time pushing method of media stream according to claim 13, wherein the media segment further encapsulates media stream index information, the media stream index information includes a plurality of media stream description information belonging to the same content, and the media stream description information includes a media stream identifier and a media stream bit rate.

15. The real-time pushing method of media stream according to claim 1, wherein the command further comprises: a media segment pull command, the media segment pull command not carrying or carrying at least one command parameter, the command parameter comprising: a first class of parameters indicative of a target media stream to be transmitted and a second class of parameters indicative of a candidate media unit characteristic to be transmitted, the method further comprising:

16. A real-time push server of a media stream, wherein the media stream is a sequence of media units generated in real-time, and wherein each media unit is associated with a generation time and/or a sequence number indicating a generation order, the server comprising:

the client interface component is used for receiving a media segment request sent by a client and returning a corresponding media segment, wherein the media segment request carries at least one command;

a media segment generating component, configured to start a push task according to a push starting command in the media segment request, where when the push starting command carries at least one of a first type parameter indicating a target media stream to be transmitted, a second type parameter indicating a candidate media unit feature to be transmitted, and a segmentation policy indicating a new media segment generating method, the media segment generating component determines the target media stream to be transmitted according to the first type parameter, allocates a push task identifier to the push task, generates an initial media segment according to the second type parameter, and continuously generates a new media segment according to the segmentation policy, and sends the push task identifier and the generated media segment to the client through the client interface component, where the media segment encapsulates the candidate media unit in the target media stream.

17. The real-time push server of media streams of claim 16,

18. The real-time push server of media stream as claimed in claim 17, wherein the default designated media unit is a media unit in which the sequence number interval of all and the latest media unit in the target media stream is smaller than a first preset value, or a media unit in which the generation time interval of all and the latest media unit in the target media stream is smaller than a second preset value.

19. The real-time push server of media stream as claimed in claim 16, wherein when the second type of parameter carried by the start push command includes a start sequence number and the start sequence number is valid, the sequence number of the candidate media unit is subsequent to the start sequence number.

20. The real-time push server of media stream as claimed in claim 16, wherein the generation time of the candidate media unit is after the start time when the second type of parameter carried by the start push command includes the start time and the start time is valid.

21. The real-time push server of media streams of claim 16, wherein each media unit is associated with a priority, and when the second type of parameter carried by the start push command includes a unit priority range, the priority of the candidate media unit is within the unit priority range.

22. The real-time media stream pushing server of claim 16, wherein the segment policy is equal sequence number interval pushing, and the parameters carried by the segment policy include: specifying a sequence number interval, the media segment generation component further configured to perform the steps of:

s803, S802 is repeatedly executed.

23. The real-time media stream pushing server according to claim 16, wherein the segmentation policy is equal time interval pushing, and parameters carried by the segmentation policy include: specifying a time interval, the media segment generation component further to:

24. The real-time push server of media stream as claimed in claim 16, wherein a plurality of media units with consecutive sequence numbers form a unit set, the segmentation policy is unit set push, and the parameters carried by the segmentation policy include: the number of the unit sets is specified, and the media segment generation component is further configured to:

25. The real-time push server of media streams of claim 16, wherein the media segment generation component is further configured to:

26. The real-time push server of media streams of claim 25, wherein the commands further comprise: continuing to push the command, wherein the continuing to push the command carries at least one command parameter: the task identifier is pushed, and the media segment generation component is further configured to:

27. The real-time push server of media streams of claim 16, wherein the commands further comprise: terminating a push command, wherein the terminating push command carries at least one command parameter: the task identifier is pushed, and the media segment generation component is further configured to:

28. The real-time push server of media streams of claim 16, wherein the commands further comprise: restarting a push command, wherein the restart push command carries a push task identifier, and the restart push command does not carry or carries at least one of the following command parameters: a first type of parameter indicating a target media stream to be transmitted, a second type of parameter indicating a candidate media unit feature to be transmitted, and a segmentation policy, the media segment generation component being further configured to:

29. The real-time push server of media stream as claimed in claim 28, wherein the media segment further encapsulates media stream index information, the media stream index information contains a plurality of media stream description information belonging to the same content, and the media stream description information includes a media stream identifier and a media stream bit rate.

30. The real-time push server of media streams of claim 16, wherein the commands further comprise: a media segment pull command, the media segment pull command not carrying or carrying at least one command parameter, the command parameter comprising: a first type of parameter indicating a target media stream to be transmitted and a second type of parameter indicating a candidate media unit feature to be transmitted, the media segment generation component further configured to:

31. Computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the program, implements a real-time push method of a media stream according to any of claims 1-15.

32. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements a method for pushing media streams in real time according to any of claims 1 to 15.