WO2016090904A1 - Transmission method and device for media data streams - Google Patents

Transmission method and device for media data streams Download PDF

Info

Publication number
WO2016090904A1
WO2016090904A1 PCT/CN2015/082946 CN2015082946W WO2016090904A1 WO 2016090904 A1 WO2016090904 A1 WO 2016090904A1 CN 2015082946 W CN2015082946 W CN 2015082946W WO 2016090904 A1 WO2016090904 A1 WO 2016090904A1
Authority
WO
WIPO (PCT)
Prior art keywords
media
negotiation
media data
session
terminal
Prior art date
Application number
PCT/CN2015/082946
Other languages
French (fr)
Chinese (zh)
Inventor
方晓琴
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2016090904A1 publication Critical patent/WO2016090904A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering

Definitions

  • the present invention relates to the field of communications, and in particular to a method and apparatus for transmitting a media data stream.
  • a single terminal In a conference television system, in an IP point-to-point conference or an MCU (Multipoint Control Unit) multipoint conference, a single terminal only supports one main video codec and one auxiliary video codec, and the input source may be multiple channels. However, the current main video can only be encoded and transmitted to the opposite end through the media channel, and the opposite end decodes and outputs the display. If the dual stream is sent in the conference, the picture is composed of at most one main video and one auxiliary video and local picture. Image, if you need to see the picture of multiple video sources, you need multiple terminals as encoders for multi-channel encoding and you need to have a built-in MCU terminal group multi-point conference or a multi-point multi-screen conference on the MCU to see multiple channels. The scene of the picture.
  • the embodiment of the invention provides a method and a device for transmitting a media data stream, so as to at least solve the problem that a single terminal cannot transmit a multi-channel media data stream in the prior art.
  • a method for transmitting a media data stream including:
  • the terminal initiates a session negotiation process with the peer end, and starts the first media transmission channel according to the negotiation result of the session negotiation process;
  • the negotiation result further carries a data stream format of the at least two media data streams
  • the method further includes:
  • the media capability information further includes a first network time protocol (NTP) timestamp of the terminal and a second NTP timestamp of the peer end;
  • NTP network time protocol
  • the process of initiating the session negotiation with the peer end includes:
  • the session negotiation process is initiated again with the peer end.
  • the process of initiating a session negotiation between the terminal and the peer end includes:
  • the terminal initiates a session negotiation process based on the Session Initiation Protocol SIP protocol with the peer.
  • the process of initiating a session negotiation between the terminal and the peer end includes:
  • the terminal initiates a session negotiation process based on the H.323 protocol with the peer end.
  • an apparatus for providing a media data stream is applied to a terminal, including:
  • the session initiation module is configured to initiate a session negotiation process with the peer end;
  • a channel control module configured to: after the session initiation module completes the session negotiation process, start a first media transmission channel according to a negotiation result of the session negotiation process;
  • the negotiation module is configured to obtain media capability information by negotiating with the peer according to the media capability parameter in the negotiation result, where the media capability parameter indicates that the terminal supports transmission of at least two media data streams, the media
  • the capability information includes the number of ways in which the at least two media data streams are multiplexed;
  • the channel control module is configured to: after the negotiation module obtains the media capability information, turn off the first media transmission channel;
  • the session initiating module is further configured to: after the channel control module closes the first media transport channel, initiate the session negotiation process according to the media capability parameter obtained by the negotiation module and the peer end;
  • the channel control module is further configured to enable the second media transport channel according to the negotiation result of the session negotiation process initiated by the session initiation module;
  • a transmission module configured to transmit the at least two media data streams on the second media transmission channel that is opened by the channel control module.
  • the negotiation result further includes a data stream format of the at least two media data streams; the device further includes:
  • a processing module configured to: after the transmission module transmits the number of channels multiplexed by the at least two media data streams on the second media transmission channel, according to the number of channels multiplexed by the at least two media data streams
  • the data stream format encodes and decodes the transmitted media data stream.
  • the media capability information further includes a first network time protocol NTP timestamp of the terminal and a second NTP timestamp of the opposite end;
  • the negotiation module is further configured to initiate the session negotiation process with the peer end when determining that the first NTP timestamp is smaller than the second NTP timestamp of the opposite end.
  • the session initiation module is configured to initiate a session negotiation process based on the session initiation protocol SIP protocol with the peer.
  • the session initiation module is configured to initiate a session negotiation process based on the H.323 protocol with the peer end.
  • the terminal initiates a session negotiation process with the peer end, and starts the first media transmission channel according to the negotiation result of the session negotiation process, and obtains media capability information by negotiating with the peer end according to the media capability parameter of the terminal, where
  • the media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the number of channels multiplexed by the at least two media data streams, the terminal closes the first media transmission channel, and according to the media data
  • the peer end initiates the session negotiation process again, and starts a second media transmission channel according to the negotiation result of the re-initiated session negotiation process, and transmits the at least two channels on the second media transmission channel.
  • the media data stream is such that, since the terminal and the peer end determine in the negotiation that the two ends support multiplexing of at least two media data streams, the problem that a single terminal cannot transmit the multi-channel media data stream is solved.
  • FIG. 1 is a flowchart of a method for transmitting a media data stream according to an embodiment of the present invention
  • 2a is a schematic diagram of a display screen layout according to an embodiment of the present invention.
  • FIG. 2b is a schematic diagram of another display screen layout according to an embodiment of the present invention.
  • 2c is a schematic diagram of another display screen layout according to an embodiment of the present invention.
  • 2d is a schematic diagram of another display screen layout according to an embodiment of the present invention.
  • 2e is a schematic diagram of another display screen layout according to an embodiment of the present invention.
  • 2f is a schematic diagram of another display screen layout according to an embodiment of the present invention.
  • 2g is a schematic diagram of another display screen layout according to an embodiment of the present invention.
  • FIG. 3 is a flowchart of another method for transmitting a media data stream according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of another method for transmitting a media data stream according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a transmission apparatus according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of another transmission apparatus according to an embodiment of the present invention.
  • FIG. 1 is a flowchart according to an embodiment of the present invention. As shown in FIG. 1, the process includes the following steps:
  • Step S102 The terminal initiates a session negotiation process with the peer end, and starts the first media transmission channel according to the negotiation result of the session negotiation process.
  • Step S104 Obtain media capability information by negotiating with the peer according to the media capability parameter in the negotiation result.
  • the media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the number of channels multiplexed by the at least two media data streams.
  • Step S106 the first media transmission channel is closed, and the session negotiation process is initiated again according to the number of channels multiplexed by the media data stream;
  • Step S108 Start a second media transport channel according to the negotiation result of the re-initiated session negotiation process, and transmit the at least two media data streams on the second media transport channel.
  • the terminal and the peer determine that the two ends support multiplexing of at least two media data streams in the negotiation, thereby solving the problem that a single terminal cannot transmit the multi-channel media data stream.
  • the peer end also performs the transmission of the media data stream according to the above steps S102 to S108.
  • the above capability parameter may be the audio capability parameter AAC_LD_3640.
  • the negotiation result further carries a data stream format of the at least two media data streams, and after transmitting the number of channels multiplexed by the at least two media data streams on the second media transmission channel, according to the The number of ways in which the at least two media streams are multiplexed and the stream format encodes and decodes the transmitted media stream.
  • the peer end also encodes and decodes the received media data stream.
  • the display device can display the main media data stream of the terminal 1, the main media data stream 2 and the main media data stream 3, and the main media data stream of the opposite end, the main media data stream 2 and the main media data stream, as needed 3, and a large screen of the auxiliary media data stream, and a multi-screen composed of 1 to 7 sub-pictures of different layouts such as Picture In Picture (PIP).
  • PIP Picture In Picture
  • the source of each screen can be arbitrarily selected, four video input interfaces on the terminal, different working modes DVI (Digital Visual Interface), HDMI (High Definition Multimedia Interface), YPbPr (color difference component) Interface), VGA (Video Graphics Array), different formats can be arbitrarily configured, the output mode can also be selected or only selected, the selected mode is displayed back and forth through the switching operation, as shown in the figure 2a to 2g show the display modes of single screen, dual screen to seven screen, respectively, and can be switched between different display modes.
  • DVI Digital Visual Interface
  • HDMI High Definition Multimedia Interface
  • YPbPr color difference component
  • VGA Video Graphics Array
  • the media capability information further includes a first network time protocol NTP timestamp of the terminal and a second NTP timestamp of the peer end, where the first NTP timestamp is determined to be smaller than the second NTP time of the opposite end.
  • the terminal and the peer end initiate a session negotiation process based on a SIP (Session Initiation Protocol) protocol, so that in a SIP-based scenario, a single terminal can transmit multiple media data streams.
  • SIP Session Initiation Protocol
  • the terminal may include a main control module, a SIP protocol module, and a TIP (Telepresence Interoperability Protocol) protocol module, and the main control module of the terminal initiates a new call to the SIP protocol module, SIP.
  • the protocol module sends a session negotiation request message to the peer end, and after receiving the session negotiation request message, the peer end sends a 200 OK message to the terminal, and the SIP protocol module of the terminal receives the 200 OK message, and the SIP protocol module sends the message to the opposite end.
  • An ACK (Acknowledgement) response message and the first media transport channel is turned on.
  • the terminal and the peer end initiate a session negotiation process based on the H.323 protocol, so that in a scenario based on the H.323 protocol, a single terminal can transmit multiple media data streams.
  • the terminal may include a main control module, an H.323 protocol module, and a TIP protocol module, where the main control module of the terminal initiates a new call to the H.323 protocol module, and the H.323 protocol module is paired.
  • the terminal sends a create message, and after receiving the setup message, the peer sends a call procedure and an alert message and a connect message to the terminal, and the call is continuously received by the H.323 protocol module.
  • the message and the prompt message and the connection message establish a call signaling channel with the opposite end, and perform capability negotiation and master slave confirmation on the call signaling channel with the peer, and complete the capability negotiation and the master After confirmation, the first media transmission channel is turned on.
  • FIG. 3 is a flowchart according to an embodiment of the present invention.
  • the embodiment of the present invention is a media data stream transmission based on a SIP protocol.
  • the terminal in the embodiment of the present invention includes a main control module, a SIP protocol module, and a TIP protocol module. The process includes the following steps:
  • Steps S301 to S304 are session negotiation processes:
  • the terminal control module initiates a new call request to the SIP protocol module.
  • the SIP protocol module of the terminal After receiving the call request, the SIP protocol module of the terminal sends a session negotiation request message to the peer end.
  • the SIP protocol module of the terminal receives a 200 OK message sent by the peer end according to the session negotiation request message.
  • the SIP protocol module of the terminal sends an ACK response message to the peer end.
  • the terminal opens a first media transmission channel with the opposite end.
  • Steps S306 to S310 are processes of TIP negotiation:
  • the media capability parameter indicates that the terminal supports transmission of at least two media data streams, and the media capability parameter may be AAC_LD_3640.
  • the TIP protocol module of the terminal sends a data stream multiplexing path number negotiation request message to the peer end.
  • the TIP protocol module of the terminal receives the data stream multiplexing path negotiation response message sent by the peer end according to the data stream multiplexing path number negotiation request message, and obtains the number of channels multiplexed by the media data stream and the NTP timestamp of the opposite end.
  • the TIP protocol module of the terminal sends a data stream format negotiation request message to the peer end.
  • the TIP protocol module of the terminal receives a data stream format negotiation response message sent by the peer end according to the data stream format negotiation request message, and obtains a format of the media data stream transmission.
  • the terminal closes the first media transmission channel with the opposite end.
  • Steps S312 to S314 are the second session negotiation process:
  • the terminal control module initiates a secondary call request to the SIP protocol module according to the number of channels multiplexed by the media data stream.
  • the SIP protocol module of the terminal After receiving the call request, the SIP protocol module of the terminal sends a session negotiation request message with the peer end.
  • the SIP protocol module of the terminal receives the 200 OK message sent by the peer end according to the session negotiation request message.
  • the SIP protocol module of the terminal sends an ACK response message to the peer end.
  • the terminal starts the second media transport channel according to the negotiation result of the second initiated session negotiation process.
  • the terminal transmits at least two media data streams on the second media transport channel according to the number of channels multiplexed by the media data stream and the format of the media data stream transmission.
  • the terminal performs encoding and decoding according to the number of channels multiplexed by the media data stream and the data stream transmitted by the format of the media data stream transmission.
  • peer end also performs negotiation and data interaction with the terminal according to the foregoing steps S301 to S315.
  • a single terminal can transmit multiple media data streams.
  • FIG. 4 is a flowchart according to an embodiment of the present invention.
  • the embodiment of the present invention is a media data stream based on the H.323 protocol.
  • the transmission is described as an example.
  • the terminal in the embodiment of the present invention includes a main control module, an H.323 protocol module, and a TIP protocol module. The process includes the following steps:
  • Steps S401 to S404 are the session negotiation process:
  • the terminal control module of the terminal initiates a new call request to the H.323 protocol module of the terminal.
  • the H.323 protocol module of the terminal sends a creation message to the peer end.
  • the H.323 protocol module of the terminal receives a call duration message and a prompt message and a connection message sent by the peer end according to the creation message, and establishes a call signaling channel with the peer end.
  • S404 The H.323 protocol module of the terminal performs capability negotiation and master-slave confirmation with the peer end on the call signaling channel.
  • the terminal turns on a first media transmission channel with the opposite end.
  • Steps S406 to S410 are processes of TIP negotiation:
  • the main control module of the terminal when determining the carrying media capability parameter according to the negotiation result of the session negotiation, the main control module of the terminal sends a TIP negotiation request message to the TIP protocol module.
  • the media capability parameter indicates that the terminal supports transmission of at least two media data streams, and the media capability parameter may be AAC_LD_3640.
  • the TIP protocol module of the terminal sends a data stream multiplexing path number negotiation request message to the peer end.
  • the TIP protocol module of the terminal receives the data stream multiplexing path negotiation response message sent by the peer end according to the data stream multiplexing path number negotiation request message, and obtains the number of channels multiplexed by the media data stream and the NTP timestamp of the opposite end.
  • the TIP protocol module of the terminal sends a data stream format negotiation request message to the peer end.
  • the TIP protocol module of the terminal receives a data stream format negotiation response message sent by the peer end according to the data stream format negotiation request message, and obtains a format of the media data stream transmission.
  • the terminal closes the first media transmission channel with the opposite end.
  • Steps S412 to S415 are the second session negotiation process:
  • the H.323 protocol module of the terminal sends a creation message to the peer end.
  • the H.323 protocol module of the terminal receives a call duration message and a prompt message and a connection message sent by the peer end according to the creation message, and establishes a call signaling channel with the peer end.
  • S415 The H.323 protocol module of the terminal performs capability negotiation and master-slave confirmation with the peer end on the call signaling channel.
  • the terminal starts the second media transport channel according to the negotiation result of the second initiated session negotiation process.
  • the terminal transmits at least two media data streams on the second media transmission channel according to the number of channels multiplexed by the media data stream and the format of the media data stream transmission.
  • the terminal performs encoding and decoding according to the number of channels multiplexed by the media data stream and the data stream transmitted by the format of the media data stream transmission.
  • peer end also performs negotiation and data interaction with the terminal according to the foregoing steps S411 to S417.
  • a single terminal can transmit multiple media data streams.
  • a device for transmitting a media data stream is also provided, and the device is applied to a terminal, and the device is used to implement the foregoing embodiments and preferred embodiments, and details are not described herein.
  • the term "module” may implement a combination of software and/or hardware of a predetermined function. Despite the description of the following embodiments The implementation is preferably implemented in software, but hardware, or a combination of software and hardware, is also possible and contemplated.
  • FIG. 5 is a structural block diagram of a device for transmitting a media data stream according to an embodiment of the present invention. As shown in FIG. 5, the device 50 includes:
  • the session initiation module 51 is configured to initiate a session negotiation process with the peer end.
  • the channel control module 52 is configured to: after the session initiation module 51 completes the session negotiation process, start the first media transmission channel according to the negotiation result of the session negotiation process;
  • the negotiation module 53 is configured to negotiate with the peer to obtain media capability information according to the media capability parameter in the negotiation result, where the media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the The number of ways to multiplex at least two media streams;
  • the channel control module 52 is configured to: after the negotiation module 53 obtains the media capability information, close the first media transmission channel;
  • the session initiation module 51 is further configured to: after the channel control module 52 closes the first media transmission channel, initiate the session negotiation process according to the media capability parameter obtained by the negotiation module 53 and the peer end;
  • the channel control module 52 is further configured to enable the second media transport channel according to the negotiation result of the session negotiation process initiated by the session initiation module 51;
  • the transmission module 54 is configured to transmit the at least two media data streams on the second media transport channel opened by the channel control module 52.
  • the negotiation result further includes a data stream format of the at least two media data streams; as shown in FIG. 6, the device further includes:
  • the processing module 55 is configured to: after the transmission module 54 transmits the number of channels multiplexed by the at least two media data streams on the second media transmission channel, the number of channels multiplexed according to the at least two media data streams and the data
  • the stream format encodes and decodes the transmitted media data stream.
  • the media capability information further includes a first network time protocol NTP timestamp of the terminal and a second NTP timestamp of the peer end;
  • the negotiation module is further configured to: when determining that the first NTP timestamp is smaller than the second NTP timestamp of the peer end, initiate the session negotiation process with the peer end, so that in the SIP protocol-based scenario, the single terminal can Transfer multiple media streams.
  • the session initiation module 51 is configured to initiate a session negotiation process based on the session initiation protocol SIP protocol with the peer.
  • the session initiation module 51 is configured to initiate a session negotiation process based on the H.323 protocol with the peer end, so that in a scenario based on the H.323 protocol, a single terminal can transmit multiple media data. flow. .
  • the terminal and the peer determine that the two ends support multiplexing of at least two media data streams in the negotiation, thereby solving the problem that a single terminal cannot transmit the multi-channel media data stream.
  • each input source can select any video input interface, any working mode, any standard as the video source, or the same source, and the input source can be flexibly configured.
  • the TIP protocol is added to the H.323 or SIP protocol to perform capability interaction and open channels.
  • the three primary video media are multiplexed to transmit media data on one RTP channel, and the auxiliary video is separated by one channel and the audio is separately.
  • the display there may be at least one display at least. This requires outputting all the local end (equivalent to the terminal), the far end (equivalent to the opposite end), the multi-picture of the auxiliary video image, or the multi-screen different layout of the sub-picture on this display.
  • Composite image For example, when the four streams of a single display, the media includes a remote 1, a remote 2, a remote 3, a secondary video four-way media, the display needs to be able to display a large screen of the remote N, the main N, the auxiliary video, and Multi-screen composed of 2 to 7 sub-pictures of different layouts such as pip and triple.
  • the initiating caller initiates a session negotiation request to the peer end, and the peer end responds 200 ok after receiving the response, and the originating end responds with an ACK to confirm.
  • the interactive capability is enabled to carry the audio capability.
  • AAC_LD_3640 initiates TIP negotiation TipNegotiate.
  • the TIP negotiates the number of media reuse channels (MUXCTRL) and format (MEDIAOPTS).
  • MUXCTRL media reuse channels
  • MEDIAOPTS format
  • the reINVITE re-invitation is initiated by the local end or the peer end depending on the MUXCTRL.
  • the NTP timestamp in the message the party with the small timestamp actively initiates the second re-invitation. After the second re-invitation succeeds, the video and audio media channel openchannel is opened, and the RTP media stream starts to send and receive. If there is no AAC_LD_3640 audio capability in the initial negotiation, it will be in the traditional terminal mode.
  • the call is established according to the standard H.323 protocol.
  • the originating caller sends a setup message to the peer, and after receiving the call, the call canceleeding, Alerting, and Connect messages are received.
  • the call signaling channel is established, and capability negotiation and master-slave confirmation are performed on the channel. If the capability carries the audio capability AAC_LD_3640, the TIP negotiation TipNegotiate is initiated, and the TIP negotiates the number of media multiplexing channels (MUXCTRL) and format (MEDIAOPTS). After the TIP negotiation is passed, the second re-invitation is performed, and the capability negotiation is confirmed again.
  • MUXCTRL media multiplexing channels
  • MEDIAOPTS format
  • the media channel opened for the first time is closed, and then the media channel is re-opened according to the result of the last negotiation, and the RTP (Real-time Transport Protocol) media is opened after the video and audio media channel is opened.
  • the stream starts to send and receive. If there is no AAC_LD_3640 audio capability in the initial negotiation, it will be in the traditional terminal mode.
  • modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein.
  • the steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module.
  • the invention is not limited to any specific combination of hardware and software.
  • the technical solution provided by the embodiment of the present invention may be applied to the process of transmitting the media data stream, and the terminal initiates a session negotiation process with the peer end, and starts the first media transmission channel according to the negotiation result of the session negotiation process, and according to the terminal,
  • the media capability parameter is negotiated with the peer to obtain the media capability information, where the media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the number of channels multiplexed by the at least two media data streams.
  • the terminal closes the first media transmission channel, and initiates the session negotiation process again according to the number of channels multiplexed by the media data stream, and starts the second media transmission channel according to the negotiation result of the re-initiated session negotiation process. And transmitting the at least two media data streams on the second media transmission channel, so that the terminal and the peer end determine that the two ends support multiplexing of at least two media data streams, thereby solving the problem that the single terminal cannot transmit more The problem of the road media data stream.

Abstract

Disclosed in the present invention are a transmission method and device for media data streams. With the invention, a session negotiation process is initiated by a terminal to an opposite end, and a first media transmission channel is opened according to the negotiation result of the session negotiation process; the media capability information is acquired by negotiating with the opposite end according to a media capability parameter in the negotiation result, wherein the media capability parameter denotes that the terminal supports the transmission of at least two paths of media data streams and the media capability information includes the multiplexing path number of the at least two paths of media data streams; the first media transmission channel is closed, and the session negotiation process is initiated again to the opposite end according to the multiplexing path number of the media data streams; a second media transmission channel is opened according to the negotiation result of the session negotiation process initiated again, and the at least two paths of media data streams are transmitted on the second media transmission channel, thus resolved is the problem existed in the prior art that the single terminal cannot transmit multi-path media data streams.

Description

媒体数据流的传输方法和装置Method and device for transmitting media data stream 技术领域Technical field
本发明涉及通信领域，具体而言，涉及一种媒体数据流的传输方法和装置。The present invention relates to the field of communications, and in particular to a method and apparatus for transmitting a media data stream.
背景技术Background technique
会议电视系统中，在IP点对点会议或MCU(Multipoint Control Unit，多点控制单元)多点会议中，单个终端只支持一路主视频的编解码和一路辅视频的编解码，输入源可以是多路，但当前主视频只能选择一路进行编码并通过媒体通道传输给对端，对端进行解码后输出显示，会议中如果发送双流，画面中最多是一路主视频和一路辅视频以及本地画面的合成图像，如果需要看多路视频源的画面，则需要多个终端作为编码器进行多路的编码并需要内置MCU终端组多点会议或者是MCU上组多点多画面会议，才能看到多路画面的场景。In a conference television system, in an IP point-to-point conference or an MCU (Multipoint Control Unit) multipoint conference, a single terminal only supports one main video codec and one auxiliary video codec, and the input source may be multiple channels. However, the current main video can only be encoded and transmitted to the opposite end through the media channel, and the opposite end decodes and outputs the display. If the dual stream is sent in the conference, the picture is composed of at most one main video and one auxiliary video and local picture. Image, if you need to see the picture of multiple video sources, you need multiple terminals as encoders for multi-channel encoding and you need to have a built-in MCU terminal group multi-point conference or a multi-point multi-screen conference on the MCU to see multiple channels. The scene of the picture.
发明内容Summary of the invention
本发明实施例提供了一种媒体数据流的传输方法及装置，以至少解决现有技术单个终端无法传输多路媒体数据流的问题。The embodiment of the invention provides a method and a device for transmitting a media data stream, so as to at least solve the problem that a single terminal cannot transmit a multi-channel media data stream in the prior art.
根据本发明的一个实施例，提供了一种媒体数据流的传输方法，包括：According to an embodiment of the present invention, a method for transmitting a media data stream is provided, including:
终端与对端发起会话协商流程，并根据所述会话协商流程的协商结果开启第一媒体传输通道；The terminal initiates a session negotiation process with the peer end, and starts the first media transmission channel according to the negotiation result of the session negotiation process;
根据所述协商结果中的媒体能力参数与所述对端协商得到媒体能力信息，其中，所述媒体能力参数表示所述终端支持至少两路媒体数据流的传输，所述媒体能力信息包括所述至少两路媒体数据流复用的路数；Obtaining media capability information according to the media capability parameter in the negotiation result, where the media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the The number of ways to multiplex at least two media streams;
关闭所述第一媒体传输信道，并根据所述媒体数据流复用的路数与所述对端再次发起所述会话协商流程；Close the first media transport channel, and initiate the session negotiation process with the peer according to the number of paths multiplexed by the media data stream;
根据所述再次发起的会话协商流程的协商结果开启第二媒体传输信道，并在所述第二媒体传输信道上传输所述至少两路媒体数据流。 And performing, according to the negotiation result of the re-initiated session negotiation process, a second media transport channel, and transmitting the at least two media data streams on the second media transport channel.
在本发明实施例中，所述协商结果中还携带有所述至少两个媒体数据流的数据流格式；In the embodiment of the present invention, the negotiation result further carries a data stream format of the at least two media data streams;
在所述第二媒体传输信道上传输所述至少两路媒体数据流复用的路数后，还包括：After the number of the multiplexed channels of the at least two media data streams is transmitted on the second media transmission channel, the method further includes:
根据所述至少两路媒体数据流复用的路数和所述数据流格式对所述传输的所述媒体数据流进行编码和解码。And encoding the transmitted media data stream according to the number of channels multiplexed by the at least two media data streams and the data stream format.
在本发明实施例中，所述媒体能力信息还包括所述终端的第一网络时间协议(Network Time Protocol，简称为NTP)时间戳和所述对端的第二NTP时间戳；In the embodiment of the present invention, the media capability information further includes a first network time protocol (NTP) timestamp of the terminal and a second NTP timestamp of the peer end;
与所述对端再次发起所述会话协商流程包括：The process of initiating the session negotiation with the peer end includes:
在确定所述第一NTP时间戳小于所述对端的第二NTP时间戳时，与所述对端再次发起所述会话协商流程。When it is determined that the first NTP timestamp is smaller than the second NTP timestamp of the peer end, the session negotiation process is initiated again with the peer end.
在本发明实施例中，终端与对端发起会话协商流程包括：In the embodiment of the present invention, the process of initiating a session negotiation between the terminal and the peer end includes:
终端与所述对端发起基于会话初始化协议SIP协议的会话协商流程。The terminal initiates a session negotiation process based on the Session Initiation Protocol SIP protocol with the peer.
在本发明实施例中，终端与对端发起会话协商流程包括：In the embodiment of the present invention, the process of initiating a session negotiation between the terminal and the peer end includes:
终端与所述对端发起基于H.323协议的会话协商流程。The terminal initiates a session negotiation process based on the H.323 protocol with the peer end.
根据本发明的另一个实施例，提供了一种媒体数据流的装置，应用于终端，包括：According to another embodiment of the present invention, an apparatus for providing a media data stream is applied to a terminal, including:
会话发起模块，设置为与对端发起会话协商流程；The session initiation module is configured to initiate a session negotiation process with the peer end;
通道控制模块，设置为在所述会话发起模块完成所述会话协商流程后，根据所述会话协商流程的协商结果开启第一媒体传输信道；a channel control module, configured to: after the session initiation module completes the session negotiation process, start a first media transmission channel according to a negotiation result of the session negotiation process;
协商模块，设置为根据所述协商结果中的媒体能力参数与所述对端协商得到媒体能力信息，其中，所述媒体能力参数表示所述终端支持至少两路媒体数据流的传输，所述媒体能力信息包括所述至少两路媒体数据流复用的路数；The negotiation module is configured to obtain media capability information by negotiating with the peer according to the media capability parameter in the negotiation result, where the media capability parameter indicates that the terminal supports transmission of at least two media data streams, the media The capability information includes the number of ways in which the at least two media data streams are multiplexed;
所述通道控制模块，设置为在所述协商模块得到所述媒体能力信息后，关闭所述第一媒体传输信道；The channel control module is configured to: after the negotiation module obtains the media capability information, turn off the first media transmission channel;
所述会话发起模块，还设置为在所述通道控制模块关闭所述第一媒体传输信道后，根据所述协商模块得到的媒体能力参数与所述对端再次发起所述会话协商流程； The session initiating module is further configured to: after the channel control module closes the first media transport channel, initiate the session negotiation process according to the media capability parameter obtained by the negotiation module and the peer end;
所述通道控制模块，还设置为根据所述会话发起模块再次发起的会话协商流程的协商结果开启第二媒体传输信道；The channel control module is further configured to enable the second media transport channel according to the negotiation result of the session negotiation process initiated by the session initiation module;
传输模块，设置为在所述通道控制模块开启的所述第二媒体传输信道上传输所述至少两路媒体数据流。And a transmission module, configured to transmit the at least two media data streams on the second media transmission channel that is opened by the channel control module.
在本发明实施例中，所述协商结果中还携带有所述至少两个媒体数据流的数据流格式；所述装置还包括：In the embodiment of the present invention, the negotiation result further includes a data stream format of the at least two media data streams; the device further includes:
处理模块，设置为在所述传输模块在所述第二媒体传输信道上传输所述至少两路媒体数据流复用的路数后，根据所述至少两路媒体数据流复用的路数和所述数据流格式对所述传输的所述媒体数据流进行编码和解码。a processing module, configured to: after the transmission module transmits the number of channels multiplexed by the at least two media data streams on the second media transmission channel, according to the number of channels multiplexed by the at least two media data streams The data stream format encodes and decodes the transmitted media data stream.
在本发明实施例中，所述媒体能力信息还包括所述终端的第一网络时间协议NTP时间戳和所述对端的第二NTP时间戳；In the embodiment of the present invention, the media capability information further includes a first network time protocol NTP timestamp of the terminal and a second NTP timestamp of the opposite end;
所述协商模块还设置为，在确定所述第一NTP时间戳小于所述对端的第二NTP时间戳时，与所述对端再次发起所述会话协商流程。The negotiation module is further configured to initiate the session negotiation process with the peer end when determining that the first NTP timestamp is smaller than the second NTP timestamp of the opposite end.
在本发明实施例中，所述会话发起模块设置为，与所述对端发起基于会话初始化协议SIP协议的会话协商流程。In the embodiment of the present invention, the session initiation module is configured to initiate a session negotiation process based on the session initiation protocol SIP protocol with the peer.
在本发明实施例中，所述会话发起模块设置为，与所述对端发起基于H.323协议的会话协商流程。In the embodiment of the present invention, the session initiation module is configured to initiate a session negotiation process based on the H.323 protocol with the peer end.
通过本发明实施例，终端与对端发起会话协商流程，并根据该会话协商流程的协商结果开启第一媒体传输通道，并根据该终端的媒体能力参数与该对端协商得到媒体能力信息，其中，该媒体能力参数表示该终端支持至少两路媒体数据流的传输，该媒体能力信息包括该至少两路媒体数据流复用的路数，终端关闭该第一媒体传输信道，并根据该媒体数据流复用的路数与该对端再次发起该会话协商流程，并根据该再次发起的会话协商流程的协商结果开启第二媒体传输信道，并在该第二媒体传输信道上传输该至少两路媒体数据流，这样，由于终端与对端在协商中确定两端支持至少两路媒体数据流的复用，从而解决了单个终端无法传输多路媒体数据流的问题。 According to the embodiment of the present invention, the terminal initiates a session negotiation process with the peer end, and starts the first media transmission channel according to the negotiation result of the session negotiation process, and obtains media capability information by negotiating with the peer end according to the media capability parameter of the terminal, where The media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the number of channels multiplexed by the at least two media data streams, the terminal closes the first media transmission channel, and according to the media data And the peer end initiates the session negotiation process again, and starts a second media transmission channel according to the negotiation result of the re-initiated session negotiation process, and transmits the at least two channels on the second media transmission channel. The media data stream is such that, since the terminal and the peer end determine in the negotiation that the two ends support multiplexing of at least two media data streams, the problem that a single terminal cannot transmit the multi-channel media data stream is solved.
附图说明DRAWINGS
此处所说明的附图用来提供对本发明的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:
图1是根据本发明实施例的一种媒体数据流传输方法的流程图；1 is a flowchart of a method for transmitting a media data stream according to an embodiment of the present invention;
图2a是根据本发明实施例的一种显示画面布局的示意图；2a is a schematic diagram of a display screen layout according to an embodiment of the present invention;
图2b是根据本发明实施例的另一种显示画面布局的示意图；2b is a schematic diagram of another display screen layout according to an embodiment of the present invention;
图2c是根据本发明实施例的另一种显示画面布局的示意图；2c is a schematic diagram of another display screen layout according to an embodiment of the present invention;
图2d是根据本发明实施例的另一种显示画面布局的示意图；2d is a schematic diagram of another display screen layout according to an embodiment of the present invention;
图2e是根据本发明实施例的另一种显示画面布局的示意图；2e is a schematic diagram of another display screen layout according to an embodiment of the present invention;
图2f是根据本发明实施例的另一种显示画面布局的示意图；2f is a schematic diagram of another display screen layout according to an embodiment of the present invention;
图2g是根据本发明实施例的另一种显示画面布局的示意图；2g is a schematic diagram of another display screen layout according to an embodiment of the present invention;
图3是根据本发明实施例的另一种媒体数据流传输方法的流程图；FIG. 3 is a flowchart of another method for transmitting a media data stream according to an embodiment of the present invention; FIG.
图4是根据本发明实施例的另一种媒体数据流传输方法的流程图；4 is a flowchart of another method for transmitting a media data stream according to an embodiment of the present invention;
图5是根据本发明实施例的一种传输装置的结构示意图；FIG. 5 is a schematic structural diagram of a transmission apparatus according to an embodiment of the present invention; FIG.
图6是根据本发明实施例的另一种传输装置的结构示意图。FIG. 6 is a schematic structural diagram of another transmission apparatus according to an embodiment of the present invention.
具体实施方式detailed description
下文中将参考附图并结合实施例来详细说明本发明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.
在本实施例中提供了一种媒体数据流的传输方法，图1是根据本发明实施例的的流程图，如图1所示，该流程包括如下步骤：In this embodiment, a method for transmitting a media data stream is provided. FIG. 1 is a flowchart according to an embodiment of the present invention. As shown in FIG. 1, the process includes the following steps:
步骤S102，终端与对端发起会话协商流程，并根据该会话协商流程的协商结果开启第一媒体传输通道； Step S102: The terminal initiates a session negotiation process with the peer end, and starts the first media transmission channel according to the negotiation result of the session negotiation process.
步骤S104，根据该协商结果中的媒体能力参数与该对端协商得到媒体能力信息；Step S104: Obtain media capability information by negotiating with the peer according to the media capability parameter in the negotiation result.
其中，该媒体能力参数表示该终端支持至少两路媒体数据流的传输，该媒体能力信息包括该至少两路媒体数据流复用的路数。The media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the number of channels multiplexed by the at least two media data streams.
步骤S106，关闭该第一媒体传输信道，并根据该媒体数据流复用的路数与该对端再次发起该会话协商流程；Step S106, the first media transmission channel is closed, and the session negotiation process is initiated again according to the number of channels multiplexed by the media data stream;
步骤S108，根据该再次发起的会话协商流程的协商结果开启第二媒体传输信道，并在该第二媒体传输信道上传输该至少两路媒体数据流。Step S108: Start a second media transport channel according to the negotiation result of the re-initiated session negotiation process, and transmit the at least two media data streams on the second media transport channel.
通过上述步骤，由于终端与对端在协商中确定两端支持至少两路媒体数据流的复用，从而解决了单个终端无法传输多路媒体数据流的问题。Through the above steps, the terminal and the peer determine that the two ends support multiplexing of at least two media data streams in the negotiation, thereby solving the problem that a single terminal cannot transmit the multi-channel media data stream.
需要说明的是，对端也会按照上述步骤S102至步骤S108进行媒体数据流的传输。另外，上述能力参数可以是音频能力参数AAC_LD_3640。It should be noted that the peer end also performs the transmission of the media data stream according to the above steps S102 to S108. In addition, the above capability parameter may be the audio capability parameter AAC_LD_3640.
在本发明实施例中，该协商结果中还携带有该至少两个媒体数据流的数据流格式，在该第二媒体传输信道上传输该至少两路媒体数据流复用的路数后，根据该至少两路媒体数据流复用的路数和该数据流格式对该传输的该媒体数据流进行编码和解码。In the embodiment of the present invention, the negotiation result further carries a data stream format of the at least two media data streams, and after transmitting the number of channels multiplexed by the at least two media data streams on the second media transmission channel, according to the The number of ways in which the at least two media streams are multiplexed and the stream format encodes and decodes the transmitted media stream.
其中，对端也会对接收到的媒体数据流进行编码和解码。The peer end also encodes and decodes the received media data stream.
需要说明的是，终端和对端对接收到的媒体数据流进行解码后，分别将解码后的媒体数据流传输至显示设备，该显示设备多画面合成布局，例如对于三路主媒体数据流和一路辅媒体数据流，显示设备根据需要能够显示终端的主媒体数据流1、主媒体数据流2和主媒体数据流3和对端的主媒体数据流1、主媒体数据流2和主媒体数据流3以及辅媒体数据流的大画面，以及画中画(Picture In Picture，简称为PIP)等不同布局的1～7个子画面组成的多画面。每一路画面的源可任意选择，终端上四个视频输入接口，不同的工作模式DVI(Digital Visual Interface，数字视频接口)、HDMI(High Definition Multimedia Interface，高清晰度多媒体接口)、YPbPr(色差分量接口)、VGA(Video Graphics Array，视频图形阵列)、不同的制式均可任意配置，输出送显的模式也可以全部选择或者只选择某几种，被选中的模式通过切换操作来回显示，如图2a至图2g所示，分别表示单画面、双画面至七画面的显示模式，不同的显示模式之间可以进行切换。 It should be noted that after the terminal and the peer decode the received media data stream, respectively, the decoded media data stream is transmitted to the display device, and the display device multi-screen composite layout, for example, for the three-way main media data stream and A secondary media data stream, the display device can display the main media data stream of the terminal 1, the main media data stream 2 and the main media data stream 3, and the main media data stream of the opposite end, the main media data stream 2 and the main media data stream, as needed 3, and a large screen of the auxiliary media data stream, and a multi-screen composed of 1 to 7 sub-pictures of different layouts such as Picture In Picture (PIP). The source of each screen can be arbitrarily selected, four video input interfaces on the terminal, different working modes DVI (Digital Visual Interface), HDMI (High Definition Multimedia Interface), YPbPr (color difference component) Interface), VGA (Video Graphics Array), different formats can be arbitrarily configured, the output mode can also be selected or only selected, the selected mode is displayed back and forth through the switching operation, as shown in the figure 2a to 2g show the display modes of single screen, dual screen to seven screen, respectively, and can be switched between different display modes.
在本发明实施例中，该媒体能力信息还包括该终端的第一网络时间协议NTP时间戳和该对端的第二NTP时间戳，在确定该第一NTP时间戳小于该对端的第二NTP时间戳时，与该对端再次发起该会话协商流程。In the embodiment of the present invention, the media capability information further includes a first network time protocol NTP timestamp of the terminal and a second NTP timestamp of the peer end, where the first NTP timestamp is determined to be smaller than the second NTP time of the opposite end. When the stamp is stamped, the session negotiation process is initiated again with the peer.
在本发明实施例中，终端与该对端发起基于SIP(Session Initiation Protocol，会话初始化协议)协议的会话协商流程，这样，在基于SIP协议的场景下，单个终端能够传输多路媒体数据流。In the embodiment of the present invention, the terminal and the peer end initiate a session negotiation process based on a SIP (Session Initiation Protocol) protocol, so that in a SIP-based scenario, a single terminal can transmit multiple media data streams.
其中，在本发明实施例中，终端可以包括主控模块，SIP协议模块和TIP(Telepresence Interoperability protocol，网真互通协议)协议模块，该终端的主控模块向SIP协议模块发起新的呼叫，SIP协议模块向对端发送会话协商请求消息，对端在接收到该会话协商请求消息后，向终端发送200OK消息，并由该终端的SIP协议模块接收该200OK消息，该SIP协议模块向对端发送ACK(Acknowledgement，确认字符)响应消息，并开启第一媒体传输信道。In the embodiment of the present invention, the terminal may include a main control module, a SIP protocol module, and a TIP (Telepresence Interoperability Protocol) protocol module, and the main control module of the terminal initiates a new call to the SIP protocol module, SIP. The protocol module sends a session negotiation request message to the peer end, and after receiving the session negotiation request message, the peer end sends a 200 OK message to the terminal, and the SIP protocol module of the terminal receives the 200 OK message, and the SIP protocol module sends the message to the opposite end. An ACK (Acknowledgement) response message and the first media transport channel is turned on.
在本发明实施例中，终端与该对端发起基于H.323协议的会话协商流程，这样，在基于H.323协议的场景下，单个终端能够传输多路媒体数据流。In the embodiment of the present invention, the terminal and the peer end initiate a session negotiation process based on the H.323 protocol, so that in a scenario based on the H.323 protocol, a single terminal can transmit multiple media data streams.
其中，在本发明实施例中，终端可以包括主控模块，H.323协议模块和TIP协议模块，该终端的主控模块向H.323协议模块发起新的呼叫，H.323协议模块向对端发送创建消息，对端在接收到该创建(setup)消息后，向终端发送呼叫持续(callproceeding)消息和提示(alerting)消息以及连接(connect)消息，并由H.323协议模块接收呼叫持续消息和提示消息以及连接消息，与对端建立呼叫信令通道，并在该呼叫信令通道上与对端进行能力(capability)协商和主从(mast slave)确认，并在完成能力协商和主从确认后，开启第一媒体传输通道。In the embodiment of the present invention, the terminal may include a main control module, an H.323 protocol module, and a TIP protocol module, where the main control module of the terminal initiates a new call to the H.323 protocol module, and the H.323 protocol module is paired. The terminal sends a create message, and after receiving the setup message, the peer sends a call procedure and an alert message and a connect message to the terminal, and the call is continuously received by the H.323 protocol module. The message and the prompt message and the connection message establish a call signaling channel with the opposite end, and perform capability negotiation and master slave confirmation on the call signaling channel with the peer, and complete the capability negotiation and the master After confirmation, the first media transmission channel is turned on.
在本实施例中提供了一种媒体数据流的传输方法，图3是根据本发明实施例的的流程图，如图3所示，本发明实施例是以基于SIP协议的媒体数据流的传输为例进行说明的，本发明实施例中的终端包括主控模块和SIP协议模块以及TIP协议模块，该流程包括如下步骤：In this embodiment, a method for transmitting a media data stream is provided. FIG. 3 is a flowchart according to an embodiment of the present invention. As shown in FIG. 3, the embodiment of the present invention is a media data stream transmission based on a SIP protocol. For example, the terminal in the embodiment of the present invention includes a main control module, a SIP protocol module, and a TIP protocol module. The process includes the following steps:
步骤S301至步骤S304为会话协商流程：Steps S301 to S304 are session negotiation processes:
S301，终端的主控模块向SIP协议模块发起新的呼叫请求。S301. The terminal control module initiates a new call request to the SIP protocol module.
S302，该终端的SIP协议模块在接收到该呼叫请求后，向对端发送会话协商请求消息。 S302. After receiving the call request, the SIP protocol module of the terminal sends a session negotiation request message to the peer end.
S303，该终端的SIP协议模块接收对端根据会话协商请求消息发送的200OK消息。S303. The SIP protocol module of the terminal receives a 200 OK message sent by the peer end according to the session negotiation request message.
S304，该终端的SIP协议模块向对端发送ACK响应消息。S304. The SIP protocol module of the terminal sends an ACK response message to the peer end.
S305，该终端开启与对端的第一媒体传输通道。S305. The terminal opens a first media transmission channel with the opposite end.
步骤S306至步骤S310为TIP协商的流程：Steps S306 to S310 are processes of TIP negotiation:
S306，在根据会话协商的协商结果确定携带媒体能力参数时，该终端的主控模块模块向TIP协议模块发送TIP协商请求消息。S306: When determining the carrying media capability parameter according to the negotiation result of the session negotiation, the main control module of the terminal sends a TIP negotiation request message to the TIP protocol module.
其中，该媒体能力参数表示该终端支持至少两路媒体数据流的传输，该媒体能力参数可以是AAC_LD_3640。The media capability parameter indicates that the terminal supports transmission of at least two media data streams, and the media capability parameter may be AAC_LD_3640.
S307，该终端的TIP协议模块向对端发送数据流复用路数协商请求消息。S307. The TIP protocol module of the terminal sends a data stream multiplexing path number negotiation request message to the peer end.
S308，该终端的TIP协议模块接收对端根据数据流复用路数协商请求消息发送的数据流复用路数协商响应消息，并得到媒体数据流复用的路数和对端的NTP时间戳。S308, the TIP protocol module of the terminal receives the data stream multiplexing path negotiation response message sent by the peer end according to the data stream multiplexing path number negotiation request message, and obtains the number of channels multiplexed by the media data stream and the NTP timestamp of the opposite end.
S309，该终端的TIP协议模块向对端发送数据流格式协商请求消息。S309. The TIP protocol module of the terminal sends a data stream format negotiation request message to the peer end.
S310，该终端的TIP协议模块接收对端根据数据流格式协商请求消息发送的数据流格式协商响应消息，并得到媒体数据流传输的格式。S310: The TIP protocol module of the terminal receives a data stream format negotiation response message sent by the peer end according to the data stream format negotiation request message, and obtains a format of the media data stream transmission.
S311，该终端关闭与对端的第一媒体传输通道。S311. The terminal closes the first media transmission channel with the opposite end.
步骤S312至步骤S314为二次会话协商流程：Steps S312 to S314 are the second session negotiation process:
S312，在终端的NTP时间戳小于对端的NTP时间戳时，终端的主控模块根据媒体数据流复用的路数向SIP协议模块发起二次呼叫请求。S312: When the NTP timestamp of the terminal is smaller than the NTP timestamp of the peer end, the terminal control module initiates a secondary call request to the SIP protocol module according to the number of channels multiplexed by the media data stream.
S313，该终端的SIP协议模块在接收到该呼叫请求后，与对端发送会话协商请求消息。S313. After receiving the call request, the SIP protocol module of the terminal sends a session negotiation request message with the peer end.
S314，该终端的SIP协议模块接收对端根据会话协商请求消息发送的200OK消息.S314, the SIP protocol module of the terminal receives the 200 OK message sent by the peer end according to the session negotiation request message.
S315，该终端的SIP协议模块向对端发送ACK响应消息。S315. The SIP protocol module of the terminal sends an ACK response message to the peer end.
S316，终端根据二次发起的会话协商流程的协商结果开启第二媒体传输信道。 S316. The terminal starts the second media transport channel according to the negotiation result of the second initiated session negotiation process.
S317，终端根据媒体数据流复用的路数和媒体数据流传输的格式在该第二媒体传输信道上传输至少两路媒体数据流。S317. The terminal transmits at least two media data streams on the second media transport channel according to the number of channels multiplexed by the media data stream and the format of the media data stream transmission.
其中，终端根据媒体数据流复用的路数和媒体数据流传输的格式传输的数据流进行编码和解码。The terminal performs encoding and decoding according to the number of channels multiplexed by the media data stream and the data stream transmitted by the format of the media data stream transmission.
需要说明的是，对端也会按照上述步骤S301至步骤S315与该终端进行协商和数据交互。It should be noted that the peer end also performs negotiation and data interaction with the terminal according to the foregoing steps S301 to S315.
这样，采用上述实施步骤，在基于SIP协议的场景下，单个终端能够传输多路媒体数据流。Thus, with the above implementation steps, in a SIP-based scenario, a single terminal can transmit multiple media data streams.
在本实施例中提供了一种媒体数据流的传输方法，图4是根据本发明实施例的的流程图，如图4所示，本发明实施例是以基于H.323协议的媒体数据流的传输为例进行说明的，本发明实施例中的终端包括主控模块和H.323协议模块以及TIP协议模块，该流程包括如下步骤：In this embodiment, a method for transmitting a media data stream is provided. FIG. 4 is a flowchart according to an embodiment of the present invention. As shown in FIG. 4, the embodiment of the present invention is a media data stream based on the H.323 protocol. The transmission is described as an example. The terminal in the embodiment of the present invention includes a main control module, an H.323 protocol module, and a TIP protocol module. The process includes the following steps:
步骤S401至步骤S404为会话协商流程：Steps S401 to S404 are the session negotiation process:
S401，终端的主控模块向该终端的H.323协议模块发起新的呼叫请求。S401. The terminal control module of the terminal initiates a new call request to the H.323 protocol module of the terminal.
S402，该终端的H.323协议模块向对端发送创建消息。S402. The H.323 protocol module of the terminal sends a creation message to the peer end.
S403，该终端的H.323协议模块接收对端根据创建消息发送的呼叫持续消息和提示消息以及连接消息，与对端建立呼叫信令通道。S403. The H.323 protocol module of the terminal receives a call duration message and a prompt message and a connection message sent by the peer end according to the creation message, and establishes a call signaling channel with the peer end.
S404，该终端的H.323协议模块在该呼叫信令通道上与对端进行能力协商和主从确认。S404: The H.323 protocol module of the terminal performs capability negotiation and master-slave confirmation with the peer end on the call signaling channel.
S405，该终端开启与对端的第一媒体传输通道。S405. The terminal turns on a first media transmission channel with the opposite end.
步骤S406至步骤S410为TIP协商的流程：Steps S406 to S410 are processes of TIP negotiation:
S406在根据会话协商的协商结果确定携带媒体能力参数时，该终端的主控模块模块向TIP协议模块发送TIP协商请求消息。S406, when determining the carrying media capability parameter according to the negotiation result of the session negotiation, the main control module of the terminal sends a TIP negotiation request message to the TIP protocol module.
其中，该媒体能力参数表示该终端支持至少两路媒体数据流的传输，该媒体能力参数可以是AAC_LD_3640。The media capability parameter indicates that the terminal supports transmission of at least two media data streams, and the media capability parameter may be AAC_LD_3640.
S407，该终端的TIP协议模块向对端发送数据流复用路数协商请求消息。 S407. The TIP protocol module of the terminal sends a data stream multiplexing path number negotiation request message to the peer end.
S408，该终端的TIP协议模块接收对端根据数据流复用路数协商请求消息发送的数据流复用路数协商响应消息，并得到媒体数据流复用的路数和对端的NTP时间戳。S408, the TIP protocol module of the terminal receives the data stream multiplexing path negotiation response message sent by the peer end according to the data stream multiplexing path number negotiation request message, and obtains the number of channels multiplexed by the media data stream and the NTP timestamp of the opposite end.
S409，该终端的TIP协议模块向对端发送数据流格式协商请求消息。S409. The TIP protocol module of the terminal sends a data stream format negotiation request message to the peer end.
S410，该终端的TIP协议模块接收对端根据数据流格式协商请求消息发送的数据流格式协商响应消息，并得到媒体数据流传输的格式。S410: The TIP protocol module of the terminal receives a data stream format negotiation response message sent by the peer end according to the data stream format negotiation request message, and obtains a format of the media data stream transmission.
S411，该终端关闭与对端的第一媒体传输通道。S411. The terminal closes the first media transmission channel with the opposite end.
步骤S412至步骤S415为二次会话协商流程：Steps S412 to S415 are the second session negotiation process:
S412，在终端的NTP时间戳小于对端的NTP时间戳时，终端的主控模块向该终端的H.323协议模块发起二次呼叫请求。S412: When the NTP timestamp of the terminal is smaller than the NTP timestamp of the peer end, the main control module of the terminal initiates a secondary call request to the H.323 protocol module of the terminal.
S413，该终端的H.323协议模块向对端发送创建消息。S413. The H.323 protocol module of the terminal sends a creation message to the peer end.
S414，该终端的H.323协议模块接收对端根据创建消息发送的呼叫持续消息和提示消息以及连接消息，与对端建立呼叫信令通道。S414. The H.323 protocol module of the terminal receives a call duration message and a prompt message and a connection message sent by the peer end according to the creation message, and establishes a call signaling channel with the peer end.
S415，该终端的H.323协议模块在该呼叫信令通道上与对端进行能力协商和主从确认。S415: The H.323 protocol module of the terminal performs capability negotiation and master-slave confirmation with the peer end on the call signaling channel.
S416，终端根据二次发起的会话协商流程的协商结果开启第二媒体传输信道。S416. The terminal starts the second media transport channel according to the negotiation result of the second initiated session negotiation process.
S417，终端根据媒体数据流复用的路数和媒体数据流传输的格式在该第二媒体传输信道上传输至少两路媒体数据流。S417. The terminal transmits at least two media data streams on the second media transmission channel according to the number of channels multiplexed by the media data stream and the format of the media data stream transmission.
其中，终端根据媒体数据流复用的路数和媒体数据流传输的格式传输的数据流进行编码和解码。The terminal performs encoding and decoding according to the number of channels multiplexed by the media data stream and the data stream transmitted by the format of the media data stream transmission.
需要说明的是，对端也会按照上述步骤S411至步骤S417与该终端进行协商和数据交互。It should be noted that the peer end also performs negotiation and data interaction with the terminal according to the foregoing steps S411 to S417.
这样，采用上述实施步骤，在基于H.323协议的场景下，单个终端能够传输多路媒体数据流。Thus, with the above implementation steps, in a scenario based on the H.323 protocol, a single terminal can transmit multiple media data streams.
在本实施例中还提供了一种媒体数据流的传输装置，该装置应用于终端，该装置用于实现上述实施例及优选实施方式，已经进行过说明的不再赘述。如以下所使用的，术语“模块”可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装 置较佳地以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能并被构想的。In this embodiment, a device for transmitting a media data stream is also provided, and the device is applied to a terminal, and the device is used to implement the foregoing embodiments and preferred embodiments, and details are not described herein. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Despite the description of the following embodiments The implementation is preferably implemented in software, but hardware, or a combination of software and hardware, is also possible and contemplated.
图5是根据本发明实施例的媒体数据流的传输装置的结构框图，如图5所示，该装置50包括：FIG. 5 is a structural block diagram of a device for transmitting a media data stream according to an embodiment of the present invention. As shown in FIG. 5, the device 50 includes:
会话发起模块51，设置为与对端发起会话协商流程；The session initiation module 51 is configured to initiate a session negotiation process with the peer end.
通道控制模块52，设置为在该会话发起模块51完成该会话协商流程后，根据该会话协商流程的协商结果开启第一媒体传输信道；The channel control module 52 is configured to: after the session initiation module 51 completes the session negotiation process, start the first media transmission channel according to the negotiation result of the session negotiation process;
协商模块53，设置为根据该协商结果中的媒体能力参数与该对端协商得到媒体能力信息，其中，该媒体能力参数表示该终端支持至少两路媒体数据流的传输，该媒体能力信息包括该至少两路媒体数据流复用的路数；The negotiation module 53 is configured to negotiate with the peer to obtain media capability information according to the media capability parameter in the negotiation result, where the media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the The number of ways to multiplex at least two media streams;
该通道控制模块52，设置为在该协商模块53得到媒体能力信息后，关闭该第一媒体传输信道；The channel control module 52 is configured to: after the negotiation module 53 obtains the media capability information, close the first media transmission channel;
该会话发起模块51，还设置为在该通道控制模块52关闭该第一媒体传输信道后，根据该协商模块53得到的媒体能力参数与该对端再次发起该会话协商流程；The session initiation module 51 is further configured to: after the channel control module 52 closes the first media transmission channel, initiate the session negotiation process according to the media capability parameter obtained by the negotiation module 53 and the peer end;
该通道控制模块52，还设置为根据该会话发起模块51再次发起的会话协商流程的协商结果开启第二媒体传输信道；The channel control module 52 is further configured to enable the second media transport channel according to the negotiation result of the session negotiation process initiated by the session initiation module 51;
传输模块54，设置为在该通道控制模块52开启的该第二媒体传输信道上传输该至少两路媒体数据流。The transmission module 54 is configured to transmit the at least two media data streams on the second media transport channel opened by the channel control module 52.
在本发明实施例中，该协商结果中还携带有该至少两个媒体数据流的数据流格式；如图6所示，该装置还包括：In the embodiment of the present invention, the negotiation result further includes a data stream format of the at least two media data streams; as shown in FIG. 6, the device further includes:
处理模块55，设置为在该传输模块54在该第二媒体传输信道上传输该至少两路媒体数据流复用的路数后，根据该至少两路媒体数据流复用的路数和该数据流格式对该传输的该媒体数据流进行编码和解码。The processing module 55 is configured to: after the transmission module 54 transmits the number of channels multiplexed by the at least two media data streams on the second media transmission channel, the number of channels multiplexed according to the at least two media data streams and the data The stream format encodes and decodes the transmitted media data stream.
在本发明实施例中，该媒体能力信息还包括该终端的第一网络时间协议NTP时间戳和该对端的第二NTP时间戳； In the embodiment of the present invention, the media capability information further includes a first network time protocol NTP timestamp of the terminal and a second NTP timestamp of the peer end;
该协商模块还设置为，在确定该第一NTP时间戳小于该对端的第二NTP时间戳时，与该对端再次发起该会话协商流程，这样，在基于SIP协议的场景下，单个终端能够传输多路媒体数据流。The negotiation module is further configured to: when determining that the first NTP timestamp is smaller than the second NTP timestamp of the peer end, initiate the session negotiation process with the peer end, so that in the SIP protocol-based scenario, the single terminal can Transfer multiple media streams.
在本发明实施例中，该会话发起模块51设置为，与该对端发起基于会话初始化协议SIP协议的会话协商流程。In the embodiment of the present invention, the session initiation module 51 is configured to initiate a session negotiation process based on the session initiation protocol SIP protocol with the peer.
在本发明实施例中，该会话发起模块51设置为，与该对端发起基于H.323协议的会话协商流程，这样，在基于H.323协议的场景下，单个终端能够传输多路媒体数据流。。In the embodiment of the present invention, the session initiation module 51 is configured to initiate a session negotiation process based on the H.323 protocol with the peer end, so that in a scenario based on the H.323 protocol, a single terminal can transmit multiple media data. flow. .
通过上述装置，由于终端与对端在协商中确定两端支持至少两路媒体数据流的复用，从而解决了单个终端无法传输多路媒体数据流的问题。Through the foregoing apparatus, the terminal and the peer determine that the two ends support multiplexing of at least two media data streams in the negotiation, thereby solving the problem that a single terminal cannot transmit the multi-channel media data stream.
综上所述，本发明实施例采用以下技术方案：In summary, the embodiment of the present invention adopts the following technical solutions:
输入源方面，各路输入源可以选择任意的视频输入接口、任意的工作模式、任意的制式作为视频源，也可选择相同的源，输入源的选择上灵活配置。For the input source, each input source can select any video input interface, any working mode, any standard as the video source, or the same source, and the input source can be flexibly configured.
信令方面，在H.323或者SIP协议上增加TIP协议进行能力交互、打开通道，三路主视频媒体复用在一路RTP通道上传输媒体数据，辅视频单独一路，音频单独一路。In terms of signaling, the TIP protocol is added to the H.323 or SIP protocol to perform capability interaction and open channels. The three primary video media are multiplexed to transmit media data on one RTP channel, and the auxiliary video is separated by one channel and the audio is separately.
显示方面，最少可能只有一个显示器，这就需要在这个显示器上输出所有本端(相当于终端)、远端(相当于对端)、辅视频图像的多画面，或者子画面的多画面不同布局的合成图像。例如单显示器的四流时，媒体共包含远端1、远端2、远端3、辅视频四路媒体，则显示器上需要能够显示远主N、本主N、辅视频的大画面，以及pip、triple等不同布局的2～7个子画面组成的多画面。In terms of display, there may be at least one display at least. This requires outputting all the local end (equivalent to the terminal), the far end (equivalent to the opposite end), the multi-picture of the auxiliary video image, or the multi-screen different layout of the sub-picture on this display. Composite image. For example, when the four streams of a single display, the media includes a remote 1, a remote 2, a remote 3, a secondary video four-way media, the display needs to be able to display a large screen of the remote N, the main N, the auxiliary video, and Multi-screen composed of 2 to 7 sub-pictures of different layouts such as pip and triple.
对于SIP协议的数据流传输，发起呼叫端给对端发起会话协商请求，对端收到后回应200ok，发起端再回应一个ACK进行确认，SIP的协议完成初始协商后，发现交互能力携带音频能力AAC_LD_3640，则发起TIP协商TipNegotiate，TIP协商出媒体复用的路数(MUXCTRL)和格式(MEDIAOPTS)，TIP协商通过后，进行二次重邀，reINVITE重邀是本端还是对端发起取决于MUXCTRL消息中的NTP时间戳，时间戳小的一方主动发起二次重邀，二次重邀成功后，打开视音频媒体通道openchannel，RTP媒体流开始收发。如果初始协商中没有AAC_LD_3640音频能力，则按传统终端模式上会。 For the data stream transmission of the SIP protocol, the initiating caller initiates a session negotiation request to the peer end, and the peer end responds 200 ok after receiving the response, and the originating end responds with an ACK to confirm. After the initial negotiation of the SIP protocol, the interactive capability is enabled to carry the audio capability. AAC_LD_3640 initiates TIP negotiation TipNegotiate. The TIP negotiates the number of media reuse channels (MUXCTRL) and format (MEDIAOPTS). After the TIP negotiation, the second re-invite is performed. The reINVITE re-invitation is initiated by the local end or the peer end depending on the MUXCTRL. The NTP timestamp in the message, the party with the small timestamp actively initiates the second re-invitation. After the second re-invitation succeeds, the video and audio media channel openchannel is opened, and the RTP media stream starts to send and receive. If there is no AAC_LD_3640 audio capability in the initial negotiation, it will be in the traditional terminal mode.
对于H.323协议的数据流传输，呼叫的时候按照标准的H.323协议建立连接，按照H225协议，发起呼叫端给对端发送setup消息，对端收到后回复callproceeding、Alerting、Connect消息后，建立呼叫信令通道，在这通道上进行能力协商和主从确认，如果能力中携带音频能力AAC_LD_3640，则发起TIP协商TipNegotiate，TIP协商出媒体复用的路数(MUXCTRL)和格式(MEDIAOPTS)，TIP协商通过后，进行二次重邀，重新进行能力协商确认，二次重邀是本端还是对端发起取决于MUCTRL消息中的NTP时间戳，时间戳小的一方主动发起二次重邀，二次重邀成功后，先关闭第一次打开的媒体通道，再按照最后一次协商的结果重新打开媒体通道，视音频媒体通道打开后，RTP(Real-time Transport Protocol，实时传输协议)媒体流开始收发。如果初始协商中没有AAC_LD_3640音频能力，则按传统终端模式上会。For the data stream transmission of the H.323 protocol, the call is established according to the standard H.323 protocol. According to the H225 protocol, the originating caller sends a setup message to the peer, and after receiving the call, the call canceleeding, Alerting, and Connect messages are received. The call signaling channel is established, and capability negotiation and master-slave confirmation are performed on the channel. If the capability carries the audio capability AAC_LD_3640, the TIP negotiation TipNegotiate is initiated, and the TIP negotiates the number of media multiplexing channels (MUXCTRL) and format (MEDIAOPTS). After the TIP negotiation is passed, the second re-invitation is performed, and the capability negotiation is confirmed again. Whether the second re-invitation is initiated by the local end or the peer depends on the NTP timestamp in the MUCTRL message, and the party with the small timestamp initiates the second re-invitation. After the second re-invitation is successful, the media channel opened for the first time is closed, and then the media channel is re-opened according to the result of the last negotiation, and the RTP (Real-time Transport Protocol) media is opened after the video and audio media channel is opened. The stream starts to send and receive. If there is no AAC_LD_3640 audio capability in the initial negotiation, it will be in the traditional terminal mode.
显然，本领域的技术人员应该明白，上述的本发明的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，可选地，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本发明不限制于任何特定的硬件和软件结合。It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.
工业实用性Industrial applicability
本发明实施例提供的技术方案，可以应用于媒体数据流的传输过程中，终端与对端发起会话协商流程，并根据该会话协商流程的协商结果开启第一媒体传输通道，并根据该终端的媒体能力参数与该对端协商得到媒体能力信息，其中，该媒体能力参数表示该终端支持至少两路媒体数据流的传输，该媒体能力信息包括该至少两路媒体数据流复用的路数，终端关闭该第一媒体传输信道，并根据该媒体数据流复用的路数与该对端再次发起该会话协商流程，并根据该再次发起的会话协商流程的协商结果开启第二媒体传输信道，并在该第二媒体传输信道上传输该至少两路媒体数据流，这样，由于终端与对端在协商中确定两端支持至少两路媒体数据流的复用，从而解决了单个终端无法传输多路媒体数据流的问题。 The technical solution provided by the embodiment of the present invention may be applied to the process of transmitting the media data stream, and the terminal initiates a session negotiation process with the peer end, and starts the first media transmission channel according to the negotiation result of the session negotiation process, and according to the terminal, The media capability parameter is negotiated with the peer to obtain the media capability information, where the media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the number of channels multiplexed by the at least two media data streams. The terminal closes the first media transmission channel, and initiates the session negotiation process again according to the number of channels multiplexed by the media data stream, and starts the second media transmission channel according to the negotiation result of the re-initiated session negotiation process. And transmitting the at least two media data streams on the second media transmission channel, so that the terminal and the peer end determine that the two ends support multiplexing of at least two media data streams, thereby solving the problem that the single terminal cannot transmit more The problem of the road media data stream.

Claims (10)

  1. 一种媒体数据流的传输方法，包括：A method for transmitting a media data stream, comprising:
    终端与对端发起会话协商流程，并根据所述会话协商流程的协商结果开启第一媒体传输通道；The terminal initiates a session negotiation process with the peer end, and starts the first media transmission channel according to the negotiation result of the session negotiation process;
    根据所述协商结果中的媒体能力参数与所述对端协商得到媒体能力信息，其中，所述媒体能力参数表示所述终端支持至少两路媒体数据流的传输，所述媒体能力信息包括所述至少两路媒体数据流复用的路数；Obtaining media capability information according to the media capability parameter in the negotiation result, where the media capability parameter indicates that the terminal supports transmission of at least two media data streams, where the media capability information includes the The number of ways to multiplex at least two media streams;
    关闭所述第一媒体传输信道，并根据所述媒体数据流复用的路数与所述对端再次发起所述会话协商流程；Close the first media transport channel, and initiate the session negotiation process with the peer according to the number of paths multiplexed by the media data stream;
    根据所述再次发起的会话协商流程的协商结果开启第二媒体传输信道，并在所述第二媒体传输信道上传输所述至少两路媒体数据流。And performing, according to the negotiation result of the re-initiated session negotiation process, a second media transport channel, and transmitting the at least two media data streams on the second media transport channel.
  2. 根据权利要求1所述的方法，其中，所述协商结果中还携带有所述至少两个媒体数据流的数据流格式；The method according to claim 1, wherein the negotiation result further carries a data stream format of the at least two media data streams;
    在所述第二媒体传输信道上传输所述至少两路媒体数据流复用的路数后，还包括：After the number of the multiplexed channels of the at least two media data streams is transmitted on the second media transmission channel, the method further includes:
    根据所述至少两路媒体数据流复用的路数和所述数据流格式对所述传输的所述媒体数据流进行编码和解码。And encoding the transmitted media data stream according to the number of channels multiplexed by the at least two media data streams and the data stream format.
  3. 根据权利要求2所述的方法，其中，所述媒体能力信息还包括所述终端的第一网络时间协议NTP时间戳和所述对端的第二NTP时间戳；The method according to claim 2, wherein the media capability information further comprises a first network time protocol NTP timestamp of the terminal and a second NTP timestamp of the opposite end;
    与所述对端再次发起所述会话协商流程包括：The process of initiating the session negotiation with the peer end includes:
    在确定所述第一NTP时间戳小于所述对端的第二NTP时间戳时，与所述对端再次发起所述会话协商流程。When it is determined that the first NTP timestamp is smaller than the second NTP timestamp of the peer end, the session negotiation process is initiated again with the peer end.
  4. 根据权利要求1至3中任一项所述的方法，其中，终端与对端发起会话协商流程包括：The method according to any one of claims 1 to 3, wherein the process of initiating a session negotiation between the terminal and the peer end comprises:
    终端与所述对端发起基于会话初始化协议SIP协议的会话协商流程。The terminal initiates a session negotiation process based on the Session Initiation Protocol SIP protocol with the peer.
  5. 根据权利要求1至3中任一项所述的方法，其中，终端与对端发起会话协商流程包括： The method according to any one of claims 1 to 3, wherein the process of initiating a session negotiation between the terminal and the peer end comprises:
    终端与所述对端发起基于H.323协议的会话协商流程。The terminal initiates a session negotiation process based on the H.323 protocol with the peer end.
  6. 一种媒体数据流的传输装置，应用于终端，所述装置包括：A transmission device for a media data stream is applied to a terminal, and the device includes:
    会话发起模块，设置为与对端发起会话协商流程；The session initiation module is configured to initiate a session negotiation process with the peer end;
    通道控制模块，设置为在所述会话发起模块完成所述会话协商流程后，根据所述会话协商流程的协商结果开启第一媒体传输信道；a channel control module, configured to: after the session initiation module completes the session negotiation process, start a first media transmission channel according to a negotiation result of the session negotiation process;
    协商模块，设置为根据所述协商结果中的媒体能力参数与所述对端协商得到媒体能力信息，其中，所述媒体能力参数表示所述终端支持至少两路媒体数据流的传输，所述媒体能力信息包括所述至少两路媒体数据流复用的路数；The negotiation module is configured to obtain media capability information by negotiating with the peer according to the media capability parameter in the negotiation result, where the media capability parameter indicates that the terminal supports transmission of at least two media data streams, the media The capability information includes the number of ways in which the at least two media data streams are multiplexed;
    所述通道控制模块，设置为在所述协商模块得到所述媒体能力信息后，关闭所述第一媒体传输信道；The channel control module is configured to: after the negotiation module obtains the media capability information, turn off the first media transmission channel;
    所述会话发起模块，还设置为在所述通道控制模块关闭所述第一媒体传输信道后，根据所述协商模块得到的媒体能力参数与所述对端再次发起所述会话协商流程；The session initiating module is further configured to: after the channel control module closes the first media transport channel, initiate the session negotiation process according to the media capability parameter obtained by the negotiation module and the peer end;
    所述通道控制模块，还设置为根据所述会话发起模块再次发起的会话协商流程的协商结果开启第二媒体传输信道；The channel control module is further configured to enable the second media transport channel according to the negotiation result of the session negotiation process initiated by the session initiation module;
    传输模块，设置为在所述通道控制模块开启的所述第二媒体传输信道上传输所述至少两路媒体数据流。And a transmission module, configured to transmit the at least two media data streams on the second media transmission channel that is opened by the channel control module.
  7. 根据权利要求6所述的装置，其中，所述协商结果中还携带有所述至少两个媒体数据流的数据流格式；所述装置还包括：The device according to claim 6, wherein the negotiation result further carries a data stream format of the at least two media data streams; the device further includes:
    处理模块，设置为在所述传输模块在所述第二媒体传输信道上传输所述至少两路媒体数据流复用的路数后，根据所述至少两路媒体数据流复用的路数和所述数据流格式对所述传输的所述媒体数据流进行编码和解码。a processing module, configured to: after the transmission module transmits the number of channels multiplexed by the at least two media data streams on the second media transmission channel, according to the number of channels multiplexed by the at least two media data streams The data stream format encodes and decodes the transmitted media data stream.
  8. 根据权利要求7所述的装置，其中，所述媒体能力信息还包括所述终端的第一网络时间协议NTP时间戳和所述对端的第二NTP时间戳；The device according to claim 7, wherein the media capability information further comprises a first network time protocol NTP timestamp of the terminal and a second NTP timestamp of the opposite end;
    所述协商模块还设置为，在确定所述第一NTP时间戳小于所述对端的第二NTP时间戳时，与所述对端再次发起所述会话协商流程。The negotiation module is further configured to initiate the session negotiation process with the peer end when determining that the first NTP timestamp is smaller than the second NTP timestamp of the opposite end.
  9. 根据权利要求6至8中任一项所述的装置，其中，所述会话发起模块设置为，与所述对端发起基于会话初始化协议SIP协议的会话协商流程。 The apparatus according to any one of claims 6 to 8, wherein the session initiation module is configured to initiate a session negotiation procedure based on a Session Initiation Protocol SIP protocol with the opposite end.
  10. 根据权利要求6至8中任一项所述的装置，其中，所述会话发起模块设置为，与所述对端发起基于H.323协议的会话协商流程。 The apparatus according to any one of claims 6 to 8, wherein the session initiation module is configured to initiate a session negotiation procedure based on an H.323 protocol with the opposite end.
PCT/CN2015/082946 2014-12-12 2015-06-30 Transmission method and device for media data streams WO2016090904A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410768139.0A CN105744209B (en) 2014-12-12 2014-12-12 The transmission method and device of media data flow
CN201410768139.0 2014-12-12

Publications (1)

Publication Number Publication Date
WO2016090904A1 true WO2016090904A1 (en) 2016-06-16

Family

ID=56106582

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/082946 WO2016090904A1 (en) 2014-12-12 2015-06-30 Transmission method and device for media data streams

Country Status (2)

Country Link
CN (1) CN105744209B (en)
WO (1) WO2016090904A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112954255A (en) * 2021-01-29 2021-06-11 深圳市捷视飞通科技股份有限公司 Transmission method and device of video conference code stream, computer equipment and storage medium

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114553839B (en) * 2022-02-25 2024-03-15 阿里巴巴（中国）有限公司 RTC data processing method and device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1476209A (en) * 2002-08-12 2004-02-18 华为技术有限公司 Method of receivel/send end ability selection
CN101073257A (en) * 2004-12-22 2007-11-14 中兴通讯股份有限公司 Method for transmitting multi-path video for conference television system
CN102883131A (en) * 2011-07-15 2013-01-16 中兴通讯股份有限公司 Signaling interaction method and device based on tele-presence system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100505864C (en) * 2005-02-06 2009-06-24 中兴通讯股份有限公司 Multi-spot video conference system and media processing method
CN103634697B (en) * 2012-08-24 2017-09-26 中兴通讯股份有限公司 Net the implementation method of true technology and net true equipment

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1476209A (en) * 2002-08-12 2004-02-18 华为技术有限公司 Method of receivel/send end ability selection
CN101073257A (en) * 2004-12-22 2007-11-14 中兴通讯股份有限公司 Method for transmitting multi-path video for conference television system
CN102883131A (en) * 2011-07-15 2013-01-16 中兴通讯股份有限公司 Signaling interaction method and device based on tele-presence system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112954255A (en) * 2021-01-29 2021-06-11 深圳市捷视飞通科技股份有限公司 Transmission method and device of video conference code stream, computer equipment and storage medium
CN112954255B (en) * 2021-01-29 2024-04-16 深圳市捷视飞通科技股份有限公司 Video conference code stream transmission method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN105744209B (en) 2019-01-25
CN105744209A (en) 2016-07-06

Similar Documents

Publication Publication Date Title
US10015440B2 (en) Multiple channel communication using multiple cameras
AU2016213760B2 (en) Media negotiation method, device, and system for multi-stream conference
US9172912B2 (en) Telepresence method, terminal and system
US8228363B2 (en) Method and system for conducting continuous presence conferences
WO2019128204A1 (en) Conference implementation method, device, apparatus and system, and computer readable storage medium
US9344475B2 (en) Media transmission method and system based on telepresence
US9554127B2 (en) Display apparatus, method for controlling the display apparatus, glasses and method for controlling the glasses
US9497390B2 (en) Video processing method, apparatus, and system
WO2015127799A1 (en) Method and device for negotiating on media capability
WO2015154608A1 (en) Method, system and apparatus for sharing video conference material
WO2012079424A1 (en) Distributed video processing method, system and multipoint control unit
US10348777B2 (en) Telehealth video chat mirroring of disparate video chat devices
WO2007140668A1 (en) Method and apparatus for realizing remote monitoring in conference television system
WO2015003532A1 (en) Multimedia conferencing establishment method, device and system
US20160241683A1 (en) Method and device for processing capability negotiation, and telepresence endpoint
WO2014177082A1 (en) Video conference video processing method and terminal
WO2016090904A1 (en) Transmission method and device for media data streams
EP2637404B1 (en) Method and device for controlling multiple auxiliary streams, and network system
US10142590B2 (en) Devices, system and method for sharing a presentation
US11102451B2 (en) Videoconferencing server for providing multi-screen videoconferencing by using a plurality of videoconferencing terminals and method therefor
KR100812964B1 (en) Video-telephony control system and method thereof
KR20160016655A (en) System and method for providing multi-sharing of multimedia
KR101962447B1 (en) Terminal apparatus and method of executing a real-time application using thereof
TW201537990A (en) Method and system for processing video data of meeting

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15868647

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15868647

Country of ref document: EP

Kind code of ref document: A1