WO2017008605A1 - 抗丢包处理方法和装置 - Google Patents

抗丢包处理方法和装置 Download PDF

Info

Publication number
WO2017008605A1
WO2017008605A1 PCT/CN2016/085559 CN2016085559W WO2017008605A1 WO 2017008605 A1 WO2017008605 A1 WO 2017008605A1 CN 2016085559 W CN2016085559 W CN 2016085559W WO 2017008605 A1 WO2017008605 A1 WO 2017008605A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio stream
message
stream
packet
receiving
Prior art date
Application number
PCT/CN2016/085559
Other languages
English (en)
French (fr)
Inventor
柳亮亮
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2017008605A1 publication Critical patent/WO2017008605A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast

Definitions

  • the present invention relates to the field of audio conferencing, and in particular to an anti-drop packet processing method and apparatus.
  • the conference server receives audio streams from multiple conference terminals.
  • the conference server needs to pass the received audio stream through the transcoding function.
  • the encoded audio stream supported by the receiving conference terminal is then sent to the receiving conference terminal, and the receiving conference terminal receives the transcoded audio stream to correctly decode and restore the sound.
  • Simultaneous simulcast technology is adopted in the current audio conference, that is, the sender conference terminal is connected to an original audio.
  • the sequence is encoded into a plurality of audio streams by using different codecs, and then the multiple audio streams are simultaneously sent to the conference server, and the conference server selects one of the multiple audio streams according to the codec mode supported by the receiver conference terminal.
  • the audio stream is sent to the receiver conference terminal.
  • the plurality of audio streams are referred to as simultaneous simulcast streams.
  • the packet may be lost.
  • the packet may also be referred to as a data packet.
  • the packet loss may be referred to as packet loss, in order to avoid packet loss.
  • the problem of the user experience is reduced.
  • the common practice is to implement the anti-drop function by using redundant packets.
  • the sender conference terminal sends the data to be sent with a certain redundant error correction code, and the conference server corrects the error.
  • the error code performs error detection on the data and packet loss detection. If an error or packet loss is found, it is corrected or restored by the conference server.
  • the anti-drop packet function is implemented through redundant packets, the bandwidth overhead is increased, and the redundancy report is additionally added.
  • the codec processing of the text has low processing efficiency and high complexity.
  • the embodiment of the invention provides an anti-drop packet processing method and device, which can effectively reduce bandwidth overhead and improve processing efficiency, and the method is simple.
  • an anti-drop packet processing method is provided.
  • the method is applied to an audio conference using a simultaneous simulcasting technology, where the method is performed by a conference server, and the method includes:
  • the simultaneous simulcast stream includes an audio stream adopting a first coding mode and an audio stream adopting a second coding mode, where the audio stream of the first coding mode is adopted and the second coding is adopted.
  • the audio stream of the mode has the same media content and the same packaging duration;
  • the audio stream in the second coding mode is used to find a message having the same media content as the lost message
  • the audio stream using the first coding mode and the audio stream using the second coding mode have the same media content and the same packetization duration, so the audio stream adopting the second coding mode may be referred to as adopting the first coding mode.
  • a homologous mapping stream of an audio stream includes a plurality of audio streams of different encoding modes in the simultaneous simulcast stream, and the foregoing plurality is at least two.
  • the audio stream using the first encoding mode may have multiple a homologous mapping stream.
  • the audio stream using the second encoding mode may be any one of the multiple homologous mapping streams, or may be from the multiple homologous mapping streams according to a preset rule.
  • an association may be established between multiple audio streams that are mutually homogenous mapping flows, and is used to indicate that the multiple audio streams are mutually the same mapping flow.
  • the method before the receiving the simultaneous simulcast stream sent by the first conference terminal, the method further includes:
  • the simultaneous simulcast capability indication information is used to indicate that the first conference terminal has the capability of sending a simultaneous simulcast stream
  • the simultaneous simulcast stream includes the audio that adopts the first coding mode.
  • the audio stream adopting the second encoding mode wherein the audio stream adopting the first encoding manner has the same media content and the same packaging duration as the audio stream adopting the second encoding manner;
  • simulcast capability confirmation information is used to indicate that the conference server has the capability of receiving the simultaneous simulcast stream.
  • the method before the performing the packet loss detection on the audio stream that uses the first coding mode, the method further includes:
  • Receiving the receiving capability indication information sent by the second conference terminal where the receiving capability indication information is used to indicate that the second conference terminal has the capability of receiving the audio stream adopting the first encoding mode and the audio stream adopting the second encoding mode, and The second conference terminal has the capability of receiving a message of an audio stream of another coding mode inserted in an audio stream of one coding mode;
  • the third possibility in the first aspect In combination with the first aspect or the first or second possible implementation of the first aspect, the third possibility in the first aspect In the implementation manner, the sequence number information of the message of the audio stream using the first coding mode having the same media content as the message is recorded in the message of the audio stream of the second coding mode;
  • the packet in the second encoding mode is used to find the packet with the same media content as the lost packet, including:
  • the sequence number of the lost message is determined
  • the message in the second encoding mode is used to search for a message in which the serial number information of the lost message is recorded in the message.
  • the packet of the audio stream that uses the second coding mode includes a first extended parameter and a second extended parameter
  • the first extended parameter is used to record that the sequence number of the message in the audio channel of the first coding mode having the same media content as the message is larger or smaller than the sequence number of the message
  • the second extended parameter is used. And recording a difference between a sequence number of the message of the audio stream of the first coding mode and the sequence number of the message having the same media content as the message.
  • a second aspect provides an anti-drop packet processing device, where the device is applied to an audio conference using a simultaneous simulcasting technology, the device is disposed in a conference server, and the device includes:
  • a receiving unit configured to receive a simultaneous simulcast stream sent by the first conference terminal, where the simultaneous simulcast stream includes an audio stream that adopts a first encoding manner and an audio stream that adopts a second encoding manner, where the audio stream of the first encoding manner is adopted.
  • a detecting unit configured to perform packet loss detection on the audio stream that is received by the receiving unit and adopts the first encoding mode
  • a searching unit configured to: when the detecting unit detects that the audio stream in the first encoding mode is lost, find and use the same media in the audio stream in the second encoding manner received by the receiving unit Content message
  • An embedding unit configured to embed, in the audio stream of the first coding mode, a packet that is found by the searching unit and has the same media content as the lost packet, to obtain the anti-loss packet processing An encoded audio stream;
  • a sending unit configured to send, by the embedded unit, the audio stream of the first coding mode that is resistant to packet loss processing to the second conference terminal.
  • the receiving unit is further configured to receive the simultaneous simulcast capability indication information sent by the first conference terminal, where the simulcasting capability indication information is used to indicate the foregoing, before the receiving unit receives the simultaneous simulcast stream sent by the first conference terminal.
  • the first conference terminal has the capability of transmitting a simultaneous simulcast stream, where the simultaneous simulcast stream includes an audio stream adopting a first encoding manner and an audio stream adopting a second encoding manner, wherein the audio stream adopting the first encoding manner and adopting the second encoding manner
  • the encoded audio stream has the same media content and the same packaging duration;
  • the sending unit is further configured to send the simultaneous simulcast capability confirmation information to the first conference terminal, where the simulcast capability confirmation information is used to indicate that the conference server has the capability of receiving the simultaneous simulcast stream.
  • the receiving unit is further configured to: before the detecting unit performs packet loss detection on the audio stream that adopts the first coding mode, receive the receiving capability indication information sent by the second conference terminal, where the receiving capability indication information is used to indicate the location Second meeting
  • the terminal has the capability of receiving an audio stream using the first encoding mode and an audio stream adopting the second encoding mode, and the second conference terminal has the function of receiving another encoding method in the audio stream adopting one encoding mode. The ability of the message of the audio stream;
  • the sending unit is further configured to send, to the second conference terminal, receive capability confirmation information, where the receive capability confirmation information is used to indicate that the conference server has another code inserted in an audio stream that adopts one coding mode. The ability of the way to stream messages.
  • the receiving unit receives the audio stream that uses the second encoding mode
  • the sequence number information of the message of the audio stream using the first coding mode having the same media content as the message is recorded in the message;
  • the searching unit includes:
  • Determining a subunit configured to determine a sequence number of the lost message when the detecting unit detects that the audio stream in the first coding mode is lost;
  • a locating sub-unit configured to search, in the audio stream that uses the second coding mode, a message in which the sequence number information of the lost message determined by the determining sub-unit is recorded in the message.
  • the receiving, the receiving, by the receiving unit, the first extended parameter is included in the packet of the audio stream in the second encoding mode.
  • a second extended parameter where the first extended parameter is used to record that the sequence number of the message of the audio stream in the first coding mode having the same media content as the message is larger or smaller than the sequence number of the message.
  • the second extended parameter is used to record the difference between the sequence number of the message of the audio stream of the first coding mode and the sequence number of the message having the same media content as the message.
  • a conference server including:
  • the memory is configured to store program instructions
  • the processor is configured to perform the following operations according to program instructions stored in the memory:
  • a simultaneous simulcast stream sent by the first conference terminal where the simultaneous simulcast stream includes an audio stream that adopts a first encoding manner and an audio stream that adopts a second encoding manner, where the first encoding manner is used.
  • the stream has the same media content and the same packing duration as the audio stream using the second encoding method;
  • the audio stream in the second coding mode is used to find a message having the same media content as the lost message
  • the audio stream of the first coding mode after the anti-drop packet processing is sent to the second conference terminal through the Ethernet port.
  • the processor is further configured to perform the following operations according to the program instructions stored in the memory:
  • the simultaneous simulcast stream includes an audio stream in a first encoding manner and an audio stream in a second encoding manner, wherein the audio stream in the first encoding mode and the audio in the second encoding manner are used. Streams have the same media content and the same packaging duration;
  • the simulcast capability confirmation information is sent to the first conference terminal by using the Ethernet interface, where the simulcast capability confirmation information is used to indicate that the conference server has the capability of receiving the simultaneous simulcast stream.
  • the processor is further configured to perform the following operations according to the program instructions stored in the memory:
  • the terminal has the capability of receiving an audio stream using the first encoding mode and an audio stream adopting the second encoding mode, and the second conference terminal has the function of receiving another encoding method in the audio stream adopting one encoding mode.
  • the receiving capability confirmation information is used to indicate that the conference server has another coding mode inserted in an audio stream adopting one coding mode. The ability of the message to stream audio.
  • the packet of the audio stream that uses the second encoding manner is recorded in the packet
  • the message has the serial number information of the message of the audio stream of the first encoding mode with the same media content
  • the processor performs the operation of searching for a packet having the same media content as the lost packet in the audio stream using the second encoding mode when detecting that the audio stream in the first encoding mode is lost, including :
  • the sequence number of the lost message is determined
  • the message in the second encoding mode is used to search for a message in which the serial number information of the lost message is recorded in the message.
  • the packet of the audio stream that uses the second encoding manner includes the first extended parameter and the second extended parameter
  • the first extended parameter is used to record that the sequence number of the message in the audio channel of the first coding mode having the same media content as the message is larger or smaller than the sequence number of the message
  • the second extended parameter is used. And recording a difference between a sequence number of the message of the audio stream of the first coding mode and the sequence number of the message having the same media content as the message.
  • an anti-drop packet processing method is provided.
  • the method is applied to an audio conference using a simultaneous simulcasting technology. Since the simultaneous simulcast stream includes at least two audio streams having the same media content, the conference server can be used by the conference server.
  • packet loss detection is performed on one of the audio streams, and when the packet loss is detected in the audio stream, the audio stream having the same media content on the other channel and the audio stream is detected. Searching for the message with the same media content as the lost message, and then embedding the found message with the same media content as the lost message into the audio stream of the channel, and obtaining the audio of the anti-drop packet processing.
  • FIG. 1 is a schematic structural diagram of a system on which an anti-drop packet processing method according to Embodiment 1 of the present invention is based;
  • FIG. 2 is a schematic structural diagram of a conference server according to Embodiment 2 of the present invention.
  • Embodiment 3 is a flowchart of an anti-drop packet processing method according to Embodiment 3 of the present invention.
  • FIG. 4 is a schematic diagram of message replacement according to an embodiment of the present invention.
  • FIG. 5 is a signal flow diagram of an anti-drop packet processing method according to Embodiment 4 of the present invention.
  • FIG. 6 is a schematic diagram of searching for associated messages according to an embodiment of the present invention.
  • FIG. 7 is a structural diagram of an anti-drop packet processing apparatus according to Embodiment 5 of the present invention.
  • FIG. 1 is a schematic structural diagram of a system based on a method for processing a packet loss prevention method according to Embodiment 1 of the present invention.
  • the system is used for performing an audio conference, and uses a simultaneous simulcast technology in an audio conference, where the system includes multiple conference terminals, where At a certain moment, one conference terminal serves as the sender conference terminal 101, and other conference terminals serve as the receiver conference terminal 102.
  • the system further includes a conference server 103, and the sender conference terminal 101 can generate a plurality of audio streams. Each of the two is at least two, and the audio communication between the plurality of conference terminals needs to pass through the conference server 103, and the conference server 103 performs functions such as anti-lost processing and forwarding.
  • the simultaneous simulcasting technology mentioned in the embodiment of the present invention that is, the sender conference terminal 101 encodes an original audio sequence into a plurality of audio streams by using different codecs, and then simultaneously sends the multiple audio streams to the conference server 103.
  • An audio stream is selected from the plurality of audio streams and sent to the receiving conference terminal 102 by the conference server 103 according to the codec mode supported by the receiver conference terminal 102.
  • the plurality of audio streams are referred to as simultaneous simulcast streams. .
  • the conference server 103 fully considers the characteristics of the simultaneous simulcast stream when the anti-drop packet processing is performed on the received audio stream. Since the plurality of audio streams included in the simulcast stream have the same media content, the codec mode is different. Therefore, when one audio stream included in the simultaneous simulating stream has packet loss, the other audio stream included in the simultaneous simulating stream searches for a data packet having the same media content as the lost data packet, and is found in another audio stream. The data packet is embedded in the audio stream instead of the lost data packet, and the anti-drop packet processing of the audio stream is completed in this way, that is, the anti-drop packet processing method in the embodiment of the present invention adopts a completely different manner from the traditional manner. Therefore, the media can resist packet loss without redundant packets, thereby effectively reducing bandwidth overhead and improving processing efficiency, and the method is simple.
  • FIG. 2 is a schematic structural diagram of a conference server according to Embodiment 2 of the present invention, where the conference server includes:
  • the memory 201 is configured to store program instructions
  • the processor 202 is configured to perform the following operations according to the program instructions stored in the memory 201:
  • the simultaneous simulcast stream sent by the first conference terminal receives, by the Ethernet interface 203, the simultaneous simulcast stream sent by the first conference terminal, where the simultaneous simulcast stream includes an audio stream that adopts a first coding mode and an audio stream that adopts a second coding mode, where the first coding mode is adopted.
  • the audio stream has the same media content and the same packing duration as the audio stream using the second encoding method;
  • the audio stream in the second coding mode is used to find a message having the same media content as the lost message
  • the audio stream of the first coding mode after the anti-drop packet processing is sent to the second conference terminal through the Ethernet port.
  • the processor 202 is further configured to perform the following operations according to the program instructions stored in the memory 201:
  • the simultaneous simulcast capability indication information sent by the first conference terminal, where the simultaneous simulcast capability indication information is used to indicate the first conference, before receiving the simultaneous simulcast stream sent by the first conference terminal
  • the terminal has the capability of transmitting the simultaneous simulcast stream, and the simultaneous simulcast stream includes an audio stream adopting a first encoding manner and an audio stream adopting a second encoding manner, wherein the audio stream adopting the first encoding manner and the second encoding manner are adopted.
  • the audio stream has the same media content and the same packaging duration;
  • the simulcast capability confirmation information is sent to the first conference terminal by using the Ethernet interface 203, where the simulcast capability confirmation information is used to indicate that the conference server has the capability of receiving the simultaneous simulcast stream.
  • the processor 202 is further configured to perform the following operations according to the program instructions stored in the memory 201:
  • the conference terminal has the capability of receiving an audio stream adopting a first coding mode and an audio stream adopting a second coding mode, and the second conference terminal has receiving another coding mode inserted in an audio stream adopting one coding mode The ability of the audio stream to message;
  • the receiving capability confirmation information is sent to the second conference terminal by using the Ethernet interface 203, where the receiving capability confirmation information is used to indicate that the conference server has another coding mode inserted in an audio stream adopting one coding mode.
  • the ability of the audio stream to message is sent to the second conference terminal by using the Ethernet interface 203, where the receiving capability confirmation information is used to indicate that the conference server has another coding mode inserted in an audio stream adopting one coding mode. The ability of the audio stream to message.
  • sequence number information of the message of the audio stream of the first coding mode having the same media content as the message is recorded in the message of the audio stream of the second coding mode;
  • the processor 202 performs an operation of searching for a packet having the same media content as the lost packet in the audio stream using the second encoding mode when detecting that the audio stream in the first encoding mode is lost.
  • the sequence number of the lost message is determined
  • the message in the second encoding mode is used to search for a message in which the serial number information of the lost message is recorded in the message.
  • the message of the audio stream in the second coding mode includes a first extended parameter and a second extended parameter, where the first extended parameter is used to record the first coding mode that has the same media content as the message.
  • the sequence number of the message of the audio stream is larger or smaller than the sequence number of the message, and the second extended parameter is used to record the message of the audio stream using the first coding mode having the same media content as the message. The difference between the serial number and the serial number of the message.
  • FIG. 3 is a flowchart of an anti-drop packet processing method according to Embodiment 3 of the present invention.
  • the method is applied to an audio conference using simultaneous simulcast technology, and may be performed by a conference server according to the system architecture shown in FIG. 1 .
  • a conference server according to the system architecture shown in FIG. 1 .
  • Step 301 Receive a simultaneous simulcast stream sent by the first conference terminal, where the simulcast simulcast stream includes an audio stream that adopts a first coding mode and an audio stream that uses a second coding mode, where the audio stream and the first coding mode are adopted.
  • the audio stream of the second encoding mode has the same media content and the same packing duration.
  • the conference server before receiving the simultaneous broadcast stream sent by the first conference terminal, the conference server may further include a signaling interaction process, and complete the signaling interaction between the conference server and the first conference terminal to complete the two.
  • the process of the capability negotiation may specifically include:
  • the conference server receives the simulcast capability indication information sent by the first conference terminal, where the simulcast capability indication information is used to indicate that the first conference terminal has the capability of sending the simultaneous simulcast stream, and the indication information indicates that the simultaneous simulcast stream includes the first An audio stream in an encoding mode and an audio stream in a second encoding mode, and indicating that the audio stream in the first encoding mode has the same media content and the same packaging duration as the audio stream in the second encoding mode;
  • the association between the audio streams may be implemented by using the indication information.
  • the indication information may carry information that the messages are mapped to each other between different audio streams, so as to implement association of packets between different audio streams.
  • the conference server After receiving the simultaneous simulating capability indication information sent by the first conference terminal, the conference server sends the simulcast capability confirmation information to the first conference terminal, where the simulcast capability confirmation information is used to indicate that the conference server has the capability of receiving the simultaneous simulcast stream.
  • the conference server can pass the first and second parties to receive the audio stream of the encoding mode supported by the second conference terminal, and the conference server can pass the The signaling interaction between the conference terminal and the second conference terminal is completed.
  • the process of the signaling interaction may include:
  • Receiving the receiving capability indication information sent by the second conference terminal where the receiving capability indication information is used to indicate that the second conference terminal has the capability of receiving the audio stream adopting the first encoding mode and the audio stream adopting the second encoding mode, and The second conference terminal has the capability of receiving a message of an audio stream of another coding mode inserted in an audio stream of one coding mode;
  • Step 302 Perform packet loss detection on the audio stream adopting the first coding mode.
  • the packet number of the packet can be used to determine whether packet loss occurs.
  • Step 303 When it is detected that the audio stream in the first coding mode is lost, the audio stream in the second coding mode is used to search for a packet having the same media content as the lost message.
  • the sequence number information of the message of the audio stream of the first coding mode having the same media content as the message may be recorded in the message of the audio stream of the second coding mode;
  • the sequence number of the lost message is first determined, and then the sequence number of the lost message is recorded in the search message in the audio stream in the second coding mode. Message message.
  • the embodiment of the present invention provides a new packet format.
  • the packet of the audio stream in the second encoding mode includes a first extended parameter and a second extended parameter, where the first extended parameter is used. Recording, if the message has the same media content, the sequence number of the message in the first encoding mode is larger or smaller than the sequence number of the message, and the second extended parameter is used to record the same as the message. The difference between the serial number of the message of the audio stream in the first encoding mode and the serial number of the packet in the media content.
  • Step 304 Embed the found message with the same media content as the lost message in the audio stream of the first coding mode, and obtain the audio stream of the first coding mode after the packet loss prevention process. .
  • Step 305 Send the audio stream of the first coding mode after the packet loss prevention process to the second conference terminal.
  • the system for performing an audio conference is composed of a conference server and four conference terminals.
  • the first conference terminal (UserA) serves as a broadcast source through the conference server (MRP) to other conference terminals.
  • the audio stream is sent, and the simultaneous simulcast technology is adopted in the audio conference.
  • the first conference terminal (UserA) sends a simultaneous simulcast stream including three audio streams to the conference server (MRP), and the three audio streams respectively use PT1 and PT2.
  • the conference server detects that there is a packet loss in the audio stream PT1, and searches for the packet of the audio stream PT2 having the same media content as the lost packet, and embeds the packet into the audio stream corresponding to the packet loss, and sends the packet to the audio stream corresponding to the packet loss.
  • Two conference terminals (UserC).
  • an anti-drop packet processing method is provided.
  • the characteristics of the simultaneous simulcast stream are fully utilized to perform anti-lost processing, so that no redundant message can be used.
  • the media is anti-lost, so it can effectively reduce bandwidth overhead and improve processing efficiency.
  • the method is simple.
  • FIG. 5 is a signal flow diagram of an anti-drop packet processing method according to Embodiment 4 of the present invention. The method is applied to an audio conference using the simultaneous simulcast technology, and may be based on the system architecture shown in FIG.
  • Step 501 The first conference terminal sends an offer message to the conference server, where the message is used to indicate the simultaneous simulcast capability of the first conference terminal.
  • two new attributes may be added in the usual offer message.
  • a homologous mapping stream can associate two streams, or can associate three or more streams.
  • RTP Real-time Transport Protocol
  • Step 502 The conference server sends an answer message to the first conference terminal, where the message is used to indicate the simultaneous simulcast capability supported by the conference server.
  • Step 503 The second conference terminal sends an offer message to the conference server, where the message is used to indicate the receiving capability of the second conference terminal, and indicates that it supports the insertion of the packet of the same-origin mapping stream in a media stream.
  • the receiving capability of the second conference terminal includes a codec mode supported by the second conference terminal.
  • Step 504 The conference server sends a reply message to the second conference terminal, where the message is used to indicate that the conference server also supports the insertion of the packet of the same-origin map stream in a media stream.
  • Step 505 The first conference terminal sends a simultaneous simulcast stream to the conference server according to the simultaneous simulcast capability supported by the conference server indicated in the answer message.
  • the simultaneous simulating stream includes a homologous mapping stream.
  • parameters are added in the RTP packet header extension parameter to implement association of the same media content.
  • the new parameters may be as shown in Table 1.
  • Step 506 detecting that a certain packet is lost, searching for the same media content according to the RTP extension, instead of the present The media content of the body.
  • the conference server does not directly discard the received packets of the same-origin mapping stream, but temporarily stores them in a buffer.
  • Step 507 Send the audio stream after the message replacement to the second conference terminal.
  • sequence number of the message in the simultaneous broadcast stream is continuous, the sequence number does not appear to be intermittent, so it is possible to detect whether there is a packet loss according to the sequence number.
  • FIG. 6 is a schematic diagram of the association message search in the embodiment of the present invention.
  • the simultaneous broadcast stream includes two audio streams, and the two audio streams are mutually identical mapping streams, wherein one audio stream has a message. 102 and the message 103, the other audio stream has a message 104 and a message 105, and the message 102 and the message 104 are associated messages, and have the same media content, and the message 103 and the message 105 are associated messages.
  • the new RTP parameter values corresponding to each packet are shown in Table 2.
  • a pointer-like function can be implemented, pointing to another message with the same media content.
  • a packet is lost, such as packet 103, it is necessary to find out whether the received packet has a packet 103, and if the subsequently received packet 105 points to the packet 103, the packet can be obtained and lost.
  • the message has the same media content.
  • the conference server needs a small buffer. Because the message with the same media content in the same conference and the simultaneous broadcast stream has a small interval, a small buffer can be set.
  • FIG. 7 is a structural diagram of an anti-drop packet processing apparatus according to Embodiment 5 of the present invention, where the apparatus is used to perform an anti-drop packet processing method provided by an embodiment of the present invention, and is specifically applied to an audio conference using a simultaneous simulcast technology, the apparatus Set in the conference server, the device includes:
  • the receiving unit 701 is configured to receive a simultaneous simulcast stream sent by the first conference terminal, where the simulcast simulcast stream includes an audio stream in a first encoding manner and an audio stream in a second encoding manner, where the first encoding manner is used.
  • the stream has the same media content and the same packing duration as the audio stream using the second encoding method;
  • the detecting unit 702 is configured to perform packet loss detection on the audio stream that is received by the receiving unit 701 and adopts the first encoding mode.
  • the searching unit 703 is configured to: when the detecting unit 702 detects that the audio stream in the first encoding mode is lost, the search and loss message in the audio stream in the second encoding manner received by the receiving unit 701 has In the same media Tolerant message;
  • the embedding unit 704 is configured to embed the message that is found by the searching unit 703 and having the same media content as the lost message into the audio stream of the first coding mode, to obtain an anti-loss packet processing An audio stream of the first encoding mode;
  • the sending unit 705 is configured to send, by the embedding unit 704, the audio stream of the first coding mode that is resistant to packet loss processing to the second conference terminal.
  • the receiving unit 701 is further configured to: receive the simultaneous simulcast capability indication information sent by the first conference terminal, and the simultaneous simulcast capability indication, before the receiving unit 701 receives the simultaneous simulcast stream sent by the first conference terminal.
  • the information is used to indicate that the first conference terminal has the capability of transmitting a simultaneous simulcast stream, where the simultaneous simulcast stream includes an audio stream that adopts a first coding mode and an audio stream that adopts a second coding mode, where the first coding mode is adopted.
  • the audio stream has the same media content and the same packing duration as the audio stream using the second encoding method;
  • the sending unit 705 is further configured to send the simulcast capability confirmation information to the first conference terminal, where the simulcast capability confirmation information is used to indicate that the conference server has the capability of receiving the simultaneous simulcast stream.
  • the receiving unit 701 is further configured to: before the detecting unit 702 performs packet loss detection on the audio stream that adopts the first coding mode, receive the receiving capability indication information sent by the second conference terminal, where the receiving capability is The indication information is used to indicate that the second conference terminal has the capability of receiving an audio stream adopting a first coding mode and an audio stream adopting a second coding mode, and the second conference terminal has a receiving manner in a coding mode. The ability to insert a message of another encoded audio stream into the audio stream;
  • the sending unit 705 is further configured to send, to the second conference terminal, receive capability confirmation information, where the receive capability confirmation information is used to indicate that the conference server has another type inserted in an audio stream that adopts one coding mode.
  • sequence information of the message of the audio stream in the first coding mode that has the same media content as the message is recorded in the message of the audio stream in the second coding mode received by the receiving unit 701;
  • the searching unit 703 includes:
  • Determining a subunit configured to determine a sequence number of the lost message when the detecting unit 702 detects that the audio stream in the first encoding mode is lost;
  • a locating sub-unit configured to search, in the audio stream that uses the second coding mode, a message in which the sequence number information of the lost message determined by the determining sub-unit is recorded in the message.
  • the detecting unit 702 may first detect that the audio stream in the first encoding mode is lost, and then determine the sequence number of the lost packet by the determining subunit included in the searching unit 703; or, may also detect The unit 702 detects that the audio stream of the first coding mode is lost, and determines the sequence number of the lost message.
  • the packet that is received by the receiving unit 701 and that uses the second encoding mode of the audio stream includes a first extended parameter and a second extended parameter, where the first extended parameter is used to record the same media as the packet.
  • the sequence number of the message of the audio stream adopting the first coding mode is larger or smaller than the sequence number of the message, and the second extended parameter is used for recording
  • the message has the same media content, and the difference between the sequence number of the message of the audio stream of the first coding mode and the sequence number of the message.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Telephonic Communication Services (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本发明实施例涉及抗丢包处理方法和装置，该方法包括：接收第一会议终端发送的同时联播流，同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；对采用第一编码方式的音频流进行丢包检测；当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文；将查找到的报文嵌入到第一编码方式的音频流中发送给第二会议终端。由上可见，本发明实施例中，充分利用了同时联播流的特点来进行抗丢包处理，可以有效减少带宽开销、提高处理效率，方法简单。

Description

抗丢包处理方法和装置
本申请要求于2015年7月16日提交中国专利局、申请号为201510418445.6、发明名称为“抗丢包处理方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及音频会议领域，尤其涉及抗丢包处理方法和装置。
背景技术
在传统的音频会议中，会议服务器会收到来自多个会议终端的音频流，当各会议终端的音频编解码能力不齐时，会议服务器需把收到的音频流通过转码功能，先转成接收方会议终端支持的编解码的音频流，然后再发送给接收方会议终端，接收方会议终端收到被转码过的音频流，才能正确地解码和还原声音。
由于会议服务器可能不具有转码的功能，为了在会议服务器不转码的情况下实现正常的音频通信，当前的音频会议中多采用同时联播(Simulcast)技术，即发送方会议终端对一个原始音频序列采用不同的编解码方式编码成多个音频流，然后将这多个音频流同时发送给会议服务器，由会议服务器根据接收方会议终端支持的编解码方式从这多个音频流中选择出一个音频流发送给接收方会议终端，为描述方便，将上述多个音频流称为同时联播流。
在发送方会议终端将同时联播流发送给会议服务器的过程中，可能会出现报文丢失的情况，其中，报文也可称为数据包，报文丢失可简称为丢包，为了避免丢包引起的用户体验下降的问题，常见的做法是通过冗余报文来实现抗丢包功能，发送方会议终端将要发送的数据附加上一定的冗余纠错码一并发送，会议服务器则根据纠错码对数据进行差错检测以及丢包检测，如发现差错或者丢包，由会议服务器进行纠正或者恢复。
由上可见，现有技术的抗丢包处理方法中，对于采用同时联播技术的音频会议，通过冗余报文来实现抗丢包功能，增加了带宽开销，并且，额外增加了对冗余报文的编解码处理，处理效率低下，复杂度高。
发明内容
本发明实施例提供抗丢包处理方法和装置，可以有效减少带宽开销、提高处理效率，方法简单。
第一方面，提供了一种抗丢包处理方法，所述方法应用于采用同时联播技术的音频会议中，所述方法由会议服务器执行，所述方法包括：
接收第一会议终端发送的同时联播流，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
对采用第一编码方式的音频流进行丢包检测；
当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文；
将查找到的与所述丢失报文具有相同的媒体内容的报文嵌入到所述第一编码方式的音频流中，得到抗丢包处理后的所述第一编码方式的音频流；
将抗丢包处理后的所述第一编码方式的音频流发送给第二会议终端。
其中，由于采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长，因此可以将采用第二编码方式的音频流称为采用第一编码方式的音频流的同源映射流，通常地，同时联播流中包含多个不同编码方式的音频流，上述多个为至少两个，本发明实施例中，采用第一编码方式的音频流可能具有多个同源映射流，此时，采用第二编码方式的音频流可以为这多个同源映射流中的任意一个同源映射流，或者，根据预设规则从这多个同源映射流中选取的一个同源映射流。
本发明实施例中，可以在互为同源映射流的多个音频流之间建立关联，用于指示上述多个音频流互为同源映射流。
结合第一方面，在第一方面的第一种可能的实现方式中，所述接收第一会议终端发送的同时联播流之前，所述方法还包括：
接收第一会议终端发送的同时联播能力指示信息，所述同时联播能力指示信息用于指示所述第一会议终端具有发送同时联播流的能力，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
向所述第一会议终端发送同时联播能力确认信息，所述同时联播能力确认信息用于指示所述会议服务器具有接收所述同时联播流的能力。
结合第一方面的第一种可能的实现方式，在第一方面的第二种可能的实现方式中，所述对采用第一编码方式的音频流进行丢包检测之前，所述方法还包括：
接收第二会议终端发送的接收能力指示信息，所述接收能力指示信息用于指示所述第二会议终端具有接收采用第一编码方式的音频流和采用第二编码方式的音频流的能力，以及，所述第二会议终端具有接收在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力；
向所述第二会议终端发送接收能力确认信息，所述接收能力确认信息用于指示所述会议服务器具有在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力。
结合第一方面或第一方面的第一种或第二种可能的实现方式，在第一方面的第三种可能 的实现方式中，采用第二编码方式的音频流的报文中记录了与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号信息；
所述当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文，包括：
当检测出采用第一编码方式的音频流发生丢包时，确定丢失报文的序列号；
在采用第二编码方式的音频流中查找报文中记录了所述丢失报文的序列号信息的报文。
结合第一方面的第三种可能的实现方式，在第一方面的第四种可能的实现方式中，采用第二编码方式的音频流的报文中包括第一扩展参数和第二扩展参数，所述第一扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号比该报文的序列号大还是小，所述第二扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号与该报文的序列号的差值。
第二方面，提供了一种抗丢包处理装置，所述装置应用于采用同时联播技术的音频会议中，所述装置设置于会议服务器中，所述装置包括：
接收单元，用于接收第一会议终端发送的同时联播流，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
检测单元，用于对所述接收单元接收的采用第一编码方式的音频流进行丢包检测；
查找单元，用于当所述检测单元检测出采用第一编码方式的音频流发生丢包时，在所述接收单元接收的采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文；
嵌入单元，用于将所述查找单元查找到的与所述丢失报文具有相同的媒体内容的报文嵌入到所述第一编码方式的音频流中，得到抗丢包处理后的所述第一编码方式的音频流；
发送单元，用于将所述嵌入单元得到的抗丢包处理后的所述第一编码方式的音频流发送给第二会议终端。
结合第二方面，在第二方面的第一种可能的实现方式中：
所述接收单元，还用于在所述接收单元接收第一会议终端发送的同时联播流之前，接收第一会议终端发送的同时联播能力指示信息，所述同时联播能力指示信息用于指示所述第一会议终端具有发送同时联播流的能力，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
所述发送单元，还用于向所述第一会议终端发送同时联播能力确认信息，所述同时联播能力确认信息用于指示所述会议服务器具有接收所述同时联播流的能力。
结合第二方面的第一种可能的实现方式，在第二方面的第二种可能的实现方式中：
所述接收单元，还用于在所述检测单元对采用第一编码方式的音频流进行丢包检测之前，接收第二会议终端发送的接收能力指示信息，所述接收能力指示信息用于指示所述第二会议 终端具有接收采用第一编码方式的音频流和采用第二编码方式的音频流的能力，以及，所述第二会议终端具有接收在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力；
所述发送单元，还用于向所述第二会议终端发送接收能力确认信息，所述接收能力确认信息用于指示所述会议服务器具有在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力。
结合第二方面或第二方面的第一种或第二种可能的实现方式，在第二方面的第三种可能的实现方式中，所述接收单元接收的采用第二编码方式的音频流的报文中记录了与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号信息；
所述查找单元包括：
确定子单元，用于当所述检测单元检测出采用第一编码方式的音频流发生丢包时，确定丢失报文的序列号；
查找子单元，用于在采用第二编码方式的音频流中查找报文中记录了所述确定子单元确定出的丢失报文的序列号信息的报文。
结合第二方面的第三种可能的实现方式，在第二方面的第四种可能的实现方式中，所述接收单元接收的采用第二编码方式的音频流的报文中包括第一扩展参数和第二扩展参数，所述第一扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号比该报文的序列号大还是小，所述第二扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号与该报文的序列号的差值。
第三方面，提供了一种会议服务器，所述会议服务器包括：
存储器、处理器和以太网口；
所述存储器，用于存储程序指令；
所述处理器，用于根据所述存储器中存储的程序指令执行以下操作：
通过所述以太网口接收第一会议终端发送的同时联播流，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
对采用第一编码方式的音频流进行丢包检测；
当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文；
将查找到的与所述丢失报文具有相同的媒体内容的报文嵌入到所述第一编码方式的音频流中，得到抗丢包处理后的所述第一编码方式的音频流；
将抗丢包处理后的所述第一编码方式的音频流通过所述以太网口发送给第二会议终端。
结合第三方面，在第三方面的第一种可能的实现方式中，所述处理器还用于根据所述存储器中存储的程序指令执行以下操作：
在所述接收第一会议终端发送的同时联播流之前，通过所述以太网口接收第一会议终端发送的同时联播能力指示信息，所述同时联播能力指示信息用于指示所述第一会议终端具有发送同时联播流的能力，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
通过所述以太网口向所述第一会议终端发送同时联播能力确认信息，所述同时联播能力确认信息用于指示所述会议服务器具有接收所述同时联播流的能力。
结合第三方面的第一种可能的实现方式，在第三方面的第二种可能的实现方式中，所述处理器还用于根据所述存储器中存储的程序指令执行以下操作：
在所述对采用第一编码方式的音频流进行丢包检测之前，通过所述以太网口接收第二会议终端发送的接收能力指示信息，所述接收能力指示信息用于指示所述第二会议终端具有接收采用第一编码方式的音频流和采用第二编码方式的音频流的能力，以及，所述第二会议终端具有接收在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力；
通过所述以太网口向所述第二会议终端发送接收能力确认信息，所述接收能力确认信息用于指示所述会议服务器具有在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力。
结合第三方面或第三方面的第一种或第二种可能的实现方式，在第三方面的第三种可能的实现方式中，采用第二编码方式的音频流的报文中记录了与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号信息；
所述处理器执行所述当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文的操作，包括：
当检测出采用第一编码方式的音频流发生丢包时，确定丢失报文的序列号；
在采用第二编码方式的音频流中查找报文中记录了所述丢失报文的序列号信息的报文。
结合第三方面的第三种可能的实现方式，在第三方面的第四种可能的实现方式中，采用第二编码方式的音频流的报文中包括第一扩展参数和第二扩展参数，所述第一扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号比该报文的序列号大还是小，所述第二扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号与该报文的序列号的差值。
本发明实施例中提供了一种抗丢包处理方法，该方法应用于采用同时联播技术的音频会议中，由于同时联播流中包括至少两路具有相同媒体内容的音频流，因此可以由会议服务器在接收到一个会议终端发送的同时联播流后，对其中一路音频流进行丢包检测，当检测出该路音频流发生丢包时，在另一路与该路音频流具有相同媒体内容的音频流中，查找与丢失报文具有相同的媒体内容的报文，再将查找到的与丢失报文具有相同的媒体内容的报文嵌入到该路音频流中，得到抗丢包处理后的一路音频流，最后将抗丢包处理后的一路音频流发送给 另一个会议终端。由上可见，本发明实施例中，在采用同时联播技术的音频会议中，充分利用了同时联播流的特点来进行抗丢包处理，从而可以在没有冗余报文的前提下，实现媒体抗丢包，因此可以有效减少带宽开销、提高处理效率，方法简单。
附图说明
图1为本发明实施例一提供的抗丢包处理方法所基于的系统架构示意图；
图2为本发明实施例二提供的会议服务器结构示意图；
图3为本发明实施例三提供的抗丢包处理方法流程图；
图4为本发明实施例提供的报文替换示意图；
图5为本发明实施例四提供的抗丢包处理方法信号流图；
图6为本发明实施例中的关联报文查找示意图；
图7为本发明实施例五提供的抗丢包处理装置结构图。
具体实施方式
下面通过附图和实施例，对本发明的技术方案做进一步的详细描述。
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚地描述。
图1为本发明实施例一提供的抗丢包处理方法所基于的系统架构示意图，该系统用于进行音频会议，在进行音频会议时采用同时联播技术，该系统中包括多个会议终端，其中，某一时刻，一个会议终端作为发送方会议终端101，其他的会议终端作为接收方会议终端102，该系统中还包括一个会议服务器103，发送方会议终端101能够产生多个音频流，上述多个均为至少两个，多个会议终端之间进行音频通信需要经过会议服务器103，由会议服务器103进行抗丢包处理和转发等功能。
本发明实施例中提到的同时联播技术，即发送方会议终端101对一个原始音频序列采用不同的编解码方式编码成多个音频流，然后将这多个音频流同时发送给会议服务器103，由会议服务器103根据接收方会议终端102支持的编解码方式从这多个音频流中选择出一个音频流发送给接收方会议终端102，为描述方便，将上述多个音频流称为同时联播流。
其中，会议服务器103在对接收到的音频流进行抗丢包处理时，充分考虑了同时联播流的特点，由于同时联播流包含的多个音频流具有相同的媒体内容，只是编解码方式不同，因此当同时联播流包含的一个音频流出现丢包时，在该同时联播流包含的另一个音频流中查找与丢失数据包具有相同媒体内容的数据包，将在另一个音频流中查找到的数据包代替丢失数据包嵌入到音频流中，以这种方式来完成音频流的抗丢包处理，也就是说，本发明实施例中的抗丢包处理方法采用了与传统方式完全不同的方式，从而可以在没有冗余报文的前提下，实现媒体抗丢包，因此可以有效减少带宽开销、提高处理效率，方法简单。
图2为本发明实施例二提供的会议服务器结构示意图，所述会议服务器包括：
存储器201、处理器202和以太网口203；
所述存储器201，用于存储程序指令；
所述处理器202，用于根据所述存储器201中存储的程序指令执行以下操作：
通过所述以太网口203接收第一会议终端发送的同时联播流，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
对采用第一编码方式的音频流进行丢包检测；
当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文；
将查找到的与所述丢失报文具有相同的媒体内容的报文嵌入到所述第一编码方式的音频流中，得到抗丢包处理后的所述第一编码方式的音频流；
将抗丢包处理后的所述第一编码方式的音频流通过所述以太网口发送给第二会议终端。
可选地，所述处理器202还用于根据所述存储器201中存储的程序指令执行以下操作：
在所述接收第一会议终端发送的同时联播流之前，通过所述以太网口203接收第一会议终端发送的同时联播能力指示信息，所述同时联播能力指示信息用于指示所述第一会议终端具有发送同时联播流的能力，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
通过所述以太网口203向所述第一会议终端发送同时联播能力确认信息，所述同时联播能力确认信息用于指示所述会议服务器具有接收所述同时联播流的能力。
可选地，所述处理器202还用于根据所述存储器201中存储的程序指令执行以下操作：
在所述对采用第一编码方式的音频流进行丢包检测之前，通过所述以太网口203接收第二会议终端发送的接收能力指示信息，所述接收能力指示信息用于指示所述第二会议终端具有接收采用第一编码方式的音频流和采用第二编码方式的音频流的能力，以及，所述第二会议终端具有接收在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力；
通过所述以太网口203向所述第二会议终端发送接收能力确认信息，所述接收能力确认信息用于指示所述会议服务器具有在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力。
可选地，采用第二编码方式的音频流的报文中记录了与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号信息；
所述处理器202执行所述当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文的操作，包括：
当检测出采用第一编码方式的音频流发生丢包时，确定丢失报文的序列号；
在采用第二编码方式的音频流中查找报文中记录了所述丢失报文的序列号信息的报文。
可选地，采用第二编码方式的音频流的报文中包括第一扩展参数和第二扩展参数，所述第一扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号比该报文的序列号大还是小，所述第二扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号与该报文的序列号的差值。
图3为本发明实施例三提供的抗丢包处理方法流程图，该方法应用于采用同时联播技术的音频会议中，可基于图1所示的系统架构，该方法由会议服务器执行，该方法包括：
步骤301，接收第一会议终端发送的同时联播流，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长。
本发明实施例中，会议服务器在接收第一会议终端发送的同时联播流之前，还可以包括一个信令交互的过程，通过会议服务器与第一会议终端之间的信令交互，完成二者之间的能力协商，该信令交互的过程具体可以包括：
会议服务器接收第一会议终端发送的同时联播能力指示信息，该同时联播能力指示信息用于指示第一会议终端具有发送同时联播流的能力，并且，通过该指示信息指示该同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，以及，指示采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；也就是说，通过该指示信息可以实现音频流之间的关联，进一步地，可以在该指示信息中携带不同音频流之间报文互相映射的信息，从而实现不同音频流之间报文的关联。
会议服务器在接收到第一会议终端发送的同时联播能力指示信息后，向第一会议终端发送同时联播能力确认信息，该同时联播能力确认信息用于指示会议服务器具有接收上述同时联播流的能力。
由于会议服务器接收的第一会议终端发送的同时联播流包含多个不同编码方式的音频流，为了将第二会议终端支持的编码方式的音频流发送给第二会议终端，会议服务器可以通过与第二会议终端的信令交互，完成会议服务器与第二会议终端之间的能力协商，该信令交互的过程具体可以包括：
接收第二会议终端发送的接收能力指示信息，所述接收能力指示信息用于指示所述第二会议终端具有接收采用第一编码方式的音频流和采用第二编码方式的音频流的能力，以及，所述第二会议终端具有接收在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力；
向所述第二会议终端发送接收能力确认信息，所述接收能力确认信息用于指示所述会议服务器具有在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力。
步骤302，对采用第一编码方式的音频流进行丢包检测。
其中，可以根据报文的序列号来判断是否发生丢包。
步骤303，当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文。
本发明实施例中，可以在采用第二编码方式的音频流的报文中记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号信息；
当检测出采用第一编码方式的音频流发生丢包时，先确定丢失报文的序列号，然后在采用第二编码方式的音频流中查找报文中记录了所述丢失报文的序列号信息的报文。
其中，本发明实施例提出了一种新的报文格式，具体地，采用第二编码方式的音频流的报文中包括第一扩展参数和第二扩展参数，所述第一扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号比该报文的序列号大还是小，所述第二扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号与该报文的序列号的差值。
步骤304，将查找到的与所述丢失报文具有相同的媒体内容的报文嵌入到所述第一编码方式的音频流中，得到抗丢包处理后的所述第一编码方式的音频流。
步骤305，将抗丢包处理后的所述第一编码方式的音频流发送给第二会议终端。
为了更加清楚地描述本发明实施例的抗丢包处理方法，下面参照图4所示的报文替换示意图进行详细说明。在图4所示的应用场景中，进行音频会议的系统由一个会议服务器和四个会议终端组成，某一时刻由第一会议终端(UserA)作为广播源通过会议服务器(MRP)向其他会议终端发送音频流，在这个音频会议中采用了同时联播技术，第一会议终端(UserA)向会议服务器(MRP)发送包含三个音频流的同时联播流，这三个音频流分别用PT1、PT2、PT3来表示，由于同时联播流中包括的这三个音频流对应同一媒体源，因此这三个音频流具有相同的媒体内容，当两个音频流具有相同媒体内容，并且具有相同的打包时长时，可以在一个音频流中找到与另一个音频流中的报文具有相同媒体内容的报文，这两个音频流互相称为对方的同源映射流。如图4所示，会议服务器在检测到音频流PT1存在丢包，查找与丢失报文具有相同媒体内容的音频流PT2的报文，将其嵌入到丢包对应的音频流中，发送给第二会议终端(UserC)。
本发明实施例中提供了一种抗丢包处理方法，在采用同时联播技术的音频会议中，充分利用了同时联播流的特点来进行抗丢包处理，从而可以在没有冗余报文的前提下，实现媒体抗丢包，因此可以有效减少带宽开销、提高处理效率，方法简单。
图5为本发明实施例四提供的抗丢包处理方法信号流图，该方法应用于采用同时联播技术的音频会议中，可基于图1所示的系统架构，该方法包括：
步骤501，第一会议终端向会议服务器发送提议(offer)消息，该消息用于指示第一会议终端具备的同时联播能力。
本发明实施例中，可以在通常的offer消息中，再增加两个新的属性。
属性一(a＝simulcast send_relation**)：将可以实现互相映射的两个同时联播流 关联起来。这两个流是同时联播流的子集，并且具备同样的打包时长。当其中一个流中的某个报文丢失之后，可以在另外一个流中找到内容相同的报文，互相称为对方的同源映射流。同源映射流可以关联两个流，也可以关联三个或者更多的流。
属性二(a＝extmap:7urn:3gpp:audio_relation)：表示是否支持实时传输协议(Real-time Transport Protocol，RTP)扩展，以便实现同源映射流相同媒体内容的报文的关联。
步骤502，会议服务器向第一会议终端发送应答(answer)消息，该消息用于指示会议服务器支持的同时联播能力。
步骤503，第二会议终端向会议服务器发送offer消息，该消息用于指示第二会议终端的接收能力，同时，指示自己支持在一个媒体流中，插入其同源映射流的报文。
其中，第二会议终端的接收能力包括第二会议终端支持的编解码方式。
步骤504，会议服务器向第二会议终端发送answer消息，该消息用于指示会议服务器同样支持在一个媒体流中，插入其同源映射流的报文。
步骤505，第一会议终端根据answer消息中指示的会议服务器支持的同时联播能力，向会议服务器发送同时联播流。
其中，同时联播流包括同源映射流，对于同源映射流，在RTP报文头扩展参数中增加参数，实现相同媒体内容的报文的关联，新增参数可以如表一所示。
Figure PCTCN2016085559-appb-000001
表一
步骤506，检测到某个报文丢失，则根据RTP扩展，查找相同媒体内容的报文，代替本 身的媒体内容。
其中，会议服务器对于收到的同源映射流的报文，完成转发功能之后，并不直接丢弃，而是暂存到一个缓冲区中。
步骤507，将报文替换后的音频流发送给第二会议终端。
由于同时联播流中报文的序列号是连续的，序列号不会出现间断，因此可以根据序列号，检测是否存在丢包。
图6为本发明实施例中的关联报文查找示意图，参照图6，同时联播流中包括两个音频流，这两个音频流互为同源映射流，其中，一个音频流中具有报文102和报文103，另一个音频流中具有报文104和报文105，报文102与报文104为关联报文，具有相同的媒体内容，报文103与报文105为关联报文，具有相同的媒体内容，各个报文对应的RTP新增参数值如表二所示。
报文 Front_Flag Distance
102 0 2
103 0 2
104 1 2
105 1 2
表二
通过报文头中的扩展参数，可以实现类似于指针的功能，指向媒体内容相同的另外一个报文。当某个报文丢失之后，比如报文103，需要查找已经收到的报文中，是否有指向报文103的，发现后续收到的报文105指向报文103，则可以得到与丢失报文具有相同媒体内容的报文。
其中，会议服务器需要一个小的缓冲区，由于同一个会议中，同时联播流中，具有同样的媒体内容的报文，其报文间隔很小，所以设置很小的缓冲区即可。
图7为本发明实施例五提供的抗丢包处理装置结构图，该装置用于执行本发明实施例提供的抗丢包处理方法，其具体应用于采用同时联播技术的音频会议中，该装置设置于会议服务器中，该装置包括：
接收单元701，用于接收第一会议终端发送的同时联播流，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
检测单元702，用于对所述接收单元701接收的采用第一编码方式的音频流进行丢包检测；
查找单元703，用于当所述检测单元702检测出采用第一编码方式的音频流发生丢包时，在所述接收单元701接收的采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内 容的报文；
嵌入单元704，用于将所述查找单元703查找到的与所述丢失报文具有相同的媒体内容的报文嵌入到所述第一编码方式的音频流中，得到抗丢包处理后的所述第一编码方式的音频流；
发送单元705，用于将所述嵌入单元704得到的抗丢包处理后的所述第一编码方式的音频流发送给第二会议终端。
可选地，所述接收单元701，还用于在所述接收单元701接收第一会议终端发送的同时联播流之前，接收第一会议终端发送的同时联播能力指示信息，所述同时联播能力指示信息用于指示所述第一会议终端具有发送同时联播流的能力，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
所述发送单元705，还用于向所述第一会议终端发送同时联播能力确认信息，所述同时联播能力确认信息用于指示所述会议服务器具有接收所述同时联播流的能力。
可选地，所述接收单元701，还用于在所述检测单元702对采用第一编码方式的音频流进行丢包检测之前，接收第二会议终端发送的接收能力指示信息，所述接收能力指示信息用于指示所述第二会议终端具有接收采用第一编码方式的音频流和采用第二编码方式的音频流的能力，以及，所述第二会议终端具有接收在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力；
所述发送单元705，还用于向所述第二会议终端发送接收能力确认信息，所述接收能力确认信息用于指示所述会议服务器具有在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力。
可选地，所述接收单元701接收的采用第二编码方式的音频流的报文中记录了与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号信息；
所述查找单元703包括：
确定子单元，用于当所述检测单元702检测出采用第一编码方式的音频流发生丢包时，确定丢失报文的序列号；
查找子单元，用于在采用第二编码方式的音频流中查找报文中记录了所述确定子单元确定出的丢失报文的序列号信息的报文。
本发明实施例中，可以先由检测单元702检测出采用第一编码方式的音频流发生丢包，然后由查找单元703包括的确定子单元确定丢失报文的序列号；或者，还可以在检测单元702检测出采用第一编码方式的音频流发生丢包的同时，确定丢失报文的序列号。
可选地，所述接收单元701接收的采用第二编码方式的音频流的报文中包括第一扩展参数和第二扩展参数，所述第一扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号比该报文的序列号大还是小，所述第二扩展参数用于记录与该 报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号与该报文的序列号的差值。
专业人员应该还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分步骤是可以通过程序来指令处理器完成，所述的程序可以存储于计算机可读存储介质中，所述存储介质是非短暂性(英文：non-transitory)介质，例如随机存取存储器，只读存储器，快闪存储器，硬盘，固态硬盘，磁带(英文：magnetic tape)，软盘(英文：floppy disk)，光盘(英文：optical disc)及其任意组合。
以上所述，仅为本发明较佳的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应该以权利要求的保护范围为准。

Claims (10)

  1. 一种抗丢包处理方法，其特征在于，所述方法应用于采用同时联播技术的音频会议中，所述方法由会议服务器执行，所述方法包括：
    接收第一会议终端发送的同时联播流，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
    对采用第一编码方式的音频流进行丢包检测；
    当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文；
    将查找到的与所述丢失报文具有相同的媒体内容的报文嵌入到所述第一编码方式的音频流中，得到抗丢包处理后的所述第一编码方式的音频流；
    将抗丢包处理后的所述第一编码方式的音频流发送给第二会议终端。
  2. 如权利要求1所述的方法，其特征在于，所述接收第一会议终端发送的同时联播流之前，所述方法还包括：
    接收第一会议终端发送的同时联播能力指示信息，所述同时联播能力指示信息用于指示所述第一会议终端具有发送同时联播流的能力，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
    向所述第一会议终端发送同时联播能力确认信息，所述同时联播能力确认信息用于指示所述会议服务器具有接收所述同时联播流的能力。
  3. 如权利要求2所述的方法，其特征在于，所述对采用第一编码方式的音频流进行丢包检测之前，所述方法还包括：
    接收第二会议终端发送的接收能力指示信息，所述接收能力指示信息用于指示所述第二会议终端具有接收采用第一编码方式的音频流和采用第二编码方式的音频流的能力，以及，所述第二会议终端具有接收在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力；
    向所述第二会议终端发送接收能力确认信息，所述接收能力确认信息用于指示所述会议服务器具有在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力。
  4. 如权利要求1至3中任一权利要求所述的方法，其特征在于，采用第二编码方式的音频流的报文中记录了与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号信息；
    所述当检测出采用第一编码方式的音频流发生丢包时，在采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文，包括：
    当检测出采用第一编码方式的音频流发生丢包时，确定丢失报文的序列号；
    在采用第二编码方式的音频流中查找报文中记录了所述丢失报文的序列号信息的报文。
  5. 如权利要求4所述的方法，其特征在于，采用第二编码方式的音频流的报文中包括第一扩展参数和第二扩展参数，所述第一扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号比该报文的序列号大还是小，所述第二扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号与该报文的序列号的差值。
  6. 一种抗丢包处理装置，其特征在于，所述装置应用于采用同时联播技术的音频会议中，所述装置设置于会议服务器中，所述装置包括：
    接收单元，用于接收第一会议终端发送的同时联播流，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
    检测单元，用于对所述接收单元接收的采用第一编码方式的音频流进行丢包检测；
    查找单元，用于当所述检测单元检测出采用第一编码方式的音频流发生丢包时，在所述接收单元接收的采用第二编码方式的音频流中查找与丢失报文具有相同的媒体内容的报文；
    嵌入单元，用于将所述查找单元查找到的与所述丢失报文具有相同的媒体内容的报文嵌入到所述第一编码方式的音频流中，得到抗丢包处理后的所述第一编码方式的音频流；
    发送单元，用于将所述嵌入单元得到的抗丢包处理后的所述第一编码方式的音频流发送给第二会议终端。
  7. 如权利要求6所述的装置，其特征在于：
    所述接收单元，还用于在所述接收单元接收第一会议终端发送的同时联播流之前，接收第一会议终端发送的同时联播能力指示信息，所述同时联播能力指示信息用于指示所述第一会议终端具有发送同时联播流的能力，所述同时联播流包括采用第一编码方式的音频流和采用第二编码方式的音频流，其中，采用第一编码方式的音频流与采用第二编码方式的音频流具有相同的媒体内容和相同的打包时长；
    所述发送单元，还用于向所述第一会议终端发送同时联播能力确认信息，所述同时联播能力确认信息用于指示所述会议服务器具有接收所述同时联播流的能力。
  8. 如权利要求7所述的装置，其特征在于：
    所述接收单元，还用于在所述检测单元对采用第一编码方式的音频流进行丢包检测之前，接收第二会议终端发送的接收能力指示信息，所述接收能力指示信息用于指示所述第二会议终端具有接收采用第一编码方式的音频流和采用第二编码方式的音频流的能力，以及，所述第二会议终端具有接收在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力；
    所述发送单元，还用于向所述第二会议终端发送接收能力确认信息，所述接收能力确认 信息用于指示所述会议服务器具有在采用一种编码方式的音频流中插入另一种编码方式的音频流的报文的能力。
  9. 如权利要求6至8中任一权利要求所述的装置，其特征在于，所述接收单元接收的采用第二编码方式的音频流的报文中记录了与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号信息；
    所述查找单元包括：
    确定子单元，用于当所述检测单元检测出采用第一编码方式的音频流发生丢包时，确定丢失报文的序列号；
    查找子单元，用于在采用第二编码方式的音频流中查找报文中记录了所述确定子单元确定出的丢失报文的序列号信息的报文。
  10. 如权利要求9所述的装置，其特征在于，所述接收单元接收的采用第二编码方式的音频流的报文中包括第一扩展参数和第二扩展参数，所述第一扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号比该报文的序列号大还是小，所述第二扩展参数用于记录与该报文具有相同媒体内容的采用第一编码方式的音频流的报文的序列号与该报文的序列号的差值。
PCT/CN2016/085559 2015-07-16 2016-06-13 抗丢包处理方法和装置 WO2017008605A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510418445.6A CN106357415B (zh) 2015-07-16 2015-07-16 抗丢包处理方法和装置
CN201510418445.6 2015-07-16

Publications (1)

Publication Number Publication Date
WO2017008605A1 true WO2017008605A1 (zh) 2017-01-19

Family

ID=57756830

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/085559 WO2017008605A1 (zh) 2015-07-16 2016-06-13 抗丢包处理方法和装置

Country Status (2)

Country Link
CN (1) CN106357415B (zh)
WO (1) WO2017008605A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021163954A1 (zh) * 2020-02-20 2021-08-26 深圳市汇顶科技股份有限公司 数据传输方法、装置、设备、系统及介质
CN112769524B (zh) * 2021-04-06 2021-06-22 腾讯科技（深圳）有限公司 语音传输方法、装置、计算机设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050081241A1 (en) * 2003-10-10 2005-04-14 Michael Chen Method, apparatus, and system for preparing images for integration and combining images into an integrated image
CN101820524A (zh) * 2010-03-22 2010-09-01 中兴通讯股份有限公司 用于电视会议的视频播放方法
CN102111644A (zh) * 2009-12-24 2011-06-29 华为终端有限公司 媒体传输控制方法、装置及系统
US20120169883A1 (en) * 2010-12-31 2012-07-05 Avermedia Information, Inc. Multi-stream video system, video monitoring device and multi-stream video transmission method

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050100049A1 (en) * 2003-04-29 2005-05-12 Siminoff James W. Multiple packet routing system (MPRS)
US8879464B2 (en) * 2009-01-29 2014-11-04 Avaya Inc. System and method for providing a replacement packet
CN102055968B (zh) * 2009-10-28 2012-08-29 中国移动通信集团公司 多视点视频中丢失的视频数据的恢复方法、系统及设备
CN103840914B (zh) * 2014-02-17 2017-04-19 天地融科技股份有限公司 一种数据通信方法、终端及信息安全设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050081241A1 (en) * 2003-10-10 2005-04-14 Michael Chen Method, apparatus, and system for preparing images for integration and combining images into an integrated image
CN102111644A (zh) * 2009-12-24 2011-06-29 华为终端有限公司 媒体传输控制方法、装置及系统
CN101820524A (zh) * 2010-03-22 2010-09-01 中兴通讯股份有限公司 用于电视会议的视频播放方法
US20120169883A1 (en) * 2010-12-31 2012-07-05 Avermedia Information, Inc. Multi-stream video system, video monitoring device and multi-stream video transmission method

Also Published As

Publication number Publication date
CN106357415A (zh) 2017-01-25
CN106357415B (zh) 2019-08-23

Similar Documents

Publication Publication Date Title
WO2017161998A1 (zh) 视频处理方法和装置、计算机存储介质
US9264481B2 (en) Responding to hypertext transfer protocol (HTTP) requests
TWI325706B (en) Method and apparatus for processing multimedia data
US9280413B2 (en) Redundant encoding
US9282133B2 (en) Communicating control information within a real-time stream
EP2847953B1 (en) Apparatus and method of transmitting and receiving packet in a broadcasting and communication system
EP2166687A1 (en) A method and apparatus for transmiting and receiving data packets
JP5021765B2 (ja) 逆方向リンクおよび順方向リンクのビデオデータエラーを区別するエラーフィルタ
CN102648584B (zh) 使用前向纠错检验可用带宽的系统、方法和介质
US20160057390A1 (en) Obtaining replay of audio during a conference session
US11363085B2 (en) In-band quality data
WO2017008605A1 (zh) 抗丢包处理方法和装置
JP5344541B2 (ja) データ送信装置、送信方法及びプログラム
US10051266B2 (en) Apparatus and method for transmitting and receiving hybrid packets in a broadcasting and communication system using error correction source blocks and MPEG media transport assets
WO2015154520A1 (zh) 通话录音方法、录音服务器、用户级交换机及录音系统
US10063643B2 (en) Media stream grouping in multimedia communication networks
US9667384B2 (en) Apparatus and method for transmitting and receiving forward error correction packet
JP2018518869A (ja) マルチプルシーケンス化フローのためのバンドルされた前方誤り訂正（ｆｅｃ）
KR20060095755A (ko) 패킷교환 네트워크를 통한 실시간 전송을 위한 미디어 패킷구조
WO2021174879A1 (zh) Ai视频通话质量分析方法、装置、计算机设备及存储介质
KR20160004860A (ko) 멀티미디어 시스템에서 미디어 패킷을 수신하는 방법 및 장치
WO2015143935A1 (zh) 一种智能信息传输方法、系统和装置
JP2012151622A (ja) 受信端末、パケットデータ受信方法、送信端末、送受信システム、中継端末およびパケットデータの中継方法
JP4344951B2 (ja) メディア符号化データ送信方法および装置ならびにプログラム
TWI523461B (zh) 通訊系統和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16823752

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16823752

Country of ref document: EP

Kind code of ref document: A1