WO2015039579A1

WO2015039579A1 - Video calling method for asymmetric networking system, terminal, server and system

Info

Publication number: WO2015039579A1
Application number: PCT/CN2014/086281
Authority: WO
Inventors: 田振华; 吴毅敏
Original assignee: 华为技术有限公司
Priority date: 2013-09-18
Filing date: 2014-09-11
Publication date: 2015-03-26
Also published as: CN104469537B; CN104469537A

Abstract

Disclosed are a video calling method for an asymmetric networking system, a terminal, a server and a system. The asymmetric networking system comprises a first terminal which supports NAT (network address translation), an intermediate server and a second terminal which does not support NAT. The video calling method comprises: when the detection of the connectivity of the network address translation between a first terminal and an intermediate server is completed, the first terminal sending a key frame request to a second terminal; and the first terminal receiving a key frame which is returned by the second terminal in response to the key frame request, and parsing a subsequent difference frame, so as to normally display a video which is sent by the second terminal. By means of the above-mentioned method, the present invention can effectively reduce the length of time for which the video cannot be normally displayed, thereby creating a better service experience for a user.

Description

Asymmetric networking system video calling method and terminal, server and system thereof

The present invention relates to the field of communications, and in particular, to a video calling method for an asymmetric networking system, and a terminal, a server, and a system. Background technique

Video codec transmission, such as H.264 media transmission using Real-time Transport Protocol (RTP), and RTP is carried on the User Datagram Protocol (UDP), due to UDP It is a connectionless, unreliable transmission protocol, so video stream packets are transmitted over the network, which is prone to packet loss.

In video transmission, a picture can be divided into key frames and differential frames (P frames) for transmission. Key frames are the key to video transmission, and subsequent P frame decoding is done by key frames. If the key frame is lost, the P frame cannot be decoded, and the decoder cannot parse the received video picture normally.

The H.264 related standard defines a media key frame request mechanism, that is, when a decoder finds that a key frame is lost, it can actively send a key frame request to another end to obtain a key frame for normal decoding. However, in this way, it is relatively long to display the video of the other party normally. In addition, for a network system where one end is a terminal that supports Network Address Translation (NAT) and the other end is a terminal that does not support NAT, NAT connectivity detection may also cause loss of key frames, if the above method is still used. To handle, the terminal that supports NAT is receiving the video.

Summary of the invention

The technical problem to be solved by the present invention is how to shorten the time for the NAT terminal to display the video of the other party after receiving the video request response in the asymmetric networking system.

In view of this, the present application provides a video calling method, a terminal, a server, and a system for an asymmetric networking system, which can effectively reduce the time when the video cannot be displayed normally, and provide a better service experience for the user.

The first aspect provides an asymmetric network system video calling method, where the asymmetric group The network system includes a first terminal supporting network address translation, an intermediate server, and a second terminal that does not support network address translation, the method comprising: completing network address translation connectivity detection between the first terminal and the intermediate server The first terminal sends a key frame request to the second terminal; the first terminal receives a key frame returned by the second terminal in response to the key frame request, parses the subsequent difference frame, and displays the normal frame. The video sent by the second terminal.

With reference to the first aspect, in a first possible implementation manner of the first aspect, the step of: sending the key frame request to the second terminal includes: sending, by the first terminal, a key frame at a media layer or a signaling layer Requesting, and forwarding to the second terminal through the intermediate server.

With reference to the first aspect, in a second possible implementation manner of the first aspect, the method further includes: before the network address translation connectivity detection between the first terminal and the intermediate server is completed, the method further includes: The first terminal sends a video request to the second terminal by using the intermediate server, where the video request carries video codec information and a network address translation connectivity detection candidate address of the first terminal; Carrying a request response message carrying the network address translation connectivity detection candidate address of the intermediate server, and starting network address translation connectivity detection with the intermediate server.

A second aspect provides a video calling method for an asymmetric networking system, where the asymmetric networking system includes a first terminal that supports network address translation, an intermediate server, and a second terminal that does not support network address translation, and the method includes The intermediate server sends a key frame request to the second terminal when the network address translation connectivity check between the first terminal and the intermediate server is completed; the intermediate server receives the second terminal response Deriving the key frame returned by the key frame request, and forwarding the key frame to the first terminal, so that the first terminal can normally display the video sent by the second terminal.

With reference to the second aspect, in a first possible implementation manner of the second aspect, the step of the intermediate server sending a key frame request to the second terminal includes: the intermediate server facing the media layer or the signaling layer The second terminal sends a key frame request.

With reference to the second aspect, in a second possible implementation manner of the second aspect, before the network address translation connectivity detection between the first terminal and the intermediate server is completed, the method further includes: the intermediate server Receiving a video request of the first terminal, the video request Carrying video codec information and a network address translation connectivity detection candidate address of the first terminal; the intermediate server deleting the network address translation connectivity detection candidate address in the video request, and forwarding the video request to the The second terminal.

With the second possible implementation of the second aspect, in a third possible implementation manner of the second aspect, after the step of forwarding the video request to the second terminal, the method further includes: The intermediate server receives the request response message from the second terminal, adds its own network address translation connectivity detection candidate address to the request response message, and then forwards the request response message to the first terminal.

The third aspect provides a terminal that supports network address translation, where the terminal that supports network address translation and the terminal that does not support network address translation make a video call through an intermediate server, where the terminal that supports network address translation includes a request module and receives a module, where: the requesting module is configured to send a key frame request to the terminal that does not support network address translation when the network address translation connectivity detection between the terminal supporting the network address translation and the intermediate server is completed; The receiving module is configured to receive a key frame returned by the terminal that does not support network address translation in response to the key frame request, parse the subsequent difference frame, and normally display the video sent by the terminal that does not support network address translation.

With reference to the third aspect, in a first possible implementation manner of the third aspect, the requesting module is configured to send a key frame request at a media layer or a signaling layer, and forward the information to the unsupported network by using the intermediate server The terminal for address translation.

With reference to the third aspect, in a second possible implementation manner of the third aspect, the requesting module is configured to send a video request to the terminal that does not support network address translation by using the intermediate server, where the video request carries a video The codec information and the network address translation connectivity detection candidate address of the terminal supporting the network address translation; the receiving module is configured to receive a request response message carrying the network address translation connectivity detection candidate address of the intermediate server, start with The intermediate server performs network address translation connectivity detection.

According to a fourth aspect, a server is provided, and a terminal that supports network address translation and a terminal that does not support network address translation perform a video call by using the server, where the server includes a requesting module and a forwarding module, where: the requesting module is used to When the network address translation connectivity between the terminal supporting the network address translation and the server is completed, the support is not supported. The terminal of the network address translation sends a key frame request; the forwarding module is configured to receive a key frame returned by the terminal that does not support network address translation in response to the key frame request, and forward the key frame to the support network The address translation terminal is configured to enable the terminal supporting the network address translation to normally display the video sent by the terminal that does not support network address translation.

In conjunction with the fourth aspect, in a first possible implementation manner of the fourth aspect, the requesting module is configured to send a key frame request to a terminal that does not support network address translation at a media layer or a signaling layer.

With reference to the fourth aspect, in a second possible implementation manner of the fourth aspect, the forwarding module is configured to receive a video request of the terminal that supports network address translation, where the video request carries video codec information and the The network address translation connectivity detection candidate address of the terminal that supports the network address translation is deleted, and the video request is forwarded to the terminal that does not support the network address translation after deleting the network address translation connectivity detection candidate address in the video request.

With the second possible implementation of the fourth aspect, in a third possible implementation manner of the fourth aspect, the forwarding module is configured to receive a request response message from the terminal that does not support network address translation, and add The network address translation connectivity of the server detects the candidate address into the request response message, and then forwards the request response message to the terminal supporting the network address translation.

According to a fifth aspect, a network system is provided, where the network system includes the terminal supporting the network address translation in any of the foregoing third aspect and the third aspect, and the fourth aspect and the fourth aspect A server in a possible implementation also includes a terminal that does not support network address translation.

The beneficial effects of the present application are as follows: Different from the prior art, the video calling method of the asymmetric networking system of the present application actively supports NAT when the NAT connectivity detection between the terminal supporting the NAT and the intermediate server is completed. The terminal sends a key frame request, thereby receiving a key frame of the terminal that does not support NAT, and realizing normal video decoding. In this way, the time for the NAT terminal to display the video request response message cannot be displayed normally after receiving the video request response message, thereby giving the user a better service experience. DRAWINGS 1 is a flowchart of a first embodiment of a video calling method for an asymmetric networking system of the present application;

2 is a flow chart of a second embodiment of a video calling method for an asymmetric networking system of the present application;

3 is a flow chart of a third embodiment of a video calling method for an asymmetric networking system of the present application;

4 is a flow chart of a fourth embodiment of a video calling method for an asymmetric networking system of the present application;

5 is a flow chart of a fifth embodiment of a video calling method for an asymmetric networking system of the present application;

6 is a schematic structural diagram of an embodiment of a terminal supporting NAT in the present application; FIG. 7 is a schematic structural diagram of an embodiment of a server according to the present application;

FIG. 8 is a schematic structural diagram of another embodiment of a terminal supporting NAT according to the present application. detailed description

At present, the transmission of video codec, such as H.264 media transmission, uses the real-time transmission protocol RTP mode, and RTP is carried on the user data packet protocol UDP. Since UDP is a connectionless, unreliable transmission protocol, the video stream Packets are transmitted over the network, which is prone to packet loss, resulting in the loss of key frames and the normal display of video from the peer.

NAT traversal, commonly used STU (Simple Traversal of UDP Through Network Address Translators) and TURN (Traversal Using Relay NAT) two ways, the two distinct difference is that TURN is an intermediate server, all packets need to traverse NAT Both are sent to the TURN server for transit. STUN does not need to do the transfer of data packets.

In the NAT traversal technology, connectivity detection is required after the terminal obtains the NAT device address or the TURN device address. The so-called connectivity detection is to detect whether or not connectivity is established between multiple NAT address preparations. The purpose is to determine which path is reachable from the local end to the opposite end. When the detection is completed to determine a path, the packet can be sent. The NAT connectivity detection candidate address for general connectivity detection is in the call signaling protocol (Session Initiation Protocol (SIP) messages are carried in the Session Description Protocol (SDP). Therefore, the completion of SIP media information exchange is the first step in NAT connectivity detection (because the request/response is completed, each other can know each other's The NAT connectivity detection candidate address can be used for connectivity detection based on the NAT connectivity detection candidate address of the other party.

In an asymmetric networking system, where one end is a terminal that supports NAT and the other end is a video call that does not support NAT, connectivity detection occurs in a terminal that supports NAT and a node in the middle (the main node of the intermediate node) The function is to terminate all NAT processes, because the called terminal does not support NAT. Therefore, as a NAT terminal is not supported, it is not known whether NAT detection between the terminal supporting NAT and the intermediate server has been completed, and whether it can be received normally. Or send data. A terminal that does not support NAT may start transmitting data after completing the media information exchange process. Therefore, due to the influence of connectivity detection, a terminal that supports NAT may not display the video image sent by the peer end after receiving the video request response for a period of time.

In view of this, the present application provides a video calling method for an asymmetric networking system, a terminal, a server, and a system, which are intended to solve a video call of an asymmetric networking system, and a terminal supporting NAT cannot be used for a long time after receiving a video request response. The problem of displaying the video image sent from the opposite end is normally displayed.

The technical solutions claimed in the present application are further described below in conjunction with the specific embodiments, but are not intended to limit the scope of the claims.

Referring to FIG. 1 , FIG. 1 is a flowchart of a first embodiment of a video calling method for an asymmetric networking system of the present application. In this implementation manner, the asymmetric networking system includes a terminal that supports NAT, an intermediate server, and does not support NAT. The terminal is described in the perspective of a terminal that supports NAT in an asymmetric networking system. The video calling method of the asymmetric networking system in this embodiment includes:

Step S101: When the network address translation connectivity detection between the first terminal supporting the network address translation and the intermediate server is completed, the first terminal sends a key frame request to the second terminal that does not support network address translation.

The first terminal supporting NAT returns the second terminal that does not support NAT in the intermediate server. After requesting the response message, the NAT connectivity candidate address of the intermediate server is known and connectivity detection with the intermediate server is started. When the detection is completed, the first terminal actively sends a key frame request to the second terminal to prevent the key frame from being lost, so that the video image sent by the second terminal cannot be normally displayed for a long time.

In the actual application process, the first terminal may send a key frame request through the media layer or the signaling layer, and forward the request to the second terminal through the intermediate server.

Step S102: The first terminal receives the key frame returned by the second terminal in response to the key frame request, parses the subsequent difference frame, and normally displays the video sent by the second terminal.

After receiving the key frame request from the first terminal, the second terminal returns a request response message to the first terminal, where the request response message carries the key frame requested by the first terminal. The second terminal returns the request response message to the intermediate server, and the intermediate server forwards the message to the first terminal.

The first terminal receives the returned key frame, and can correctly parse the subsequent P frame to display that the video sent by the second terminal is normal.

Through the description of the foregoing embodiments, it can be understood that the video calling method of the asymmetric networking system of the present application actively sends a key frame request to a terminal that does not support NAT when the NAT connectivity detection between the terminal supporting the NAT and the intermediate server is completed. Therefore, the key frame of the terminal that does not support NAT is received to achieve normal video decoding. In this way, the terminal that supports the NAT cannot effectively display the time of the peer video after receiving the video request response message, thereby giving the user a better service experience.

Referring to FIG. 2, FIG. 2 is a flowchart of a second embodiment of a video calling method for an asymmetric networking system of the present application. In this implementation manner, the asymmetric networking system includes a terminal that supports NAT, an intermediate server, and an interface that does not support NAT. The terminal is described in the perspective of a terminal that supports NAT in an asymmetric networking system. The video calling method of the asymmetric networking system in this embodiment includes:

Step S201: The first terminal supporting network address translation sends a video request to the second terminal that does not support network address translation through the intermediate server.

The first terminal supporting the NAT initiates a video request, and carries the video codec information and the self-NAT connectivity detection candidate address in the SDP. After receiving the video request carrying the NAT connectivity detection candidate address, the intermediate server removes the NAT in the video request. Connectivity detection candidate The address is then sent to the second terminal that does not support NAT.

The second terminal that does not support the NAT sends a request response message that does not carry the NAT connectivity detection candidate address to the intermediate server. After receiving the request response message, the intermediate server adds its own NAT connectivity detection candidate address information to the request response message. The request response message is forwarded to the first terminal supporting the NAT.

Step S202: The first terminal receives a request response message carrying a network address translation connectivity detection candidate address of the intermediate server, and starts network address translation connectivity detection with the intermediate server.

The first terminal supporting the NAT receives the request response message forwarded by the intermediate server, completes the media information exchange, obtains the NAT connectivity detection candidate addresses of each other, and starts NAT connectivity detection with the intermediate server.

The second terminal that does not support NAT may start transmitting video data before the completion of the NAT connectivity detection of the first terminal and the intermediate server, that is, transmitting the first key frame. Since the NAT connectivity detection of the first terminal and the intermediate server has not been completed, the first key frame of the second terminal may be discarded.

Step S203: When the network address translation connectivity detection between the first terminal and the intermediate server is completed, the first terminal sends a key frame request to the second terminal.

After the discovery of the NAT detection, the first terminal supporting the NAT establishes a path with the peer to start transmitting data. At this time, the first terminal starts transmitting video to the second terminal. At the same time, the first terminal actively sends a key frame request message at the media level or the signaling level, for example, actively transmitting a key frame requesting Real-time Transport Control Protocol (RTCP) message at the media level, or signaling The layer actively sends an INFO message (the purpose of the INFO message is to carry the application layer message along the SIP signaling path) and forwards it to the second terminal that does not support NAT through the intermediate server.

Step S204: The first terminal receives the key frame returned by the second terminal in response to the key frame request, parses the subsequent difference frame, and normally displays the video sent by the second terminal.

After receiving the key frame request message, the second terminal that does not support the NAT actively sends a key frame to the intermediate server, and the intermediate server forwards the key frame sent by the second terminal to the first terminal. After the first terminal receives the key frame, Can correctly parse subsequent P frames, display peer video The picture is normal.

Referring to FIG. 3, FIG. 3 is a flowchart of a third embodiment of a video calling method for an asymmetric networking system of the present application. In this implementation manner, the asymmetric networking system includes a terminal that supports NAT, an intermediate server, and an interface that does not support NAT. The terminal is described in the perspective of an intermediate server of the asymmetric networking system. The video calling method of the asymmetric networking system in this embodiment includes:

Step S301: When the network address translation connectivity detection between the first terminal supporting the network address translation and the intermediate server is completed, the intermediate server sends a key frame request to the second terminal that does not support network address translation.

The first terminal supporting the NAT starts the connectivity check with the intermediate server after the intermediate server returns the request response message that does not support the NAT terminal, knows the NAT connectivity candidate address of the intermediate server. When the detection is completed, the intermediate server actively sends a key frame request to the second terminal to prevent the key frame from being lost, so that the first terminal cannot display the video image sent by the second terminal for a long time.

In the actual application process, the intermediate server can issue a key frame request to the second terminal through the media layer or the signaling layer.

Step S302: The intermediate server receives the key frame returned by the second terminal in response to the key frame request and forwards the key frame to the first terminal.

After receiving the key frame request, the second terminal returns a request response message to the intermediate server, where the request response message carries the key frame requested by the intermediate server. The intermediate server forwards the key frame to the first terminal.

Referring to FIG. 4, FIG. 4 is a flowchart of a fourth embodiment of a video calling method for an asymmetric networking system of the present application. In this implementation manner, the asymmetric networking system includes a terminal that supports NAT, an intermediate server, and an interface that does not support NAT. The terminal is described in the perspective of an intermediate server of the asymmetric networking system. The video calling method of the asymmetric networking system in this embodiment includes:

Step S401: The intermediate server receives the video of the first terminal that supports network address translation. begging;

The first terminal supporting the NAT initiates a video request, carries the video codec information, and the self-NAT connectivity detection candidate address is in the SDP, and the intermediate server receives the video request of the first terminal.

Step S402: The intermediate server deletes the network address translation connectivity detection candidate address in the video request, and then forwards the video request to the second terminal that does not support network address translation.

After receiving the video request from the first terminal that carries the NAT connectivity detection candidate address, the intermediate server removes the NAT connectivity detection candidate address in the video request, and then sends the video request to the second terminal that does not support NAT.

Step S403: The intermediate server receives the request response message from the second terminal, adds its own network address translation connectivity detection candidate address to the request response message, and then forwards the request response message to the first terminal.

After receiving the video request, the second terminal that does not support the NAT sends a request response message that does not carry the NAT connectivity detection candidate address to the intermediate server. After receiving the request response message, the intermediate server adds its own NAT connectivity detection candidate address information to After requesting the response message, the request response message is forwarded to the first terminal supporting the NAT.

Step S404: When the network address translation connectivity detection between the first terminal and the intermediate server is completed, the intermediate server sends a key frame request to the second terminal.

The NAT connection detection between the first terminal and the intermediate server supporting NAT is completed, and the first terminal establishes a path with the opposite end to start transmitting data. At this time, the first terminal starts to send video to the second terminal through the intermediate server, and the second terminal can normally display the video image sent by the first terminal. At the same time, the intermediate server actively sends a key frame at the media level or signaling level. The RTCP message is sought for the second terminal that does not support NAT.

Step S405: The intermediate server receives the key frame returned by the second terminal in response to the key frame request and forwards the key frame to the first terminal.

After receiving the key frame request message, the second terminal that does not support the NAT actively sends a key frame to the intermediate server, and the intermediate server forwards the key frame sent by the second terminal to the first terminal. After the first terminal receives the key frame, The subsequent P frames can be parsed normally, and the peer video picture is displayed normally.

To further explain the video calling method of the asymmetric networking system of the present application, the following describes a specific implementation manner of the NAT-enabled terminal and the terminal that does not support the NAT through the intermediate server, and specifically explains the present application. Symmetric networking system video calling method:

Referring to FIG. 5, FIG. 5 is a flowchart of a fifth implementation manner of a video calling method for an asymmetric networking system according to the present application. The implementation manner includes the following steps:

S1: Supporting the NAT terminal to initiate a video request;

The NAT terminal is enabled to initiate a video request, and the video codec information and its own NAT connectivity detection candidate address are forwarded in the SDP through the intermediate server to the unsupported NAT terminal.

S2: The intermediate server receives the video request from the supporting NAT terminal, removes the NAT connectivity detection candidate address, and sends the unsupported NAT terminal;

After receiving the video request carrying the NAT connectivity check candidate address, the intermediate server removes the NAT connectivity detection candidate address that supports the NAT terminal and sends the candidate address to the terminal that does not support NAT.

S3: The NAT terminal is not allowed to send a response that does not carry the NAT connectivity detection candidate address to the intermediate server;

The NAT terminal does not support sending a response that does not carry the NAT connectivity detection candidate address to the intermediate server, and the intermediate server forwards it to the supporting NAT terminal.

S4: After receiving the response, the intermediate server adds the NAT connectivity check candidate address to the SDP and forwards it to the supporting NAT terminal.

After receiving the response, the intermediate server adds its own NAT connectivity detection candidate address to In the SDP, the response is forwarded to the supporting NAT terminal through the SDP.

S5: The NAT terminal is not supported to send the first key frame to the intermediate server;

Since the NAT terminal is not supported, it is not known whether the connectivity detection between the NAT terminal and the intermediate server is completed. Therefore, it may be possible to start sending media to the supporting NAT terminal after completing the media information exchange (ie, receiving the video request and returning the response message). . Therefore, the behavior of this step is to send the first key frame to the intermediate server and forward it to the supporting NAT terminal through the intermediate server.

S6: performing connectivity detection;

Since the media information exchange is completed between the intermediate server and the supporting NAT terminal, the NAT connectivity detection candidate addresses are obtained, and connectivity detection is started.

S7: The first key frame may be discarded;

Since the NAT connectivity check between the intermediate server and the supporting NAT terminal has not been completed, the first key frame that does not support the NAT terminal may be discarded.

S8: The connectivity detection is completed;

The connectivity detection between the intermediate server and the supporting NAT terminal is completed.

S9: Supporting the NAT terminal to send the first key frame;

After the connectivity detection is completed, the supporting NAT terminal starts to send video to the unsupported NAT terminal through the intermediate server, that is, the first key frame is sent.

S10: The intermediate server sends the first key frame from the supporting NAT terminal to the unsupported NAT terminal, and does not support the NAT terminal to display that the video supported by the NAT terminal is normal;

The intermediate server forwards the key frames sent by the NAT terminal to the peer end, that is, does not support the NAT terminal, and does not support the NAT terminal to correctly parse the subsequent P frames after receiving the key frame. NAT terminal display is not supported. The video sent by the NAT terminal is normal.

S11: Supporting NAT terminal discovery After the NAT detection is completed, the UE actively sends a key frame request message to the unsupported NAT terminal at the media level;

Supporting NAT terminal discovery After NAT detection is completed, a critical frame request RTCP message is actively sent at the media level, and is forwarded to the unsupported NAT terminal through the intermediate server.

It should be noted that after the terminal detects that the NAT detection is completed, it can also actively send a key frame request message, such as a SIP INFO message, at the signaling level, and then transit through the intermediate server. Issued to not support NAT terminals.

S12: The intermediate server forwards the key frame request message to the unsupported NAT terminal; after receiving the key frame request message through the media layer or the signaling layer, the intermediate server forwards the key frame request message to the unsupported NAT terminal.

S13: The NAT terminal is not supported to send the second key frame to the intermediate server after receiving the key frame request;

After receiving the key frame request, the NAT needs to send a key frame, that is, the second key frame to the intermediate server, and forward it to the supporting NAT terminal through the intermediate server.

S14: After receiving the second key frame, the intermediate server forwards to the supporting NAT terminal, and supports the NAT terminal to display that the peer video is normal.

After receiving the second key frame sent by the NAT terminal, the intermediate server forwards the packet to the supporting NAT terminal. After receiving the second key frame, the NAT server can normally parse the subsequent P frame to display the normal video of the peer end. .

It should be noted that the execution step of the transmission key frame request in this embodiment may also be initiated by the intermediate server. That is, after completing the connectivity check with the supporting NAT terminal, the intermediate server can actively send a key frame request message to the unsupported NAT terminal at the media level or the signaling level, for example, sending a key frame request RTCP message through the media layer, or passing The signaling layer sends a key frame request message, such as an INFO message of the SIP. The steps before the step and the steps after the step are the same as the implementation process of the embodiment.

Referring to FIG. 6, FIG. 6 is a structural caller of an embodiment of the NAT-enabled terminal of the present application for making a video call. The NAT-enabled terminal 100 includes a requesting module 11 and a receiving module 12, where:

The requesting module 11 is configured to send a key frame request to a terminal that does not support NAT when the NAT connectivity check between the terminal supporting the NAT and the intermediate server is completed;

After the intermediate server returns the request response message of the terminal that does not support NAT, the NAT-enabled terminal learns the NAT connectivity candidate address of the intermediate server and starts connectivity detection with the intermediate server. When the detection is completed, the request module 11 actively sends an unsupported NAT. The terminal sends a key frame request to avoid the loss of the key frame, which may cause the video image sent by the terminal that does not support NAT to be displayed normally for a long time.

In the actual application process, the requesting module 11 may issue a key frame request through the media layer or the signaling layer, and forward it to the terminal that does not support NAT through the intermediate server.

The receiving module 12 is configured to receive a key frame returned by the terminal that does not support the NAT in response to the key frame request, parse the subsequent difference frame, and normally display the video sent by the terminal that does not support the NAT.

After receiving the key frame request, the terminal that does not support the NAT returns a request response message to the terminal supporting the NAT, and the request response message carries the requested key frame. A terminal that does not support NAT returns a request response message to the intermediate server, which is forwarded to the receiving module 12 by the intermediate server.

The receiving module 12 receives the returned key frame, and can correctly parse the subsequent P frame, and display that the video sent by the terminal that does not support NAT is normal.

The requesting module 11 is further configured to send a video request to the terminal that does not support the NAT through the intermediate server, where the video request carries the video codec information and the NAT connectivity detection candidate address of the terminal supporting the NAT;

The requesting module 11 initiates a video request, and carries the video codec information and the self-NAT connectivity detection candidate address in the SDP. After receiving the video request carrying the NAT connectivity detection candidate address, the intermediate server removes the NAT connectivity detection in the video request. The candidate address is then sent to the terminal that does not support NAT.

The terminal that does not support the NAT sends a request response message that does not carry the NAT connectivity detection candidate address to the intermediate server. After receiving the request response message, the intermediate server adds its own NAT connectivity detection candidate address information to the request response message, and then requests the response. The message is forwarded to the receiving module 12.

The receiving module 12 is further configured to receive a request response message carrying a network address translation connectivity detection candidate address of the intermediate server, and start network address translation connectivity detection with the intermediate server.

The receiving module 12 receives the request response message forwarded by the intermediate server, completes the media information exchange, obtains the NAT connectivity detection candidate addresses of each other, and starts NAT connectivity detection with the intermediate server.

Please refer to FIG. 7. FIG. 7 is a schematic structural diagram of an embodiment of a server according to the present application. The server 200 of the present embodiment includes a requesting module 21 and a forwarding module 22, where: the requesting module 21 is configured to send a key to a terminal that does not support NAT when the network address translation connectivity detection between the terminal supporting the NAT and the server is completed. Frame request

The NAT-enabled terminal starts the connectivity check with the server after the server returns a request response message that does not support the NAT terminal, knows the NAT connectivity candidate address of the intermediate server. When the detection is completed, the requesting module 21 actively sends the video image sent by the key frame to the terminal that does not support the NAT.

In the actual application process, the requesting module 21 may issue a key frame request to a terminal that does not support NAT through a media layer or a signaling layer.

The forwarding module 22 is configured to receive a key frame returned by the terminal that does not support the NAT in response to the key frame request, and forward the key frame to the terminal that supports the NAT, so that the terminal supporting the NAT can normally display the video sent by the terminal that does not support the NAT. .

After receiving the key frame request, the terminal that does not support the NAT returns a request response message to the forwarding module 22, where the request response message carries the key frame requested by the intermediate server. The forwarding module 22 forwards the key frames to the terminal that supports the NAT.

The terminal that supports the NAT receives the returned key frame, and can correctly parse the subsequent P frame, and display the video sent by the terminal that does not support NAT.

The forwarding module 22 is configured to receive a video request of the terminal that supports the NAT, and the video request carries the video codec information and the NAT connectivity detection candidate address of the terminal that supports the NAT, and deletes the NAT connectivity detection candidate address in the video request. Request to forward to a terminal that does not support NAT.

The NAT-enabled terminal initiates a video request, carries the video codec information, and the self-NAT connectivity detection candidate address is in the SDP, and the forwarding module 22 receives the video request from the NAT-enabled terminal that carries the NAT connectivity detection candidate address, and then removes the video. The NAT connectivity in the request detects the candidate address, and then sends the video request to the terminal that does not support NAT.

The forwarding module 22 receives the request response message from the terminal that does not support the NAT, adds the NAT connectivity detection candidate address of the server to the request response message, and then cancels the request response. The information is forwarded to the terminal that supports NAT.

After receiving the video request, the terminal that does not support the NAT sends a request response message that does not carry the NAT connectivity check candidate address to the forwarding module 22 of the server. After receiving the request response message, the forwarding module 22 adds the NAT connectivity detection candidate address of the server. After the message reaches the request response message, the request response message is forwarded to the terminal supporting the NAT.

The NAT-enabled terminal receives the request response message forwarded by the forwarding module 22, completes the media information exchange, obtains the NAT connectivity detection candidate addresses of each other, and starts NAT connectivity detection with the server.

Please refer to FIG. 8. FIG. 8 is a structural server of another embodiment of the NAT-enabled terminal of the present application for making a video call. The terminal 300 supporting the NAT in this embodiment includes a processor 31, a receiver 32, a transmitter 33, and a random access memory. 34. Read only memory 35, bus 36, and network interface unit 37. The processor 31 is coupled to the receiver 32, the transmitter 33, the random access memory 34, the read only memory 35, and the network interface unit 37 via a bus 36, respectively. When the terminal 300 supporting NAT is required to be run, the booting system in the basic input/output system or the embedded system in the read-only memory 34 is booted, and the terminal 300 supporting the NAT is booted into a normal running state. After the NAT-enabled terminal 300 enters a normal operating state, the application and operating system are run in the random access memory 33, and data is received from the network or transmitted to the network through the network interface unit 36, such that:

When the NAT connectivity detection of the terminal supporting the NAT and the intermediate server is completed, the processor 31 sends a key frame request to the terminal that does not support the NAT, and receives the key frame returned by the terminal that does not support the NAT in response to the key frame request, and the subsequent analysis is performed normally. The P frame shows that the video image sent by the terminal that does not support NAT is normal.

On the other hand, the processor 31 is further configured to send a video request to the terminal that does not support the NAT through the intermediate server, where the video request carries the video codec information and the NAT connectivity detection candidate address of the terminal supporting the NAT, and receives the returned video. Requesting a response message to support the present embodiment, the processor 31 may be a central processing unit CPU, or An Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

After the NAT connectivity detection is completed, the transmitter 33 sends the video data to the intermediate server, and the intermediate server forwards the video data to the terminal that does not support the NAT;

The receiver 32 receives the video data forwarded by the terminal that does not support the NAT through the intermediate server.

The present application further provides a network system, including the terminal supporting NAT according to any one of the foregoing embodiments, and any of the foregoing embodiments, on the basis of the foregoing embodiments of the NAT-enabled terminal and the server provided by the present application. The server further includes a terminal that does not support NAT. A terminal that supports NAT performs a video call between a server and a terminal that does not support NAT, and implements interactive transmission of video data.

In the foregoing embodiment, the video calling method, the terminal, the server, and the system of the asymmetric networking system of the present application actively send a key frame request to an unsupported NAT terminal when the NAT connectivity detection between the terminal supporting the NAT and the intermediate server is completed. Therefore, the key frame of the terminal that does not support NAT is received to achieve normal video decoding. In this way, the time for the NAT terminal to display the video request response message cannot be displayed normally after receiving the video request response message, thereby giving the user a better service experience.

In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be used. Combined or can be integrated into another system, or some features can be ignored, or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form. The components displayed as the unit may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment. In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application, in essence or the contribution to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. The instructions include a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like, which can store program codes. .

The above is only the embodiment of the present application, and thus does not limit the scope of the patent application, and the equivalent structure or equivalent process transformation made by using the specification and the drawings of the present application, or directly or indirectly applied to other related technologies. The scope of the invention is included in the scope of patent protection of this application.

Claims

Rights request

An asymmetric network system video calling method, the asymmetric networking system comprising a first terminal supporting network address translation, an intermediate server, and a second terminal not supporting network address translation, wherein the method Includes:

When the network address translation connectivity check between the first terminal and the intermediate server is completed, the first terminal sends a key frame request to the second terminal;

The first terminal receives the key frame returned by the second terminal in response to the key frame request, parses the subsequent difference frame, and normally displays the video sent by the second terminal.

2. The method of claim 1 wherein

The step of sending a key frame request to the second terminal includes: the first terminal sends a key frame request at a media level or a signaling level, and forwards the request to the second terminal by using the intermediate server.

3. The method of claim 1 wherein

The method further includes: before the network address translation connectivity detection between the first terminal and the intermediate server is completed, the method further includes:

The first terminal sends a video request to the second terminal by using the intermediate server, where the video request carries video codec information and a network address translation connectivity detection candidate address of the first terminal;

The first terminal receives a request response message carrying a network address translation connectivity detection candidate address of the intermediate server, and starts network address translation connectivity detection with the intermediate server.

A video calling method for an asymmetric networking system, the asymmetric networking system comprising a first terminal supporting network address translation, an intermediate server, and a second terminal not supporting network address translation, wherein the method Includes:

After the network address translation connectivity detection between the first terminal and the intermediate server is completed, the intermediate server sends a key frame request to the second terminal;

The intermediate server receives the key frame returned by the second terminal in response to the key frame request, and forwards the key frame to the first terminal, so that the first terminal can normally display the second terminal The video that came.

5. The method of claim 4, wherein

The step of the intermediate server sending a key frame request to the second terminal includes: the intermediate server sending a key frame request to the second terminal at a media level or a signaling layer.

6. The method of claim 4, wherein

Before the network address translation connectivity check between the first terminal and the intermediate server is completed, the method further includes:

The intermediate server receives a video request of the first terminal, where the video request carries video codec information and a network address translation connectivity detection candidate address of the first terminal;

And deleting, by the intermediate server, the network address translation connectivity detection candidate address in the video request, and forwarding the video request to the second terminal.

7. The method of claim 6 wherein:

After the step of forwarding the video request to the second terminal, the method further includes: the intermediate server receiving a request response message from the second terminal, adding its own network address translation connectivity detection candidate address to the In the request response message, the request response message is then forwarded to the first terminal.

8. A terminal for supporting network address translation, wherein: the terminal supporting the network address translation and the terminal not supporting the network address translation perform a video call through the intermediate server, where the terminal supporting the network address translation includes a request module and a receiving module, where: the requesting module is configured to send a key frame request to the terminal that does not support network address translation when the network address translation connectivity detection between the terminal supporting the network address translation and the intermediate server is completed ;

The receiving module is configured to receive a key frame returned by the terminal that does not support network address translation in response to the key frame request, parse the subsequent difference frame, and normally display the video sent by the terminal that does not support network address translation.

9. The terminal of claim 8 wherein:

The requesting module is configured to issue a key frame request at a media level or a signaling level, and forwarded by the intermediate server to the terminal that does not support network address translation.

10. The terminal of claim 8 wherein: The requesting module is configured to send a video request to the terminal that does not support network address translation by using the intermediate server, where the video request carries video codec information and network address translation connectivity detection of the terminal that supports network address translation Candidate address

The receiving module is configured to receive a request response message carrying a network address translation connectivity detection candidate address of the intermediate server, and start network address translation connectivity detection with the intermediate server.

A server, wherein a terminal that supports network address translation and a terminal that does not support network address translation make a video call through the server, the server includes a request module and a forwarding module, where:

The requesting module is configured to send a key frame request to the terminal that does not support network address translation when the network address translation connectivity detection between the terminal supporting the network address translation and the server is completed;

The forwarding module is configured to receive a key frame returned by the terminal that does not support network address translation in response to the key frame request, and forward the key frame to the terminal that supports network address translation, so that the support is supported. The terminal of the network address translation can normally display the video sent by the terminal that does not support network address translation.

The server according to claim 11, wherein

The requesting module is configured to send a key frame request to the terminal that does not support network address translation at a media level or a signaling layer.

13. The server of claim 11 wherein:

The forwarding module is configured to receive a video request of the terminal that supports network address translation, where the video request carries video codec information and a network address translation connectivity detection candidate address of the terminal that supports network address translation, and deletes the After the network address translation connectivity detection candidate address in the video request, the video request is forwarded to the terminal that does not support network address translation.

14. The server of claim 13 wherein:

The forwarding module is configured to receive a request response message from the terminal that does not support network address translation, add a network address translation connectivity detection candidate address of the server to the request response message, and then forward the request response message to the The support network address translation Terminal.

A network system, comprising: the network address translation enabled terminal according to any one of claims 8-10, and the server and unsupported network according to any one of claims 11-14. The terminal for address translation.