CN117914817A

CN117914817A - Real-time communication method, client, server, storage medium and electronic device

Info

Publication number: CN117914817A
Application number: CN202311865431.XA
Authority: CN
Inventors: 白宇
Original assignee: Netease Youdao Information Technology Beijing Co Ltd
Current assignee: Netease Youdao Information Technology Beijing Co Ltd
Priority date: 2023-12-29
Filing date: 2023-12-29
Publication date: 2024-04-19

Abstract

The application relates to a real-time communication method, a client, a server, a storage medium and electronic equipment. The method comprises the following steps: calling a server-side real-time communication SDK of the server to establish reliable communication connection with a client-side real-time communication SDK of the client, wherein the reliable communication connection is used for reestablishing the bottom communication connection and retransmitting lost messages under the condition that the client-side is disconnected from the bottom communication connection of the server so as to ensure reliable transmission of communication messages; and transmitting the communication message with the client through the established reliable communication connection, wherein the time for transmitting the communication message is not constrained. The application solves the technical problem that the accurate transmission of the message can not be ensured in real-time communication.

Description

Real-time communication method, client, server, storage medium and electronic device

Technical Field

The present application relates to the field of communications, and in particular, to a real-time communication method, a client, a server, a storage medium, and an electronic device.

Background

In the prior art, the real-time communication technology, IM (Instant Messaging) for short, can send and receive messages instantly. The system architecture of the real-time communication technology generally consists of a client and a server, wherein the client is a terminal device and can send and receive messages, and a network communication function is built in to communicate with the server. The server transmits the message between the clients through the network.

However, if the message fails to be sent due to network abnormality, disconnection, etc., part of the message may be discarded, and the message cannot be transmitted to the receiver, so that accurate transmission of the message cannot be ensured.

Disclosure of Invention

The application provides a real-time communication method, a client, a server, a storage medium and electronic equipment, which are used for solving the technical problem that accurate transmission of a message cannot be guaranteed in real-time communication.

In a first aspect, the present application provides a real-time communication method, applied to a server, including: calling the server side real-time communication SDK of the server side to establish reliable communication connection with the client side real-time communication SDK of the client side, wherein the reliable communication connection is used for reestablishing the bottom communication connection and retransmitting lost messages under the condition that the client side is disconnected with the bottom communication connection of the server side so as to ensure reliable transmission of communication messages; and transmitting a communication message with the client through the established reliable communication connection, wherein the time for transmitting the communication message is not restricted.

As an optional example, the server side real-time communication SDK includes a server side management module and a server side communication module, the server side management module includes a server side sending function and a server side receiving function, the client side real-time communication SDK includes a client side management module and a client side communication module, and the client side management module includes a client side sending function and a client side receiving function; the calling the server-side real-time communication SDK of the server to establish reliable communication connection with the client-side real-time communication SDK of the client comprises the following steps: the reliable communication connection is established between the server-side sending function and the client-side receiving function, and the reliable communication connection is established between the server-side receiving function and the client-side sending function, wherein the server-side communication module and the client-side communication module are used for establishing the bottom communication connection, the server-side sending function is used for sending communication information to the client-side receiving function through the bottom communication connection, and the server-side receiving function is used for receiving the communication information sent by the client-side sending function through the bottom communication connection.

As an alternative example, the method further comprises: under the condition that the bottom communication connection is disconnected, reestablishing the bottom communication connection with a client management module of the client real-time communication SDK through the server management module of the server real-time communication SDK; and retransmitting the communication message which is not successfully transmitted through the server management module.

As an optional example, the server receiving function in the server management module is configured to perform at least one of an acknowledgement operation, a retransmission request operation, a sorting operation, and a deduplication operation on a communication message sent by the sending function of the client management module, in a case where the communication message is received.

As an optional example, before invoking the server-side real-time communication SDK of the server to establish a network communication connection with the client-side real-time communication SDK of the client, the method further includes: and inserting the SDK for real-time communication of the server in the server.

As an alternative example, the transmitting the communication message with the client through the established reliable communication connection includes: when the service end is a message sender and the communication message is sent but the acknowledgement message of the first data packet in the communication message is not received, retransmitting the first data packet every a preset time period until the acknowledgement message of the first data packet is received, wherein the first data packet is any data packet in the communication message; or when the server side is a message receiver, when receiving a second data packet in the communication message, returning an acknowledgement message of the second data packet to the client side, and when repeatedly receiving the second data packet, repeatedly returning the acknowledgement message of the second data packet to the client side and uploading the second data packet only once.

As an optional example, the retransmitting, by the server management module, the communication message that is not successfully transmitted includes: retransmitting the communication message which is not successfully transmitted by a sliding window mechanism.

As an alternative example, the transmitting the communication message with the client through the established reliable communication connection includes: and when the server side is a message receiver, returning an acknowledgement message to the client side when receiving a plurality of data packets in the communication message, wherein the returned acknowledgement message is used for indicating that a plurality of continuous data packets in the communication message are received.

As an optional example, the server management module further includes a server processing function, and the method further includes: and under the condition that the server side is a message receiver and the client side management module of the client side fails to send the communication message and retransmits the communication message successfully, the server side processing function in the server side management module is called by the server side receiving function to process the communication message, wherein the server side processing function is a self-defined function.

As an optional example, the server is deployed in a distributed cluster, and a gateway for forwarding a communication message is included between the server and the client; transmitting a communication message with the client over the established reliable communication connection, comprising: and transmitting the communication message with the client through the established reliable communication connection and the gateway.

As an alternative example, forwarding the communication message through the gateway while transmitting the communication message with the client includes: and transmitting the communication message with the client with the target client label through the gateway, wherein the client labels of a plurality of clients communicating with the current server are all the target client labels, and the gateway is used for transmitting the communication message of the clients with the same client label to the same server.

As an optional example, the client includes a first client and a target client, the first client is a client that establishes the reliable communication connection with the server, the target client is a client that establishes the reliable communication connection with a distribution gateway, the gateway includes a forwarding gateway and the distribution gateway, the forwarding gateway is a gateway that forwards communication messages between the first client and the server, and the distribution gateway is a client that establishes the reliable communication connection with the server.

As an alternative example, the method further comprises: the server sends the communication message to other clients in the first client through the reliable communication connection, and sends the communication message to the distribution gateway through the reliable communication connection, so that the distribution gateway distributes the communication message to the target client through the reliable communication connection, wherein the other clients are clients except the client which sends the communication message to the server in the first client.

In a second aspect, the present application provides a real-time communication method, applied to a client, including: calling the client real-time communication SDK of the client to establish reliable communication connection with the server real-time communication SDK of the server, wherein the reliable communication connection is used for reestablishing the bottom communication connection and retransmitting lost messages under the condition that the server is disconnected from the bottom communication connection of the client so as to ensure reliable transmission of communication messages; and transmitting a communication message with the server through the established reliable communication connection, wherein the time for transmitting the communication message is not restricted.

As an optional example, the client real-time communication SDK includes a client management module and a client communication module, the client management module includes a client sending function and a client receiving function, the server real-time communication SDK includes a server management module and a server communication module, and the server management module includes a server sending function and a server receiving function; the calling the client real-time communication SDK of the client to establish reliable communication connection with the server real-time communication SDK of the server comprises the following steps: the reliable communication connection is established between the client-side sending function and the server-side receiving function, and the reliable communication connection is established between the client-side receiving function and the server-side sending function, wherein the client-side communication module and the server-side communication module are used for establishing the bottom communication connection, the client-side sending function is used for sending communication information to the server-side receiving function through the bottom communication connection, and the client-side receiving function is used for receiving the communication information sent by the server-side sending function through the bottom communication connection.

As an alternative example, the method further comprises: under the condition that the bottom layer communication connection is disconnected, reestablishing the bottom layer communication connection through the client management module of the client real-time communication SDK and the server management module of the server real-time communication SDK; and retransmitting the communication message which is not successfully transmitted through the client management module.

As an optional example, the client receiving function in the client management module is configured to perform at least one of an acknowledgement operation, a retransmission request operation, a sorting operation, and a deduplication operation on a communication message sent by the sending function of the server management module, in a case where the communication message is received.

As an optional example, before invoking the client-side real-time communication SDK of the client to establish a network communication connection with the server-side real-time communication SDK of the server, the method further includes: inserting the client real-time communication SDK in the client.

As an optional example, the transmitting the communication message with the server through the established reliable communication connection includes: when the client is a message sender and the communication message is sent but the acknowledgement message of the first data packet in the communication message is not received, retransmitting the first data packet every a preset time period until the acknowledgement message of the first data packet is received, wherein the first data packet is any data packet in the communication message; or when the client is a message receiver, when receiving a second data packet in the communication message, returning an acknowledgement message of the second data packet to the server, and when repeatedly receiving the second data packet, repeatedly returning the acknowledgement message of the second data packet to the server and uploading the second data packet only once.

As an optional example, the retransmitting, by the client management module, the communication message that is not successfully transmitted includes: retransmitting the communication message which is not successfully transmitted by a sliding window mechanism.

As an optional example, the transmitting the communication message with the server through the established reliable communication connection includes: and when the client is a message receiver, returning an acknowledgement message to the server when receiving a plurality of data packets in the communication message, wherein the returned acknowledgement message is used for indicating that a plurality of continuous data packets in the communication message are received.

As an optional example, the client management module further includes a client processing function, and the method further includes: and under the condition that the client is a message receiver and the server management module of the server fails to send the communication message and retransmits the communication message successfully, invoking the client processing function in the client management module by the client receiving function to process the communication message, wherein the client processing function is a custom function.

In a third aspect, the present application provides a real-time communication server, configured to implement the real-time communication method applied to the server.

In a fourth aspect, the present application provides a real-time communication client, configured to implement the real-time communication method applied to the client.

In a fifth aspect, the present application provides an electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor coupled to the at least one bus; at least one memory coupled to the at least one bus, wherein the memory stores a computer program, and the processor is configured to implement any one of the real-time communication methods described above when executing the computer program.

In a sixth aspect, the present application also provides a computer storage medium storing computer executable instructions for performing any one of the above-described real-time communication methods of the present application.

Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the method provided by the embodiment of the application, the server side real-time communication SDK of the server side is called to establish reliable communication connection with the client side real-time communication SDK of the client side, wherein the reliable communication connection is used for reestablishing the bottom communication connection and retransmitting lost messages under the condition that the client side is disconnected from the bottom communication connection of the server side so as to ensure reliable transmission of communication messages; and transmitting the communication message with the client through the established reliable communication connection, wherein the time for transmitting the communication message is not constrained, thereby establishing the reliable communication connection between the client and the server, being capable of double-ended communication and ensuring the reliable transmission of the message.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.

One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.

Fig. 1 is a flowchart of a real-time communication method according to an embodiment of the present application;

FIG. 2 is a flow chart of another method of real-time communication according to an embodiment of the present application;

FIG. 3 is a diagram of a reliable communication connection provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of an SDK according to an embodiment of the present application;

FIG. 5 is a diagram of a cluster deployment server provided by an embodiment of the present application;

FIG. 6 is a diagram of a lateral deployment server for a physical machine according to an embodiment of the present application;

fig. 7 is a gateway forwarding diagram provided in an embodiment of the present application;

FIG. 8 is a diagram of a reliable communication connection provided by an embodiment of the present application;

Fig. 9 is a diagram of a transmission communication message according to an embodiment of the present application;

fig. 10 is a diagram of a retransmission communication message according to an embodiment of the present application;

fig. 11 is a diagram of another retransmission communication message provided in an embodiment of the present application;

FIG. 12 is a sliding window diagram provided by an embodiment of the present application;

FIG. 13 is a message batch acknowledgement diagram provided by an embodiment of the present application;

FIG. 14 is a diagram of yet another reliable communication transmission provided by an embodiment of the present application;

FIG. 15 is a large scale distribution scenario diagram provided by an embodiment of the present application;

FIG. 16 is another mass distribution scenario diagram provided by an embodiment of the present application;

Fig. 17 is a schematic diagram of an electronic device according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The following disclosure provides many different embodiments, or examples, for implementing different structures of the invention. In order to simplify the present disclosure, components and arrangements of specific examples are described below. They are, of course, merely examples and are not intended to limit the invention. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.

In order to solve the technical problem that accurate transmission of messages cannot be guaranteed in real-time communication in the prior art, the application provides a real-time communication method which can realize double-end communication and reliable transmission.

Fig. 1 is a flowchart of a real-time communication method applied to a server according to an embodiment of the present application. As shown in fig. 1, the real-time communication method includes:

S102, calling a server side real-time communication SDK of the server side to establish reliable communication connection with a client side real-time communication SDK of a client side, wherein the reliable communication connection is used for reestablishing the bottom communication connection and retransmitting lost messages under the condition that the client side is disconnected with the bottom communication connection of the server side so as to ensure reliable transmission of communication messages;

S104, transmitting a communication message with the client through the established reliable communication connection, wherein the time for transmitting the communication message is not constrained.

Fig. 2 is a flow chart of a real-time communication method applied at a client. As shown in fig. 2, the real-time communication method includes:

S202, calling a client-side real-time communication SDK of the client-side to establish reliable communication connection with a server-side real-time communication SDK of a server-side, wherein the reliable communication connection is used for reestablishing the bottom communication connection and retransmitting lost messages under the condition that the server-side is disconnected from the bottom communication connection of the client-side so as to ensure reliable transmission of communication messages;

S204, transmitting a communication message with the server through the established reliable communication connection, wherein the time for transmitting the communication message is not restricted.

The server is configured with the server real-time communication SDK, the client is configured with the client real-time communication SDK, reliable communication connection is established between the server and the client through the SDK, and the reliable communication connection reestablishes the bottom communication connection and retransmits lost messages when the bottom communication connection is disconnected. The client and the server communicate through reliable communication connection, and the time for sending the communication message is not constrained, namely, the client and the server can send the communication message to each other at any time.

The reliable communication connection is a network transmission mechanism, which can ensure that when data is transmitted through the bottom communication connection, the data is transmitted from a message sender to a message receiver accurately, and even if the bottom communication connection is disconnected in an unstable network environment, the bottom communication connection can be recovered and the data is retransmitted, so that the integrity of the data is ensured.

Fig. 3 is a schematic diagram of reliable transmission communication in this embodiment. And the client and the server establish reliable transmission communication with each other through respective SDKs, and send and receive communication messages through the reliable transmission communication.

According to the method provided by the embodiment of the application, the server side real-time communication SDK of the server side is called to establish reliable communication connection with the client side real-time communication SDK of the client side, wherein the reliable communication connection is used for reestablishing the bottom communication connection and retransmitting lost messages under the condition that the client side is disconnected with the bottom communication connection of the server side so as to ensure reliable transmission of communication messages; and transmitting the communication message with the client through the established reliable communication connection, wherein the time for transmitting the communication message is not constrained, thereby establishing the reliable communication connection between the client and the server, being capable of double-ended communication and ensuring the reliable transmission of the message.

As an optional example, the server side real-time communication SDK includes a server side management module and a server side communication module, the server side management module includes a server side sending function and a server side receiving function, the client side real-time communication SDK includes a client side management module and a client side communication module, and the client side management module includes a client side sending function and a client side receiving function; the calling the server-side real-time communication SDK of the server to establish reliable communication connection with the client-side real-time communication SDK of the client comprises: the reliable communication connection is established between the server-side sending function and the client-side receiving function, and the reliable communication connection is established between the server-side receiving function and the client-side sending function, wherein the server-side communication module and the client-side communication module are used for establishing the bottom communication connection, the server-side sending function is used for sending communication information to the client-side receiving function through the bottom communication connection, and the server-side receiving function is used for receiving the communication information sent by the client-side sending function through the bottom communication connection.

In this embodiment, the real-time communication SDK includes a management module and a communication module, where the management module includes a sending function and a receiving function, the sending function of the server sends a communication message to be received by the receiving function of the client, the sending function of the client sends a communication message to be received by the receiving function of the server, a reliable communication connection is established between the SDK of the server and the SDK of the client, the sending function of the server sends a communication message to the receiving function of the client through the reliable communication connection, and the sending function of the client sends a communication message to the receiving function of the server through the reliable communication connection. The communication module of the server and the communication module of the client are responsible for establishing the bottom communication connection.

As shown in fig. 4, one of the client and the server is a sender, the other is a receiver, SDKs of the sender and the receiver both include a management module stream and a communication module connection, the management module is used for establishing a reliable communication connection, and the communication module is used for establishing a bottom communication connection. Reliable communication connection transport can ensure that the communication messages which are not successfully transmitted are retransmitted after the lower communication connection is disconnected.

In this embodiment, due to network influence, the underlying communication connection may be disconnected, and if the underlying communication connection is disconnected, the management module of the real-time communication SDK of the server and the management module of the real-time communication SDK of the client reestablish the underlying communication connection, and retransmit the communication message that is not successfully transmitted until the transmission is successful.

In this embodiment, after receiving data sent by the sending function of the message sender, the receiving function in the management modules of the server and the client performs at least one of a confirmation operation, a retransmission request operation, a sorting operation, and a deduplication operation. For example, performing an acknowledgement operation, i.e. returning a reply message to the sending function that the message has received, e.g. requesting a retransmission operation, is sending a reply message to the sending function that the message is to be retransmitted.

In this embodiment, the real-time communication SDK may be inserted into the server and the client in the form of a plug-in or a code. If the code package is imported, the real-time communication SDK is installed on the client or the server. Or the function codes of the real-time communication SDK are put into the processing chips of the client and the server.

In this embodiment, in the case where the client transmits a communication message to the server or the server transmits a communication message to the client, if the communication message is transmitted without receiving an acknowledgement message, the communication message is retransmitted every predetermined time, and the communication message having received the acknowledgement message does not need to be retransmitted. And for the receiver, if the communication message is received, the confirmation message is returned, and the confirmation message is returned every time the communication message is received, but the data packet of the repeatedly received communication message is subjected to deduplication and is uploaded only once.

In this embodiment, in order to ensure that the message transmission does not occupy too much bandwidth, a sliding window mechanism is used to ensure that the number of messages sent simultaneously is not too large.

In this embodiment, when the server and the client serve as the receiving sides of the message, batch confirmation may be performed on the data sent by the sending side. For example, after the sender sends a communication message, the receiver receives a plurality of continuous data packets, and when the receiver receives one data packet, the receiver does not reply to an acknowledgement message, but replies the received plurality of continuous data packets to an acknowledgement message, which indicates that the plurality of continuous data packets have been received.

As an optional example, the server management module further includes a server processing function, and the method further includes: and under the condition that the server side is a message receiver and the client side management module of the client side fails to send the communication message and retransmits the communication message to be transmitted successfully, the server side processing function in the server side management module is called by the server side receiving function to process the communication message, wherein the server side processing function is a self-defined function.

In this embodiment, for the server and the client, after the receiving functions of the server and the client successfully receive the communication message, the processing function may be invoked. The processing function is a custom function, and the custom function is used for processing the communication message, so that custom operation can be performed on the communication message.

As an optional example, the server is deployed in a distributed cluster, and a gateway for forwarding a communication message is included between the server and the client; the transmitting communication messages with the client over the established reliable communication connection includes: and transmitting the communication message with the client through the established reliable communication connection and the gateway.

In this embodiment, the server may be deployed in a distributed cluster, where the distributed cluster includes the server and the gateway. The deployment format is shown in fig. 4. The server and the gateway expose an address to the outside, and the client accesses the address. And the gateway forwards the communication message of the client to a server, which processes the communication message of the client.

In this embodiment, the communication message of the client may carry a client tag, and after receiving the communication message of the client, the gateway calculates a hash value according to the client tag, and then forwards the communication message with the same hash value to a server.

As an optional example, the client includes a first client and a target client, where the first client is a client that establishes the reliable communication connection with the server, the target client is a client that establishes the reliable communication connection with a distribution gateway, and the gateway includes a forwarding gateway and a distribution gateway, where the forwarding gateway is a gateway that forwards communication messages between the first client and the server, and the distribution gateway is a client that establishes the reliable communication connection with the server.

In this embodiment, for some scenarios, there may be batches of clients present. And if the batch clients all establish reliable communication connection with the server, the server is under excessive pressure. Therefore, a deployment manner shown in fig. 5 may be adopted, where the cluster may be a k8s service cluster, where the server 1 and the server 2 are deployed in the cluster, the gateway is a forwarding gateway, and is responsible for forwarding the communication message, the client accesses the server in the cluster through the address exposed by the cluster, and the forwarding gateway forwards the communication message of the client to the server.

With continued reference to fig. 5, in this embodiment, the first client may send a communication message to the server through an uplink channel, and the server sends the communication message to other clients through a downlink channel. And other clients may also send communication messages to the first client via the server via the upstream channel. The number of other clients and the first client is small. For example, in the playing process, the first client is one client, and the other clients are one client, and the two clients play. And the playing process is sent to the target client, which is the audience. Because the audience is too many, reliable communication connection is established between the server and the distribution gateway, and reliable communication connection is established between the distribution gateway and the target client, so that the connection pressure of the server is reduced.

The description is made with reference to examples. The service system of the embodiment is composed of a service end and a client end, wherein the service end can communicate with a plurality of client ends, but the client ends do not communicate with each other. The service end and the client end are developed by the service party to realize business logic, and three-party services such as synchronous service and the like are not involved in communication. Wherein:

The real-time communication capability is provided for the service party in the form of SDK, the service party invokes the SDK to establish reliable transmission communication between the service end and the client end, and the communication form is direct communication without relying on three-party service for message forwarding.

The SDK provides only message transfer capability without restriction to the business logic, without expressing the business logic in the form of a data container. When a business party needs to communicate information, calling the SDK to send information, wherein the information consists of information types and information contents, and the meaning is defined by the business party (for example, joining classrooms, going away by players and answering by contestants); the SDK allows the business party to register callbacks for received messages to define how to handle a particular message type. (receiving function in SDK of receiver calls custom function)

The message transmission of the SDK is reliable. As long as the server and the client survive, the network connection disconnection will be automatically reconnected, the automatic reconnection is automatically initiated by the management module, and the reconnection is executed by the communication module. Message loss can be automatically retransmitted and the processing sequence is ensured, and the caller does not need to additionally realize an exception handling mechanism.

Based on the real-time communication SDK, the business party can construct a high-throughput real-time communication service cluster. The cluster structure is as shown in fig. 5:

The cluster comprises a plurality of service end nodes, and the deployment of the service end nodes is compatible with the mainstream containerization technology (kubernetes); may be deployed in physical machines if special needs are required. The physical machine deployment is as shown in fig. 6, and a plurality of service ends are deployed transversely, and the communication message of the client end is transmitted to the service end through reliable transmission communication. Whether the cluster deployment or the physical machine deployment, the server supports lateral expansion to increase the number of nodes of the server and improve the throughput of the system without modifying codes. The server node may be stateful, and messages for a certain group of clients must be handled by the server-specific node. The same client identifier may be added to a group of client segments, and communication messages of the same client identifier may be sent to the same server.

A gateway can be introduced between the client and the server, as shown in fig. 7, the client is a client in different classrooms, the server has a plurality of clients, and the gateway forwards the clients in each classroom to the corresponding server. A group of clients can be identified by a partition key, that is, a group unique identifier is added to clients of different groups, the gateway forwards according to the partition key, and the messages with the same partition key are necessarily forwarded to the same node of the server. For example, in a classroom interaction scene, students in the same class and messages sent by classrooms have the same partition key, namely, the virtual classrooms in which the students and the classrooms are located; and each gateway forwards the messages to the corresponding service end node of the classroom according to the partition key no matter which gateway the teacher and the students access, so that the teacher and the students in the classroom interact in the same node. The real-time communication SDK proposed in this embodiment establishes a reliability protocol between end-to-end, and spans an intermediate network path to achieve reliability of the whole system.

In a real-time communication system, the SDK functions to provide a set of codes that can be multiplexed for a service party to establish reliable transport communication (transport) between a service end and a client end. the communication of transport is bidirectional, and both the server and the client can actively send messages; the transport communication is reliable, ensuring the sequence and integrity. "sequentiality" means that the message processing order and the sending order are consistent, and not out of order; "integrity" means that the transmitted message is processed and not lost.

Transport provides message sending and receiving functions for the service party, and the pseudocode is as follows:

And (3) transmitting: the caller calls send () to send any one message. The message format is defined by the service party, consists of the message type and the message content, and is not constrained by the SDK. Based on promise design patterns, the caller can register callbacks to obtain the send result; if the transmission is successful (i.e., the exception does not exist), the recipient has successfully received the message (transport built-in necessary acknowledgement and retransmission mechanisms) rather than just placing the message in the sender's network buffer.

resultPromise＝transport.send(message)；

resultPromise.await(e->…)

And (3) receiving: caller registration callback processes received messages and ensures sequencing and integrity. Transport. MessageListable. AddListener (message e- > …)

Transport is distinguished from the underlying physical network connection (TCP for example, below). As shown in fig. 8, the network connection is not directly established between the server and the client, and may be forwarded through intermediate nodes such as a nginx and a gateway during the period, and both ends only establish TCP connection with adjacent intermediate nodes. While transport is end-to-end (server, client), a communication protocol is established between the server and client across complex network paths, with the necessary acknowledgement and retransmission of messages.

In such an indirect, complex network environment, it is a common phenomenon that network failures cause message loss; relying solely on point-to-point protocols such as TCP cannot meet reliability requirements and end-to-end communication protocols must be introduced by transport. For example, the established TCP connection may be disconnected and reconnected between the client and the gateway, and between the gateway and the server, and other gateway nodes may be connected during reconnection. TCP cannot recover lost messages because there is no end-to-end TCP connection. transport introduces an end-to-end communication protocol to solve this problem, and the sender will try to resend the message continuously, possibly via different network paths (TCP connections, intermediate nodes) until the acknowledgement message reaches the opposite end. Wherein, any node on the network path does not participate in the communication protocol, and is only used as a means for transmitting messages by the communication protocol, so that transport allows the underlying network to be unreliable and changeable.

Functionally, SDK transport is unconstrained with respect to message handling (presentation, transport, upper layer business logic). The business can define the message type and the message content at will without applying the data structure of the three-party service. The communication endpoint may actively send messages at any time, independent of message timing and control signaling. The business party may register callbacks for the specified message type, executing any business logic.

In terms of reliability, SDK transport is able to recover lost messages when a network failure is encountered. the transport communication protocol continues to perform the necessary acknowledgements and retransmissions without additional processing by the service party. Therefore, the service side mainly considers the functional requirements when implementing the communication scene, translates the communication scene into message sending and receiving, and does not need to put a great deal of development effort in terms of exception handling.

The transport is internally composed of two components, connection and stream, each of which implements a role. As shown in fig. 4, to ensure reliability, the service message cannot be directly transmitted by connection, but must be retransmitted, sequenced, and de-duplicated by stream control.

Connection: underlying network transport such as TCP. Completing the establishment and reconnection of the network connection to provide basic message transmission capability; but reliability is not guaranteed and messages may be lost when network failures are encountered. connection encapsulates details of network communications, for example, TCP, and implements functions such as TCP connection establishment, long connection keep-alive, message serialization (e.g., json, protobuf), codec format (e.g., SSL, web socket), etc.

Stream: reliable communication protocols, such as selective retransmission protocols. When encountering a network failure, making necessary retransmission to recover the lost message; meanwhile, mechanisms such as ordering, de-duplication and the like of the messages need to be provided, and the messages are prevented from being disordered and repeated due to retransmission. And submitting the received message to service logic for processing after the lost message is recovered and the sequence and the integrity of the message are ensured.

Stream is a key component to ensure reliability, implemented using the "selective retransmission protocol". In order to solve the problem of message loss, selecting a retransmission protocol to uniquely number each message (seq), and sending a corresponding acknowledgement message (ack) to a sender after receiving the message by a receiver; the sender retransmits the same message at regular intervals (e.g. 5 s) until the message is acknowledged, thus eventually defining a message that can recover the lost message. To improve performance, the retransmission protocol is selected to allow multiple messages to be sent in succession, rather than the next message being sent after the previous message is acknowledged; message retransmission is "selective" in that acknowledged messages do not need to be retransmitted to save duplicate message transmissions.

The selection of retransmission protocol can handle several abnormal situations, the scenario is as follows:

1. Normal transmission, no message loss. As shown in fig. 9. The sender continuously sends 5 messages, and the seq of the messages is 1-5 respectively; the receiver receives the message successfully, finds out the meeting of the sequence and the integrity, and submits the message to the upper business logic for processing. Meanwhile, the receiving side sends corresponding acknowledgement messages to the sender, wherein acks of the acknowledgement messages are respectively 1-5 and represent messages which have received the same seq; after the sender receives the acknowledgement, it is no longer necessary to resend the messages.

2. The sender message is lost. As shown in fig. 10. The sender sends a message seq=1-5; but encountering a network failure (e.g., a TCP connection disconnection reconnection, and possibly reconnection to other gateway nodes) results in the loss of seq=3, and the recipient receives only the remaining messages. Wherein, the message seq=1, 2 satisfies the sequence and the integrity and is submitted to the upper business logic processing; but since the message seq=3 is lost, the integrity is destroyed and no further commit is made up from this message. Meanwhile, the receiver transmits acknowledgement messages ack=1, 2,4,5, but does not transmit acknowledgement ack=3 for lost messages. The sender retransmits the message seq=3 at regular intervals until it is acknowledged, since no acknowledgement ack=3 is received, thereby recovering the lost message; note that there is no need to resend the acknowledged message, i.e. any of seq=1, 2,4, 5. After receiving the message seq=3, the receiver recovers the integrity, so that the retransmitted message seq=3 and the previously received messages seq=4, 5 are submitted upwards in sequence.

3. The acknowledgement message is lost. As shown in fig. 11. The sender sends messages seq=1-5 and all received successfully, the receiver submits the message up and sends an acknowledgement message, but the acknowledgement message ack=3 is lost. The sender repeatedly sends the message seq=3 after a period of time because the sender does not receive the acknowledgement; the receiver discovery message seq repeats without submitting upwards, but will still respond to this message so that the sender knows that the message has been successfully received and stops retransmitting.

The number of messages that are consecutively transmitted at one time by the selective retransmission protocol is limited for the purpose of congestion control. When multiple transports share the same network connection, a single transport cannot send excessive messages to drain network resources, thereby affecting the communication of other transports. To achieve this, the sender needs to maintain messaging in a "sliding window" fashion, as shown in fig. 12. Wherein "window size" indicates the number of messages allowed to be continuously transmitted (set to 5), i.e., the number of messages being transmitted without being acknowledged; the sender may repeatedly resend these messages at intervals before being acknowledged. If the sender has a plurality of messages to send, setting from seq=1; due to the sliding window limitation, only the first 5 messages can be sent consecutively, i.e. the message seq=1-5, and the subsequent messages are not sent for a while starting from seq=6. When the earliest message in the window is acknowledged, the window is moved one cell so that the next message enters the window and begins to be sent. For example, when message seq=1 is acknowledged, the window moves one frame and enters message seq=6 into the window and performs transmission; after which when message seq=2 is acknowledged the window is moved again and the message seq=7 is sent, and so on. If message seq=2 is acknowledged but seq=1 has not yet been acknowledged, only stop retransmitting message seq=2, but the window does not move, since seq=2 is not the earliest message in the window; until message seq=1 is acknowledged, the window may be continuously moved 2 cells, sending message seq=6, 7.

Each message selecting a retransmission protocol is to be acknowledged; when the sender frequently sends messages, the receiver also needs to frequently send acknowledgement messages, thereby wasting network resources and cpu resources and reducing system throughput. To optimize this, SDK STREAM introduces a "batch acknowledgment" mechanism to save the number of acknowledgment messages. After the activation, the receiver does not send an acknowledgement message immediately after receiving the message, but sends an acknowledgement message at intervals (e.g. 1 s), which indicates that the messages received during this period are acknowledged. As shown in fig. 13, batch validation can save significantly on the number of message bars:

a. The sender sends the message seq=1-100 in a short time, wherein the message seq=50, 51 is lost, the rest messages are successfully received (98 pieces in total), and the receiver needs to acknowledge the received messages.

B. If a single acknowledgement is used by the recipient, then acknowledgement messages ack=1, 2..49 and acknowledgement messages ack=52, 53..100 (98 total) need to be sent.

C. if the receiver uses batch confirmation, only one confirmation message is needed to be sent, the continuous interval of the seq of the received message is included, and the start and stop seq of each interval is carried. In this scenario, the batch acknowledgment message contains 2 intervals, 1-49, 52-100, respectively; a batch acknowledgment message carries only the 4 seq described above, i.e. is equivalent to acknowledging 98 messages received.

The batch confirmation mechanism can save the number of messages, but can reduce the timeliness of confirmed messages, and is not applicable to all business scenes. In view of this, the SDK allows transport to create multiple streams inside, each stream can use a different communication protocol, so that one stream uses a single acknowledgement and another stream uses a batch acknowledgement; and each message can specify a different stream to send when the service invokes the SDK.

The embodiment can deploy the server side in the cluster. For example, a classroom interaction system maintains a class state at a server, and a teacher client and a student client of the same class (called a partition key) must be connected to the same node of the server to commonly access the class state. To achieve this, a deployment as shown in fig. 6 is performed.

1. The system consists of a server and a gateway, and a plurality of nodes are allowed to be deployed. The server is stateful and is used for realizing the functional requirements of the service scene; the gateways are stateless, and any gateway can forward messages to a server-specific node according to the partition key. Both the gateway and the server are inside the k8s cluster, so the gateway can establish a network connection to a designated server node without exposing the server pod to the public network.

2. The server workload is stateful set in type so that each pod has a fixed name within the cluster, such as server-0, server-1, and so on; the gateway can identify the pod of the server stateful set through DNS in the cluster, so as to select a node according to the partition key, and forward the message of the same partition key to the node. Wherein DNS in the cluster is a mechanism provided by k8s, and can use the pod name as a destination address (the address actually consists of pod, service, namespace, and the fields are known and not changed for the specific pod of the server) of the network connection (such as TCP) for establishing the connection. The mechanism is a k8s native mechanism, and the business side does not need to customize and develop any code when using the mechanism.

3. The gateway workload is of the type discover, a generic stateless service. The gateway exposes k8s services for clients to access, and the clients access any gateway through the services, and the gateway can forward the message to the correct node of the server.

In the system architecture, several node types of the server, the client and the gateway can be realized by using the real-time communication SDK. As shown in fig. 14, a transport is established between the server and the client, but a network connection (connection) is not directly established, but a connection is established with the gateway, respectively, as a means of transport message transmission. When the network connection fails, reconnection and other gateways can be connected; and the stream in the transport can complete the retransmission of the message through the new network connection, so that the reliable transmission between the server and the client can still be maintained. In this process, the gateway only performs network forwarding without understanding service logic or guaranteeing reliability, so that it can rely on connection components only, and does not need to rely on complete transport.

In some business scenarios, the server needs to send data to a large number of clients (referred to as "mass distribution"), and the system architecture in such scenarios can be further optimized. For example, a go game is live broadcast through a real-time communication system, two players connect the same node of the server to play games, and the server calculates the go rule and simultaneously live broadcast the two players to 9000 spectators. Such a scenario, if implemented simply using the architecture described above, requires the server to repeatedly send the same message content (player mover) to a large number of clients (spectators), with a large performance overhead. To solve this problem, as shown in fig. 15, the uplink and downlink of the system architecture may be designed differently, so that the downlink avoids repeated transmission of the same message as much as possible.

1. The ascending part of the system is used for realizing the game of players. A transport is established from a client (chess player) to a server, and the transport is transmitted through a gateway in the middle; the gateway relies only on connection and not on full transport.

2. The downstream part of the system is used to realize spectator sightseeing. Unlike the upstream part, the client (viewer) directly establishes a transport to the gateway (instead of the server), and the gateway then establishes a transport to the server; when the server distributes (rebroadcasts) the data, the gateway sends the information to the gateway of the opposite terminal through each transport, and then the gateway sends the information to the audience of the opposite terminal through each transport. Because the server only establishes the transport with the gateway, the number of the required transport is equal to the number of the gateway and is far less than the number of audiences, thereby reducing the sending times of the server during data distribution; and each gateway only establishes transport with a part of audience, so that load balancing is carried out among the gateways, and the sending times are limited within a reasonable range.

Assuming 3 gateways are deployed in the downstream portion of the system, each gateway needs to send live messages to 3000 viewers; the network transmission load of each node of the two architectures can be calculated, and the performance bottlenecks of the two architectures can be analyzed:

a. The basic architecture does not use large-scale distribution. The server establishes a transport with each client (audience), and needs to send a message to each transport in turn; the whole network has 9000 audiences in total, so the server needs to continuously send 9000 messages. As the number of viewers increases, the messaging load increases as the service assumes the full network of message distribution, which may become a performance bottleneck.

B. And (5) large-scale distribution. The server establishes a transport with each gateway, and also needs to send a message to each transport in turn; the downlink part has 3 gateways in total, so that the server needs to continuously send 3 messages, which is far lower than the basic architecture and has negligible load. As the number of viewers increases, the number of gateways in the system can be increased; the load of the single gateway is unchanged, and the load of the server side for sending messages to the gateways is negligible, so that each component in the system has no performance bottleneck. In the scenario of massive distribution, as shown in fig. 16, the server and the gateway only send one communication message, which greatly reduces the load compared with 3000 messages sent using the TCP protocol.

In this embodiment, the real-time communication SDK transport provides an extension point, and the internal components connection, stream are pluggable. The connection implementation may also enable SSL and web socket to support encryption, support browser access. TCP can be replaced by KCP, quic and other transmission layer protocols, so that higher availability and instantaneity are achieved. The message serialization mode in the connection component can be readable json text or a binary format such as protobuf so as to reduce the message length and reduce the network load. In addition, the stream may be implemented as a selective retransmission protocol, or may be implemented as follows:

a. for recovery of lost messages, FEC (forward error correction) mechanisms may be introduced. The FEC may include redundant data when sending a set of messages, and according to the redundant data, it is possible to directly calculate the lost message content without retransmission, so that the delay caused by retransmission is avoided.

B. For strong real-time scenarios, unreliable message transmission may be allowed, a portion of the message is allowed to be lost while retransmissions are reduced, sacrificing reliability in exchange for real-time. For example, in a 'multi-user whiteboard collaboration' business scenario, each path point traversed by a whiteboard brush may construct a message; message transmission may be unreliable without affecting the overall visual effect of the handwriting even if part of the waypoints are lost. Therefore, the stream of the scene can be customized and developed, and incomplete handwriting can be directly displayed without waiting for retransmission when the message is lost, so that the visual effect of the handwriting is smoother.

C. Aiming at congestion control, if SDK depends on controlled protocols such as TCP, stream only needs to realize a simple window mechanism; if the SDK relies on uncontrolled protocols such as UDP, then the team can introduce congestion control algorithms such as cubic, bbr, etc.

Besides the cluster deployment server, the server can be deployed on a physical machine. Physical machine deployment does not depend on kubernetes and other containerization technologies. The method is mainly used for off-line local area network communication scenes, such as on-line teaching indoor lessons. The present embodiment provides a set of real-time communication SDKs in the form of callable code rather than services, thus not restricting the deployment approach. The business side can develop a real-time communication service end and a client end based on the SDK and is installed on physical equipment (such as a personal computer, a mobile phone and a tablet) in the teaching room, and the communication is based on the local area network. The SDK connection component needs to rely on the lan address to establish a network connection, and may introduce a lan broadcast mechanism, so that the client automatically discovers the server address.

In this embodiment, for stateful services, the gateway is mainly used to forward the messages, which ensure that the messages with the same partition key are necessarily forwarded to the designated server node. While in certain traffic scenarios, if the state constraints are relaxed, the gateway may be replaced with kubernetes "session hold" feature. In some scenarios, the server has multiple nodes, but the nodes share data (e.g. access the same database together), so that any node of the client access to the server can realize the service function, and the selection of the server node is not restricted by partition keys. However, the server single node is stateful (e.g., SDK transport requires maintenance of message state to ensure reliability), and once the client selects a certain node on the server, the client must then repeatedly select the same node. In this scenario kubernetes session hold may be enabled instead of a gateway. Firstly, a service party deploys a service end node by kubernetes and creates service exposure public network access; the service enables session maintenance, so that the same node of the service is repeatedly accessed when the same client is reconnected. If it is assumed that a pod within the cluster cannot expose public network access, it is necessary to introduce gateway nodes for forwarding and expose services for clients to access. If the limitation is relaxed, the server-side pod is allowed to directly expose public network access, a gateway node is not needed, and only one http service is deployed as a directory for the client-side to query the address of the server-side pod according to the partition key. The http directory service belongs to a short connection query service, and is simpler to realize than a gateway because the http directory service does not need to maintain the state of network connection and also does not need to establish network connection with nodes in a cluster to communicate.

The embodiment also provides a real-time communication server and a real-time communication client, which are used for executing the real-time communication method applied to the server and the real-time communication method applied to the client.

As shown in fig. 17, an embodiment of the present application provides an electronic device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 perform communication with each other through the communication bus 114,

A memory 113 for storing a computer program;

In one embodiment of the present application, the processor 111 is configured to implement the real-time communication method provided in any one of the foregoing method embodiments when executing the program stored in the memory 113.

The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the real-time communication method provided by any one of the method embodiments described above.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus a general purpose hardware platform, or may be implemented by hardware. Based on such understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the related art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the method described in the respective embodiments or some parts of the embodiments.

It is to be understood that the terminology used herein is for the purpose of describing particular example embodiments only, and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises," "comprising," "includes," "including," and "having" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order described or illustrated, unless an order of performance is explicitly stated. It should also be appreciated that additional or alternative steps may be used.

The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A real-time communication method, wherein the application is at a server, comprising:

Invoking a server-side real-time communication SDK of the server to establish a reliable communication connection with a client-side real-time communication SDK of a client, wherein the reliable communication connection is used for reestablishing the bottom communication connection and retransmitting lost messages under the condition that the client-side is disconnected from the bottom communication connection of the server so as to ensure reliable transmission of communication messages;

and transmitting a communication message with the client through the established reliable communication connection, wherein the time for transmitting the communication message is not constrained.

2. The method of claim 1, wherein the server side real-time communication SDK includes a server side management module and a server side communication module, the server side management module includes a server side sending function and a server side receiving function, the client side real-time communication SDK includes a client side management module and a client side communication module, and the client side management module includes a client side sending function and a client side receiving function;

The calling the server-side real-time communication SDK of the server to establish reliable communication connection with the client-side real-time communication SDK of the client comprises the following steps:

The reliable communication connection is established between the server-side sending function and the client-side receiving function, and the reliable communication connection is established between the server-side receiving function and the client-side sending function, wherein the server-side communication module and the client-side communication module are used for establishing the bottom communication connection, the server-side sending function is used for sending communication information to the client-side receiving function through the bottom communication connection, and the server-side receiving function is used for receiving the communication information sent by the client-side sending function through the bottom communication connection.

3. The method according to claim 2, wherein the method further comprises:

Under the condition that the bottom communication connection is disconnected, reestablishing the bottom communication connection with a client management module of the client real-time communication SDK through the server management module of the server real-time communication SDK;

And retransmitting the communication message which is not successfully transmitted through the server management module.

4. The method of claim 3, wherein the server receiving function in the server management module is configured to perform at least one of an acknowledge operation, a request retransmission operation, a sort operation, and a deduplication operation on a communication message sent by the sending function of the client management module, in case that the communication message is received.

5. The method of claim 1, wherein before invoking the server-side real-time communication SDK of the server to establish a network communication connection with the client-side real-time communication SDK of the client, the method further comprises:

and inserting the SDK for real-time communication of the server in the server.

6. A method of real-time communication, applied to a client, comprising:

invoking a client-side real-time communication SDK of the client to establish a reliable communication connection with a server-side real-time communication SDK of a server, wherein the reliable communication connection is used for reestablishing the bottom communication connection and retransmitting lost messages under the condition that the server-side is disconnected from the bottom communication connection of the client so as to ensure reliable transmission of communication messages;

and transmitting a communication message with the server through the established reliable communication connection, wherein the time for transmitting the communication message is not restricted.

7. A real-time communication server, wherein the real-time communication server is configured to perform the real-time communication method according to any one of claims 1 to 5.

8. A real-time communication client, characterized in that the real-time communication client is adapted to perform the real-time communication method of claim 6.

9. An electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor coupled to the at least one bus; at least one memory connected to the at least one bus, wherein the memory stores a computer program, and the processor implements the real-time communication method of any one of claims 1-5 or 6 when executing the computer program.

10. A computer-readable storage medium storing computer-executable instructions for performing the real-time communication method of any one of the above claims 1-5 or 6 of the present application.