WO2023035779A1

WO2023035779A1 - Data transmission method, data processing method, sending end device, receiving end device, conference terminal, and computer-readable storage medium

Info

Publication number: WO2023035779A1
Application number: PCT/CN2022/106656
Authority: WO
Inventors: 李帆
Original assignee: 中兴通讯股份有限公司
Priority date: 2021-09-10
Filing date: 2022-07-20
Publication date: 2023-03-16
Also published as: CN115801739A

Abstract

Provided in the present disclosure is a data transmission method, which method is applied to a sending end device. The data transmission method comprises: establishing a plurality of media transmission links with a receiving end device, wherein the plurality of media transmission links correspond to various types of networks that a sending end device accesses; and by means of the plurality of media transmission links, transmitting data to be transmitted. Further provided in the present disclosure are a data processing method, a sending end device, a receiving end device, a conference terminal, and a computer-readable storage medium.

Description

Data transmission method, data processing method, sending end device, receiving end device, conference terminal, and computer-readable storage medium

Cross References to Related Applications

This application claims priority to Chinese Patent Application No. 202111062989.5 filed on September 10, 2021, the contents of which are hereby incorporated by reference in their entirety.

technical field

The present disclosure relates to the field of communication technologies, and in particular, to a data transmission method, a data processing method, a sending end device, a receiving end device, a conference terminal, and a computer-readable storage medium.

Background technique

With the development of web conferencing related technologies, the voice clarity and video resolution of web conferencing are rapidly improving, and the functions of web conferencing are becoming more and more abundant. For example, in some related technologies, web conferencing has supported virtual background, video editing and file sharing , Web conferencing functions based on virtual reality (VR, Virtual Reality) and augmented reality (AR, Augmented Reality) technologies are also under development.

However, limited by network conditions, there are still some deficiencies in effectively guaranteeing the quality of network meetings.

public content

In the first aspect, an embodiment of the present disclosure provides a data transmission method applied to a sending end device, including:

Establishing multiple media transmission links with the receiving end device, where the multiple media transmission links correspond to various networks accessed by the sending end device; and

The data to be transmitted is transmitted through multiple media transmission links.

In the second aspect, an embodiment of the present disclosure provides a data processing method applied to a receiving end device, including:

receiving the data packet to be processed through multiple media transmission links established with the sending end device, where the multiple media transmission links correspond to various networks accessed by the receiving end device; and

Process the data packets to be processed.

In a third aspect, an embodiment of the present disclosure provides a sender device, including:

one or more processors;

Memory, on which one or more computer programs are stored, when said one or more computer programs are executed by said one or more processors, said one or more processors implement the first aspect of the present disclosure The data transfer method provided by the example; and

One or more I/O interfaces, connected between the processor and the memory, configured to realize information exchange between the processor and the memory.

In a fourth aspect, an embodiment of the present disclosure provides a receiver device, including:

one or more processors;

Memory, on which one or more computer programs are stored, when the one or more computer programs are executed by the one or more processors, causing the one or more processors to implement the second aspect of the present disclosure The data processing method provided by the example; and

In a fifth aspect, an embodiment of the present disclosure provides a conference terminal, including:

one or more processors;

Memory, on which one or more computer programs are stored, when said one or more computer programs are executed by said one or more processors, said one or more processors implement the first aspect of the present disclosure At least one of the data transmission method provided by the example and the data processing method provided by the second aspect of the present disclosure; and

In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, at least one of the following methods is implemented:

The data transmission method provided by the embodiment of the present disclosure in the first aspect; and

The second aspect is the data processing method provided by the embodiments of the present disclosure.

Description of drawings

FIG. 1 is a flowchart of a data transmission method in an embodiment of the present disclosure;

FIG. 2 is a flowchart of some steps in a data transmission method in an embodiment of the present disclosure;

FIG. 3 is a flowchart of some steps in a data transmission method in an embodiment of the present disclosure;

Fig. 4 is a flowchart of some steps in a data transmission method in an embodiment of the present disclosure;

Fig. 5 is a flowchart of some steps in a data transmission method in an embodiment of the present disclosure;

FIG. 6 is a flowchart of a data processing method in an embodiment of the present disclosure;

FIG. 7 is a flowchart of some steps in a data processing method in an embodiment of the present disclosure;

Fig. 8 is a flowchart of some steps in a data processing method in an embodiment of the present disclosure;

FIG. 9 is a flowchart of some steps in a data processing method in an embodiment of the present disclosure;

FIG. 10 is a block diagram of a sending end device in an embodiment of the present disclosure;

FIG. 11 is a block diagram of a receiver device in an embodiment of the present disclosure;

FIG. 12 is a block diagram of a conference terminal in an embodiment of the present disclosure;

Fig. 13 is a composition block diagram of a computer-readable storage medium in an embodiment of the present disclosure;

FIG. 14 is a schematic diagram of a network architecture of a network meeting provided by an example of the present disclosure;

FIG. 15 is a schematic diagram of a conference terminal provided by an example of the present disclosure;

FIG. 16 is a schematic diagram of a process for establishing multiple media transmission links provided by an example of the present disclosure;

FIG. 17 is a schematic flow diagram of sending media data by a conference terminal at the sending end provided by an example of the present disclosure; and

Fig. 18 is a schematic flowchart of receiving and processing media data by a conference terminal at a receiving end provided by an example of the present disclosure.

Detailed ways

In order for those skilled in the art to better understand the technical solution of the present disclosure, the data transmission method, data processing method, sending end device, receiving end device, conference terminal and computer-readable storage medium provided by the present disclosure are described below in conjunction with the accompanying drawings A detailed description.

Example embodiments will be described more fully hereinafter with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and the disclosure should not be construed as limited to the embodiments set forth herein. These embodiments are provided to make the present disclosure more thorough and complete, and to enable those skilled in the art to fully understand the scope of the present disclosure.

In the case of no conflict, various embodiments of the present disclosure and various features in the embodiments can be combined with each other.

As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The terms used herein are for describing particular embodiments only, and do not limit the present disclosure. As used herein, the singular forms "a" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that when the terms "comprising" and/or "consisting of" are used in this specification, the presence of specific features, integers, steps, operations, elements and/or components is specified but not excluded or may be present. Add one or more other features, integers, steps, operations, elements, components and/or groups thereof.

Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant art and the present disclosure, and will not be interpreted as having idealized or excessive formal meanings unless This article expressly so limits.

Packet loss and delay are important factors affecting the quality of online conferences. As network conditions deteriorate, packet loss and delay will become more serious. Some related technologies can solve the packet loss problem when the network condition is bad, so as to guarantee the quality of the network meeting.

For example, in some related technologies, the packet loss problem is solved by packet loss retransmission and forward error correction (FEC, Forward Error Correction). Packet loss retransmission means that the sending end device caches data packets and retransmits the lost data packets when it is notified that the data packets are lost; forward error correction refers to carrying FEC redundant packets while sending data packets, and the receiving end detects When the packet is lost, some data packets can be recovered from the FEC redundant packet. However, both packet loss retransmission and forward error correction can only solve part of the packet loss problem, and are still insufficient in solving the delay problem.

In a first aspect, referring to FIG. 1 , an embodiment of the present disclosure provides a data transmission method applied to a sending end device, including steps S100 and S200.

S100. Establish multiple media transmission links with the receiving end device, and the multiple media transmission links correspond to various networks accessed by the sending end device.

S200. Transmit data to be transmitted through multiple media transmission links.

It should be noted that, in the data transmission method provided in the present disclosure, the sending end device refers to a device that sends data during a data transmission process; the receiving end device refers to a device that receives data during a data transmission process. During a complete data transfer, the same device can be both a sender and a receiver. In some implementation manners, the sending end device and the receiving end device are conference terminals.

In the data transmission method provided in the present disclosure, the media transmission link is used to transmit media data between the sending end device and the receiving end device. The media data may refer to audio data, may also be video data, and may also be audio and video data. The present disclosure makes no special limitation on this. In some implementations, the media transmission link can also be used to transmit data other than media data, such as text and documents.

In the data transmission method provided in this disclosure, both the sending end device and the receiving end device have multiple network communication capabilities, that is, both the sending end device and the receiving end device can access various networks, and the sending end device and the receiving end device are respectively connected to Into any kind of network, the data sent by the sending end device can finally reach the receiving end device. The present disclosure makes no special limitation on various networks. For example, the various networks include at least two of wired communication networks, wireless communication networks, and mobile communication networks. It should be noted that, in the data transmission method provided in the present disclosure, the network resources of various networks accessed by the sending end device are independent of each other, and sending data through different networks will not compete for network resources.

In the data transmission method provided in this disclosure, multiple media transmission links corresponding to multiple networks means that in the sending end device, one media transmission link is established on a network connected to the sending end device, and multiple media transmission links The media transmission links are respectively established on various networks accessed by the sending end device. In the data transmission method provided in the present disclosure, in the receiving end device, the multiple media transmission links also correspond to various networks accessed by the receiving end device. The network corresponding to the same media transmission link in the sending end device and the corresponding network in the receiving end device may be the same or different, which is not specifically limited in the present disclosure.

It should also be noted that steps S100 and S200 are executed at the kernel level, and the application layer and the user are not aware of the process of establishing multiple media transmission links and transmitting data through multiple media transmission links.

In the data transmission method provided by the present disclosure, multiple media transmission links established on various networks serve as mutual backups, and when transmitting data through multiple media transmission links, the multiple media transmission links will not compete with each other for network resources , when transmitting data, there will always be a media transmission link with the smallest delay among the multiple media transmission links, which means that data can always be transmitted through the media transmission link with the There are other media transmission links to supplement the package. When the data transmission method is applied to a network conference, it can greatly shorten the delay and avoid packet loss, thereby effectively ensuring the quality of the network conference and improving user experience.

In some implementations, referring to FIG. 2 , step S200 includes steps S210 and S220.

S210. Add a label field to the data packet of the data to be transmitted, and the label fields of different data packets are different.

S220. Copy the data packet added with the label field to each of the media transmission links for transmission.

In the data transmission method provided by the present disclosure, the tag field is used to uniquely identify the data packet. The present disclosure makes no special limitation on the label field. For example, the sending end device numbers the data packets, and the sequence number of each data packet is the label field of the data packets.

In the data transmission method provided by the present disclosure, a label field is added to the data packet, and the data packet with the label field added is copied to each media transmission link for transmission, so that the receiving end device receives the data packet from multiple media transmission links Afterwards, the data packets received repeatedly can be screened out according to the label fields of the data packets, that is, for the same data packets in multiple media transmission links, the receiving end device only selects the one that arrives first, and the repeated data packets received subsequently will be discarded to shorten the delay.

It should be noted that, in the data transmission method provided in the present disclosure, the application layer is not aware of the establishment of multiple media transmission links and the process of transmitting data through the multiple media transmission links. When the media data needs to be sent, the application layer writes the data to be transmitted into the sending buffer, and sends a socket (socket) call signal to the kernel; the kernel responds to the socket sending call signal to obtain the data to be transmitted from the send buffer .

In some implementations, a label field is added to the packet header of the data packet.

In some implementations, referring to FIG. 3 , step S210 includes steps S211 and S212.

S211. Determine whether the data packet of the data to be transmitted is a media data packet.

S212. When the data packet of the data to be transmitted is a media data packet, add a tag field to the data packet of the data to be transmitted.

In the data transmission method provided in the present disclosure, the media data packet may refer to an audio data packet, may also be a video data packet, and may also be an audio and video data packet. The present disclosure makes no special limitation on this.

It should be noted that during the online conference, in addition to transmitting media data that is sensitive to delay and packet loss, non-media data that is relatively insensitive to delay and packet loss, such as text and documents, can also be transmitted. Through steps S211 and S212, when the data to be transmitted is media data, the media data is copied to multiple media transmission links for transmission; when the data to be transmitted is non-media data, through one of the multiple media transmission links The transmission of non-media data can save network resources as a whole, avoid network congestion caused by the transmission of non-media data, and further guarantee the quality of network meetings.

In some implementations, referring to FIG. 4 , step S100 includes steps S111 to S113.

S111. Establish a first link with the receiving end device.

S112. Acquire network configuration information of the receiving end device through the first link.

S113. In the case that the receiving end device accesses multiple networks, establish at least one second link according to the network configuration information of the sending end device and the network configuration information of the receiving end device, and the multiple The media transmission link includes the first link and the at least one second link.

In some embodiments, the first link is established at the application layer, and the second link is established at the kernel level. In some implementation manners, the sending end device establishes the first link with the receiving end device using a primary Internet Protocol address (IP, Internet Protocol).

In some implementations, referring to FIG. 4 , after step S111 , step S100 further includes step S114 .

S114. Transmit the network configuration information of the sending-end device to the receiving-end device through the first link.

The network configuration information of the sending end device is sent to the receiving end device through step S114, so that the receiving end device can establish multiple media transmission links with the sending end device according to the network configuration information of the sending end device.

In some implementations, referring to FIG. 5 , step S100 includes steps S121 and S122.

S121. Create multiple sockets to establish multiple media transmission links with the receiving device.

S122. Bind the multiple sockets in a one-to-one correspondence with multiple network cards used to access various networks in the sending end device.

It should be noted that using multiple sockets to establish multiple media transmission links with the receiving end device can make the application layer unaware of the creation of multiple media transmission links and the process of transmitting data through multiple media transmission links .

In some implementation manners, the various networks accessed by the sending end device include at least two of a wired communication network, a wireless communication network, and a mobile communication network.

In a second aspect, referring to FIG. 6 , an embodiment of the present disclosure provides a data processing method applied to a receiving end device, including steps S300 and S400.

S300. Receive data packets to be processed through multiple media transmission links established with the sending end device, where the multiple media transmission links correspond to various networks accessed by the receiving end device.

S400. Process the data packets to be processed.

It should be noted that, in the data processing method provided in the present disclosure, the sending device refers to a device that sends data during a data transmission; the receiving device refers to a device that receives data during a data transmission. During a complete data transfer, the same device can be both a sender and a receiver. In some implementation manners, the sending end device and the receiving end device are conference terminals.

In the data processing method provided in the present disclosure, the media transmission link is used to transmit media data between the sending end device and the receiving end device. The media data may refer to audio data, may also be video data, and may also be audio and video data. The present disclosure makes no special limitation on this. In some implementations, the media transmission link can also be used to transmit data other than media data, such as text and documents.

In the data processing method provided in this disclosure, both the sending end device and the receiving end device have multiple network communication capabilities, that is, both the sending end device and the receiving end device can access various networks, and the sending end device and the receiving end device are respectively connected to Into any kind of network, the data sent by the sending end device can finally reach the receiving end device. The present disclosure makes no special limitation on various networks. For example, the various networks include at least two of a wired communication network, a wireless communication network, and a mobile communication network. It should be noted that, in the data processing method provided in the present disclosure, the network resources of various networks accessed by the sending end device are independent of each other, and sending data through different networks will not compete for network resources.

In the data processing method provided in the present disclosure, multiple media transmission links corresponding to multiple networks means that in the receiving end device, one media transmission link is established on a network accessed by the receiving end device, and multiple media transmission links The media transmission links are respectively established on various networks accessed by the receiving end device. In the data processing method provided in the present disclosure, in the sender device, the multiple media transmission links also correspond to various networks accessed by the sender device. The network corresponding to the same media transmission link in the sending end device and the corresponding network in the receiving end device may be the same or different, which is not specifically limited in the present disclosure.

It should also be noted that step S300 is executed at the kernel level, and the application layer and the user are unaware of the process of receiving data packets to be processed through multiple media transmission links.

In the data processing method provided by the present disclosure, multiple media transmission links established on various networks serve as backups for each other, and multiple media transmission links will not compete with each other when receiving data packets to be processed through multiple media transmission links resources, when receiving data packets to be processed, there will always be a media transmission link with the smallest delay among the multiple media transmission links, which means that the data packets to be processed can always be received through the media transmission link with the smallest relative delay. And when a media transmission link loses packets, there are other media transmission links to supplement it. When the data processing method is applied to a network conference, it can greatly shorten the delay and avoid packet loss, thereby effectively ensuring the quality of the network conference and improving user experience.

In some implementation manners, the receiving end device receives data packets from multiple media transmission links according to the principle of "whoever arrives first receives". The principle of "whoever arrives first" means that the sending device copies the data packets to each media transmission link for transmission, and the receiving device filters out the duplicate received data packets after receiving data packets from multiple media transmission links , that is, for the same data packet in multiple media transmission links, the receiving end device only selects the one that arrives first, and discards the repeated data packets received subsequently, so that the delay can be shortened.

Correspondingly, in some implementations, referring to FIG. 7 , step S400 includes steps S410 to S430.

S410. Extract the label field of the data packet to be processed, and the label fields of different data packets to be processed are different.

S420. If the label field of the data packet to be processed is the same as the label field of any received data packet, discard the data packet to be processed.

S430. In a case where the label field of the data packet to be processed is different from the label field of each received data packet, perform application layer processing on the data packet to be processed.

In the data processing method provided by the present disclosure, the tag field is used to uniquely identify the data packet, and the receiving end device can screen out the repeatedly received data packets according to the tag field. The present disclosure makes no special limitation on the label field. For example, the sending end device numbers the data packets, and the sequence number of each data packet is the label field of the data packets.

In some implementations, performing application layer processing on the pending data packet includes: removing the label field of the pending data packet, and sending the pending data packet from which the label field is removed to the application layer program for further processing.

In the data processing method provided by the present disclosure, during the network conference, in addition to transmitting media data that is sensitive to delay and packet loss, the sending device can also transmit non-media data that is relatively insensitive to delay and packet loss, for example text, documents, etc. In some embodiments, the sending end device only adds a label field to the media data packet, and copies it to multiple media transmission links for transmission; non-media data does not add a label field, and passes through one of the multiple media transmission links to transfer.

In the data processing method provided in the present disclosure, the media data may refer to audio data packets, video data packets, or audio and video data packets. The present disclosure makes no special limitation on this.

Correspondingly, in some implementations, referring to FIG. 8 , step S410 includes steps S411 and S412.

S411. Determine whether the data packet to be processed is a media data packet.

S412. If the data packet to be processed is a media data packet, extract a label field of the data packet to be processed.

In some embodiments, the sending end device first establishes a first link with the receiving end device, obtains the network configuration information of the receiving end device through the first link, and transmits the network configuration information of the sending end device to the receiving end device through the first link configuration information.

Correspondingly, in some implementations, referring to FIG. 9 , before step S300, the data processing method further includes steps S501 and S502.

S501. Receive network configuration information of the sending end device.

S502. Establish multiple media transmission links with the sending device according to the network configuration information of the sending device and the network configuration information of the receiving device.

In a third aspect, referring to FIG. 10 , an embodiment of the present disclosure provides a sender device, including one or more processors 101 (only one is shown in FIG. 10 ), memory 102, and one or more I/O interfaces 103 ( Only one is shown in Figure 10).

One or more computer programs are stored on the memory 102, and when the one or more computer programs are executed by the one or more processors 101, the one or more processors 101 realize the first aspect of the present disclosure The data transmission method provided by the embodiment.

The one or more I/O interfaces 103 are connected between the processor 101 and the memory 102 and are configured to realize information exchange between the processor 101 and the memory 102 .

Processor 101 is a device with data processing capability, including but not limited to central processing unit (CPU) etc.; memory 102 is a device with data storage capability, including but not limited to random access memory (RAM, more specifically SDRAM, DDR etc.), read-only memory (ROM), electrified erasable programmable read-only memory (EEPROM), flash memory (FLASH); I/O interface (read-write interface) 103 is connected between processor 101 and memory 102, can realize processing Information interaction between the device 101 and the memory 102, including but not limited to a data bus (Bus).

In some implementations, the processor 101 , the memory 102 and the I/O interface 103 are connected to each other through the bus 104 , and are further connected to other components of the computing device.

In the fourth aspect, referring to FIG. 11 , an embodiment of the present disclosure provides a receiver device, including one or more processors 201 (only one is shown in FIG. 11 ), memory 202, and one or more I/O interfaces 203 ( Only one is shown in Figure 11).

One or more computer programs are stored on the memory 202, and when the one or more computer programs are executed by the one or more processors 201, the one or more processors 201 realize the second aspect of the present disclosure The data processing method provided by the embodiment.

The one or more I/O interfaces 203 are connected between the processor 201 and the memory 202 and are configured to realize information exchange between the processor 201 and the memory 202 .

Processor 201 is a device with data processing capability, including but not limited to central processing unit (CPU) etc.; memory 202 is a device with data storage capability, including but not limited to random access memory (RAM, more specifically SDRAM, DDR etc.), read-only memory (ROM), electrified erasable programmable read-only memory (EEPROM), flash memory (FLASH); I/O interface (read-write interface) 203 is connected between processor 201 and memory 202, can realize processing Information interaction between the device 201 and the memory 202, including but not limited to a data bus (Bus).

In some implementations, the processor 201 , the memory 202 and the I/O interface 203 are connected to each other through the bus 204 , and further connected to other components of the computing device.

In the fifth aspect, referring to FIG. 12 , an embodiment of the present disclosure provides a conference terminal, including one or more processors 301 (only one is shown in FIG. 12 ), memory 302, and one or more I/O interfaces 303 (FIG. 12 only one is shown).

One or more computer programs are stored on the memory 302, and when the one or more computer programs are executed by the one or more processors 301, the one or more processors 301 implement the first aspect of the present disclosure The data transmission method provided by the embodiments and the second aspect At least one of the data processing methods provided by the embodiments of the present disclosure.

The one or more I/O interfaces 303 are connected between the processor 301 and the memory 302 and are configured to implement information exchange between the processor 301 and the memory 302 .

Processor 301 is a device with data processing capability, including but not limited to central processing unit (CPU) etc.; memory 302 is a device with data storage capability, including but not limited to random access memory (RAM, more specifically SDRAM, DDR etc.), read-only memory (ROM), electrified erasable programmable read-only memory (EEPROM), flash memory (FLASH); I/O interface (read-write interface) 303 is connected between processor 301 and memory 302, can realize processing Information interaction between the device 301 and the memory 302, including but not limited to a data bus (Bus).

In some implementations, the processor 301 , the memory 302 and the I/O interface 303 are connected to each other through the bus 304 , and further connected to other components of the computing device.

In the sixth aspect, referring to FIG. 13 , an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, at least one of the following methods is implemented:

In order to enable those skilled in the art to understand the technical solutions provided by the embodiments of the present disclosure more clearly, the technical solutions provided by the embodiments of the present disclosure will be described in detail below through specific examples.

Fig. 14 is a schematic diagram of the network architecture of the network meeting provided by the disclosed example.

In this example, the conference terminals support three types of network communication capabilities: wired network, wireless network, and 5G network. The conference terminal can be connected to any network, and the data sent by the conference terminal at the sending end can reach the conference terminal at the receiving end.

In this example, three media transmission links are established between conference terminals based on three types of networks: wired network, wireless network and 5G network: media transmission link 1, media transmission link 2 and media transmission link 3.

Fig. 15 is a schematic structural diagram of a conference terminal provided by an example of the present disclosure.

The application layer of the conference terminal includes a media module configured to process media data and non-media data, and the application layer sends the code stream to the kernel of the conference terminal through the User Datagram Protocol (UDP, User Datagram Protocol).

The kernel of the conference terminal includes a transmission control protocol (TCP, Transmission Control Protocol) stack, a link building module, and a distribution module. The link building module is configured to create multiple media transmission links, and the distribution module is configured to add a label field to the data packet, copy the data packet, and distribute the data packet to multiple media transmission links for forwarding.

The physical layer of the conference terminal includes wired network card, wireless network card, and 5G module. The kernel establishes media transmission link 1, media transmission link 2, and media transmission link 3 through the wired network card, wireless network card, and 5G module of the physical layer.

Fig. 16 is a schematic diagram of a process for establishing multiple media transmission links provided by an example of the present disclosure. As shown in FIG. 16, the process of establishing multiple media transmission links includes steps E101 to E103.

E101. The application layer media module in the conference terminal at the sending end initiates the establishment of the first link (main channel), and obtains the network communication capabilities of the conference terminal at the receiving end through the main channel, for example, whether the conference terminal at the receiving end supports wired networks, wireless networks, 5G network.

E102. Determine whether the wired network, wireless network, and 5G network of the conference terminal at the sending end are available; if at least two of the wired network, wireless network, and 5G network are available, execute E103; otherwise, interact with the conference terminal at the receiving end through the main channel .

E103. Determine whether the conference terminal at the receiving end supports at least two of a wired network, a wireless network, and a 5G network; The conference terminal establishes multiple media transmission links; otherwise, it interacts with the conference terminal at the receiving end through the main channel.

Fig. 17 is a schematic flowchart of sending media data by a conference terminal at a sending end provided by an example of the present disclosure. As shown in FIG. 17 , the process of sending media data by the conference terminal at the sending end includes steps E201 to E204.

E201. The application layer sends the media data to the kernel protocol stack for processing.

E202. The kernel judges whether multiple media transmission links have been established between the current sending end and the receiving end; if multiple media transmission links have not been established, then initiate the establishment of multiple media transmission links; if multiple media transmission links have been established If there is a path, number each data packet of the media data, and add a sequence number to the header of the data packet.

E203. Multiple copies of the data packet added with the sequence number are copied, and forwarded through multiple media transmission links respectively.

E204. The wired network card, wireless network card, and 5G module of the physical layer of the sending end send the data packet with the added serial number to the network corresponding to each media transmission link, and complete the sending task of the conference terminal at the sending end.

Fig. 18 is a schematic flowchart of receiving and processing media data by a conference terminal at a receiving end provided by an example of the present disclosure. As shown in FIG. 18 , the process of receiving and processing media data by the conference terminal at the receiving end includes steps E301 to E303 .

E301, the wired network card, wireless network card, and 5G module of the physical layer of the receiving end receive media data successively, and then send the data up to the kernel protocol stack for processing;

E302, after the kernel protocol stack receives the media data, it parses the serial number in the header of the data packet to determine whether the data packet with the same serial number has been received; if the data packet with the same serial number has been received, discard the data packet and continue Wait for the next data packet; otherwise, send the data packet up to the application layer program for the next step of processing;

E303. After the application layer program (mainly the media module) receives the data packet, it performs corresponding decoding and display processing. For the application layer, the process of receiving multiple copies of the same data packet through multiple media transmission links at the kernel level No perception.

Those of ordinary skill in the art can understand that all or some of the steps in the method disclosed above and the functional modules/units in the device can be implemented as software, firmware, hardware and an appropriate combination thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components. Components cooperate to execute. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Example embodiments have been disclosed herein, and while specific terms have been employed, they are used and should be construed in a generic descriptive sense only and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics and/or elements described in connection with a particular embodiment may be used alone or in combination with features, characteristics described in connection with other embodiments, unless expressly stated otherwise. and/or elements in combination. Accordingly, it will be understood by those of ordinary skill in the art that various changes in form and details may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

A data transmission method applied to a sending device, including:

Establishing multiple media transmission links with the receiving end device, where the multiple media transmission links correspond to various networks accessed by the sending end device; and

The data to be transmitted is transmitted through multiple media transmission links.
The data transmission method according to claim 1, wherein said transmitting the data to be transmitted through a plurality of said media transmission links comprises:

Adding a label field to the data packet of the data to be transmitted, the label fields of different data packets are different; and

Copying the data packet added with the label field to each of the media transmission links for transmission.
The data transmission method according to claim 2, wherein said adding a label field to the data packet of the data to be transmitted comprises:

judging whether the data packet of the data to be transmitted is a media data packet; and

When the data packet of the data to be transmitted is a media data packet, a label field is added to the data packet of the data to be transmitted.
The data transmission method according to any one of claims 1 to 3, wherein said establishing multiple media transmission links with the receiving end device comprises:

establishing a first link with the receiving end device;

Obtaining network configuration information of the receiver device through the first link; and

When the receiving end device accesses multiple networks, at least one second link is established according to the network configuration information of the sending end device and the network configuration information of the receiving end device, and the multiple media transmissions The links include the first link and the at least one second link.
The data transmission method according to claim 4, wherein said setting up multiple media transmission links with the receiving end device also includes:

After the first link is established with the receiving end device, the network configuration information of the sending end device is transmitted to the receiving end device through the first link.
The data transmission method according to any one of claims 1 to 3, wherein said establishing multiple media transmission links with the receiving end device comprises:

creating a plurality of sockets to establish a plurality of media transmission links with the receiving device; and

Binding the multiple sockets in one-to-one correspondence with multiple network cards used to access various networks in the sending end device.
The data transmission method according to any one of claims 1 to 3, wherein the various networks accessed by the sending end device include at least two of a wired communication network, a wireless communication network, and a mobile communication network.
A data processing method applied to a receiving end device, comprising:

receiving the data packet to be processed through multiple media transmission links established with the sending end device, where the multiple media transmission links correspond to various networks accessed by the receiving end device; and

Process the data packets to be processed.
The data processing method according to claim 8, wherein said processing said data packet to be processed comprises:

Extracting the label fields of the data packets to be processed, wherein the label fields of different data packets to be processed are different;

In the case that the label field of the data packet to be processed is the same as the label field of any received data packet, discarding the data packet to be processed; and

In a case where the label field of the data packet to be processed is different from the label field of each received data packet, application layer processing is performed on the data packet to be processed.
The data processing method according to claim 9, wherein said extracting the label field of said data packet to be processed comprises:

judging whether the data packet to be processed is a media data packet; and

In the case that the data packet to be processed is a media data packet, the label field of the data packet to be processed is extracted.
The data processing method according to any one of claims 8 to 10, further comprising:

Before receiving the data packets to be processed through the multiple media transmission links established with the sender device, receiving network configuration information of the sender device; and

Establishing multiple media transmission links with the sender device according to the network configuration information of the sender device and the network configuration information of the receiver device.
A sender device, comprising:

at least one processor;

memory having stored thereon at least one computer program which, when executed by said at least one processor, causes said at least one processor to implement data according to any one of claims 1 to 7 method of transmission; and

At least one I/O interface is connected between the processor and the memory and is configured to realize information exchange between the processor and the memory.
A receiver device, comprising:

at least one processor;

memory having stored thereon at least one computer program which, when executed by said at least one processor, causes said at least one processor to implement data according to any one of claims 8 to 11 the method of treatment; and

At least one I/O interface is connected between the processor and the memory and is configured to realize information exchange between the processor and the memory.
A conference terminal, comprising:

at least one processor;

memory having stored thereon at least one computer program which, when executed by said at least one processor, causes said at least one processor to implement data according to any one of claims 1 to 7 at least one of a transmission method and a data processing method according to any one of claims 8 to 11; and

At least one I/O interface is connected between the processor and the memory and is configured to realize information exchange between the processor and the memory.
A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, at least one of the following methods is implemented:

A data transmission method according to any one of claims 1 to 7; and

The data processing method according to any one of claims 8-11.