CN116668365A

CN116668365A - Audio and video data transmission system and method based on multi-network combination

Info

Publication number: CN116668365A
Application number: CN202310402755.3A
Authority: CN
Inventors: 龚谏君; 邓薇; 王锦
Original assignee: Zhongchuang Technology Group Ltd
Current assignee: Zhongchuang Technology Group Ltd
Priority date: 2023-04-11
Filing date: 2023-04-11
Publication date: 2023-08-29
Anticipated expiration: 2043-04-11
Also published as: CN116668365B

Abstract

The invention provides an audio and video data transmission system and method based on multi-network combination, comprising a first intelligent terminal, a second intelligent terminal, a plurality of communication gateways, a plurality of Internet of things servers for managing the first intelligent terminal and/or the second intelligent terminal and a cloud server for managing the plurality of Internet of things servers; the first intelligent terminal comprises a plurality of first broadband/narrowband communication modules, a first positioning module, a first detection module and a first control module; the communication gateway includes a plurality of wideband/narrowband communication channels and a channel management module. According to the attribute information of the audio data and the video data of the data to be transmitted, the corresponding transmission paths are selected from a plurality of communication gateways and a plurality of broadband/narrowband communication channels on the communication gateways, and the audio/video data are transmitted by selecting different transmission paths according to different characteristics, so that the transmission range of narrowband communication is enlarged, and the communication efficiency and the utilization rate of communication resources are improved.

Description

Audio and video data transmission system and method based on multi-network combination

Technical Field

The invention relates to the technical field of communication, in particular to an audio and video data transmission system and method based on multi-network combination.

Background

Private network refers to a private network, a network within a certain system, which serves only that system, e.g., railway system private network, flood control private network, military private network, etc. The narrow-band private network communication equipment has wide application in the fields of public safety, public utilities and the like, and is necessary equipment for commanding, dispatching and emergency communication of industry users. However, with the development of private network industry, private network users have increasingly demanded to provide reliable voice services, and simultaneously, large data services such as video monitoring and multimedia broadcasting have also been demanded. The traditional narrow-band system is limited by a standard and a bandwidth, has limited data transmission capability and limited application, and particularly, how to combine a broadband system and a narrow-band system to carry out high-efficiency transmission of audio and video data in the transmission process of the audio and video data is a problem to be solved.

Disclosure of Invention

The invention is based on the above problems, and provides an audio/video data transmission system and method based on multi-network association.

In view of this, an aspect of the present invention proposes an audio/video data transmission system based on multi-network association, including: the system comprises a first intelligent terminal, a second intelligent terminal, a plurality of communication gateways, a plurality of Internet of things servers for managing the first intelligent terminal and/or the second intelligent terminal and cloud servers for managing the plurality of Internet of things servers; the first intelligent terminal comprises a plurality of first broadband communication modules, a plurality of first narrowband communication modules, a first positioning module for acquiring position information, a first detection module and a first control module; the second intelligent terminal comprises a plurality of second broadband communication modules, a plurality of second narrowband communication modules, a second positioning module for acquiring position information, a second detection module and a second control module; the communication gateway comprises a plurality of broadband communication channels, a plurality of narrowband communication channels and a channel management module; wherein,

the first intelligent terminal is configured to:

collecting first audio data and first video data;

when the first audio data and the first video data need to be transmitted to the second intelligent terminal, a first data transmission instruction is generated by the first control module, and the first positioning module is activated; the first data transmission instruction comprises first attribute information of the first audio data to be transmitted, second attribute information of the first video data and second related information belonging to the second intelligent terminal; the first/second attribute information includes, but is not limited to, data type, data size, data security level, and data transmission aging requirements; the second related information includes, but is not limited to, a unique identifier of the second intelligent terminal;

The first positioning module is configured to:

acquiring first position information of the first intelligent terminal;

the first detection module is configured to:

searching a first communication gateway of the plurality of communication gateways and a first Internet of things server of the plurality of Internet of things servers according to the first data transmission instruction and the first position information;

the first control module is configured to: the first data transmission instruction is sent to the first Internet of things server;

the first internet of things server is configured to:

receiving the first data transmission instruction sent by the first intelligent terminal;

respectively determining a first transmission requirement for transmitting the first audio data and a second transmission requirement for transmitting the first video data according to the first data transmission instruction, and sending the first transmission requirement and the second transmission requirement to the first intelligent terminal; the first transmission requirement comprises information of a second communication gateway in the communication gateways, and the second transmission requirement comprises information of a third communication gateway in the communication gateways;

the first control module is configured to:

transmitting the first transmission requirement and the second related information to the second communication gateway, and transmitting the second transmission requirement and the second related information to the third communication gateway;

Establishing a connection with the second communication gateway and the third communication gateway;

transmitting the first audio data to the second intelligent terminal by using the second communication gateway;

and transmitting the first video data to the second intelligent terminal by using the third communication gateway.

Optionally, the first channel management module of the second communication gateway is configured to:

determining a first wideband communication channel from a plurality of wideband communication channels in the second communication gateway according to the first transmission requirement, and determining a first narrowband communication channel from a plurality of narrowband communication channels in the second communication gateway;

establishing a first broadband transmission connection between the first intelligent terminal and the first Internet of things server by utilizing the first broadband communication channel;

establishing a first narrowband transmission connection between the first intelligent terminal and the first Internet of things server by using the first narrowband communication channel;

the first control module is configured to:

and sending the first audio data to the first internet of things server through the first broadband transmission connection and the first narrowband transmission connection.

The invention further provides an audio and video data transmission method based on multi-network association, which is applied to an audio and video data transmission system based on multi-network association, wherein the audio and video data transmission system based on multi-network association comprises: the system comprises a first intelligent terminal, a second intelligent terminal, a plurality of communication gateways, a plurality of Internet of things servers for managing the first intelligent terminal and/or the second intelligent terminal and cloud servers for managing the plurality of Internet of things servers; the first intelligent terminal comprises a plurality of first broadband communication modules, a plurality of first narrowband communication modules, a first positioning module for acquiring position information, a first detection module and a first control module; the second intelligent terminal comprises a plurality of second broadband communication modules, a plurality of second narrowband communication modules, a second positioning module for acquiring position information, a second detection module and a second control module; the communication gateway comprises a plurality of broadband communication channels, a plurality of narrowband communication channels and a channel management module; the audio and video data transmission method based on the multi-network association comprises the following steps:

The first intelligent terminal collects first audio data and first video data;

the first positioning module acquires first position information of the first intelligent terminal;

the first detection module searches a first communication gateway in the plurality of communication gateways and a first Internet of things server in the plurality of Internet of things servers according to the first data transmission instruction and the first position information;

the first control module sends the first data transmission instruction to the first Internet of things server;

The first internet of things server receives the first data transmission instruction sent by the first intelligent terminal;

the first control module sends the first transmission requirement and the second related information to the second communication gateway, and sends the second transmission requirement and the second related information to the third communication gateway;

Optionally, the audio and video data transmission method based on multi-network association further comprises the following steps:

A first channel management module of the second communication gateway determines a first wideband communication channel from a plurality of wideband communication channels in the second communication gateway according to the first transmission requirement, and determines a first narrowband communication channel from a plurality of narrowband communication channels in the second communication gateway;

the first control module sends the first audio data to the first internet of things server through the first broadband transmission connection and the first narrowband transmission connection.

a second channel management module of the third communication gateway determines a second wideband communication channel from a plurality of wideband communication channels in the third communication gateway according to the second transmission requirement, and determines a second narrowband communication channel from a plurality of narrowband communication channels in the third communication gateway;

Establishing a second broadband transmission connection between the first intelligent terminal and the first Internet of things server by using the second broadband communication channel;

establishing a second narrowband transmission connection between the first intelligent terminal and the first Internet of things server by using the second narrowband communication channel;

the first control module sends the first video data to the first internet of things server through the second broadband transmission connection and the second narrowband transmission connection.

the first internet of things server sends the first data transmission instruction to the cloud server;

the cloud server receives the first data transmission instruction;

determining a second internet server for managing the second intelligent terminal according to the second related information in the first data transmission instruction;

determining address information of the second intelligent terminal from the second internet server according to the second related information;

receiving the first audio data and the first video data sent by the first Internet of things server;

and transmitting the first audio data and the first video data to the second internet server;

The second internet server determines a fourth communication gateway and a fifth communication gateway from a plurality of communication gateways according to the address information, and establishes a fourth data transmission connection with the fourth communication gateway and a fifth data transmission connection with the fifth communication gateway;

receiving the first audio data and the first video data;

transmitting the first audio data to the fourth communication gateway using the fourth data transmission connection and transmitting the first video data to the fifth communication gateway using the fifth data transmission connection;

the fourth communication gateway establishes a sixth data transmission connection with the second intelligent terminal and transmits the first audio data to the second intelligent terminal through the sixth data transmission connection;

and the fifth communication gateway establishes a seventh data transmission connection with the second intelligent terminal and transmits the first video data to the second intelligent terminal through the seventh data transmission connection.

Optionally, the step of determining a first wideband communication channel from a plurality of wideband communication channels in the second communication gateway according to the first transmission requirement, and determining a first narrowband communication channel from a plurality of narrowband communication channels in the second communication gateway includes:

The first channel management module extracts a first audio data type, a first audio data size, a first audio data security level and a first audio data transmission timeliness requirement of the first audio data from the first transmission requirement;

splitting a first group of data which needs to be transmitted by utilizing a narrow-band communication channel in the second communication gateway from the first audio data according to the first audio data type, the first audio data size, the first audio data security level and the first audio data transmission timeliness requirement, and taking the rest of data as a second group of data which is transmitted by utilizing a broadband communication channel in the second communication gateway;

determining a first narrowband communication channel from a plurality of narrowband communication channels in the second communication gateway according to a first data type, a first data size, a first data security level and a first data transmission timeliness requirement of the first group of data and communication states of the plurality of narrowband communication channels in the second communication gateway;

determining a first wideband communication channel from a plurality of wideband communication channels in the second communication gateway according to a second data type, a second data size, a second data security level and a second data transmission age requirement of the second set of data and communication states of the plurality of wideband communication channels in the second communication gateway;

The second channel management module is configured to, in the step of determining a second wideband communication channel from a plurality of wideband communication channels in the third communication gateway and determining a second narrowband communication channel from a plurality of narrowband communication channels in the third communication gateway according to the second transmission requirement:

extracting a first video data type, a first video data size, a first video data security level and a first video data transmission timeliness requirement of the first video data from the second transmission requirement;

splitting a third group of data which needs to be transmitted by utilizing a narrowband communication channel in the third communication gateway from the first video data according to the first video data type, the first video data size, the first video data security level and the first video data transmission timeliness requirement, and taking the rest of data as a fourth group of data which is transmitted by utilizing a wideband communication channel in the third communication gateway;

determining a second narrowband communication channel from a plurality of narrowband communication channels in the third communication gateway according to a third data type, a third data size, a third data security level, a third data transmission age requirement and communication states of the plurality of narrowband communication channels in the third communication gateway of the third group of data;

Determining a second broadband communication channel from the plurality of broadband communication channels in the third communication gateway according to a fourth data type, a fourth data size, a fourth data security level, a fourth data transmission age requirement and communication states of the plurality of broadband communication channels in the third communication gateway of the fourth group of data;

the first internet of things server receives the first set of data, the second set of data, the third set of data and the fourth set of data and sends the first set of data, the second set of data, the third set of data and the fourth set of data to the cloud server;

the cloud server receives the first set of data, the second set of data, the third set of data, and the fourth set of data, and sends the first set of data, the second set of data, the third set of data, and the fourth set of data to the second internet of things server.

Optionally, the step of determining the address information of the second intelligent terminal from the second internet server according to the second related information includes:

the cloud server screens the data transmission path of the second intelligent terminal from the data transmission historical data of the second internet server according to the unique identifier;

Determining a plurality of selectable addresses of the second intelligent terminal according to the data transmission path;

and controlling the second internet server to respectively send verification information to the plurality of optional addresses to determine the address information of the second intelligent terminal.

Optionally, the step of establishing a first broadband transmission connection with the first intelligent terminal and the first internet of things server by using the first broadband communication channel includes:

the first channel management module determines corresponding broadband communication modules from a plurality of first broadband communication modules of the first intelligent terminal to establish data transmission connection according to first broadband attribute information of the first broadband communication channels, selects corresponding channels from the communication channels of the internet of things server on the first internet of things server to establish data transmission connection, and forms the first broadband transmission connection;

the step of establishing a first narrowband transmission connection between the first intelligent terminal and the first internet of things server by using the first narrowband communication channel comprises the following steps:

the first channel management module determines that a corresponding narrowband communication module establishes data transmission connection from a plurality of first narrowband communication modules of the first intelligent terminal according to first narrowband attribute information of the first narrowband communication channel, selects a corresponding channel from the communication channels of the internet of things server on the first internet of things server to establish the data transmission connection, and forms the first narrowband transmission connection.

Optionally, the step of sending the first audio data to the fourth communication gateway using the fourth data transmission connection and sending the first video data to the fifth communication gateway using the fifth data transmission connection includes:

the second internet of things server receives the first set of data, the second set of data, the third set of data and the fourth set of data, restores the first set of data and the second set of data to the first audio data, and restores the third set of data and the fourth set of data to the first video data;

acquiring second communication state data of a plurality of second broadband communication modules and a plurality of second narrowband communication modules on the second intelligent terminal;

acquiring the first audio data type, the first audio data size, the first audio data security level and the first audio data transmission timeliness requirement;

splitting a fifth group of data which needs to be transmitted by using a narrow-band communication channel in the fourth communication gateway from the first audio data according to the first audio data type, the first audio data size, the first audio data security level, the first audio data transmission timeliness requirement and the second communication state data, and taking the rest of data as a sixth group of data which is transmitted by using a wide-band communication channel in the fourth communication gateway;

Determining a third narrowband communication channel from a plurality of narrowband communication channels in the fourth communication gateway according to a fifth data type, a fifth data size, a fifth data security level and a fifth data transmission timeliness requirement of the fifth group of data and communication states of the plurality of narrowband communication channels in the fourth communication gateway;

determining a third wideband communication channel from the plurality of wideband communication channels in the fourth communication gateway according to a sixth data type, a sixth data size, a sixth data security level and a sixth data transmission age requirement of the sixth set of data and communication states of the plurality of wideband communication channels in the fourth communication gateway;

acquiring the first video data type, the first video data size, the first video data security level and the first video data transmission timeliness requirement;

splitting a seventh group of data which needs to be transmitted by using a narrowband communication channel in the fifth communication gateway from the first video data according to the first video data type, the first video data size, the first video data security level, the first video data transmission aging requirement and the second communication state data, and taking the rest of data as an eighth group of data which is transmitted by using a wideband communication channel in the fifth communication gateway;

Determining a fourth narrowband communication channel from a plurality of narrowband communication channels in the fifth communication gateway according to a seventh data type, a seventh data size, a seventh data security level, a seventh data transmission age requirement, and a communication state of the plurality of narrowband communication channels in the fifth communication gateway for the seventh set of data;

determining a fourth wideband communication channel from a plurality of wideband communication channels in the fifth communication gateway according to an eighth data type, an eighth data size, an eighth data security level and an eighth data transmission age requirement of the eighth group of data and communication states of the plurality of wideband communication channels in the fifth communication gateway;

establishing a narrowband communication channel with the second intelligent terminal through the third narrowband communication channel to transmit the fifth set of data;

establishing a broadband communication channel with the second intelligent terminal through the third broadband communication channel to transmit the sixth set of data;

establishing a narrowband communication channel with the second intelligent terminal through the fourth narrowband communication channel to transmit the seventh set of data;

and establishing a broadband communication channel with the second intelligent terminal through the fourth broadband communication channel to transmit the eighth group of data.

By adopting the technical scheme, the audio and video data transmission system based on multi-network combination comprises a first intelligent terminal, a second intelligent terminal, a plurality of communication gateways, a plurality of Internet of things servers for managing the first intelligent terminal and/or the second intelligent terminal and a cloud server for managing the plurality of Internet of things servers; the first intelligent terminal comprises a plurality of first broadband communication modules, a plurality of first narrowband communication modules, a first positioning module for acquiring position information, a first detection module and a first control module; the second intelligent terminal comprises a plurality of second broadband communication modules, a plurality of second narrowband communication modules, a second positioning module for acquiring position information, a second detection module and a second control module; the communication gateway includes a plurality of wideband communication channels, a plurality of narrowband communication channels, and a channel management module. According to the embodiment of the invention, corresponding transmission paths are selected from a plurality of communication gateways and a plurality of broadband/narrowband communication channels on the communication gateways according to the attribute information of the audio data and the video data of the data to be transmitted, and the audio data and the video data are transmitted by selecting different transmission paths according to different characteristics, so that the transmission range of narrowband communication is enlarged, the data can be flexibly transmitted in a grouping way, and the communication efficiency and the utilization rate of communication resources are improved.

Drawings

Fig. 1 is a schematic block diagram of an audio/video data transmission system based on multi-network association according to an embodiment of the present application;

fig. 2 is a flowchart of an audio/video data transmission method based on multi-network association according to an embodiment of the present application.

Detailed Description

In order that the above-recited objects, features and advantages of the present application will be more clearly understood, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, without conflict, the embodiments of the present application and features in the embodiments may be combined with each other.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, however, the present application may be practiced otherwise than as described herein, and therefore the scope of the present application is not limited to the specific embodiments disclosed below.

The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

An audio and video data transmission system and method based on multi-network association according to some embodiments of the present application are described below with reference to fig. 1 to 2.

As shown in fig. 1, an embodiment of the present application provides an audio/video data transmission system based on multi-network association, including: the system comprises a first intelligent terminal, a second intelligent terminal, a plurality of communication gateways, a plurality of Internet of things servers for managing the first intelligent terminal and/or the second intelligent terminal and cloud servers for managing the plurality of Internet of things servers; the first intelligent terminal comprises a plurality of first broadband communication modules, a plurality of first narrowband communication modules, a first positioning module for acquiring position information, a first detection module and a first control module; the second intelligent terminal comprises a plurality of second broadband communication modules, a plurality of second narrowband communication modules, a second positioning module for acquiring position information, a second detection module and a second control module; the communication gateway comprises a plurality of broadband communication channels, a plurality of narrowband communication channels and a channel management module; wherein,

The first intelligent terminal is configured to:

collecting first audio data and first video data;

the first positioning module is configured to:

acquiring first position information of the first intelligent terminal;

the first detection module is configured to:

the first internet of things server is configured to:

determining a first transmission requirement for transmitting the first audio data and a second transmission requirement for transmitting the first video data (generated according to attribute information and the like in the transmission instruction) according to the first data transmission instruction, and sending the first transmission requirement and the second transmission requirement to the first intelligent terminal; the first transmission requirement comprises information of a second communication gateway in the communication gateways, and the second transmission requirement comprises information of a third communication gateway in the communication gateways;

the first control module is configured to:

It may be appreciated that in the embodiment of the present invention, the first detection module searches, according to the first data transmission instruction (including first attribute information of the first audio data to be transmitted and second attribute information of the first video data, where the first/second attribute information includes, but is not limited to, related information such as a data type, a data size, a data security level, and a data transmission aging requirement of the first audio data/first video data, which may be used to determine requirements in terms of functions, performances, and the like of the communication gateway) and the first location information, for a first communication gateway among the plurality of communication gateways that is within a communication range of the first intelligent terminal and meets data transmission requirements, and a first internet of things server among the plurality of internet of things servers that manages the first communication gateway.

The internet of things server is used for carrying out communication management and data management on the communication terminals in a certain area, stores communication path data of the intelligent terminals which are accessed into the communication gateway managed by the internet of things server in a history mode, and can determine common communication paths of the intelligent terminals according to unique identifiers of the intelligent terminals so that verification information can be sent through the common communication paths to obtain current address information of the intelligent terminals.

The first internet of things server respectively determines a first transmission requirement for transmitting the first audio data and a second transmission requirement for the first video data according to the first data transmission instruction, namely, generates the first/second transmission requirement according to first/second attribute information (such as data type, data size, data security level and data transmission aging requirement of the first audio data/first video data and the like) in the first data transmission instruction, and further, can more accurately determine information of a communication gateway to be used, such as information of a second/communication gateway, according to the information of the communication gateway managed by the first internet of things server; the first transmission requirement includes information of a second one of the communication gateways, the second transmission requirement includes information of a third one of the communication gateways.

According to the embodiment of the invention, corresponding transmission paths are selected from a plurality of communication gateways and a plurality of broadband/narrowband communication channels on the communication gateways according to the attribute information of the audio data and the video data of the data to be transmitted, and the audio data and the video data are transmitted by selecting different transmission paths according to different characteristics, so that the transmission range of narrowband communication is enlarged, the data can be flexibly transmitted in a grouping way, and the communication efficiency and the utilization rate of communication resources are improved.

In some possible embodiments of the present invention, the first channel management module of the second communication gateway is configured to:

the first control module is configured to:

and sending the first audio data to the first internet of things server through the first broadband transmission connection and the first narrowband transmission connection. (separate transmission of audio data of different objects, or separate transmission according to different security levels)

It can be understood that, in order to ensure the security of data transmission and the rapidness of data transmission, in this embodiment, audio data generated for different objects may be separately transmitted according to actual needs, for example, after the audio data is preprocessed, monotone/simple background sound is transmitted by using a first narrowband transmission connection, and other complex and large amount of sound data may be transmitted by using a first wideband transmission connection; or the audio data related to the sensitive words can be transmitted by selecting a route with higher security from the first broadband transmission connection and the first narrowband transmission connection, and other conventional audio data can be transmitted by using another route; in addition, according to different voiceprint information, a corresponding transmission route can be selected from the first broadband transmission connection and the first narrowband transmission connection.

It should be noted that, in the present invention, there are various possible embodiments of the audio/video data transmission system based on the multi-network association, and the description will be made in the following embodiments of the audio/video data transmission method based on the multi-network association, which are applied to the audio/video data transmission system based on the multi-network association.

Referring to fig. 2, another embodiment of the present invention provides an audio/video data transmission method based on multi-network association, which is applied to an audio/video data transmission system based on multi-network association, wherein the audio/video data transmission system based on multi-network association includes: the system comprises a first intelligent terminal, a second intelligent terminal, a plurality of communication gateways, a plurality of Internet of things servers for managing the first intelligent terminal and/or the second intelligent terminal and cloud servers for managing the plurality of Internet of things servers; the first intelligent terminal comprises a plurality of first broadband communication modules, a plurality of first narrowband communication modules, a first positioning module for acquiring position information, a first detection module and a first control module; the second intelligent terminal comprises a plurality of second broadband communication modules, a plurality of second narrowband communication modules, a second positioning module for acquiring position information, a second detection module and a second control module; the communication gateway comprises a plurality of broadband communication channels, a plurality of narrowband communication channels and a channel management module; the audio and video data transmission method based on the multi-network association comprises the following steps:

The first intelligent terminal collects first audio data and first video data;

the first internet of things server respectively determines a first transmission requirement for transmitting the first audio data and a second transmission requirement for transmitting the first video data according to the first data transmission instruction, and sends the first transmission requirement and the second transmission requirement to the first intelligent terminal; the first transmission requirement comprises information of a second communication gateway in the communication gateways, and the second transmission requirement comprises information of a third communication gateway in the communication gateways;

establishing (the first control module) a connection (with the first intelligent terminal) with the second communication gateway and the third communication gateway;

(the first intelligent terminal) transmitting the first audio data to the second intelligent terminal using the second communication gateway;

the first video data is transmitted to the second intelligent terminal by the third communication gateway.

In some possible embodiments of the present invention, the method for transmitting audio and video data based on multi-network association further includes:

the first channel management module establishes a first broadband transmission connection with the first intelligent terminal and the first Internet of things server by utilizing the first broadband communication channel;

the first channel management module establishes a first narrowband transmission connection with the first intelligent terminal and the first Internet of things server by utilizing the first narrowband communication channel;

It will be appreciated that in order to ensure the security of data transmission and the high efficiency of data transmission, in this embodiment, video data may be separately transmitted according to its data characteristics, for example, for background data in video data, the background may not change or may change little in the whole video data occurring period, so that the background data in video data may be transmitted using the second narrowband transmission connection, and other dynamic data may be transmitted using the second broadband transmission connection. For another example, for the image data with higher security level in the video data, the image data can be transmitted through a second broadband transmission connection and a route with higher security in a second narrowband transmission connection, and other image data can be transmitted through another line; the video data can be classified according to different face images in the video data, and then the corresponding transmission route is selected from the second broadband transmission connection and the second narrowband transmission connection.

the cloud server receives the first data transmission instruction;

the cloud server determines address information of the second intelligent terminal from the second internet server according to the second related information;

the second internet server receives the first audio data and the first video data;

The second internet server sends the first audio data to the fourth communication gateway by using the fourth data transmission connection, and sends the first video data to the fifth communication gateway by using the fifth data transmission connection;

It can be understood that, in order to ensure the high efficiency and security of the data transmission and the mutual noninterference of the data transmission, in the embodiment of the present invention, for the second intelligent terminal of the receiving terminal, the first audio data and the first video data are received by respectively establishing a sixth data transmission connection and a seventh data transmission connection with the fourth communication gateway and the fifth communication gateway.

In this embodiment, the second internet server determines a fourth communication gateway and a fifth communication gateway from the plurality of communication gateways managed by the second internet server according to the address information, and establishes a fourth data transmission connection with the fourth communication gateway and a fifth data transmission connection with the fifth communication gateway. Further, in order to more precisely determine the corresponding communication gateway, the second internet server may determine a fourth communication gateway and a fifth communication gateway, respectively, according to the first/second attribute information.

In some possible embodiments of the present invention, the step of determining a first wideband communication channel from a plurality of wideband communication channels in the second communication gateway according to the first transmission requirement, and determining a first narrowband communication channel from a plurality of narrowband communication channels in the second communication gateway includes:

It can be understood that, in this embodiment, according to the data type, the data size, the data security level and the data transmission aging requirement, the specific implementation procedure of the steps of splitting the first/third group of data that needs to be transmitted by using the narrowband communication channel from the first audio/video data, and using the remaining data as the second/fourth group of data that is transmitted by using the wideband communication channel is as follows:

training the first convolutional neural network to classify the data by using wideband communication characteristic data (including but not limited to bandwidth, data type suitable for transmission, data size, data security level, data transmission age, transmission rate, transmission success rate, etc.), narrowband communication characteristic data (including but not limited to bandwidth, data type suitable for transmission, data size, data security level, data transmission age, transmission rate, transmission success rate, etc.);

respectively sending the data type, the data size, the data security level and the data transmission timeliness requirement corresponding to the audio/video data as input values to an input layer of the first convolutional neural network, wherein the input layer and an initial layer are connected through matrix operation; the initial layer obtains a first output result, activates the first output result to obtain a second output result, and sends the activated second output result to the analog output layer; the analog output layer calculates an analog output value through a matrix, and inputs the analog output value into a secondary initial layer; the secondary initial layer obtains a verification output result through matrix calculation; the input value of the input layer is in data connection with a secondary initial layer; the secondary initial layer calculates a secondary output result through a matrix, and sends the secondary output result and the verification output result to the verification coefficient layer for verification, and a normalization coefficient is obtained; the normalization coefficient and the analog output value are sent to an output layer, and the output layer carries out normalization processing on the analog output value to obtain a mimicry result; collecting positive feedback and/or reverse feedback information; and learning and correcting the mimicry result according to the positive feedback and/or the inverse feedback information, so that the first audio/video data are divided into a first group of data and a second group of data (or a third group of data and a fourth group of data), the data transmission is performed by utilizing the broadband and narrowband combined networking, the respective advantages of different networks are fully utilized, and the transmission scheme of the audio/video data is more efficient and intelligent.

In some possible embodiments of the present invention, the step of determining address information of the second intelligent terminal from the second internet server according to the second related information includes:

It can be understood that, because the second intelligent terminal as the receiving end may be a mobile terminal, its physical location may change, in order to increase flexibility and efficiency of data transmission, in this embodiment, a data transmission path of the second intelligent terminal is screened out from the data transmission history data of the second internet server according to the unique identifier, and a plurality of selectable addresses of the second intelligent terminal are determined according to the data transmission path; and respectively sending verification information to the plurality of optional addresses, and carrying out verification feedback on the verification information by the second intelligent terminal, so that the address information of the second intelligent terminal can be determined, and the communication safety can be ensured. Further, a third communication gateway and a fourth communication gateway existing within a communication range/managed by a second internet server may be determined from among the plurality of communication gateways according to the address information.

In some possible embodiments of the present invention, the step of establishing a first broadband transmission connection with the first intelligent terminal and the first internet of things server by using the first broadband communication channel includes:

It may be understood that in this embodiment, according to the attribute information (such as the communication protocol type, the communication power, etc.) of the first wideband communication channel and the first narrowband communication channel, a data transmission connection is established from a plurality of first wideband communication modules and a plurality of first narrowband communication modules, and a corresponding channel is selected from the communication channels on the first internet of things server to establish a data transmission connection, so as to realize fusion and connection of narrowband and wideband of different types and different transmission network segments, and ensure smoothness of data transmission.

In some possible embodiments of the present invention, the step of sending the first audio data to the fourth communication gateway using the fourth data transmission connection and sending the first video data to the fifth communication gateway using the fifth data transmission connection includes:

It will be appreciated that in order to enhance compatibility/flexibility, in this embodiment, the second internet server receives the first set of data, the second set of data, the third set of data, and the fourth set of data, and restores the first set of data and the second set of data to the first audio data, and restores the third set of data and the fourth set of data to the first video data; acquiring second communication state data of a plurality of second broadband communication modules and a plurality of second narrowband communication modules on the second intelligent terminal; splitting fifth group data which is required to be transmitted by one or more of a plurality of narrowband communication channels from the first audio data according to the data type, the data size, the data security level and the data transmission aging requirement corresponding to the first audio/video data and the second communication state data, taking the rest data as sixth group data which is required to be transmitted by one or more of a plurality of wideband communication channels, splitting seventh group data which is required to be transmitted by one or more of a plurality of narrowband communication channels from the first video data, and taking the rest data as eighth group data which is required to be transmitted by one or more of a plurality of wideband communication channels; establishing a narrowband communication channel with the second intelligent terminal to transmit a fifth/seventh set of data; and establishing a broadband communication channel between the second intelligent terminal and the second intelligent terminal to transmit the sixth/eighth group of data, restoring the fifth group of data and the sixth group of data by the second intelligent terminal to obtain the first audio data, and restoring the seventh group of data and the eighth group of data to obtain the first video data.

It should be noted that, in order to ensure synchronization of audio data and video data, in the embodiment of the present invention, synchronization determination may be performed on the received first audio data and first video data by the first internet of things server or the second internet of things server or the cloud server (for example, uniform timestamps are marked in the first audio data and the first video data, and whether synchronization is performed may be determined by comparing the timestamps of the two data), and when the two data are not synchronized, synchronization adjustment is performed, for example, when the first audio data is earlier than the first video data, a preset image generation model and the video data that have been transmitted may be combined, and supplementary video data (i.e., predictive video) that is synchronized with the first audio data may be generated, so that the received video data and the audio data are synchronized (for example, in the course of a live lecture, because the audio data is smaller and is fast in transmission, and for this case, a situation that sound is earlier than action may occur, and for this case, a supplementary video that is played according to the received video and the image generation model that is trained by big data, supplementary video data may be generated, so that sound and user experience may be synchronized. And if the difference between the later-transmitted late video data and the supplementary video data exceeds a preset threshold value, the later-transmitted late video data is stored instead of the supplementary video data.

It will be appreciated that the entity or modified entity represented by the "first broad band" and the entity or modified entity represented by the "second broad band" may be the same or different in all embodiments of the present application; the entity or modified entity represented by the "first narrowband" may be the same as or different from the entity or modified entity represented by the "second narrowband"; the embodiments of the present application are not limited in this regard.

It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, such as the above-described division of units, merely a division of logic functions, and there may be additional manners of dividing in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the above-mentioned method of the various embodiments of the present application. And the aforementioned memory includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and the program may be stored in a computer readable memory, which may include: flash disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.

The foregoing has outlined rather broadly the more detailed description of embodiments of the application, wherein the principles and embodiments of the application are explained in detail using specific examples, the above examples being provided solely to facilitate the understanding of the method and core concepts of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Although the present application is disclosed above, the present application is not limited thereto. Variations and modifications, including combinations of the different functions and implementation steps, as well as embodiments of the software and hardware, may be readily apparent to those skilled in the art without departing from the spirit and scope of the application.

Claims

1. An audio and video data transmission system based on multi-network association, comprising: the system comprises a first intelligent terminal, a second intelligent terminal, a plurality of communication gateways, a plurality of Internet of things servers for managing the first intelligent terminal and/or the second intelligent terminal and cloud servers for managing the plurality of Internet of things servers; the first intelligent terminal comprises a plurality of first broadband communication modules, a plurality of first narrowband communication modules, a first positioning module for acquiring position information, a first detection module and a first control module; the second intelligent terminal comprises a plurality of second broadband communication modules, a plurality of second narrowband communication modules, a second positioning module for acquiring position information, a second detection module and a second control module; the communication gateway comprises a plurality of broadband communication channels, a plurality of narrowband communication channels and a channel management module; wherein,

The first intelligent terminal is configured to:

collecting first audio data and first video data;

the first positioning module is configured to:

acquiring first position information of the first intelligent terminal;

the first detection module is configured to:

the first internet of things server is configured to:

the first control module is configured to:

2. The multi-network association based audio-video data transmission system of claim 1, wherein the first channel management module of the second communication gateway is configured to:

the first control module is configured to:

3. The audio and video data transmission method based on the multi-network combination is characterized by being applied to an audio and video data transmission system based on the multi-network combination, wherein the audio and video data transmission system based on the multi-network combination comprises: the system comprises a first intelligent terminal, a second intelligent terminal, a plurality of communication gateways, a plurality of Internet of things servers for managing the first intelligent terminal and/or the second intelligent terminal and cloud servers for managing the plurality of Internet of things servers; the first intelligent terminal comprises a plurality of first broadband communication modules, a plurality of first narrowband communication modules, a first positioning module for acquiring position information, a first detection module and a first control module; the second intelligent terminal comprises a plurality of second broadband communication modules, a plurality of second narrowband communication modules, a second positioning module for acquiring position information, a second detection module and a second control module; the communication gateway comprises a plurality of broadband communication channels, a plurality of narrowband communication channels and a channel management module; the audio and video data transmission method based on the multi-network association comprises the following steps:

The first intelligent terminal collects first audio data and first video data;

4. The method for transmitting audio and video data based on multi-network association according to claim 3, wherein the method for transmitting audio and video data based on multi-network association further comprises:

5. The method for transmitting audio and video data based on multi-network association according to claim 4, further comprising:

6. The method for transmitting audio and video data based on multi-network association according to claim 5, further comprising:

the cloud server receives the first data transmission instruction;

receiving the first audio data and the first video data;

7. The method of claim 6, wherein the step of determining a first wideband communication channel from a plurality of wideband communication channels in the second communication gateway and determining a first narrowband communication channel from a plurality of narrowband communication channels in the second communication gateway according to the first transmission requirement comprises:

8. The method for transmitting audio and video data based on multi-network association according to claim 7, wherein the step of determining address information of the second intelligent terminal from the second internet server according to the second related information comprises:

9. The method for transmitting audio and video data based on multi-network association according to claim 8, wherein the step of establishing a first broadband transmission connection with the first intelligent terminal and the first internet of things server by using the first broadband communication channel comprises:

10. The method of claim 9, wherein the step of sending the first audio data to the fourth communication gateway via the fourth data transmission connection and the step of sending the first video data to the fifth communication gateway via the fifth data transmission connection comprises: