CN110943972A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN110943972A CN110943972A CN201911041326.8A CN201911041326A CN110943972A CN 110943972 A CN110943972 A CN 110943972A CN 201911041326 A CN201911041326 A CN 201911041326A CN 110943972 A CN110943972 A CN 110943972A
- Authority
- CN
- China
- Prior art keywords
- data
- multimedia data
- channel
- transmission
- application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 30
- 230000005540 biological transmission Effects 0.000 claims abstract description 200
- 238000012545 processing Methods 0.000 claims abstract description 49
- 238000000034 method Methods 0.000 claims abstract description 21
- 238000012546 transfer Methods 0.000 claims description 12
- 238000004806 packaging method and process Methods 0.000 claims description 10
- 238000012544 monitoring process Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000012856 packing Methods 0.000 claims description 2
- 238000007726 management method Methods 0.000 abstract description 9
- 238000013523 data management Methods 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 25
- 238000010586 diagram Methods 0.000 description 10
- 238000012986 modification Methods 0.000 description 8
- 230000004048 modification Effects 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/08—Protocols for interworking; Protocol conversion
Abstract
The present disclosure provides a data processing method and apparatus, relating to the technical field of computer multimedia, wherein the method comprises: acquiring application multimedia data; splitting the application multimedia data into a plurality of data packets, and adding media transmission layer headers to the data packets respectively to generate transmission multimedia data; and sending the transmission multimedia data to a channel encoder, so that the channel encoder sends the channel multimedia data obtained by encoding the transmission multimedia data to a receiving end. The method and the device can decouple the multimedia data management and the channel management, and can ensure the stability in the multimedia data processing process.
Description
Technical Field
The present disclosure relates to the field of computer multimedia technologies, and in particular, to a data processing method and apparatus.
Background
An existing multimedia data processing system may be as shown in fig. 1, where the processing system includes an Application module Application of an upper module and a control Controller module of a bottom module, where the Controller module is used for transmission data management of APP _ Metadata (Application multimedia data) from the Application module and transmission channel management. It is understood that Channel _ Metadata (Channel multimedia data) transmitted on a link includes various Channel management related messages in addition to the split APP _ Metadata.
Because the transmission channel management is strongly related to the link type, different types of links and different transmission channel management methods are different, for example, new creation processes of a UDP link and a TCP link are different. Therefore, once the link type in the transmission system changes or the transmission path of the link changes, the Controller module needs to be adaptively modified, and the modification is complex and has a large workload, which is not only detrimental to the multiplexing of the module, but also affects the stability in the transmission process.
Disclosure of Invention
The embodiment of the disclosure provides a data processing method and device, and the stability in a multimedia data processing process can be improved. The technical scheme is as follows:
according to a first aspect of embodiments of the present disclosure, there is provided a data processing method, including:
acquiring application multimedia data;
splitting the application multimedia data into a plurality of data packets, and adding media transmission layer headers to the data packets respectively to generate transmission multimedia data;
and sending the transmission multimedia data to a channel encoder, so that the channel encoder sends the channel multimedia data obtained by encoding the transmission multimedia data to a receiving end.
In one embodiment, the dividing the application multimedia data into a plurality of data packets, and adding media transport layer headers to the plurality of data packets, respectively, and the generating the transmission multimedia data includes:
and splitting the application multimedia data into a plurality of data packets according to a Maximum Transmission Unit (MTU) of a transmission channel, and adding media transmission layer headers to the plurality of data packets respectively to generate transmission multimedia data.
In one embodiment, obtaining application multimedia data comprises:
acquiring application multimedia data through a data source interface of an upper application;
the sending the transmission multimedia data to a channel encoder so that the channel encoder sends the channel multimedia data after encoding the transmission multimedia data to a receiving end comprises:
and sending the transmission multimedia data to a channel encoder through a data source callback function, so that the channel encoder sends the channel multimedia data after encoding the transmission multimedia data to a receiving end.
In one embodiment, the method further comprises:
acquiring application feedback information from a channel encoder; the application feedback information is obtained by decoding the received channel feedback information from the receiving end by the channel encoder;
and controlling the sending rate and/or the frame rate of the transmission multimedia data according to the application feedback information.
In one embodiment, the method further comprises:
the channel encoder encodes the transmission multimedia data to generate channel multimedia data; wherein encoding the transmission multimedia data comprises: adding header information to the multimedia data, wherein the header information carries channel type information;
and transmitting the channel multimedia data to a receiving end.
In one embodiment, sending the channel multimedia data to a receiving end, so that the channel multimedia data encoded by the channel encoder is sent to the receiving end comprises:
negotiating transmission attribute information with a transit server through a channel encoder;
and sending the channel multimedia data to the transfer server through a channel encoder according to the transmission attribute information, so that the transfer server sends the channel multimedia data to a receiving end according to the transmission attribute information.
According to a second aspect of the embodiments of the present disclosure, there is provided a data processing method, including:
acquiring transmission multimedia transmission data, packaging the transmission multimedia to obtain application multimedia data, and sending the application multimedia data to an upper application;
acquiring application feedback information; the feedback information is generated by the upper layer application monitoring the receiving state data of the multimedia data according to the receiving state data;
and sending the application feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information after encoding the application feedback information to a sending end.
In one embodiment, sending the feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information after encoding the application feedback information to a sending end includes:
and preprocessing the feedback information according to a preset format and then sending the preprocessed feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information obtained by encoding the application feedback information to a sending end.
In one embodiment, acquiring transmission multimedia transmission data, performing packet packing processing on the transmission multimedia to obtain application multimedia data, and sending the application multimedia data to an upper layer application includes:
and acquiring the transmission multimedia data through a data pool interface of a channel encoder, performing packet processing on the transmission multimedia to obtain application multimedia data, sending the application multimedia data to an upper-layer application, performing packet processing on the transmission multimedia to obtain application multimedia data, and sending the application multimedia data to the upper-layer application through a data pool callback function.
In one embodiment, the feedback information includes at least packet loss rate or delay.
According to a third aspect of the embodiments of the present disclosure, there is provided a data processing apparatus including:
the acquisition module is used for acquiring application multimedia data;
the media transmission module is used for splitting the application multimedia data into a plurality of data packets, and respectively adding media transmission layer headers to the plurality of data packets to generate transmission multimedia data;
and the sending module is used for sending the transmission multimedia data to a channel encoder, so that the channel encoder sends the channel multimedia data after encoding the transmission multimedia data to a receiving end.
In one embodiment, the media transport module comprises:
the splitting submodule is used for splitting the application multimedia data into a plurality of data packets according to a Maximum Transmission Unit (MTU) of a transmission channel;
and the adding submodule is used for respectively adding media transmission layer headers to the data packets to generate transmission multimedia data.
In one embodiment, the obtaining module is specifically configured to:
acquiring application multimedia data through a data source interface of an upper application;
the sending module is specifically configured to:
and sending the transmission multimedia data to a channel encoder through a data source callback function, so that the channel encoder sends the channel multimedia data after encoding the transmission multimedia data to a receiving end.
In one embodiment, the above apparatus further comprises:
a feedback receiving module for receiving application feedback information from the channel encoder; the application feedback information is obtained by decoding the received channel feedback information from the receiving end by the channel encoder;
and the control module is used for controlling the sending rate and/or the frame rate of the transmission multimedia data according to the application feedback information.
In one embodiment, the above apparatus further comprises:
a generating module, configured to encode the transmission multimedia data through a channel encoder to generate channel multimedia data; the generating module is specifically configured to add header information to the multimedia data, where the header information carries channel type information;
and the channel sending module is used for sending the channel multimedia data to a receiving end.
In one embodiment, the channel transmitting module includes:
the negotiation submodule is used for negotiating the transmission attribute information through the channel encoder transfer server;
and the transfer sub-module is used for sending the channel multimedia data to the transfer server through a channel encoder according to the transmission attribute information, so that the transfer server sends the channel multimedia data to a receiving end according to the transmission attribute information.
According to a fourth aspect of the embodiments of the present disclosure, there is provided a data processing apparatus including:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring transmission multimedia transmission data, packaging the transmission multimedia to obtain application multimedia data and sending the application multimedia data to an upper application;
the second acquisition module is used for acquiring application feedback information; the feedback information is generated by the upper layer application monitoring the receiving state data of the multimedia data according to the receiving state data;
and the sending module is used for sending the application feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information after encoding the application feedback information to a sending end.
In one embodiment, the first obtaining module is specifically configured to:
and receiving the transmission multimedia data through a data pool interface of a channel encoder, performing packet packaging processing on the transmission multimedia to obtain application multimedia data, and sending the application multimedia data to an upper application through a data pool callback function.
In one embodiment, the feedback information includes at least packet loss rate or delay.
In order to avoid frequently modifying the Controller control module when the channel changes in the background technology, the transmission data management and the transmission channel management can be decoupled, the transmission data management is given to the data processing device for execution, and the transmission channel management is executed by the channel encoder. In the embodiment of the present disclosure, a module responsible for managing transmission data is denoted as a media transmission VMTL module, that is, a multimedia data transmission device, and a module responsible for managing transmission channels is denoted as a Channel module, that is, a Channel encoder.
Therefore, under the condition that the Channel changes, for example, the link type or the transmission path changes, only the Channel module needs to be adaptively modified without influencing the VMTL module, so that the multiplexing of the VMTL module is realized, the modification amount of the Channel module is small, and the stability in the multimedia data processing process can be ensured.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a diagram of a background art multimedia data processing system architecture for the present disclosure;
fig. 2 is a flowchart of a data processing method provided by an embodiment of the present disclosure;
fig. 3 is a flowchart of a data processing method provided by an embodiment of the present disclosure;
fig. 4 is a flowchart of transmitting channel multimedia data according to an embodiment of the disclosure;
fig. 5 is a flowchart of a data processing method provided by an embodiment of the present disclosure;
FIG. 6 is an architecture diagram of a multimedia data processing system provided by an embodiment of the present disclosure;
fig. 7 is a flowchart of a data processing method provided by an embodiment of the present disclosure;
fig. 8 is a schematic diagram of a multimedia data process provided by an embodiment of the present disclosure;
FIG. 9 is an architecture diagram of a multimedia data processing system provided by an embodiment of the present disclosure;
fig. 10 is a flowchart of a data processing method provided by an embodiment of the present disclosure;
FIG. 11 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;
FIG. 12 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;
fig. 13 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;
fig. 14 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;
fig. 15 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
An embodiment of the present disclosure provides a data processing method, as shown in fig. 2, the data processing method includes the following steps:
acquiring application multimedia data comprises:
and acquiring application multimedia data through a data source interface of the upper application.
splitting the application multimedia data into a plurality of data packets, and adding media transport layer headers to the plurality of data packets, respectively, to generate transmission multimedia data, including:
and splitting the application multimedia data into a plurality of data packets according to a Maximum Transmission Unit (MTU) of a transmission channel, and adding media transmission layer headers to the plurality of data packets respectively to generate transmission multimedia data.
The sending the transmission multimedia data to a channel encoder so that the channel encoder sends the channel multimedia data after encoding the transmission multimedia data to a receiving end comprises:
and sending the transmission multimedia data to a channel encoder through a data source callback function, so that the channel encoder sends the channel multimedia data after encoding the transmission multimedia data to a receiving end.
As shown in fig. 3, the method further includes:
Specifically, as shown in fig. 4, the step 207 of sending the channel multimedia data to the receiving end includes:
An embodiment of the present disclosure provides a data processing method, as shown in fig. 5, the data processing method includes the following steps:
acquiring transmission multimedia transmission data, packaging the transmission multimedia to obtain application multimedia data, and sending the application multimedia data to an upper application, wherein the method comprises the following steps:
and acquiring the transmission multimedia data through a data pool interface of a channel encoder, packaging the transmission multimedia data to obtain application multimedia data, and sending the application multimedia data to an upper application through a data pool callback function.
the feedback information at least comprises packet loss rate or time delay.
Sending the feedback information to a channel encoder, so that transmitting the channel feedback information after the channel encoder encodes the application feedback information to a sending end comprises:
and preprocessing the feedback information according to a preset format and then sending the preprocessed feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information obtained by encoding the application feedback information to a sending end.
Fig. 6 is an architecture diagram of a multimedia data processing system according to an embodiment of the present disclosure, where the multimedia data processing system shown in fig. 6 includes a transmitting end and a receiving end, where the transmitting end and the receiving end are both of a layered design and are structurally equivalent. The transmitting end and the receiving end are divided into an Application layer (Application, upper layer Application), a media transport layer (VMTL) and a Channel layer (Channel), and this three-layer structure is defined as a VMTL model in this Application. Wherein, VMTL is used for transmission data management of APP _ Metadata from application module, and Channel is used for transmission Channel management. The user can carry out custom design on the protocol of the VMTL model according to the requirement of the user.
For example, the sending VMTL layer unpacks the data to be sent and then adds its own protocol header. The protocol header may include, but is not limited to, the following:
a protocol field may be included to indicate that the packet is generated by the VMTL layer to distinguish data of the Channel layer;
may contain a send _ no field indicating the original data, e.g., video frame number, before entering the VMTL module;
may contain a total _ seq field indicating that a frame of raw data sent to the VMTL module is split into several small packets for transmission;
may contain a seq field indicating the sequence number of the packet split by the VMTL module;
the combination of the above fields can enable the VMTL module at the receiving end to combine the packets into raw data that is fed into the VMTL module.
In the multimedia processing system in this embodiment, the transmission systems at the transmitting end and the receiving end have completely equivalent structures, the protocols of the modules are equivalent, the codes of the modules are uniform, and management is facilitated.
Based on the multimedia processing system, an embodiment of the present disclosure provides a data processing method, as shown in fig. 7, the data processing method includes the following steps:
specifically, the application layer sends the application multimedia data to the media transmission layer through a data source data _ source interface. As shown in fig. 8, the data source interface is a data output interface of the application layer.
specifically, as shown in fig. 8, the media transport layer sends the transmission multimedia data to the channel layer through the data source callback function data _ source _ cb.
At a sending end, a Channel layer performs special processing on some data called back by a data _ source _ cb callback function (callback) of the VMTL, such as changing header information, adapting a Channel type, and the like, so that the VMTL of the sending end can set a data structure for describing the data called back by the data _ source _ cb callback function (callback), and a user can refer to the data structure to modify the data called back by the data _ source _ cb callback function (callback) without knowing a packet structure of the VMTL, thereby meeting the requirement of special processing.
For example, the UDP-based Channel layer is not the same for some network parameters for transmission of audio video. The Channel layer can set appropriate Channel parameters for transmitting data according to the type of data to be transmitted. Generally, the receive buffer of a video frame needs to be a little larger, such as 8M. When the Channel recognizes that the data packet to be transmitted is a video frame according to the structure, the data is transmitted through a Channel with a receiving buffer area of 8M.
It should be noted that the data structure is used to give a description to the Channel, for example, to describe the data to be transmitted as a video frame, and the Channel is set for determining what Channel the video frame is transmitted with.
The VMTL module and the Channel module are mutually transparent, the change of the application module does not influence the Channel module, the change of the transmission Channel does not influence the VMTL module, and the user of the Channel module possibly does not know the protocol of the VMTL module, so that a data structure and a self message analysis function are respectively arranged at the sending end and the receiving end of the transmission system disclosed by the invention in order to facilitate the modification of the user of the Channel module.
In one embodiment, the multimedia processing system as shown in fig. 9, transmitting the channel multimedia data to a receiving end comprises:
the channel layer negotiates with the transfer server for transmission attribute information;
the transmission attribute information includes an incoming address and an incoming port number of the relay server. And the receiving end establishes connection with the relayServer in the same way.
The channel layer sends the channel multimedia data to a corresponding transfer server according to the transmission attribute information;
and the transfer server sends the channel multimedia data to a channel layer of a receiving end according to the transmission attribute information.
In this embodiment, the relayServer only needs to directly forward the transmission data from the Channel layer of the sending end to the Channel layer of the corresponding receiving end, and the Channel layer is handed over to the VMTL layer. Thus, the VMTL module may not care about whether the data is directly transmitted through the Channel layer or transmitted and received after being forwarded through one or more relay servers, that is, the relay servers are transparent to the VMTL layer of the transmitting end and the receiving end.
It can be seen that the VMTL module not only supports direct interfacing, but also can support server relay forwarding without any modification. Under the condition that the transmission path of the channel changes, the VMTL module does not need to be modified.
In one embodiment, the media transport layer splits the application multimedia data into a plurality of packets, and generating the transport multimedia data includes:
the media Transmission layer splits the application multimedia data into a plurality of data packets according to a Maximum Transmission Unit (MTU) of a Transmission channel, and generates Transmission multimedia data.
It should be noted that the MTU is the size of a split data packet, and the MTU is typically 500 bytes to 1500 bytes. In the image transfer scenario, the MTU may be 576. The value may be dynamically adjusted according to network bandwidth to improve bandwidth utilization.
In one embodiment, the channel layer is configured to encode the transport multimedia data, and generating the channel multimedia data comprises:
and the channel layer adds header information to the multimedia data, wherein the header information carries channel type information.
In particular, the Channel layer (Channel module) may be of a different type, for example, a UDP based Channel module. The Channel module can perform processing such as header information adding on the VMTL _ Metadata according to the Channel type to obtain the Channel _ Metadata, so that the Channel _ Metadata can be transmitted in the Channel module.
In one embodiment, the method further comprises:
step 706, the channel layer acquires and decodes the encoded feedback information from the receiving end to acquire the decoded feedback information;
step 707, the channel layer sends the decoded feedback information to a multimedia transmission layer;
step 708, the multimedia transmission layer controls the sending rate and/or the frame rate of the transmission multimedia data according to the feedback information.
Step 709, the media transmission layer sends the feedback information to the application layer;
step 710, controlling the transmission quality QOS parameter of the transmission multimedia data according to the feedback information. Wherein the QOS parameters include: second-start parameter, low-delay parameter, code stream adaptive parameter.
It should be noted that the Channel module does not care about the content of the VMTL _ Metadata, that is, the VMTL _ Metadata is invisible to the Channel module, and the Channel module transmits the data packet in the Channel as long as the VMTL module discards the data packet to the Channel module.
It should be noted that the Channel _ Metadata mentioned here includes not only VMTL _ Metadata processed by the Channel module, but also data (such as heartbeat message) associated with the Channel module itself.
Based on the multimedia processing system, an embodiment of the present disclosure provides a data processing method, as shown in fig. 10, the data processing method includes the following steps:
specifically, as shown in fig. 9, the channel layer receives channel multimedia data from the relay server.
specifically, the channel layer performs header removal processing on the multimedia data to generate transmission multimedia data.
Optionally, the channel layer performs header removal processing on the multimedia data, and then filters out data of the channel layer to generate transmission multimedia data. The data of the device itself refers to various interactive messages for maintaining the channel, such as heartbeat messages and the like.
specifically, as shown in fig. 8, the channel layer sends the transmission multimedia data to the multimedia transmission layer through the data pool data _ sink interface.
Specifically, as shown in fig. 8, the multimedia transport layer sends the multimedia data to the application layer through the data pool callback function data _ sink _ cb.
In one embodiment, the method further comprises:
it can be understood that the data received by each layer of the receiving end is the same as the data generated by the layer corresponding to the sending end, for example, the VMTL layer of the sending end generates VMTL _ Metadata for transmitting multimedia data, and the VMTL module of the receiving end also receives VMTL _ Metadata.
specifically, the VMTL layer processes the feedback information APP _ Message into a format capable of being transmitted in a Channel according to a preset format, and sends the feedback information APP _ Message to the Channel layer.
And the Channel layer adds header information on the feedback information and sends the feedback information to the sending end.
In one embodiment, the feedback information comprises at least packet loss rate and/or time delay.
It should be noted that, to implement the above function, it is necessary that the protocol description in the packet header sent by the Channel module and the VMTL module are different, because the self message parsing function distinguishes data according to the packet header.
On the one hand, the VMTL layer at the receiving end only receives the multimedia VMTL _ Metadata data transmitted by the VMTL layer at the transmitting end. Specifically, Channel _ Metadata received by the Channel layer at the receiving end includes not only Channel multimedia data processed by the Channel layer but also data transmitted by the Channel layer to maintain a Channel, such as heartbeat detection data, packet loss detection data, and the like.
In order to enable the VMTL layer at the receiving end to only receive the VMTL _ Metadata sent by the VMTL at the sending end, the data source callback function can distinguish the VMTL _ Metadata sent by the VMTL module at the sending end from the Channel _ Metadata, and directly filters the data of the Channel layer to enable the data not to enter the VMTL module.
Therefore, the data acquired by the VMTL at the two ends are the same, and the data of the Channel layer cannot be seen, so that the data are really transparent with the Channel layer.
By true transparency, it is meant that the VMTL layer is as if it were unaware of the Channel layer, as if the two VMTL layers were directly sending data, what the sender sent, and what the receiver received. The receiving end does not get data that is not sent by the sending end.
And on the other hand, the Channel layer user is convenient to use.
Because the Channel layer user does not know the structure of the VMTL protocol header, the Channel layer user can realize certain requirements through the self message analysis function.
For example, the VMTL layer at the receiving end needs to know whether the video frame data is transmitted directly through the Channel layer or is relayed through the relayser, but the VMTL does not know the transmission path of the Channel for which the Channel layer is responsible, and the Channel layer does not know the structure of the VMTL protocol header; therefore, the VMTL needs to provide a function modification value in its message parsing function, and the Channel layer knows the meaning of the function modification value, so that the Channel layer can modify the function modification value according to the transmission path of the Channel (whether passing through the relayServer), and thus, the VMTL at the receiving end can determine whether the video frame data is directly transmitted through the Channel layer or is relayed through the relayServer from the function modification value in its message parsing function.
In the present disclosure, the receiving end counts the received application multimedia data APP _ Metadata and feeds the counted data back to the sending end. The user can count the required data at the receiving end according to the characteristics of the user, such as packet loss rate, time delay and the like, and feed back the data to the sending end, and the sending end adjusts the sending rate of the multimedia data according to the strategy of the user, so that the purpose of feedback control is achieved.
Specifically, the VMTL layer controls the sending speed of the split packet data according to the feedback information of the receiving end, so as to achieve the purpose of congestion control.
Therefore, when the Channel layer is changed (Channel type or transmission path is changed, and the like), only the designer of the Channel needs to correspondingly modify the Channel layer, and the VMTL layer does not need to be modified, so that the reusability of the code is greatly improved.
Fig. 11 discloses a data processing apparatus, the data processing apparatus 110 shown in fig. 11 includes an obtaining module 1101, a media transmission module 1102 and a sending module 1103, wherein the obtaining module 1101 is used for obtaining application multimedia data; a media transmission module 1102, configured to split the application multimedia data into multiple data packets, add a media transmission layer header to each of the multiple data packets, and generate transmission multimedia data; a sending module 1103, configured to send the transmission multimedia data to a channel encoder, so that the channel encoder sends the channel multimedia data obtained by encoding the transmission multimedia data to a receiving end.
Optionally, the obtaining module 1101 is specifically configured to obtain application multimedia data through a data source interface of an upper layer application; the sending module 1103 is specifically configured to send the transmission multimedia data to a channel encoder through a data source callback function, so that the channel encoder sends the channel multimedia data obtained by encoding the transmission multimedia data to a receiving end.
Fig. 12 discloses a data processing apparatus, such as the data processing apparatus 120 shown in fig. 12, which includes an obtaining module 1201, a media transmission module 1202 and a sending module 1203, wherein, in an embodiment, the media transmission module 1203 includes: a splitting submodule 12031, configured to split the application multimedia data into multiple data packets according to a maximum transmission unit MTU of a transmission channel; the adding submodule 12032 is configured to add a media transport layer header to each of the plurality of data packets, and generate transmission multimedia data.
Fig. 13 discloses a data processing apparatus, such as the data processing apparatus 130 shown in fig. 13, which includes an obtaining module 1301, a media transmission module 1302, a sending module 1303, a feedback receiving module 1304, and a control module 1305, where the feedback receiving module 1304 is configured to receive application feedback information from a channel encoder; the application feedback information is obtained by decoding the received channel feedback information from the receiving end by the channel encoder; the control module 1305 is configured to control the sending rate and/or the frame rate of the transmission multimedia data according to the application feedback information.
Fig. 14 discloses a data processing apparatus, such as the data processing apparatus 140 shown in fig. 14, which includes an obtaining module 1401, a media transmission module 1402, a transmission module 1403, a generating module 1404 and a channel transmission module 1405, wherein the generating module 1404 is configured to encode the transmission multimedia data through a channel encoder to generate channel multimedia data; the generating module is specifically configured to add header information to the multimedia data, where the header information carries channel type information; a channel sending module 1405, configured to send the channel multimedia data to a receiving end.
Optionally, the channel transmitting module 1405 includes:
a negotiation submodule 14051 for negotiating transmission attribute information by the channel encoder relay server;
the middle rotator module 14052 is configured to send the channel multimedia data to the relay server through a channel encoder according to the transmission attribute information, so that the relay server sends the channel multimedia data to a receiving end according to the transmission attribute information.
Fig. 15 discloses a data processing apparatus, such as the data processing apparatus 150 shown in fig. 15, which includes a first obtaining module 1501, a second obtaining module 1502, and a sending module 1503, where the first obtaining module 1501 is configured to obtain transmission multimedia transmission data, perform packet processing on the transmission multimedia, obtain application multimedia data, and send the application multimedia data to an upper application; a second obtaining module 1502, configured to obtain application feedback information; the feedback information is generated by the upper layer application monitoring the receiving state data of the multimedia data according to the receiving state data; a sending module 1503, configured to send the application feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information after encoding the application feedback information to a sending end.
In one embodiment, the first obtaining module 1501 is specifically configured to: and receiving the transmission multimedia data through a data pool interface of a channel encoder, performing packet packaging processing on the transmission multimedia to obtain application multimedia data, and sending the application multimedia data to an upper application through a data pool callback function.
In one embodiment, the feedback information includes at least packet loss rate or delay.
In the embodiment of the present disclosure, the function of the channel layer may be implemented by a channel encoder.
Based on the data processing method described in the embodiment corresponding to fig. 2, an embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the data processing method described in the embodiment corresponding to fig. 2, which is not described herein again.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
Claims (12)
1. A method of data processing, the method comprising:
acquiring application multimedia data;
splitting the application multimedia data into a plurality of data packets, and adding media transmission layer headers to the data packets respectively to generate transmission multimedia data;
and sending the transmission multimedia data to a channel encoder, so that the channel encoder sends the channel multimedia data obtained by encoding the transmission multimedia data to a receiving end.
2. The data processing method according to claim 1, wherein the splitting the application multimedia data into a plurality of packets and adding media transport layer headers to the plurality of packets, respectively, and generating the transmission multimedia data comprises:
and splitting the application multimedia data into a plurality of data packets according to a Maximum Transmission Unit (MTU) of a transmission channel, and adding media transmission layer headers to the plurality of data packets respectively to generate transmission multimedia data.
3. The data processing method of claim 1, wherein the obtaining application multimedia data comprises:
acquiring application multimedia data through a data source interface of an upper application;
the sending the transmission multimedia data to a channel encoder so that the channel encoder sends the channel multimedia data after encoding the transmission multimedia data to a receiving end comprises:
and sending the transmission multimedia data to a channel encoder through a data source callback function, so that the channel encoder sends the channel multimedia data after encoding the transmission multimedia data to a receiving end.
4. The data processing method of claim 1, wherein the method further comprises:
acquiring application feedback information from a channel encoder; the application feedback information is obtained by decoding the received channel feedback information from the receiving end by the channel encoder;
and controlling the sending rate and/or the frame rate of the transmission multimedia data according to the application feedback information.
5. The data processing method of claim 1, wherein the method further comprises:
encoding the transmission multimedia data through a channel encoder to generate channel multimedia data; wherein encoding the transmission multimedia data comprises: adding header information to the multimedia data, wherein the header information carries channel type information;
and transmitting the channel multimedia data to a receiving end.
6. The data processing method of claim 5, wherein the transmitting the channel multimedia data to a receiving end comprises:
negotiating transmission attribute information with a transit server through a channel encoder;
and sending the channel multimedia data to the transfer server through a channel encoder according to the transmission attribute information, so that the transfer server sends the channel multimedia data to a receiving end according to the transmission attribute information.
7. A method of data processing, the method comprising:
acquiring transmission multimedia transmission data, packaging the transmission multimedia to obtain application multimedia data, and sending the application multimedia data to an upper application;
acquiring application feedback information; the feedback information is generated by the upper layer application monitoring the receiving state data of the multimedia data according to the receiving state data;
and sending the application feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information after encoding the application feedback information to a sending end.
8. The data processing method of claim 7, wherein the sending the feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information after encoding the application feedback information to a transmitting end comprises:
and preprocessing the feedback information according to a preset format and then sending the preprocessed feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information obtained by encoding the application feedback information to a sending end.
9. The data processing method of claim 7, wherein the obtaining transmission multimedia transmission data, performing packet packing processing on the transmission multimedia to obtain application multimedia data, and sending the application multimedia data to an upper application comprises:
and acquiring the transmission multimedia data through a data pool interface of a channel encoder, packaging the transmission multimedia data to obtain application multimedia data, and sending the application multimedia data to an upper application through a data pool callback function.
10. The data processing method of claim 7, wherein the feedback information at least includes a packet loss rate or a delay time.
11. A data processing apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring application multimedia data;
the media transmission module is used for splitting the application multimedia data into a plurality of data packets, and respectively adding media transmission layer headers to the plurality of data packets to generate transmission multimedia data;
and the sending module is used for sending the transmission multimedia data to a channel encoder, so that the channel encoder sends the channel multimedia data after encoding the transmission multimedia data to a receiving end.
12. A data processing apparatus, characterized in that the apparatus comprises:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring transmission multimedia transmission data, packaging the transmission multimedia to obtain application multimedia data and sending the application multimedia data to an upper application;
the second acquisition module is used for acquiring application feedback information; the feedback information is generated by the upper layer application monitoring the receiving state data of the multimedia data according to the receiving state data;
and the sending module is used for sending the application feedback information to a channel encoder, so that the channel encoder transmits the channel feedback information after encoding the application feedback information to a sending end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911041326.8A CN110943972A (en) | 2019-10-30 | 2019-10-30 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911041326.8A CN110943972A (en) | 2019-10-30 | 2019-10-30 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110943972A true CN110943972A (en) | 2020-03-31 |
Family
ID=69906801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911041326.8A Pending CN110943972A (en) | 2019-10-30 | 2019-10-30 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110943972A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111935102A (en) * | 2020-07-16 | 2020-11-13 | 西安万像电子科技有限公司 | Data processing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20010022785A1 (en) * | 2000-03-10 | 2001-09-20 | Pessi Pekka Tapio | Control of application-specific quality of service optimizations |
| CN101335851A (en) * | 2008-07-18 | 2008-12-31 | 深圳创维－Rgb电子有限公司 | Network television receiver capable of watching video live broadcast through network |
| CN101707701A (en) * | 2009-11-26 | 2010-05-12 | 上海交通大学 | Multi-stream information transmission system and transmission method thereof |
| CN101909208A (en) * | 2010-08-05 | 2010-12-08 | 浙江工业大学 | Video wireless transmission control method suitable for CDMA2000 |
| CN102404348A (en) * | 2011-12-29 | 2012-04-04 | 王怿忻 | Multimedia data transport protocol |
| CN103875225A (en) * | 2011-10-14 | 2014-06-18 | 高通股份有限公司 | Feedback channel for wireless display devices |
-
2019
- 2019-10-30 CN CN201911041326.8A patent/CN110943972A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20010022785A1 (en) * | 2000-03-10 | 2001-09-20 | Pessi Pekka Tapio | Control of application-specific quality of service optimizations |
| CN101335851A (en) * | 2008-07-18 | 2008-12-31 | 深圳创维－Rgb电子有限公司 | Network television receiver capable of watching video live broadcast through network |
| CN101707701A (en) * | 2009-11-26 | 2010-05-12 | 上海交通大学 | Multi-stream information transmission system and transmission method thereof |
| CN101909208A (en) * | 2010-08-05 | 2010-12-08 | 浙江工业大学 | Video wireless transmission control method suitable for CDMA2000 |
| CN103875225A (en) * | 2011-10-14 | 2014-06-18 | 高通股份有限公司 | Feedback channel for wireless display devices |
| CN102404348A (en) * | 2011-12-29 | 2012-04-04 | 王怿忻 | Multimedia data transport protocol |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111935102A (en) * | 2020-07-16 | 2020-11-13 | 西安万像电子科技有限公司 | Data processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3917137B1 (en) | Apparatus for transmitting and apparatus for receiving media contents in a multimedia system | |
| US10659502B2 (en) | Multicast streaming | |
| KR102301333B1 (en) | Method and apparatus for streaming dash content over broadcast channels | |
| EP3247121B1 (en) | Streaming media data transmission method, client and server | |
| KR102077883B1 (en) | Data communication system and method | |
| US10367873B2 (en) | Method of communicating data packets within data communication systems | |
| WO2017000720A1 (en) | Multicast transmission method, apparatus, and system for ott media | |
| AU2012207704B2 (en) | Apparatus and method for transmitting multimedia data in a broadcast system | |
| US9413797B2 (en) | Data communication system and method | |
| US20130308638A1 (en) | Synchronizing Multiple Transcoding Devices Utilizing Simultaneity of Receipt of Multicast Packets | |
| US20150249835A1 (en) | Method for adaptively transmitting fec parity data using cross-layer optimization | |
| US11363085B2 (en) | In-band quality data | |
| US10924524B2 (en) | Communication devices, communication data generation method, and communication data processing method | |
| US20090219919A1 (en) | Data transport container for transferring data in a high speed internet protocol network | |
| JP2009296164A (en) | Data transmitting device, control method therefor, and program | |
| CN110943972A (en) | Data processing method and device | |
| CN110557370B (en) | Method, system, electronic equipment and storage medium for pamir synchronization of terminal information | |
| CN114221909B (en) | Data transmission method, device, terminal and storage medium | |
| Pourmohammadi et al. | Streaming MPEG-4 over IP and Broadcast Networks: DMIF based architectures | |
| CN115733898A (en) | Data transmission method and related equipment | |
| CN115150476B (en) | EIGRP protocol message compression method, system, equipment and medium | |
| WO2024001621A1 (en) | Multimedia system with dynamic adaptation | |
| JP2005072973A (en) | Multimedia information relay system | |
| WO2009117905A1 (en) | Message filtering method, apparatus and system | |
| CN115085790A (en) | Channel environment self-adaption method and narrow bandwidth conference system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200331 |