CN109951254B - Data processing method and device and computer readable storage medium - Google Patents
Data processing method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN109951254B CN109951254B CN201910218961.2A CN201910218961A CN109951254B CN 109951254 B CN109951254 B CN 109951254B CN 201910218961 A CN201910218961 A CN 201910218961A CN 109951254 B CN109951254 B CN 109951254B
- Authority
- CN
- China
- Prior art keywords
- ith
- packet
- packet loss
- data
- redundant
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a data processing method and device and a computer readable storage medium; the method comprises the following steps: acquiring the ith multimedia data to be transmitted; when i is larger than 2, acquiring the (i-1) th packet loss rate and the (i-2) th packet loss level; obtaining an ith packet loss level according to the i-1 th packet loss rate, the i-2 th packet loss level, the fast-up and slow-down relation between preset adjacent packet loss levels and the mapping relation between the preset packet loss level, the packet loss rate, the data packet quantity and the redundant packet quantity, and further determining the ith data packet quantity and the ith redundant packet quantity corresponding to the ith packet loss level; coding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity to obtain an ith data packet and an ith redundant packet; and sending the ith data packet and the ith redundant packet to a receiving terminal.
Description
Technical Field
The embodiment of the invention relates to a data processing technology in the field of internet, in particular to a data processing method and device and a computer readable storage medium.
Background
With the development of multimedia internet, the transmission efficiency and anti-interference of multimedia data in the transmission process are receiving more and more attention.
At present, for the transmission of multimedia data, packet loss resistance processing and the like are adopted to ensure the efficiency of data transmission. For example, the anti-packet loss processing of Voice over IP (VoIP) is generally implemented by using Forward Error Correction (FEC).
However, the current methods for processing packet loss resistance all adopt a single packet loss resistance strategy, and cannot meet the processing requirements under different transmission conditions. For example, when the FEC technology is adopted, the FEC strategy is to perform data processing on multimedia data by adopting a fixed delay and a fixed redundancy rate, but values of the delay size and the redundancy rate are difficult to take, and if the delay and the redundancy rate are too small, the anti-packet loss capability is poor; the delay is high, the experience is poor, the redundancy rate is too large, the bandwidth is wasted, a good anti-packet loss effect cannot be achieved according to the actual transmission condition, and the accuracy of data transmission is low.
Disclosure of Invention
Embodiments of the present invention provide a data processing method and apparatus, and a computer-readable storage medium, which can improve an anti-packet loss effect, thereby improving accuracy of data transmission.
The technical scheme of the embodiment of the invention is realized as follows:
the embodiment of the invention provides a data processing method, which comprises the following steps:
acquiring the ith multimedia data to be transmitted; wherein i is a positive integer;
when i is larger than 2, acquiring the (i-1) th packet loss rate and the (i-2) th packet loss level; the i-1 th packet loss rate is the packet loss condition of the i-1 th to-be-transmitted multimedia data in the transmission process, and the i-2 th packet loss level is the severity corresponding to the packet loss condition of the i-1 th to-be-transmitted multimedia data in the transmission process, and is determined according to the acquired i-2 th packet loss rate when the i-1 th to-be-transmitted multimedia data is transmitted;
obtaining an ith packet loss level according to the ith-1 packet loss rate, the ith-2 packet loss levels, a fast-up and slow-down relation between preset adjacent packet loss levels and a mapping relation between the preset packet loss level, the packet loss rate, the number of data packets and the number of redundant packets, and further determining the number of ith data packets and the number of ith redundant packets corresponding to the ith packet loss level; the fast-up and slow-down relation between the preset adjacent packet loss levels represents the relation between the current packet loss level and the fast-up and slow-down of the last two packet loss levels;
coding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity to obtain an ith data packet and an ith redundant packet; the number of data packets in the ith data packet is the number of the ith redundant packet, and the number of the redundant packets in the ith redundant packet is the number of the ith redundant packet;
and sending the ith data packet and the ith redundant packet to a receiving terminal.
The embodiment of the invention also provides a data processing method, which comprises the following steps:
receiving the ith data packet, and counting the actual number of the ith data packet sent by the sending terminal in a preset time period;
obtaining the ith packet loss rate according to the actual number and the corresponding preset data packet number in the preset time period;
and sending the ith packet loss rate to the sending terminal for the sending terminal to use when the sending terminal carries out the (i + 1) th multimedia data to be transmitted.
An embodiment of the present invention provides a data processing apparatus, including:
the first acquisition unit is used for acquiring the ith multimedia data to be transmitted; wherein i is a positive integer;
the first receiving unit is used for acquiring the (i-1) th packet loss rate when i is larger than 2, wherein the (i-1) th packet loss rate is the packet loss condition of the (i-1) th multimedia data to be transmitted in the transmission process;
the first obtaining unit is further configured to obtain an i-2 th packet loss level, where the i-2 th packet loss level is a severity corresponding to a packet loss condition of the i-1 th multimedia data to be transmitted in a transmission process, and is determined according to the obtained i-2 th packet loss rate when the i-1 th multimedia data to be transmitted is transmitted;
a determining unit, configured to obtain an ith packet loss level according to the i-1 th packet loss rate, the i-2 th packet loss level, a fast-slow-down relationship between preset adjacent packet loss levels, and a mapping relationship between a preset packet loss level, a preset packet loss rate, a preset data packet number, and a preset redundant packet number, and further determine an ith data packet number and an ith redundant packet number corresponding to the ith packet loss level;
the encoding unit is used for encoding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity to obtain an ith data packet and an ith redundant packet; the number of data packets in the ith data packet is the number of the ith redundant packet, and the number of the redundant packets in the ith redundant packet is the number of the ith redundant packet;
and the first sending unit is used for sending the ith data packet and the ith redundant packet to a receiving terminal.
In the foregoing scheme, the determining unit is specifically configured to obtain the i-1 th packet loss level according to the preset packet loss level, a packet loss ratio, a mapping relationship between the number of data packets and the number of redundant packets, and the i-1 th packet loss ratio; obtaining the ith packet loss level according to a fast up-and-slow down relation among preset adjacent packet loss levels, the (i-1) th packet loss level and the (i-2) th packet loss level; and determining the ith data packet quantity and the ith redundant packet quantity according to the preset packet loss level, the packet loss rate, the mapping relation between the data packet quantity and the redundant packet quantity and the ith packet loss level.
In the foregoing scheme, the determining unit is further specifically configured to, when the i-1 th packet loss level is smaller than the i-2 th packet loss level, characterize the i-th packet loss level by using the fast-up-and-slow-down relationship between the preset adjacent packet loss levels, and obtain the i-th packet loss level by using the i-2 th packet loss level as a reference and reducing the preset level; when the ith-1 packet loss level is greater than or equal to the ith-2 packet loss level, the fast up-slow down relation between the preset adjacent packet loss levels represents that the ith packet loss level is the same as the ith-1 packet loss level, so that the ith packet loss level is obtained.
In the above scheme, the encoding unit is specifically configured to perform division encoding on the ith multimedia data to be transmitted according to the number of the ith data packet, so as to obtain the ith data packet; and according to the number of the ith redundant packets, performing preset coding processing on the ith data packet to generate the ith redundant packet corresponding to the ith data packet.
In the above scheme, the first sending unit is further configured to, after the ith to-be-transmitted multimedia data is obtained, when i is equal to 1, divide and encode the ith to-be-transmitted multimedia data according to the number of preset initial data packets, and transmit the encoded ith to-be-transmitted multimedia data to the receiving terminal; alternatively, the first and second electrodes may be,
the obtaining unit is further configured to obtain an i-1 th packet loss rate when i is equal to 2 after the i-th multimedia data to be transmitted is obtained;
the determining unit is further configured to determine the ith data packet number and the ith redundant packet number according to the i-1 th packet loss rate and the mapping relationship among the preset packet loss level, the preset packet loss rate, the preset data packet number and the preset redundant packet number;
the encoding unit is used for encoding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity to obtain an ith data packet and an ith redundant packet;
the first sending unit is configured to send the ith data packet and the ith redundant packet to the receiving terminal.
An embodiment of the present invention further provides a data processing apparatus, including:
the second receiving unit is used for receiving the ith data packet and counting the actual number of the ith data packet sent by the sending terminal received in a preset time period;
the second obtaining unit is used for obtaining the ith packet loss rate according to the actual number and the corresponding preset data packet number in the preset time period;
and the second sending unit is used for sending the ith packet loss rate to the sending terminal for the sending terminal to use when the ith +1 th multimedia data to be transmitted is carried out.
In the foregoing solution, the second receiving unit is specifically configured to obtain, within a preset delay time, an actual data packet actually received in the ith data packet; the preset delay time is used for representing the sum of the transmission time of the ith data packet and the transmission time of the reserved data packet.
In the above solution, the data processing apparatus further includes: a storage unit, a clearing unit and a decoding unit;
the storage unit is used for caching the actual data packet in a cache region after the ith data packet is received;
the second receiving unit is further configured to receive an ith redundant packet and cache the ith redundant packet in the cache region;
the decoding unit is used for decoding the actual data packet in combination with the actual redundant packet actually received in the ith redundant packet to obtain actual multimedia data when the preset delay time is reached;
the clearing unit is used for clearing the cache region.
In the above scheme, the preset delay time is a product of a sum of an ith data packet number corresponding to the ith data packet and a reserved data packet number and a preset time length of one data packet, and the ith data packet number is obtained from the sending terminal.
In the foregoing scheme, the decoding unit is specifically configured to obtain, when the preset delay time arrives, an actual packet loss number according to the number of the actual data packets and the preset number of data packets; acquiring a final redundant packet with an actual packet loss quantity from the actual redundant packet actually received from the ith redundant packet; and decoding the actual data packet by combining the final redundant packet to recover the actual multimedia data.
An embodiment of the present invention further provides a data processing apparatus, including:
a first memory for storing executable data processing instructions;
the first processor is used for realizing the data processing method provided by the embodiment of the invention when executing the executable data processing instruction stored in the first memory.
An embodiment of the present invention further provides a data processing apparatus, including:
a second memory for storing executable data processing instructions;
and the second processor is used for realizing the data processing method provided by the embodiment of the invention when executing the executable data processing instruction stored in the second memory.
The embodiment of the invention provides a storage medium, which stores executable data processing instructions and is used for causing a first processor to execute so as to realize the data processing method provided by the embodiment of the invention; or, executable data processing instructions are stored, and are used for causing the second processor to execute, so as to implement the data processing method provided by the embodiment of the invention.
The embodiment of the invention has the following beneficial effects:
the data processing device acquires the ith multimedia data to be transmitted; when i is greater than 2, determining the ith data packet number and the ith redundant packet number according to the i-1 th packet loss rate, the i-2 th packet loss level, the preset fast-slow-down relation between adjacent packet loss levels and the mapping relation between the preset packet loss level, the preset packet loss rate, the preset data packet number and the preset redundant packet number; and coding the ith multimedia data to be transmitted based on the ith data packet quantity and the ith redundant packet quantity to obtain the ith data packet and the ith redundant packet, and then transmitting. The data processing device can adjust the data packet and the redundant packet of the multimedia data to be transmitted at the time based on the fast-increasing and slow-decreasing relation between the transmission condition of the multimedia data to be transmitted and the preset adjacent packet loss level, namely, the redundancy rate is adjusted to resist the packet loss of network transmission, the redundancy rate adjustment of dynamic transmission is carried out based on the actual condition, the packet loss resistance is improved, and the accuracy of data transmission is improved.
Drawings
FIG. 1 is an alternative architectural diagram of a data processing system architecture according to an embodiment of the present invention;
FIG. 2 is a first diagram illustrating an alternative structure of a data processing apparatus according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an alternative structure of a data processing apparatus according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an alternative structure of a data processing apparatus according to another embodiment of the present invention;
FIG. 5 is a first flowchart illustrating an alternative data processing method according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an alternative flow chart of a data processing method according to an embodiment of the present invention;
fig. 7 is a first schematic flow chart of an alternative data processing method according to another embodiment of the present invention;
fig. 8 is a schematic diagram of an alternative flow chart of a data processing method according to another embodiment of the present invention;
FIG. 9 is a diagram of an exemplary data decoding process provided by an embodiment of the present invention;
FIG. 10 is a diagram of an exemplary real-time audio call interface provided by an embodiment of the present invention;
fig. 11 is a structural diagram of an exemplary real-time audio call scenario provided in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.
It should be noted that the terms "first \ second \ third" related to the embodiments of the present invention merely distinguish similar objects and do not represent specific ordering for the objects, and it should be understood that "first \ second \ third" may exchange specific orders or sequences where allowed, so that the embodiments of the present invention described herein can be implemented in an order other than that shown or described herein.
Before further detailed description of the embodiments of the present invention, terms and expressions mentioned in the embodiments of the present invention are explained, and the terms and expressions mentioned in the embodiments of the present invention are applied to the following explanations.
1) QoS: quality of Service (QoS) refers to a network that can provide better Service capability for specified network communication by using various basic technologies, and is a technology for solving the problems of network delay and congestion.
2) FEC: forward Error Correction, also called Forward Error Correction (FEC) is a method for increasing the reliability of data communication. In a one-way communication channel, once an error is found, its receiver will not be entitled to a transmission again. FEC is a method of transmitting redundant information using data that will allow a receiver to reconstruct the data when an error occurs in the transmission.
3) RTP: the Real-time Transport Protocol (RTP) is a network Transport Protocol. The RTP protocol is commonly used in streaming media systems, video conferencing systems (in conjunction with h.323 or SIP), making it the technological base of the IP telephony industry.
4) RTCP: the Real-time Transport Control Protocol (Real-time Transport Control Protocol or RTPControl Protocol or abbreviated RTCP) is a sister Protocol of the Real-time Transport Protocol (RTP). RTCP provides out-of-band control for RTP media streams.
The following describes an exemplary application of the data processing apparatus to implement the embodiment of the present invention, where the data processing apparatus provided in the embodiment of the present invention may be divided into a receiving end and a sending end, and one data processing apparatus may serve as the receiving end to receive multimedia data to be transmitted and may also serve as the sending end to transmit the multimedia data to be transmitted. Next, an exemplary application covering a receiving terminal and a transmitting terminal when the data processing apparatus is implemented as a terminal will be described. One terminal may serve as a receiving terminal to receive the multimedia data to be transmitted, and may also serve as a sending terminal to transmit the multimedia data to be transmitted.
In the embodiment of the present invention, the receiving terminal and the sending terminal are both data processing devices, and the receiving terminal and the sending terminal may transmit the multimedia data through the server, or may directly perform transmission of the multimedia data through the network, for example, Peer-To-Peer (P2P for short) connection between the sending terminal and the receiving terminal.
System architecture for relay transmission through server referring to fig. 1, fig. 1 is an optional architecture diagram of a system 100 provided in an embodiment of the present invention, in order to support an exemplary application, a data processing apparatus 400 (a sending terminal 400-1 and a receiving terminal 400-2 are exemplarily shown) is connected to a server 300 through a network 200, and the network 200 may be a wide area network or a local area network, or a combination of the two, and uses a wireless link to implement data transmission.
The sending terminal 400-1 is configured to obtain the ith multimedia data to be transmitted; wherein i is a positive integer; when i is larger than 2, acquiring the (i-1) th packet loss rate and the (i-2) th packet loss level; obtaining an ith packet loss level according to the i-1 th packet loss rate, the i-2 th packet loss level, the fast-up and slow-down relation between preset adjacent packet loss levels and the mapping relation between the preset packet loss level, the packet loss rate, the data packet quantity and the redundant packet quantity, and further determining the ith data packet quantity and the ith redundant packet quantity corresponding to the ith packet loss level; coding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity to obtain an ith data packet and an ith redundant packet; the number of data packets in the ith data packet is the number of the ith redundant packet, and the number of the redundant packets in the ith redundant packet is the number of the ith redundant packet; and sending the ith data packet and the ith redundant packet to a server.
And the server 300 is configured to send the ith data packet and the ith redundant packet to the receiving terminal.
The receiving terminal 400-2 is configured to receive the ith data packet, and count the actual number of the ith data packet sent by the sending terminal within a preset time period; obtaining the ith packet loss rate according to the actual number and the corresponding preset data packet number in a preset time period; and sending the ith packet loss rate to a sending terminal for the sending terminal to use when the (i + 1) th multimedia data to be transmitted is carried out.
The data processing apparatus provided in the embodiments of the present invention may be implemented as hardware or a combination of hardware and software, and various exemplary implementations of the data processing apparatus provided in the embodiments of the present invention are described below.
Referring to fig. 2, fig. 2 is a schematic diagram of an alternative structure of a data processing apparatus 400 according to an embodiment of the present invention, where the data processing apparatus 400 may be a mobile phone, a computer, a digital broadcast terminal, an information transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like, an exemplary structure of the apparatus implemented as a terminal may be foreseen according to the structure of the data processing apparatus, and therefore, the structure described herein should not be considered as a limitation, for example, some components described below may be omitted, or components not described below may be added to adapt to special requirements of some applications.
The data processing apparatus 400 shown in fig. 2 includes: at least one processor 410, memory 440, at least one network interface 420, and a user interface 430. The various components in the data processing device 400 are coupled together by a bus system 450. It is understood that the bus system 450 is used to enable connected communication between these components. The bus system 450 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 450 in fig. 2.
The user interface 430 may include a display, keyboard, mouse, trackball, click wheel, keys, buttons, touch pad or touch screen, etc.
The memory 440 in embodiments of the present invention is capable of storing data to support the operation of the data processing apparatus 400. Examples of such data include: any computer program for operation on data processing apparatus 400, such as operating system 441 and application programs 442. Operating system 441 comprises various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and for processing hardware-based tasks. The application programs 442 may include various application programs such as executable data processing instructions and the like.
As an example that the data processing method provided by the embodiment of the present invention is implemented by combining software and hardware, the data processing method provided by the embodiment of the present invention may be directly embodied as a combination of software modules executed by the processor 410, the software modules may be located in a storage medium located in the memory 440, and the processor 410 reads executable instructions included in the software modules in the memory 440, and completes the data processing method provided by the embodiment of the present invention in combination with necessary hardware (for example, including the processor 410 and other components connected to the bus 450).
By way of example, the Processor 410 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor or the like.
In addition, as an example that the data processing method provided by the embodiment of the present invention is implemented by hardware, the data processing method provided by the embodiment of the present invention may be implemented by directly using the processor 410 in the form of a hardware decoding processor, for example, by being executed by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components, to implement the data processing method provided by the embodiment of the present invention.
Illustratively, when the data processing apparatus is a sending terminal, the data processing apparatus provided in an embodiment of the present invention includes:
a first memory for storing executable data processing instructions;
and the first processor is used for realizing the data processing method corresponding to the transmitting terminal side when executing the executable data processing instruction stored in the first memory.
Illustratively, when the data processing apparatus is a receiving terminal, the data processing apparatus provided in an embodiment of the present invention includes:
a second memory for storing executable data processing instructions;
and the second processor is used for realizing the method corresponding to the receiving terminal side when executing the executable data processing instruction stored in the second memory.
An exemplary structure of software modules is explained below, and in some embodiments, as shown in fig. 3, the software modules in the memory 440 of the data processing apparatus may include, for the transmitting terminal 1: a first acquiring unit 10, a first receiving unit 11, a determining unit 12, an encoding unit 13 and a first transmitting unit 14.
A first obtaining unit 10, configured to obtain an ith multimedia data to be transmitted; wherein i is a positive integer;
the first receiving unit 11 is configured to obtain an i-1 th packet loss rate when i is greater than 2, where the i-1 th packet loss rate is a packet loss condition of the i-1 th multimedia data to be transmitted in a transmission process;
the first obtaining unit 10 is further configured to obtain an i-2 th packet loss level, where the i-2 th packet loss level is a severity corresponding to a packet loss condition of the i-1 th to-be-transmitted multimedia data in a transmission process, and is determined according to the obtained i-2 th packet loss rate when the i-1 th to-be-transmitted multimedia data is transmitted;
a determining unit 12, configured to obtain an ith packet loss level according to the i-1 th packet loss rate, the i-2 th packet loss level, a fast-slow-down relationship between preset adjacent packet loss levels, and a mapping relationship between a preset packet loss level, a preset packet loss rate, a preset data packet number, and a preset redundant packet number, and further determine an ith data packet number and an ith redundant packet number corresponding to the ith packet loss level;
the encoding unit 13 is configured to encode the ith to-be-transmitted multimedia data according to the ith data packet number and the ith redundant packet number to obtain an ith data packet and an ith redundant packet; the number of data packets in the ith data packet is the number of the ith redundant packet, and the number of the redundant packets in the ith redundant packet is the number of the ith redundant packet;
a first sending unit 14, configured to send the ith data packet and the ith redundant packet to a receiving terminal.
In some embodiments of the present invention, the determining unit 12 is specifically configured to obtain the i-1 th packet loss level according to the preset packet loss level, the packet loss ratio, a mapping relationship between the number of data packets and the number of redundant packets, and the i-1 th packet loss ratio; obtaining the ith packet loss level according to a fast up-and-slow down relation among preset adjacent packet loss levels, the (i-1) th packet loss level and the (i-2) th packet loss level; and determining the ith data packet quantity and the ith redundant packet quantity according to the preset packet loss level, the packet loss rate, the mapping relation between the data packet quantity and the redundant packet quantity and the ith packet loss level.
In some embodiments of the present invention, the determining unit 12 is further specifically configured to, when the i-1 th packet loss level is smaller than the i-2 th packet loss level, characterize the i-th packet loss level by using the fast ramping down relationship between the preset adjacent packet loss levels as a reference, and obtain the i-th packet loss level after a preset level is reduced, so as to obtain the i-th packet loss level; when the ith-1 packet loss level is greater than or equal to the ith-2 packet loss level, the fast up-slow down relation between the preset adjacent packet loss levels represents that the ith packet loss level is the same as the ith-1 packet loss level, so that the ith packet loss level is obtained.
In some embodiments of the present invention, the encoding unit 13 is specifically configured to perform division encoding on the ith multimedia data to be transmitted according to the number of the ith data packet, so as to obtain the ith data packet; and according to the number of the ith redundant packets, performing preset coding processing on the ith data packet to generate the ith redundant packet corresponding to the ith data packet.
In some embodiments of the present invention, the first sending unit 14 is further configured to, after the ith to-be-transmitted multimedia data is obtained, when i is equal to 1, divide and encode the ith to-be-transmitted multimedia data according to the number of preset initial data packets, and transmit the divided and encoded data to the receiving terminal; alternatively, the first and second electrodes may be,
the obtaining unit 10 is further configured to, after obtaining the ith multimedia data to be transmitted, obtain an i-1 th packet loss ratio when i is equal to 2;
the determining unit 12 is further configured to determine the ith data packet number and the ith redundant packet number according to the i-1 th packet loss ratio and the mapping relationship among the preset packet loss level, the preset packet loss ratio, the preset data packet number, the preset redundant packet number, and the preset mapping relationship among the preset data packet number and the preset redundant packet number;
the encoding unit 13 is configured to encode the ith to-be-transmitted multimedia data according to the ith data packet number and the ith redundant packet number to obtain an ith data packet and an ith redundant packet;
the first sending unit 14 is configured to send the ith data packet and the ith redundant packet to the receiving terminal.
In some embodiments, as shown in fig. 4, the software modules in the memory 440 of the data processing apparatus may include, for the receiving terminal 2: a second receiving unit 20, a second obtaining unit 21, a second sending unit 22, a storing unit 23, a clearing unit 24 and a decoding unit 25.
A second receiving unit 20, configured to receive the ith data packet, and count the actual number of the ith data packet sent by the sending terminal and received within a preset time period;
a second obtaining unit 21, configured to obtain an ith packet loss ratio according to the actual number and the number of corresponding preset data packets in the preset time period;
a second sending unit 22, configured to send the ith packet loss ratio to the sending terminal, so that the sending terminal can use the ith +1 multimedia data to be transmitted.
In some embodiments of the present invention, the second receiving unit 20 is specifically configured to obtain, within a preset delay time, an actual data packet actually received in the ith data packet; the preset delay time is used for representing the sum of the transmission time of the ith data packet and the transmission time of the reserved data packet.
In some embodiments of the invention, the data processing apparatus 400 further comprises: a storage unit 23, a clearing unit 24, and a decoding unit 25;
the storage unit 23 is configured to cache the actual data packet in a cache region after the ith data packet is received;
the second receiving unit 20 is further configured to receive an ith redundant packet, and cache the ith redundant packet in the cache region;
the decoding unit 25 is configured to, when a preset delay time arrives, perform decoding processing on the actual data packet in combination with an actual redundant packet actually received in the ith redundant packet to obtain actual multimedia data;
the clearing unit 24 is configured to clear the cache area.
In some embodiments of the present invention, the preset delay time is a product of a sum of an ith data packet number corresponding to the ith data packet and a reserved data packet number and a preset time length of one data packet, and the ith data packet number is obtained from the sending terminal.
In some embodiments of the present invention, the decoding unit 25 is specifically configured to, when the preset delay time arrives, obtain an actual packet loss number according to the number of the actual data packets and the preset number of data packets; acquiring a final redundant packet with an actual packet loss quantity from the actual redundant packet actually received from the ith redundant packet; and decoding the actual data packet by combining the final redundant packet to recover the actual multimedia data.
A data processing method implementing an embodiment of the present invention will be described in connection with the foregoing exemplary application and implementation of a data processing apparatus implementing an embodiment of the present invention.
Referring to fig. 5, fig. 5 is an optional flowchart of a data processing method according to an embodiment of the present invention, and is applied to a sending terminal, which will be described with reference to the steps shown in fig. 5.
S101, obtaining the ith multimedia data to be transmitted, wherein i is a positive integer.
S102, when i is larger than 2, obtaining an i-1 th packet loss rate and an i-2 th packet loss level, wherein the i-1 th packet loss rate is the packet loss condition of the i-1 th multimedia data to be transmitted in the transmission process, and the i-2 th packet loss level is the severity corresponding to the packet loss condition of the i-1 th multimedia data to be transmitted in the transmission process and is determined according to the obtained i-2 th packet loss rate when the i-1 th multimedia data to be transmitted is transmitted;
s103, obtaining an ith packet loss level according to the i-1 th packet loss rate, the i-2 th packet loss level, the fast-up and slow-down relation between the preset adjacent packet loss levels and the mapping relation between the preset packet loss level, the preset packet loss rate, the data packet quantity and the redundant packet quantity, and further determining the ith data packet quantity and the ith redundant packet quantity corresponding to the ith packet loss level.
S104, according to the number of the ith data packet and the number of the ith redundant packet, coding the ith multimedia data to be transmitted to obtain an ith data packet and an ith redundant packet; the number of the data packets in the ith data packet is the number of the ith redundant packet, and the number of the redundant packets in the ith redundant packet is the number of the ith redundant packet.
And S105, sending the ith data packet and the ith redundant packet to a receiving terminal.
In this embodiment of the present invention, the data processing apparatus performs data processing on multimedia data, where the multimedia data may include audio data, video data, and the like, and the embodiment of the present invention is not limited thereto.
The data processing method provided by the embodiment of the invention is applied to a scene of multimedia data transmission in real time, such as audio and video communication, live network broadcast, brand sharing, remote control and the like, and the embodiment of the invention is not limited.
It should be noted that the multimedia data in the embodiment of the present invention may be digital multimedia data, and in a scene of real-time multimedia data transmission, the multimedia data is acquired and transmitted in one segment.
In S101, the data processing apparatus acquires real-time multimedia data to be transmitted for a time period, and in the embodiment of the present invention, the data processing apparatus sorts, from a start time, one piece of multimedia data to be transmitted for each segment of multimedia data to be transmitted according to time, and performs data processing on each acquired piece of multimedia data to be transmitted.
In the embodiment of the invention, each section of multimedia data to be transmitted, which is acquired by the data processing device in real time, can be original multimedia data, the data processing device acquires the original multimedia data through the acquisition device, and then can perform data preprocessing on the original multimedia data and then compress the data, so that the network transmission is facilitated. The ith multimedia data is to sort the multimedia data to be transmitted according to time and has time sequentiality. Wherein i is a positive integer of 1 or more.
In the embodiment of the present invention, the acquisition device may be a microphone for acquiring audio data, or may be a camera for acquiring video data, and the like. The original multimedia data collected by the data processing device can be data in a non-compressed POM format.
It should be noted that, in the embodiment of the present invention, the data preprocessing may include: echo cancellation, noise reduction, and automatic gain processing, which utilize the quality of multimedia data, embodiments of the present invention are not limited.
In S102, the data processing device may obtain the ith to-be-transmitted multimedia data, and because the ith to-be-transmitted multimedia data has time sequence, the ith to-be-transmitted multimedia data obtained by the data processing device may be the 1 st to-be-transmitted multimedia data, may also be the 2 nd to-be-transmitted multimedia data, and so on, where i is a positive integer greater than or equal to 1. When i is larger than 2, the characterization data processing device has processed the first i-1 multimedia data to be transmitted. The data processing device already obtains the (i-1) th packet loss rate corresponding to the (i-1) th multimedia data to be transmitted and the (i-2) th packet loss level analyzed when the (i-2) th multimedia data to be transmitted are transmitted in the i-1 th multimedia data to be transmitted before processing. The method comprises the steps that the (i-1) th packet loss rate is the packet loss condition of the (i-1) th to-be-transmitted multimedia data in the transmission process, the (i-2) th packet loss level is the severity degree corresponding to the packet loss condition of the (i-1) th to-be-transmitted multimedia data in the transmission process, and the (i-1) th to-be-transmitted multimedia data are determined according to the obtained (i-2) th packet loss rate when the (i-1) th to-be-transmitted multimedia data are transmitted.
In the embodiment of the present invention, the level or hierarchy classification may be performed in advance for the severity of the packet loss condition in the data transmission process, and the packet loss level is used for representing that the packet loss condition is more severe when the packet loss level is higher.
It should be noted that, in the embodiment of the present invention, the packet loss rate is calculated in the receiving terminal and then transmitted back to the sending terminal. The specific calculation process will be described in the implementation process of the receiving terminal.
In S103, the data processing apparatus may determine the i-1 th packet loss level according to the i-1 th packet loss rate and the mapping relationship between the preset packet loss level, the packet loss rate, the number of data packets, and the number of redundant packets, and obtain the i-th packet loss level by combining the i-2 th packet loss level and the fast-increasing and slow-decreasing relationship between the preset adjacent packet loss levels, thereby determining the number of the i-th data packets and the number of the i-th redundant packets.
In the embodiment of the present invention, the preset packet loss levels, the packet loss rates, the mapping relationship between the number of data packets and the number of redundant packets, and the preset fast-slow-down relationship between the adjacent packet loss levels are all pre-designed relationships, wherein the preset fast-slow-down relationship between the adjacent packet loss levels adopts a setting principle that when determining the current packet loss level, the former two packet loss levels are determined, if the packet loss rates of the former two packet loss levels suddenly drop, which may be caused by an unstable factor, and the transmission condition of the former packet loss is possibly problematic, and the determination standard of the packet loss level cannot be directly determined according to the former packet loss level, the former packet loss level above the current packet loss level is taken as a reference, and the current packet loss level is obtained after slowly dropping a certain level, and the former two packet loss levels suddenly increase, which represents that the packet loss is possibly caused by the instability of transmission, and then the current packet loss level is determined by the former packet loss, showing that the transmission packet loss situation is serious at this time.
In some embodiments of the present invention, the data processing apparatus obtains an i-1 th packet loss level according to a preset packet loss level, a packet loss ratio, a mapping relationship between the number of data packets and the number of redundant packets, and the i-1 th packet loss ratio; obtaining an ith packet loss level according to a fast up-and-slow down relation among preset adjacent packet loss levels, the ith-1 th packet loss level and the ith-2 nd packet loss level; and determining the ith data packet quantity and the ith redundant packet quantity according to the preset packet loss level, the packet loss rate, the mapping relation between the data packet quantity and the redundant packet quantity and the ith packet loss level.
It should be noted that, in the embodiment of the present invention, the higher the packet loss rate, the higher the redundancy rate (the higher the ratio of the number of the redundancy packets to the number of the data packets), and the higher the packet loss level.
In the embodiment of the invention, the packet loss rate between 0 and 100 percent can be divided into intervals, and different intervals correspond to different packet loss levels and different data packet quantities and redundant packet quantities.
For example, the mapping relationship between the preset packet loss level, the packet loss rate, the number of data packets, and the number of redundant packets is shown in table 1.
TABLE 1
| Packet loss level L | Packet loss rate Rlost | Number of data packets N | Number of redundant packets M |
| 0 | (0％，5％] | N0 | M0 |
| 1 | (5％，10％] | N1 | M1 |
| 2 | (10％，15％] | N2 | M2 |
| 3 | (15％，20％] | N3 | M3 |
| 4 | (20％，25％] | N4 | M4 |
| 5 | (25％，30％] | N5 | M5 |
| 6 | (30％，100％] | N6 | M6 |
Wherein, the packet loss rate RlostDivided into 7 intervals, whereby the following relationship is obtained:
(0％，5％]corresponding to the packet loss level 0 and the number of the corresponding data packets N0The number of redundant packets is M0；
(5％，10％]Corresponding to the packet loss level 0 and the number of the corresponding data packets N1The number of redundant packets is M1；
(10％，15％]Corresponding to the packet loss level 0 and the number of the corresponding data packets N2The number of redundant packets is M2；
(15％，20％]Corresponding to the packet loss level 0 and the number of the corresponding data packets N3The number of redundant packets is M3；
(20％，25％]Corresponding to the packet loss level 0 and the number of the corresponding data packets N4The number of redundant packets is M4；
(25％，30％]Corresponding to the packet loss level 0 and the number of the corresponding data packets N5The number of redundant packets is M5；
(30％，100％]Corresponding to the packet loss level 0 and the number of the corresponding data packets N6The number of redundant packets is M6。
It should be noted that, in the embodiment of the present invention, the division interval and the principle of the packet loss rate interval, the representation form of the packet loss level, and the like may be designed according to experiments or actual requirements, and the embodiment of the present invention is not limited.
It can be understood that the packet loss ratios, packet loss levels, the number of data packets and the number of redundant packets are different, that is, different redundancy rates can be adopted for each transmitted multimedia data according to the difference of the packet loss ratios (i.e., different redundancy rates can be adopted for each transmitted multimedia data (i.e., different packet loss levels and different redundancy rates are adopted for each transmitted multimedia data)
M represents the number of redundant packets, and N represents the number of data packets) to implement data processing, so as to meet the actual requirement of each transmitted multimedia data, dynamically adjust the effect of packet loss resistance, and improve the accuracy of data transmission.
In the embodiment of the present invention, the process of the data processing device obtaining the ith packet loss level according to the fast-up-slow-down relationship between the preset adjacent packet loss levels, the ith-1 st packet loss level and the ith-2 nd packet loss level is as follows: when the (i-1) th packet loss level is smaller than the (i-2) th packet loss level, presetting a fast-up and slow-down relation between adjacent packet loss levels to represent that the (i) th packet loss level is obtained by reducing the preset levels on the basis of the (i-2) th packet loss level, so as to obtain the (i) th packet loss level; and when the ith-1 packet loss level is greater than or equal to the ith-2 packet loss level, presetting a fast-up and slow-down relation between adjacent packet loss levels to represent that the ith packet loss level is the same as the level of the ith-1 packet loss level, thereby obtaining the ith packet loss level.
And presetting a fast-up and slow-down relation among adjacent packet loss levels to represent a relation among the ith packet loss level, the (i-1) th packet loss level and the (i-2) th packet loss level. And presetting a fast and slow down relation between adjacent packet loss levels to represent a fast and slow down relation between the current packet loss level (i-th packet loss) and the last two packet loss levels (i-1-th packet loss and i-2-th packet loss), and obtaining the i-th packet loss level based on the relation.
Because the packet loss rate statistics time is short, a certain time window (usually much longer than the packet loss statistics time) is usually used for the occurrence of the packet loss event, and statistics can only reflect the packet loss at the previous moment, the jitter statistics error is easy to occur. Therefore, in the embodiment of the present invention, the packet loss level is determined by using a fast-slow-down method, where slow-down means that when the i-1 st packet loss level is a level that is fast lower than the i-2 nd packet loss level, the characteristic that the packet loss rate of data may suddenly become good is abnormal, and the i-2 nd packet loss level is subtracted by the preset level number to obtain the i-th packet loss level, instead of determining the number of data packets and the number of redundant packets by using the i-1 th packet loss level as a reference for next data transmission, the preset level number is certainly a smaller value, so that the variation of the packet loss level is smaller, a jitter error is smoothed, an extreme case that the packet loss level is lowered from a high packet loss level to a low packet loss level is prevented, and the packet loss resistance is greatly enhanced. And when the i-1 th packet loss level is higher than the i-2 th packet loss level, the representation data transmission is better, the packet loss rate is less, the subsequent processing can be directly carried out according to the i-1 th packet loss level without carrying out slow increase processing.
For example, the fast-up and slow-down relationship between preset adjacent packet loss levels can be expressed as formula (1), as follows:
wherein L isrealAnd the packet loss level is represented as ith, the packet loss level is represented as ith-1, the packet loss level is represented as ith-2, and K is a preset level number.
For example, assuming that the preset number of levels is 1, the fast-up and slow-down relationship between the preset adjacent packet loss levels can be expressed as formula (2), as follows:
it can be understood that the preset fast-up and fast-down relationship between adjacent packet loss levels uses the packet loss levels of the first 2 multimedia data to be transmitted, so that i needs to be greater than 2. In the following embodiments, the case where i is 2 or less will be described.
In S104 and S105, after acquiring the ith data packet quantity and the ith redundant packet quantity, that is, acquiring the redundancy rate at this time, the data processing device may perform data encoding processing according to the redundancy rate, that is, encode the ith to-be-transmitted multimedia data according to the ith data packet quantity and the ith redundant packet quantity, to obtain the ith data packet and the ith redundant packet.
In the embodiment of the invention, the data processing device divides and encodes the ith multimedia data to be transmitted according to the quantity of the ith data packet to obtain the ith data packet; and according to the quantity of the ith redundant packets, carrying out preset coding processing on the ith data packet to generate the ith redundant packet corresponding to the ith data packet.
In the embodiment of the present invention, the data processing apparatus may use RTP/RTCP and FEC techniques to implement packet loss prevention processing in a weak network environment. Compressing and encapsulating the ith multimedia data to be transmitted into a packet in an RTP/RTC P format, and then sending the packet to a receiving terminal through a network transmission module.
In the embodiment of the present invention, the data processing apparatus divides the ith multimedia data to be transmitted into the ith data packet in the ith data packet number, and then may generate the ith redundant packet in the ith redundant packet number by using a CRS (Cauthy Reed-Solomon Co des) coding method.
It should be noted that the ith data packet includes data packets of the ith data packet quantity; the ith redundant packet includes ith redundant packets of the ith redundant packet quantity.
In the embodiment of the invention, the data processing device encapsulates the acquired ith data packet and ith redundant packet into RTP/RTCP format and then transmits the RTP/RTCP format to the receiving terminal.
In some embodiments, referring to fig. 6, fig. 6 is an optional flowchart of the data processing method provided in the embodiments of the present invention, and based on fig. 5, after S101, S106-S110 may also be executed, which will be described with reference to each step.
And S106, when i is equal to 1, dividing and coding the ith multimedia data to be transmitted according to the number of the preset initial data packets, and then transmitting the ith multimedia data to be transmitted to the receiving terminal.
S107, when i is equal to 2, obtaining the i-1 th packet loss rate.
S108, determining the ith data packet quantity and the ith redundant packet quantity according to the (i-1) th packet loss rate and the mapping relation among the preset packet loss level, the preset packet loss rate, the preset data packet quantity and the preset redundant packet quantity.
S109, according to the ith data packet quantity and the ith redundant packet quantity, coding the ith multimedia data to be transmitted to obtain the ith data packet and the ith redundant packet.
And S110, sending the ith data packet and the ith redundant packet to a receiving terminal.
In the embodiment of the present invention, when the data processing apparatus performs the 1 st to-be-transmitted multimedia data and the 2 nd to-be-transmitted multimedia data, since the acquisition of the i-2 th packet loss level is not involved, it needs to perform additional processing.
In the embodiment of the invention, when i is equal to 1, after the 1 st multimedia data to be transmitted is divided according to the number of the preset initial data packets, the divided data packets with the number of the preset initial data packets are transmitted to the receiving terminal. When i is equal to 2, obtaining 1 st packet loss rate, determining the number of 2 nd data packets and the number of 2 nd redundant packets according to the 1 st packet loss rate and the mapping relation among the preset packet loss level, the packet loss rate, the number of data packets and the number of redundant packets, and coding the 2 nd multimedia data to be transmitted according to the number of the 2 nd data packets and the number of the 2 nd redundant packets to obtain the 2 nd data packet and the 2 nd redundant packet; and sending the 2 nd data packet and the 2 nd redundant packet to a receiving terminal.
It should be noted that, the specific data processing apparatus performs encoding processing on the 2 nd to-be-transmitted multimedia data according to the 2 nd data packet quantity and the 2 nd redundant packet quantity, so as to obtain the 2 nd data packet and the 2 nd redundant packet; the process of sending the 2 nd data packet and the 2 nd redundant packet to the receiving terminal is consistent with the principle of the ith multimedia data to be transmitted, and details are not repeated here.
It can be understood that, because the data processing apparatus can adjust the data packet and the redundant packet of the multimedia data to be transmitted at this time based on the transmission condition of the previous multimedia data to be transmitted, that is, the redundancy rate is adjusted to counteract the packet loss of network transmission, the redundancy rate adjustment of dynamic transmission is performed based on the actual condition, the packet loss resistance is improved, and thus the accuracy of data transmission is improved.
Referring to fig. 7, fig. 7 is an alternative flowchart of a data processing method provided in an embodiment of the present invention, which is applied in a receiving terminal and will be described with reference to the steps shown in fig. 7.
S201, receiving the ith data packet, and counting the actual number of the ith data packet sent by the sending terminal in a preset time period.
S202, obtaining the ith packet loss rate according to the actual number and the corresponding preset data packet number in the preset time period.
S203, the ith packet loss rate is sent to the sending terminal for the sending terminal to use when the ith +1 th multimedia data to be transmitted is carried out.
In S201, within a preset delay time, acquiring an actual data packet actually received from the ith data packet; the preset delay time is used for representing the sum of the time of the ith data packet transmission and the transmission time of the reserved data packet.
In the embodiment of the invention, in the transmission process of one-time multimedia data, the sending terminal sends the ith data packet and the ith redundant packet, and the condition of damage of priority degree or packet loss may occur in the data transmission process. Here, the receiving terminal may count an actual number of ith packets transmitted by the transmitting terminal, that is, an actually received number of packets, which are received within a period of time (that is, within a preset time period), during the receiving process, that is, during the receiving process of the ith packet. Here, the data processing apparatus (i.e., the receiving terminal) knows the packets corresponding to the time stamp in advance, for example, 50 packets for 1 second, and therefore, the data processing apparatus can know the number of the packets to be received in the preset time period, that is, the number of the corresponding preset packets in the preset time period. Therefore, the data processing device can calculate the ith packet loss rate according to the actual number and the corresponding preset data packet number in the preset time period.
It should be noted that, in the embodiment of the present invention, a manner of calculating the packet loss rate by the data processing apparatus may be expressed as formula (3), and the present invention is as follows:
wherein R islostIndicates packet loss rate, NeIndicating the corresponding number of preset data packets, N, within a preset time periodrRepresenting the actual number of packets received.
In the embodiment of the present invention, the data processing apparatus may send the ith packet loss rate to the sending terminal, so that the sending terminal may use the ith +1 th multimedia data to be transmitted as a coding reference.
It should be noted that, in the embodiment of the present invention, the data processing apparatus may encapsulate the packet loss rate into data in an RTCP format through an RTCP packet, and then send the data to the sending terminal, and when the sending terminal receives the packet loss rate, the sending terminal needs to decode the data in the same format, and then use the packet loss rate.
Further, since the sending terminal may encapsulate the ith data packet or the ith redundant packet according to the RTP/RTCP format when sending the ith data packet or the ith redundant packet, when the receiving terminal receives the ith data packet or the ith redundant packet, the receiving terminal needs to decode the ith data packet according to the same encapsulation format and then perform data processing.
It can be understood that, because the data processing apparatus can be based on the ith packet loss rate of the transmission of the ith to-be-transmitted multimedia data, the sending terminal can dynamically adjust the data packet and the redundant packet according to the ith packet loss rate when the ith +1 to-be-transmitted multimedia data is performed, that is, the redundant rate is adjusted, so as to counter the packet loss of network transmission, that is, the redundant rate adjustment of dynamic transmission is performed based on the actual situation, thereby improving the packet loss resistance, and further improving the accuracy of data transmission.
In some embodiments, referring to fig. 8, fig. 8 is an optional flowchart of the data processing method according to the embodiment of the present invention, based on fig. 7, a specific implementation of receiving the ith data packet in S201 is S2011, and based on the implementation of S2011, S2011 may be followed by executing S204-S207, which will be described with reference to each step.
S2011, acquiring an actual data packet actually received from the ith data packet within a preset delay time; the preset delay time is used for representing the sum of the time of the ith data packet transmission and the transmission time of the reserved data packet.
In the embodiment of the invention, the data processing device is provided with a delayed data receiving mechanism, and within the preset delay time set by the delayed data receiving mechanism, the data processing device acquires an actual data packet actually received in the ith data packet; the preset delay time is used for representing the sum of the time of the ith data packet transmission and the transmission time of the reserved data packet.
In the embodiment of the present invention, the reason why the receiving terminal introduces the mechanism of delaying receiving data is to wait for the redundant packet to recover when the packet is lost (if the packet is not lost, the mechanism of delaying receiving data is not present).
In the embodiment of the invention, the preset delay time is the product of the sum of the number of the ith data packet corresponding to the ith data packet and the number of the reserved data packets and the preset time length of one data packet, and the number of the ith data packet is obtained from the sending terminal.
It should be noted that, because the transmission of the data packet needs time, the preset delay time needs to consider the situation that the entire ith data packet can be received, and further, because the situation that the data packet is damaged or lost may occur during the transmission process, the redundant packet in the ith redundant packet may need to be used for data recovery, and therefore, the transmission time of the preset number of data packets needs to be considered for receiving the redundant packet.
The number of reserved data packets may be set according to an experiment or an actual situation, and the embodiment of the present invention is not limited.
In some embodiments of the present invention, the preset delay time may be obtained by using equation (4), as follows:
D＝(Nmax+X)×S (4)
wherein D is the delay time, NmaxThe number of ith data packets is represented, X represents the number of reserved data packets, and S represents the data duration contained in one data packet or redundant packet.
For example, when X is equal to 1, the preset delay time may be obtained by using equation (5), as follows:
D＝(Nmax+1)×S (5)
it can be understood that the delay time set by the data processing device reserves the time for receiving the redundant packet, so that the data packet receiving insufficiency caused by network jitter can be remedied by the received redundant packet, and the data recovery of the data packet is realized.
And S204, caching the actual data packet in a cache region.
S205, receiving the ith redundant packet, and caching the ith redundant packet in a cache region.
And S206, when the preset delay time is reached, decoding the actual data packet by combining the actual data packet with the actual redundant packet actually received in the ith redundant packet to obtain the actual multimedia data.
And S207, clearing the cache region.
In the embodiment of the present invention, in order to prevent jitter errors, a buffer area for buffering data is further provided in the data processing apparatus, and the buffer area may also be referred to as a jitter buffer area.
And the data processing device receives the ith actual data packet of the multimedia data to be transmitted within the preset delay time, decodes the ith actual data packet according to the same packaging format as that of the sending terminal, and buffers the ith actual data packet in the buffer area, namely buffers the ith actual data packet in the buffer area. Since the delay time is set to the reception time of the reserved packet, the data processing apparatus may receive the redundant packet of the i-th redundant packet after receiving the actual packet. Therefore, when the ith redundant packet is received, the ith redundant packet is cached in the cache region, and when the delay time is reached, the transmission of the ith to-be-transmitted multimedia data is ended, the actual data packet is transmitted to the decoding module to be decoded by combining with the actual redundant packet actually received in the ith redundant packet, and the cache region is emptied so as to be used when the (i + 1) th to-be-transmitted multimedia data is received.
In some embodiments of the present invention, when the preset delay time is reached, the data processing apparatus obtains the actual packet loss number according to the number of actual data packets and the preset number of data packets; acquiring a final redundant packet with the actual packet loss quantity from an actual redundant packet actually received from the ith redundant packet; and decoding the actual data packet by combining the final redundant packet to recover the actual multimedia data.
For example, as shown in fig. 9, assuming that the ith data packet includes N data packets, and the ith redundancy packet includes M redundancy packets, the transmitting terminal sends out N data packets: d0, D1, D2 and D3 and M redundant packets: r0 and R1, the receiving terminal receives D1, D2 and D3, and R0 within the delay time, so the receiving terminal can use R0 instead of D0, and recover the transmission data of this time by decoding according to D1, D2, D3, and R0. Wherein, D0 and R1 indicate that the data packet is lost or the redundant packet is lost.
In some embodiments of the present invention, when the delay time arrives and the ith redundant packet is not received, the data processing apparatus directly transmits the actual data packet to the decoding module for decoding processing, and clears the buffer area, so as to be used when receiving the (i + 1) th multimedia data to be transmitted.
It can be understood that, in the data transmission process, when a packet loss occurs in a data packet, a redundant packet corresponding to the packet loss data packet may be used to replace the packet loss data packet, so as to implement decoding of the originally transmitted multimedia data and improve the accuracy of data transmission.
In the following, an exemplary application of the embodiments of the present invention in a practical application scenario will be described.
The real-time audio call based on VoIP as shown in fig. 10 is used as an application scenario for explanation, and the process of the real-time audio call is a process of always transmitting multimedia data to be transmitted.
As shown in fig. 11, the transmitting terminal 1 (e.g., the mobile phone 1) realizes transmission of audio data with the receiving terminal 2 (e.g., the mobile phone 2) through the interface shown in fig. 10. The method comprises the steps that a sending terminal 1 acquires ith original multimedia data through an audio acquisition module 3, performs data preprocessing (such as echo cancellation, noise reduction, automatic gain and the like) on the ith original multimedia data through an audio preprocessing module 4, performs data compression coding through an audio coding module 5 to obtain ith multimedia data to be transmitted, and finally performs FEC coding through a QoS module 6, namely, when i is larger than 2, the i-1 th packet loss rate and the i-2 th packet loss level are obtained; obtaining an ith packet loss level according to the i-1 th packet loss rate, the i-2 th packet loss level, the fast-up and slow-down relation between preset adjacent packet loss levels and the mapping relation between the preset packet loss level, the packet loss rate, the data packet quantity and the redundant packet quantity, and further determining the ith data packet quantity and the ith redundant packet quantity corresponding to the ith packet loss level; coding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity to obtain an ith data packet and an ith redundant packet; the number of data packets in the ith data packet is the number of the ith redundant packet, and the number of the redundant packets in the ith redundant packet is the number of the ith redundant packet; and packaging the ith data packet and the ith redundant packet in an RTP/RTCP format and transmitting the packaged data packet and the ith redundant packet to the receiving terminal 2 through a network. The receiving terminal 2 acquires an actual data packet actually received in the ith data packet within a preset delay time; caching the actual data packet in a cache region 10; receiving an ith redundant packet, and caching the ith redundant packet in a cache region 10; when the preset delay time is reached, decoding the actual data packet by combining the actual data packet with the actual redundant packet actually received in the ith redundant packet through the audio decoding module 7 to obtain actual multimedia data, and playing the actual multimedia data through the audio playing module 9; the buffer area 10 is also emptied at the same time. The data packets or redundant packets received by the receiving terminal 2 need to be buffered in the buffer area 10 after FEC decoding and decoding in RTP/RTCP format are performed by the QoS module 8.
In an actual test, under the environment that 30% of packet loss exists in a simulation, the packet loss rate of the multimedia data recovered by the existing FEC technology is about 4%, while the packet loss rate of the multimedia data recovered by the method provided by the embodiment of the present invention is reduced to about 2%, and the accuracy of the obviously transmitted data is high.
The embodiment of the invention provides a storage medium stored with executable data processing instructions, wherein the executable data processing instructions are stored, and when being executed by a first processor, the executable data processing instructions cause the first processor to execute the data processing method on the side of a sending terminal provided by the embodiment of the invention; when the executable data processing instructions are executed by the second processor, the second processor is caused to execute the data processing method on the receiving terminal side provided by the embodiment of the invention.
In some embodiments, the storage medium may be memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.
In some embodiments, executable instructions may be written in any form of programming language (including compiled or interpreted languages), in the form of programs, software modules, scripts or code, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
By way of example, executable instructions may correspond, but do not necessarily have to correspond, to files in a file system, and may be stored in a portion of a file that holds other programs or data, such as in one or more scripts in a hypertext markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).
By way of example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.
The above description is only an example of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.
Claims (14)
1. A data processing method, comprising:
acquiring the ith multimedia data to be transmitted; wherein i is a positive integer;
when i is larger than 2, acquiring the (i-1) th packet loss rate and the (i-2) th packet loss level; the i-1 th packet loss rate is the packet loss condition of the i-1 th to-be-transmitted multimedia data in the transmission process, and the i-2 th packet loss level is the severity corresponding to the packet loss condition of the i-1 th to-be-transmitted multimedia data in the transmission process, and is determined according to the acquired i-2 th packet loss rate when the i-1 th to-be-transmitted multimedia data is transmitted;
obtaining an ith packet loss level according to the ith-1 packet loss rate, the ith-2 packet loss levels, a fast-up and slow-down relation between preset adjacent packet loss levels and a mapping relation between the preset packet loss level, the packet loss rate, the number of data packets and the number of redundant packets, and further determining the number of ith data packets and the number of ith redundant packets corresponding to the ith packet loss level; the fast-up and slow-down relation between the preset adjacent packet loss levels represents the relation between the current packet loss level and the fast-up and slow-down of the last two packet loss levels;
coding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity to obtain an ith data packet and an ith redundant packet; the number of data packets in the ith data packet is the number of the ith redundant packet, and the number of the redundant packets in the ith redundant packet is the number of the ith redundant packet;
sending the ith data packet and the ith redundant packet to a receiving terminal;
obtaining an ith packet loss level according to the i-1 th packet loss rate, the i-2 th packet loss level, a fast-up and slow-down relation between preset adjacent packet loss levels, and a mapping relation between the preset packet loss level, the preset packet loss rate, the preset data packet number and the preset redundant packet number, wherein the method comprises the following steps of:
determining the ith-1 packet loss level according to the mapping relation between the ith-1 packet loss rate and the preset packet loss level, the packet loss rate, the number of data packets and the number of redundant packets, and obtaining the ith packet loss level by combining the fast-up and slow-down relation between the ith-2 packet loss level and the preset adjacent packet loss level.
2. The method according to claim 1, wherein the obtaining an ith packet loss level according to the i-1 th packet loss rate, the i-2 th packet loss level, a preset fast-slow-down relationship between adjacent packet loss levels, and a preset mapping relationship between the packet loss level, the packet loss rate, the number of data packets, and the number of redundant packets, and further determining the ith number of data packets and the ith number of redundant packets corresponding to the ith packet loss level comprises:
obtaining the (i-1) th packet loss level according to the preset packet loss level, the packet loss rate, the mapping relation between the number of the data packets and the number of the redundant packets, and the (i-1) th packet loss rate;
obtaining the ith packet loss level according to a fast up-and-slow down relation among preset adjacent packet loss levels, the (i-1) th packet loss level and the (i-2) th packet loss level;
and determining the ith data packet quantity and the ith redundant packet quantity according to the preset packet loss level, the packet loss rate, the mapping relation between the data packet quantity and the redundant packet quantity and the ith packet loss level.
3. The method according to claim 2, wherein the obtaining the ith packet loss level according to a fast-slow-down relationship between preset adjacent packet loss levels, the i-1 st packet loss level and the i-2 nd packet loss level comprises:
when the i-1 th packet loss level is smaller than the i-2 th packet loss level, the fast-up-slow-down relation between the preset adjacent packet loss levels represents that the i-th packet loss level is obtained by reducing the preset level on the basis of the i-2 th packet loss level, so that the i-th packet loss level is obtained;
when the ith-1 packet loss level is greater than or equal to the ith-2 packet loss level, the fast up-slow down relation between the preset adjacent packet loss levels represents that the ith packet loss level is the same as the ith-1 packet loss level, so that the ith packet loss level is obtained.
4. The method of claim 1, wherein the encoding the ith multimedia data to be transmitted according to the ith data packet number and the ith redundant packet number to obtain an ith data packet and an ith redundant packet comprises:
dividing and coding the ith multimedia data to be transmitted according to the number of the ith data packets to obtain the ith data packets;
and according to the number of the ith redundant packets, performing preset coding processing on the ith data packet to generate the ith redundant packet corresponding to the ith data packet.
5. The method according to any one of claims 1 to 4, wherein after the obtaining of the ith multimedia data to be transmitted, the method further comprises:
when i is equal to 1, dividing and coding the ith multimedia data to be transmitted according to the number of preset initial data packets, and transmitting the coded data to the receiving terminal; alternatively, the first and second electrodes may be,
when i is equal to 2, acquiring the (i-1) th packet loss rate;
determining the ith data packet quantity and the ith redundant packet quantity according to the (i-1) th packet loss rate and the mapping relation among the preset packet loss level, the packet loss rate, the data packet quantity and the redundant packet quantity;
coding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity to obtain an ith data packet and an ith redundant packet;
and sending the ith data packet and the ith redundant packet to the receiving terminal.
6. A data processing method, comprising:
receiving the ith data packet, and counting the actual number of the ith data packet sent by the sending terminal in a preset time period;
obtaining the ith packet loss rate according to the actual number and the corresponding preset data packet number in the preset time period;
sending the ith packet loss rate to the sending terminal for the sending terminal to use when the sending terminal carries out the (i + 1) th multimedia data to be transmitted;
wherein the receiving the ith data packet comprises:
acquiring an actual data packet actually received in the ith data packet within a preset delay time; the preset delay time is used for representing the sum of the transmission time of the ith data packet and the transmission time of the reserved data packet;
when i is greater than 2, when the ith data packet is the ith multimedia data to be transmitted by the sending terminal, determining the (i-1) th packet loss level according to the (i-1) th packet loss rate and the mapping relation between the preset packet loss level, the packet loss rate, the number of data packets and the number of redundant packets, and obtaining the ith packet loss level by combining the (i-2) th packet loss level and the fast-increasing and slow-decreasing relation between the preset adjacent packet loss levels; determining the ith data packet quantity and the ith redundant packet quantity corresponding to the ith packet loss level; and coding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity.
7. The method of claim 6, wherein after receiving the ith packet, the method further comprises:
caching the actual data packet in a cache region;
receiving an ith redundant packet, and caching the ith redundant packet in the cache region;
when the preset delay time is reached, decoding the actual data packet in combination with the actual redundant packet actually received in the ith redundant packet to obtain actual multimedia data;
and emptying the cache region.
8. The method according to claim 6 or 7,
the preset delay time is the product of the sum of the number of the ith data packet corresponding to the ith data packet and the number of the reserved data packets and the preset duration of one data packet, and the number of the ith data packet is obtained from the sending terminal.
9. The method of claim 7, wherein the decoding the actual data packet in combination with the actual redundancy packet actually received in the ith redundancy packet to obtain the actual multimedia data when the predetermined delay time is reached comprises:
when the preset delay time is up, obtaining the actual packet loss number according to the actual data packet number and the preset data packet number;
acquiring a final redundant packet with an actual packet loss quantity from the actual redundant packet actually received from the ith redundant packet;
and decoding the actual data packet by combining the final redundant packet to recover the actual multimedia data.
10. A data processing apparatus, comprising:
the first acquisition unit is used for acquiring the ith multimedia data to be transmitted; wherein i is a positive integer;
the first receiving unit is used for acquiring the (i-1) th packet loss rate when i is larger than 2, wherein the (i-1) th packet loss rate is the packet loss condition of the (i-1) th multimedia data to be transmitted in the transmission process;
the first obtaining unit is further configured to obtain an i-2 th packet loss level, where the i-2 th packet loss level is a severity corresponding to a packet loss condition of the i-1 th multimedia data to be transmitted in a transmission process, and is determined according to the obtained i-2 th packet loss rate when the i-1 th multimedia data to be transmitted is transmitted;
a determining unit, configured to obtain an ith packet loss level according to the i-1 th packet loss rate, the i-2 th packet loss level, a fast-slow-down relationship between preset adjacent packet loss levels, and a mapping relationship between a preset packet loss level, a preset packet loss rate, a preset data packet number, and a preset redundant packet number, and further determine an ith data packet number and an ith redundant packet number corresponding to the ith packet loss level;
the encoding unit is used for encoding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity to obtain an ith data packet and an ith redundant packet; the number of data packets in the ith data packet is the number of the ith redundant packet, and the number of the redundant packets in the ith redundant packet is the number of the ith redundant packet;
the first sending unit is used for sending the ith data packet and the ith redundant packet to a receiving terminal;
the determining unit is further configured to determine the (i-1) th packet loss level according to the mapping relationship between the (i-1) th packet loss rate and the preset packet loss level, the packet loss rate, the number of data packets and the number of redundant packets, and obtain the (i) th packet loss level by combining the (i-2) th packet loss level and the fast-up and slow-down relationship between the preset adjacent packet loss levels.
11. A data processing apparatus, comprising:
the second receiving unit is used for receiving the ith data packet and counting the actual number of the ith data packet sent by the sending terminal received in a preset time period;
the second obtaining unit is used for obtaining the ith packet loss rate according to the actual number and the corresponding preset data packet number in the preset time period;
a second sending unit, configured to send the ith packet loss rate to the sending terminal, so that the sending terminal can use the ith +1 multimedia data to be transmitted;
the second receiving unit is further configured to obtain an actual data packet actually received from the ith data packet within a preset delay time; the preset delay time is used for representing the sum of the transmission time of the ith data packet and the transmission time of the reserved data packet;
when i is greater than 2, when the ith data packet is the ith multimedia data to be transmitted by the sending terminal, determining the (i-1) th packet loss level according to the (i-1) th packet loss rate and the mapping relation between the preset packet loss level, the packet loss rate, the number of data packets and the number of redundant packets, and obtaining the ith packet loss level by combining the (i-2) th packet loss level and the fast-increasing and slow-decreasing relation between the preset adjacent packet loss levels; determining the ith data packet quantity and the ith redundant packet quantity corresponding to the ith packet loss level; and then coding the ith multimedia data to be transmitted according to the ith data packet quantity and the ith redundant packet quantity.
12. A data processing apparatus, comprising:
a first memory for storing executable data processing instructions;
a first processor arranged to implement the method of any one of claims 1 to 5 when executing executable data processing instructions stored in said first memory.
13. A data processing apparatus, comprising:
a second memory for storing executable data processing instructions;
a second processor arranged to implement the method of any of claims 6 to 9 when executing executable data processing instructions stored in said second memory.
14. A computer-readable storage medium having stored thereon executable data processing instructions for causing a first processor to perform the method of any one of claims 1 to 5 when executed; alternatively, executable data processing instructions are stored for causing the second processor, when executed, to implement the method of any one of claims 6 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910218961.2A CN109951254B (en) | 2019-03-21 | 2019-03-21 | Data processing method and device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910218961.2A CN109951254B (en) | 2019-03-21 | 2019-03-21 | Data processing method and device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109951254A CN109951254A (en) | 2019-06-28 |
| CN109951254B true CN109951254B (en) | 2020-11-06 |
Family
ID=67011092
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910218961.2A Active CN109951254B (en) | 2019-03-21 | 2019-03-21 | Data processing method and device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109951254B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110890945B (en) * | 2019-11-20 | 2022-02-22 | 腾讯科技（深圳）有限公司 | Data transmission method, device, terminal and storage medium |
| CN111464262B (en) * | 2020-03-18 | 2022-03-25 | 腾讯科技（深圳）有限公司 | Data processing method, device, medium and electronic equipment |
| CN111818231B (en) * | 2020-07-06 | 2021-02-09 | 全时云商务服务股份有限公司 | Packet loss compensation method, device, data message transmission system and storage medium |
| CN113242473B (en) * | 2021-04-15 | 2023-04-21 | 深圳锐取信息技术股份有限公司 | Data processing method, system, data acquisition equipment and readable storage medium |
| CN113992304A (en) * | 2021-10-26 | 2022-01-28 | 新华三信息安全技术有限公司 | Packet loss resistance rate updating method and device, electronic equipment and storage medium |
| CN114039703B (en) * | 2021-11-26 | 2023-10-20 | 北京字节跳动网络技术有限公司 | Data transmission method, device, equipment and medium |
| CN114363214A (en) * | 2021-12-31 | 2022-04-15 | 中国信息通信研究院 | Method and device for switch anti-seismic test, computer equipment and storage medium |
| CN116015563A (en) * | 2022-12-30 | 2023-04-25 | 深圳市爱博医疗机器人有限公司 | Method, device, equipment and medium for processing packet loss of data transmission |
| CN117896546B (en) * | 2024-03-14 | 2024-06-07 | 浙江华创视讯科技有限公司 | Data transmission method, system, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107196746A (en) * | 2016-03-15 | 2017-09-22 | 中兴通讯股份有限公司 | Anti-dropout methods, devices and systems in real-time Communication for Power |
| CN107257265A (en) * | 2017-04-27 | 2017-10-17 | 深圳市鹏创软件有限公司 | Real-time transmission data stream forward error correction, equipment and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1659743B1 (en) * | 2004-11-19 | 2007-06-06 | Research In Motion Limited | Method and system for identifying degradation of a media service by determining a packet loss rate |
| JP4434242B2 (en) * | 2007-07-11 | 2010-03-17 | ソニー株式会社 | Transmission device, reception device, error correction system, transmission method, and error correction method |
| CN101505202B (en) * | 2009-03-16 | 2011-09-14 | 华中科技大学 | Adaptive error correction method for stream media transmission |
| CN103716718B (en) * | 2013-12-16 | 2017-03-01 | 广州华多网络科技有限公司 | The transmission method of packet and device |
| CN105704580B (en) * | 2016-01-21 | 2018-09-11 | 深圳比特新技术有限公司 | A kind of video transmission method |
| CN105897378A (en) * | 2016-04-06 | 2016-08-24 | 上海华为技术有限公司 | Method and device of reducing transmission packet loss rate |
-
2019
- 2019-03-21 CN CN201910218961.2A patent/CN109951254B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107196746A (en) * | 2016-03-15 | 2017-09-22 | 中兴通讯股份有限公司 | Anti-dropout methods, devices and systems in real-time Communication for Power |
| CN107257265A (en) * | 2017-04-27 | 2017-10-17 | 深圳市鹏创软件有限公司 | Real-time transmission data stream forward error correction, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109951254A (en) | 2019-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109951254B (en) | Data processing method and device and computer readable storage medium | |
| US11349900B2 (en) | Voice encoding and sending method and apparatus | |
| CN109729439B (en) | Real-time video transmission method | |
| WO2017157303A1 (en) | Anti-packet loss method, device and system for use in real-time communication | |
| JP6476197B2 (en) | Congestion control bit rate algorithm | |
| CN106341738B (en) | Bandwidth calculation method, server side and system for streaming media network transmission | |
| US8516346B2 (en) | Packet transmission apparatus, communication system and program | |
| CN111836079A (en) | Video code stream transmission method and device | |
| US20220021612A1 (en) | Data stream transmission method and device | |
| EP2312787A1 (en) | Method and device of data transmission | |
| US9451248B2 (en) | Data processing device and data processing method | |
| KR20180063223A (en) | Method and apparatus for controlling video data redundancy | |
| WO2006105713A1 (en) | Video transmission protection method based on h.264 | |
| US8347189B2 (en) | Data transmission system, program and method | |
| JP2008527862A (en) | Adaptive information delivery system using FEC feedback | |
| CN106105141A (en) | Realize the delivery acceleration device of extension transmission control function | |
| US20060130104A1 (en) | Network video method | |
| CN101521813B (en) | Method and device for processing media stream | |
| JP2010119133A (en) | Packet transmission device, communication system, and program | |
| CN115037416A (en) | Data forward error correction processing method, device, electronic equipment and storage medium | |
| JPWO2005122455A1 (en) | Two-way communication method, device, system and program | |
| CN113573003A (en) | Weak network-based audio and video real-time communication method, device and equipment | |
| CN116318545A (en) | Video data transmission method, device, equipment and storage medium | |
| CN112187817A (en) | Streaming media data transmission method, device, equipment and storage medium | |
| WO2021164405A1 (en) | Data encoding and decoding methods, and related device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |