CN113079103A - Audio transmission method, audio transmission device, electronic equipment and storage medium - Google Patents
Audio transmission method, audio transmission device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113079103A CN113079103A CN202110296197.8A CN202110296197A CN113079103A CN 113079103 A CN113079103 A CN 113079103A CN 202110296197 A CN202110296197 A CN 202110296197A CN 113079103 A CN113079103 A CN 113079103A
- Authority
- CN
- China
- Prior art keywords
- code rate
- audio
- receiving end
- redundant
- feedback
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/12—Avoiding congestion; Recovering from congestion
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/18—Vocoders using multiple modes
- G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/26—Flow control; Congestion control using explicit feedback to the source, e.g. choke packets
- H04L47/263—Rate modification at the source after receiving feedback
Abstract
The disclosure relates to an audio transmission method, an audio transmission device, an electronic device and a storage medium. The audio transmitting method includes: the method comprises the steps that a sending end sends audio coded with a first code rate to a receiving end according to the first code rate, wherein the first code rate is obtained by reducing the code rate after receiving network congestion feedback when the sending end sends the audio with a second code rate; and when the network congestion-free feedback is received from the receiving end, sending the audio which is coded by the first code rate and carries the redundant audio packet to the receiving end at a third code rate, wherein the third code rate is smaller than the second code rate and larger than the first code rate.
Description
Technical Field
The present disclosure relates to the field of signal processing, and in particular, to an audio transmitting method, an audio transmitting device, an electronic device, and a storage medium.
Background
In the audio communication process, the sending end sends the encoded audio packet to the receiving end through the network. And after receiving the audio packet, the receiving end decodes the audio packet and plays the audio packet. When the network state is poor, the available bandwidth of the network is very small (i.e., the network bandwidth is limited), and if the available bandwidth is smaller than the bandwidth required by the audio packet, congestion occurs, so that the audio packet cannot normally reach the receiving end, and normal conversation cannot be performed. At this time, the rate-limiting process is required, i.e. the sending end reduces the code rate of the audio. And if the network is improved, the audio coding rate is increased to a normal level.
The problem here is that "no speed limit is detected" may be due to a reduced code rate, and when the code rate is increased, the speed limit may be triggered again, which may result in the network being unavailable for a short time, and then a loop is entered in which the speed limit is detected > the reduced code rate > the speed limit is detected > the increased code rate > the speed limit is detected again. When network congestion feedback of a receiving end is received, a period of time may pass, and a transmitting end sends a lot of data to the network at the increased code rate, so that the network is seriously blocked.
Disclosure of Invention
The present disclosure provides an audio transmission method, apparatus, electronic device, and storage medium to at least solve a problem of network congestion due to an overshoot in a case where a network bandwidth is limited.
According to a first aspect of the embodiments of the present disclosure, there is provided an audio transmission method, including: the method comprises the steps that a sending end sends audio coded with a first code rate to a receiving end according to the first code rate, wherein the first code rate is obtained by reducing the code rate after receiving network congestion feedback when the sending end sends the audio with a second code rate; and when the network congestion-free feedback is received from the receiving end, sending the audio which is coded by the first code rate and carries the redundant audio packet to the receiving end at a third code rate, wherein the third code rate is smaller than the second code rate and larger than the first code rate.
Optionally, the method further comprises: when network congestion feedback is received from a receiving end after audio carrying redundant audio packets is sent at a third code rate, sending the audio which is coded at a first code rate and does not carry the redundant audio packets to the receiving end at the first code rate; and when the network congestion free feedback is received from the receiving end after the audio carrying the redundant audio packets is sent at the third code rate, sending the audio carrying a larger number of redundant audio packets to the receiving end at a fourth code rate, wherein the audio is coded at the first code rate and is coded at the first code rate than the audio carrying the redundant audio packets at the third code rate, and the fourth code rate is greater than the third code rate and less than or equal to the second code rate.
Optionally, the method further comprises: and when the network congestion free feedback is received from the receiving end after the audio which is coded by the first code rate and carries the redundant audio packet is sent by the fourth code rate which is equal to the second code rate, the audio which is coded by the second code rate and does not carry the redundant audio packet is sent by the second code rate.
Optionally low, the third code rate is calculated according to the following equation: the third code rate is the first code rate + mx (second code rate-first code rate)/N, where N is a positive integer greater than or equal to 1, m is a positive integer, and 1 ≦ m < N.
And the optional low and carried redundant audio packets are used for carrying out packet loss compensation on the audio.
Optionally, the method further comprises: when network congestion feedback is received from a receiving end after transmitting audio at a first code rate, the audio is transmitted at a lower code rate than the first code rate.
Optionally, m redundant audio packets are carried by every N packets of audio.
Optionally, when receiving a network congestion free feedback from a receiving end, sending, to the receiving end, an audio encoded at a first code rate and carrying a redundant audio packet at a third code rate, where the audio includes: and when the network congestion feedback is received from the receiving end within a preset time period after the audio is sent at the first code rate, sending the audio which is coded at the first code rate and carries the redundant audio packets to the receiving end at the third code rate.
Optionally, when receiving a network congestion free feedback from a receiving end after transmitting audio carrying redundant audio packets at a third code rate, transmitting audio carrying a greater number of redundant audio packets encoded at the first code rate to the receiving end at a fourth code rate than when transmitting audio at the third code rate previously, the method includes: and when network congestion free feedback is received from the receiving end within a preset time period after the audio carrying the redundant audio packets is sent at the third code rate, sending the audio carrying a larger number of redundant audio packets encoded at the first code rate to the receiving end at the fourth code rate than when the audio is sent at the third code rate previously.
Optionally, when receiving a network congestion free feedback from a receiving end after transmitting the audio carrying the redundant audio packets encoded at the first code rate at a fourth code rate equal to the second code rate, the transmitting the audio not carrying the redundant audio packets encoded at the second code rate includes: and when network congestion free feedback is received from a receiving end within a preset time period after the audio which carries the redundant audio packets and is coded by the first code rate is sent by the fourth code rate which is equal to the second code rate, the audio which does not carry the redundant audio packets and is coded by the second code rate is sent by the second code rate.
According to a second aspect of the embodiments of the present disclosure, there is provided an audio transmitting apparatus, the apparatus including: a feedback receiving unit configured to receive feedback on a network status from a receiving end; an audio transmitting unit configured to: sending the audio coded with the first code rate to a receiving end according to the first code rate, wherein the first code rate is obtained by reducing the code rate after receiving network congestion feedback when the sending end sends the audio with the second code rate; and when the network congestion-free feedback is received from the receiving end, sending the audio which is coded by the first code rate and carries the redundant audio packet to the receiving end at a third code rate, wherein the third code rate is smaller than the second code rate and larger than the first code rate.
Optionally, the audio transmitting unit is further configured to: when network congestion feedback is received from a receiving end after audio carrying redundant audio packets is sent at a third code rate, sending the audio which is coded at a first code rate and does not carry the redundant audio packets to the receiving end at the first code rate; and when the received audio carrying the redundant audio packets from the receiving end is the network congestion free feedback after the audio carrying the redundant audio packets is sent at the third code rate, sending the audio carrying a larger number of redundant audio packets coded at the first code rate to the receiving end at a fourth code rate, wherein the fourth code rate is greater than the third code rate and less than or equal to the second code rate.
Optionally, the audio transmitting unit is further configured to: and when network congestion free feedback is received from a receiving end after the audio which carries the redundant audio packets and is coded by the first code rate is sent by the fourth code rate which is equal to the second code rate, the audio which does not carry the redundant audio packets and is coded by the second code rate is sent by the second code rate.
Optionally, the third code rate is calculated according to the following equation: the third code rate is the first code rate + mx (second code rate-first code rate)/N, where N is a positive integer greater than or equal to 1, m is a positive integer, and 1 ≦ m < N.
Optionally, the carried redundant audio packets are used for packet loss compensation of the audio.
Optionally, the audio transmitting unit is further configured to: when network congestion feedback is received from a receiving end after transmitting audio at a first code rate, the audio is transmitted at a lower code rate than the first code rate.
Optionally, every N packets of audio carry m redundant audio packets.
Optionally, when receiving a network congestion free feedback from a receiving end, sending, to the receiving end, an audio encoded at a first code rate and carrying a redundant audio packet at a third code rate, where the audio includes: and when the network congestion feedback is received from the receiving end within a preset time period after the audio is sent at the first code rate, sending the audio which is coded at the first code rate and carries the redundant audio packets to the receiving end at the third code rate.
Optionally, when receiving a network congestion free feedback from a receiving end after transmitting audio carrying redundant audio packets at a third code rate, transmitting audio carrying a greater number of redundant audio packets encoded at the first code rate to the receiving end at a fourth code rate than when transmitting audio at the third code rate previously, the method includes: and when network congestion free feedback is received from the receiving end within a preset time period after the audio carrying the redundant audio packets is sent at the third code rate, sending the audio carrying a larger number of redundant audio packets encoded at the first code rate to the receiving end at the fourth code rate than when the audio is sent at the third code rate previously.
Optionally, when receiving a network congestion free feedback from a receiving end after transmitting the audio carrying the redundant audio packets encoded at the first code rate at a fourth code rate equal to the second code rate, the transmitting the audio not carrying the redundant audio packets encoded at the second code rate includes: and when network congestion free feedback is received from a receiving end within a preset time period after the audio which carries the redundant audio packets and is coded by the first code rate is sent by the fourth code rate which is equal to the second code rate, the audio which does not carry the redundant audio packets and is coded by the second code rate is sent by the second code rate.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the method as described above.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform the method as described above.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising computer instructions which, when executed by a processor, implement the method as described above.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects: the embodiment of the disclosure can effectively avoid network congestion caused by over-sending under the condition that the network bandwidth is limited by sending the audio carrying the redundant audio packet encoded at the first code rate at the third code rate (which is less than the second code rate and greater than the first code rate) instead of directly sending the audio encoded at the second code rate when the network is not congested.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is an exemplary system architecture to which exemplary embodiments of the present disclosure may be applied;
fig. 2 is a flowchart of an audio transmission method of an exemplary embodiment of the present disclosure;
FIG. 3 is a schematic diagram illustrating carrying redundant audio packets of an exemplary embodiment of the present disclosure;
fig. 4 is a block diagram of an audio transmitting apparatus of an exemplary embodiment of the present disclosure;
fig. 5 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The embodiments described in the following examples do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
In this case, the expression "at least one of the items" in the present disclosure means a case where three types of parallel expressions "any one of the items", "a combination of any plural ones of the items", and "the entirety of the items" are included. For example, "include at least one of a and B" includes the following three cases in parallel: (1) comprises A; (2) comprises B; (3) including a and B. For another example, "at least one of the first step and the second step is performed", which means that the following three cases are juxtaposed: (1) executing the step one; (2) executing the step two; (3) and executing the step one and the step two.
Fig. 1 illustrates an exemplary system architecture 100 in which exemplary embodiments of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few. A user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages (e.g., an audio-video data upload request, an audio-video data acquisition request), etc. Various communication client applications, such as a singing application, audio/video recording software, an audio/video player, an instant messaging tool, a mailbox client, social platform software, and the like, may be installed on the terminal devices 101, 102, and 103. The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a display screen and capable of playing and recording audio and video, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like. When the terminal device 101, 102, 103 is software, it may be installed in the electronic devices listed above, it may be implemented as a plurality of software or software modules (for example, to provide distributed services), or it may be implemented as a single software or software module. And is not particularly limited herein.
The terminal devices 101, 102, 103 may be equipped with an image capturing device (e.g., a camera) to capture video data. In practice, the smallest visual unit that makes up a video is a Frame (Frame). Each frame is a static image. Temporally successive sequences of frames are composited together to form a motion video. Further, the terminal apparatuses 101, 102, 103 may also be mounted with a component (e.g., a speaker) for converting an electric signal into sound to play the sound, and may also be mounted with a device (e.g., a microphone) for converting an analog audio signal into a digital audio signal to pick up the sound.
The server 105 may be a server providing various services, such as a background server providing support for multimedia applications installed on the terminal devices 101, 102, 103. The background server can analyze, store and the like the received audio and video data uploading request and other data, and can also receive the audio and video data acquisition request sent by the terminal equipment 101, 102 and 103 and feed back the audio and video data indicated by the audio and video data acquisition request to the terminal equipment 101, 102 and 103.
The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be noted that the audio transmission method provided by the embodiment of the present disclosure is generally executed by the terminal apparatuses 101, 102, and 103, and accordingly, the audio transmission device is generally disposed in the terminal apparatuses 101, 102, and 103.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation, and the disclosure is not limited thereto.
Fig. 2 is a flowchart of an audio transmission method of an exemplary embodiment of the present disclosure.
In step S201, the transmitting end transmits audio encoded at a first code rate to the receiving end according to the first code rate. Here, the audio is transmitted at the first code rate, and the audio is encoded at the first code rate, and thus, the transmission code rate of the audio is equal to the encoding code rate of the audio in step S201. The transmission code rate refers to the number of bits transmitted per unit time. The coding rate refers to the number of bits generated per unit time. According to an exemplary embodiment, the first code rate may be obtained by reducing the code rate after receiving the network congestion feedback when the transmitting end transmits the audio at the second code rate. Therefore, the first code rate is smaller than the second code rate. As an example, the network may be defaulted to a normal state when audio is transmitted at the second code rate (e.g., audio encoded at the second code rate), i.e., the network is not congested. For convenience of description, hereinafter, the first code rate may be denoted as BRn, and the second code rate may be denoted as BRn-1. If the received network state feedback has a value of true, network congestion is indicated, that is, the network congestion feedback is received, and if the call is not available due to the fact that BRn-1 is continuously used, the audio code rate needs to be reduced, for example, audio is sent at a code rate BRn smaller than BRn-1. At this time, since the code rate of BRn is lower than that of BRn-1, the carried audio information is less, so the sound quality is inferior to that of BRn-1.
When receiving audio (audio packets), the receiving end may determine whether the current network is congested using a congestion control algorithm (e.g., a GCC algorithm of WebRTC may be used), and send a network status feedback to the sending end. Any congestion control algorithm may be used to determine whether the current network is congested, and the present disclosure is not limited in this respect.
After the audio is sent at the first code rate, the network state feedback may continue to be received from the receiving end, for example, if the value of the network state feedback is true, it indicates that the network is congested, that is, sending the audio at the current code rate may cause network congestion, and at this time, the code rate needs to be further reduced. That is, when network congestion feedback is received from a receiving end after transmitting audio at a first code rate, the audio is transmitted at a lower code rate than the first code rate.
However, if the value of the network status feedback is false, it indicates that the network is not congested, that is, the network non-congestion feedback is received at this time, which indicates that the current bitrate does not cause congestion. At this time, the code rate needs to be increased, however, if the code rate is directly increased to the code rate used before the current code rate, the network may be immediately unavailable, and since the network state feedback is fed back from the receiving end (far end), a certain time is required, the sending end continues to send data to the network with the increased code rate during this time, which may cause network congestion. Therefore, according to the embodiment of the present disclosure, in step S202, when the network congestion free feedback is received from the receiving end, the audio carrying the redundant audio packets encoded at the first code rate is transmitted to the receiving end at the third code rate. Here, the third code rate is smaller than the second code rate and larger than the first code rate.
Because the audio transmission method of the present disclosure does not directly increase the code rate to the second code rate in step S202, but transmits the audio carrying the redundant audio packet at the third code rate that is smaller than the second code rate and larger than the first code rate while maintaining the current coding code rate (the first code rate) (that is, the coding code rate is maintained unchanged but the actual transmission code rate is increased), the network congestion caused by the over-transmission under the condition of limited network bandwidth can be effectively avoided.
Furthermore, since the feedback received from the receiving end may not be necessarily accurate, optionally, according to an exemplary embodiment, in step S202, when all the received feedback from the receiving end within a predetermined time period (hereinafter, may be denoted as T) after the audio is transmitted at the first code rate is network congestion free feedback, the audio carrying the redundant audio packets encoded at the first code rate is transmitted to the receiving end at the third code rate. Because the audio carrying the redundant audio packet is sent to the receiving end at the third code rate under the condition that the feedback received in the preset time period T after the audio is sent indicates that the network is not congested, the accuracy of judging the network state can be improved, and the audio is further sent under the condition that the network is really good.
Specifically, in step S202, a code rate difference BRd between the second code rate BRn-1 and the first code rate BRn, BRn-1-BRn, may be first calculated, and then, the audio carrying the redundant audio packets encoded at the first code rate BRn may be transmitted at a third code rate between the second code rate BRn-1 and the first code rate BRn. Fig. 3 is a schematic diagram illustrating carrying redundant audio packets of an exemplary embodiment of the present disclosure. As shown in fig. 3 (a), one more redundant audio packet may be carried in every N packets. And if the size of the packet coded by the second code rate BRn is Sn, the size Sn of the redundant audio packet is BRd/BRn, and the reduced code rate is BRd/N. At this time, although the coding rate is BRn, the actual transmission rate is BRn + (BRd/N), that is, the third rate is BRn + (BRd/N). Although in the above example, one more redundant audio packet is carried out in every N packets, the present disclosure is not limited thereto, and more than one redundant audio packet may be carried in every N packets, for example, two redundant audio packets, three redundant audio packets, and so on are carried in every N packets. According to an exemplary embodiment, the third code rate (i.e., the actual transmission code rate) is the first code rate + m × (second code rate — first code rate)/N. Here, N is a positive integer greater than or equal to 1, m is a positive integer and 1. ltoreq. m < N. That is, every N packets of audio may carry m redundant audio packets.
According to an exemplary embodiment of the present disclosure, the carried redundant audio packets may be used for packet loss compensation. For example, if the redundant audio packet carried in every N packets is one of the N packets, when the packet is lost, packet loss compensation can be performed using the redundant audio packet carried, so that the influence caused by data loss can be prevented.
Optionally, after step S202, the audio transmitting method transmits, to the receiving end, the audio encoded at the first code rate and not carrying the redundant audio packet, when the network congestion feedback is received from the receiving end after the audio carrying the redundant audio packet is transmitted at the third code rate. For example, if the network status feedback received after step S203 is true, i.e. indicates network congestion, it indicates that the network bandwidth is not recovered at this time, the bitrate of the audio should be stopped immediately, and the audio is still transmitted at the original first bitrate BRn. In addition, after the predetermined time period T elapses, the code rate of the audio may be increased to the third code rate, that is, the audio encoded at the first code rate and carrying the redundant audio packets may be continuously transmitted to the receiving end at the third code rate, so as to probe the network status.
Optionally, after step S202, when network congestion free feedback (for example, the value of the network status feedback is false) is received from the receiving end after the audio carrying the redundant audio packets is transmitted at the third code rate, the audio carrying a greater number of redundant audio packets encoded at the first code rate is transmitted to the receiving end at the fourth code rate than when the audio was previously transmitted at the third code rate. Here, the fourth code rate may be greater than the third code rate and less than or equal to the second code rate. Assuming that step S203 carries one redundant audio packet every N packets as shown in (a) of fig. 3, at this time, for example, two redundant audio packets may be carried every N packets as shown in (b) of fig. 3, but the number of redundant audio packets carried at this time is not limited to two, and may also be three, four, and so on, just as long as there are more redundant audio packets than are carried when the audio is originally transmitted at the third code rate in step S202.
Further, as described above, since the feedback received from the receiving end may not be necessarily accurate, optionally, according to an exemplary embodiment, when all of the network non-congestion feedback is received from the receiving end within a predetermined time period after the audio carrying the redundant audio packets is transmitted at the third code rate, the audio carrying a greater number of redundant audio packets encoded at the first code rate is transmitted to the receiving end at the fourth code rate than when the audio was previously transmitted at the third code rate. Because the audio carrying a larger number of redundant audio packets is sent to the receiving end at the fourth code rate under the condition that the feedback received in the preset time period T after the audio is sent indicates that the network is not congested, the accuracy of judging the network state can be improved, and the audio is further sent under the condition that the network is really good.
Optionally, when network congestion free feedback (for example, a value of network status feedback is false) is received from the receiving end after the audio carrying the redundant audio packets and encoded at the first code rate is transmitted at a fourth code rate equal to the second code rate, the audio carrying no redundant audio packets and encoded at the second code rate is transmitted at the second code rate. That is to say, after the sending code rate of the audio is gradually increased to the second code rate, if the received value of the network state feedback is still false, the coding code rate of the audio is increased to the second code rate BRn-1, and at this time, the redundant audio packet is not carried any more. The voice quality and continuous availability of the call can be maintained because the coding rate is increased from the lower second code rate to the higher second code rate when the network bandwidth is sufficient.
Similarly, in order to ensure the accuracy of the network state judgment, when network congestion free feedback is received from a receiving end within a preset time period after the audio carrying the redundant audio packets and encoded at the first code rate is transmitted at the fourth code rate which is equal to the second code rate, the audio carrying no redundant audio packets and encoded at the second code rate is transmitted at the second code rate.
Thus, if the network bandwidth becomes better than the second code rate BRn-1, the code rate can be increased from the first code rate BRn to the second code rate BRn-1 after time N x T. If the network bandwidth is between the first code rate BRn and the second code rate BRn-1, then only a small amount of data is overflowed onto the network when feedback indicating network congestion is received, and no serious network congestion is caused.
Fig. 4 is a block diagram of an audio transmitting apparatus of an exemplary embodiment of the present disclosure.
Referring to fig. 4, the audio transmitting apparatus 400 may include a feedback receiving unit 401 and an audio transmitting unit 402. Specifically, the feedback receiving unit 401 may be configured to receive feedback on the network status from the receiving end. The audio transmitting unit 402 may be configured to: sending the audio coded with the first code rate to a receiving end according to the first code rate, wherein the first code rate is obtained by reducing the code rate after receiving network congestion feedback when the sending end sends the audio with the second code rate; and when the network congestion-free feedback is received from the receiving end, transmitting the audio which is coded by the first code rate and carries the redundant audio packet to the receiving end at the third code rate. Here, the third code rate is smaller than the second code rate and larger than the first code rate. According to the audio transmitting device of the exemplary embodiment of the disclosure, since the code rate is not directly increased to the second code rate when the congestion-free feedback of the network is received, but the audio carrying the redundant audio packet is transmitted at the third code rate (smaller than the second code rate and larger than the first code rate) under the condition of keeping the current coding code rate (the first code rate) (that is, the coding code rate is kept unchanged but the actual transmission code rate is increased), the network congestion caused by the over-sending under the condition of limited network bandwidth can be effectively avoided.
Since the audio transmission method shown in fig. 2 can be performed by the audio transmission apparatus 400 shown in fig. 4, and the audio transmission unit 402 can respectively perform operations corresponding to step S201 and step S202 in fig. 2, any relevant details related to the operations performed by the audio transmission unit 402 in fig. 4 can be referred to the corresponding description related to fig. 2, and are not repeated here. In addition, optionally, the operations that are also executable after step S202 may also be executed by the audio sending unit, and details of these operations may also refer to what is mentioned above when these operations are described, which is also not described herein again.
Further, it should be noted that although the audio transmitting apparatus 400 is described above as being divided into units for respectively performing the corresponding processes, it is clear to those skilled in the art that the processes performed by the units described above may also be performed without any specific unit division by the audio transmitting apparatus 400 or without explicit demarcation between the units. In addition, the audio transmitting apparatus 400 may further include other units, for example, a storage unit and the like.
Fig. 5 is a block diagram of an electronic device 500 according to an embodiment of the disclosure. Referring to fig. 5, an electronic device 500 may include at least one memory 501 having a set of computer-executable instructions stored therein that, when executed by the at least one processor, perform an audio transmission method according to an embodiment of the disclosure and at least one processor 502.
By way of example, the electronic device may be a PC computer, tablet device, personal digital assistant, smartphone, or other device capable of executing the set of instructions described above. The electronic device need not be a single electronic device, but can be any collection of devices or circuits that can execute the above instructions (or sets of instructions) either individually or in combination. The electronic device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces with local or remote (e.g., via wireless transmission).
In an electronic device, a processor may include a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a programmable logic device, a special-purpose processor system, a microcontroller, or a microprocessor. By way of example, and not limitation, processors may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, and the like.
The processor may execute instructions or code stored in the memory, which may also store data. The instructions and data may also be transmitted or received over a network via a network interface device, which may employ any known transmission protocol.
The memory may be integral to the processor, e.g., RAM or flash memory disposed within an integrated circuit microprocessor or the like. Further, the memory may comprise a stand-alone device, such as an external disk drive, storage array, or any other storage device usable by a database system. The memory and the processor may be operatively coupled or may communicate with each other, such as through an I/O port, a network connection, etc., so that the processor can read files stored in the memory.
In addition, the electronic device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device may be connected to each other via a bus and/or a network.
According to an embodiment of the present disclosure, there may also be provided a computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform an audio transmission method according to an exemplary embodiment of the present disclosure. Examples of the computer-readable storage medium herein include: read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD + R, CD-RW, CD + RW, DVD-ROM, DVD-R, DVD + R, DVD-RW, DVD + RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or compact disc memory, Hard Disk Drive (HDD), solid-state drive (SSD), card-type memory (such as a multimedia card, a Secure Digital (SD) card or a extreme digital (XD) card), magnetic tape, a floppy disk, a magneto-optical data storage device, an optical data storage device, a hard disk, a magnetic tape, a magneto-optical data storage device, a, A solid state disk, and any other device configured to store and provide a computer program and any associated data, data files, and data structures to a processor or computer in a non-transitory manner such that the processor or computer can execute the computer program. The computer program in the computer-readable storage medium described above can be run in an environment deployed in a computer apparatus, such as a client, a host, a proxy device, a server, and the like, and further, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed fashion by one or more processors or computers.
According to an embodiment of the present disclosure, there may also be provided a computer program product comprising computer instructions which, when executed by a processor, implement an audio transmission method according to an embodiment of the present disclosure.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. An audio transmission method, comprising:
the method comprises the steps that a sending end sends audio coded with a first code rate to a receiving end according to the first code rate, wherein the first code rate is obtained by reducing the code rate after receiving network congestion feedback when the sending end sends the audio with a second code rate;
when the network congestion free feedback is received from the receiving end, the audio which is coded by the first code rate and carries the redundant audio packet is sent to the receiving end by the third code rate,
and the third code rate is less than the second code rate and greater than the first code rate.
2. The method of claim 1, further comprising:
when network congestion feedback is received from a receiving end after audio carrying redundant audio packets is sent at a third code rate, sending the audio which is coded at a first code rate and does not carry the redundant audio packets to the receiving end at the first code rate;
and when the network congestion free feedback is received from the receiving end after the audio carrying the redundant audio packets is sent at the third code rate, sending the audio carrying a larger number of redundant audio packets to the receiving end at a fourth code rate, wherein the audio is coded at the first code rate and is coded at the first code rate than the audio carrying the redundant audio packets at the third code rate, and the fourth code rate is greater than the third code rate and less than or equal to the second code rate.
3. The method of claim 2, further comprising:
and when the network congestion free feedback is received from the receiving end after the audio which is coded by the first code rate and carries the redundant audio packet is sent by the fourth code rate which is equal to the second code rate, the audio which is coded by the second code rate and does not carry the redundant audio packet is sent by the second code rate.
4. The method of claim 1, wherein the third code rate is calculated according to the following equation:
the third code rate is the first code rate + mx (second code rate-first code rate)/N, where N is a positive integer greater than or equal to 1, m is a positive integer, and 1 ≦ m < N.
5. The method of claim 1, wherein the carried redundant audio packets are used for packet loss compensation of audio.
6. The method of claim 1, further comprising:
when network congestion feedback is received from a receiving end after transmitting audio at a first code rate, the audio is transmitted at a lower code rate than the first code rate.
7. The method of claim 4, wherein every N packets of audio carry m redundant audio packets.
8. An audio transmission apparatus, comprising:
a feedback receiving unit configured to receive feedback on a network status from a receiving end;
an audio transmitting unit configured to: sending the audio coded with the first code rate to a receiving end according to the first code rate, wherein the first code rate is obtained by reducing the code rate after receiving network congestion feedback when the sending end sends the audio with the second code rate; when the network congestion free feedback is received from the receiving end, the audio which is coded by the first code rate and carries the redundant audio packet is sent to the receiving end by the third code rate,
and the third code rate is less than the second code rate and greater than the first code rate.
9. An electronic device, comprising:
at least one processor;
at least one memory storing computer-executable instructions,
wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 7.
10. A computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110296197.8A CN113079103A (en) | 2021-03-19 | 2021-03-19 | Audio transmission method, audio transmission device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110296197.8A CN113079103A (en) | 2021-03-19 | 2021-03-19 | Audio transmission method, audio transmission device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113079103A true CN113079103A (en) | 2021-07-06 |
Family
ID=76612844
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110296197.8A Pending CN113079103A (en) | 2021-03-19 | 2021-03-19 | Audio transmission method, audio transmission device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113079103A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024051426A1 (en) * | 2022-09-07 | 2024-03-14 | 腾讯科技（深圳）有限公司 | Video stream code rate adjustment method and apparatus, computer device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106488265A (en) * | 2016-10-12 | 2017-03-08 | 广州酷狗计算机科技有限公司 | A kind of method and apparatus sending Media Stream |
| CN107295395A (en) * | 2017-06-27 | 2017-10-24 | 武汉斗鱼网络科技有限公司 | Code check adaptive regulation method, device and electronic equipment |
| GB201800060D0 (en) * | 2014-06-20 | 2018-02-14 | Starleaf Ltd | A telecommuincation end-point device data transmission controller |
| CN112203138A (en) * | 2020-10-16 | 2021-01-08 | 深圳乐播科技有限公司 | Projection screen data transmission method, device, equipment and storage medium based on UDP protocol |
| CN112449208A (en) * | 2020-11-17 | 2021-03-05 | 北京达佳互联信息技术有限公司 | Voice processing method and device |
-
2021
- 2021-03-19 CN CN202110296197.8A patent/CN113079103A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201800060D0 (en) * | 2014-06-20 | 2018-02-14 | Starleaf Ltd | A telecommuincation end-point device data transmission controller |
| CN106488265A (en) * | 2016-10-12 | 2017-03-08 | 广州酷狗计算机科技有限公司 | A kind of method and apparatus sending Media Stream |
| CN107295395A (en) * | 2017-06-27 | 2017-10-24 | 武汉斗鱼网络科技有限公司 | Code check adaptive regulation method, device and electronic equipment |
| CN112203138A (en) * | 2020-10-16 | 2021-01-08 | 深圳乐播科技有限公司 | Projection screen data transmission method, device, equipment and storage medium based on UDP protocol |
| CN112449208A (en) * | 2020-11-17 | 2021-03-05 | 北京达佳互联信息技术有限公司 | Voice processing method and device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024051426A1 (en) * | 2022-09-07 | 2024-03-14 | 腾讯科技（深圳）有限公司 | Video stream code rate adjustment method and apparatus, computer device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20110055656A1 (en) | Systems, Methods, and Media for Checking Available Bandwidth Using Forward Error Correction | |
| US10178143B2 (en) | Selecting bitrate to stream encoded media based on tagging of important media segments | |
| CN112153415B (en) | Video transcoding method, device, equipment and storage medium | |
| CN115643449A (en) | Video display method, device, equipment, storage medium and system of cloud service | |
| CN113923397A (en) | Method and device for detecting state of conference room equipment, electronic equipment and storage medium | |
| US20110270941A1 (en) | File decoding system and method | |
| CN112562638A (en) | Voice preview method and device and electronic equipment | |
| CN113079103A (en) | Audio transmission method, audio transmission device, electronic equipment and storage medium | |
| CN112449208B (en) | Voice processing method and device | |
| CN114051120A (en) | Video alarm method, device, storage medium and electronic equipment | |
| CN113973103A (en) | Audio processing method and device, electronic equipment and storage medium | |
| CN113490055A (en) | Data processing method and device | |
| CN114221870B (en) | Bandwidth allocation method and device for server | |
| EP4117294A1 (en) | Method and device for adjusting bit rate during live streaming | |
| CN112702198B (en) | Abnormal root cause positioning method and device, electronic equipment and storage medium | |
| CN107342981B (en) | Sensor data transmission method and device and virtual reality head-mounted equipment | |
| US8791981B2 (en) | Bit rate control apparatus and method thereof | |
| CN114025190A (en) | Multi-code rate scheduling method and multi-code rate scheduling device | |
| CN102959531A (en) | Method and system for cloud-based media adaptation and transcoding service | |
| CN112769643A (en) | Resource scheduling method and device, electronic equipment and storage medium | |
| CN113873577A (en) | Data sending method and device | |
| CN112533029A (en) | Video time-sharing transmission method, camera device, system and storage medium | |
| CN114158089A (en) | Audio transmission method, terminal, electronic device and storage medium | |
| CN113364672B (en) | Method, device, equipment and computer readable medium for determining media gateway information | |
| CN113241086B (en) | Audio processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |