TWI237481B - Duplex aware adaptive playout method and communications device - Google Patents

Abstract

A duplex aware adaptive playout method includes detecting a packet communication mode as either a full-duplex mode or a half-duplex mode, calculating a playout delay for a current packet based on the detected packet communication mode, and delaying playout of the current packet by the calculated playout delay. When the packet communication mode is detected to be the half-duplex mode, the calculated playout delay for the current packet is set longer than when the packet communication mode is detected to be the full-duplex mode. A duplex aware adaptive communications device includes a playout buffer, a playout controller, a network delay estimator, and an active detector that detects a packet communication mode. The playout controller determines playout delays of packets in the playout buffer from estimated network delays and a detected packet communication mode being a full-duplex mode or a half-duplex mode.

Description

1237481

V. Description of the invention (1) Field of the invention: The present invention provides a method for transmitting network data, especially a method for optimizing the playback delay of a packet transmitted on the network based on a duplex mode of an instant interactive communication. method. Background: Due to the prevalence of the Internet and wireless communications, modern technology has now evolved to enable people to communicate with each other using real-time digital communications. In recent years, modern communications through the Internet and wireless communication networks have developed more new technologies, such as the Voice Internet Protocol and other instant interactive communication systems. Packets are used to transmit voice protocols in the network. The most important problem encountered is generally known as network delay. Network delay disturbance can be reduced. Since the amount of network delay disturbance is not the same as the network delay, it is necessary to make an accurate estimation of the playback delay. Because the purpose of side transmission is to reach the playback delay, it violates the project system. For example, the communication method through the Internet language data. For example, the trouble of the Internet is the change of network delay. 5, Jitter. As for a playback delay # $, the playback of the delayed packet is significantly determined by 111. Therefore, it is necessary to reduce the reasonable measurement of network delay during the transmission process, and however, the playback delay cannot be too long to the effect of real-time transmission, but it is too long. As a result, when users use Internet audio protocols, wireless phones,

Page 5 1237481 V. Description of the invention (2) When talking with each other, or online games, it is especially important to minimize the playback delay to avoid communication problems caused by the delay. Please refer to FIG. 1, which is a schematic diagram of transmitting sound data 20 in a packet through a network 10. For example, the data 20 may be a media output device via the Internet, such as an Internet voice protocol transmission, a personal voice data transmitted over a wireless telephone network, or other similar communication data. The material 20 includes audible sections 20a, 20c, and 20e with audible audio information and silent sections 20b and 20d with inaudible audio information. A transmitter 12 is shown in FIG. 1, which transmits packets P 1 -P 1 5 at regular intervals. The transmitter 12 can be a personal computer, wireless phone, or other device, but the packet will be delayed due to network delay. The delay in the transmission of P1P1 5 caused a part of the packet P15 to arrive at a receiver 14 at different times. In order to complete the incorporation of sound data 2 2, a personal computer or similar device must be delayed With different amounts of delay. The audio data 22 includes audible sections 22a, 22c, and 22e and stationary sections 2 2b and 2 2d corresponding to the sections 20a-20e of the transmitted data 20. The transmitter 1 2 will transmit the packet P in a given time. The packet P 1 will be delayed by the network 10 for many possible reasons. The shaded box in Figure 1 shows this delay and other further delay factors. The packet P1 will be further delayed by the receiver 14 so that the packet P 1 can be continuously played with the packet P 2 which has been delayed by the network 10. If packet P1 is not received

1237481

The receiver 1 4 further delays, and the packet P 1 and at the same time in the data 2 2 will be heard, and the audio interruption is interrupted by the user 2 on the receiver 1 2 with poor sound quality. P2 can't play continuously, it is off. The hearing in the data 22 causes the playback of the data, packet delay, and delay. However, before receiving, the additional information will not be affected. The P9 will be played and due to the delay. Similar receivers P2-P5 are all received by the Internet. The packets = P2-P5 are not necessarily received by the receiver. 14 J 1 j = "packet" arrives earlier than Γ. Ί = post broadcast receiver 14 must delay the packet ρ7] seal d6 break the static f 俨 o〇h of the material 22, play. The delay is delayed by 1 22b so that the audible section β = due to the network delay and the check of the packet c packets P10 and P11 arrive at the same time. The packet ^ is delayed, however, there will be no further steps-step: information and materials ~ 1: row ίί / 13 and J14 suffer from the same as the packet? 6 and P7. Ruthenium M2 and P15 are counterfeit in the normal order: The description is a simplified process. The packet P and P15 time. The integer product of the length ^ Gives the network M that arrives at the receiver with a delay. / In the predetermined transmission process, when the network delay and the configuration t are disturbed, i is continuously transmitted in time, and the length of the packet is several eight. It must be delayed.

Page 71237481 V. Description of the invention (4) Figure 1 shows the entire received data 2 2 series delay three sections, these three sections include a network delay and additional playback delay added by the receiver 1 4 . If the receiver 14 does not attach the additional delay, some packets may be out of order and other packets may not be played at all. Conventional techniques have shown many ways how to estimate the time to be delayed by the receiver 14e. However, excessive playback delays can cause inconvenience and misunderstanding when both parties cooperate. A basic and effective but controversial method for estimating playback delay is the Mean Delay Variation Method (MDV).

Ramjee, J. Kurose, D. Towsley, and Η ·

Schulzrinne I. I. Adaptable Playout Mechanisms for Packetized Audio Applications in Wide-area Networks is described here, and is hereby provided as a reference for the prior art of the present invention. The average delay variation method is further detailed in Marco Rocce11 i Vittorio Ghini, Giovanni Pau, Paola Salomoni, and Maria Elena Bonf igl i. Design and Experimental Evaluation of an Adaptive Delay / control Mechanism for Packetized Audio for use over the Internet " Technical reference materials. In short, the average delay variation method is an average network delay variation combined with a smoothing factor to estimate the playback delay. This simple dynamic method provides more than other non-dynamic methods Important key improvements.

1237481 V. Description of the invention (5) Other methods for estimating the playback delay are described in the real-time transport protocol. Η ·

Schulzrinne ^ S. Casner, R. Frederick, and V. Jacobson, "RTP: a Transport Pr0t0C0l f〇r Real-Time Appl i cat ions" describes in detail the standard specifications of the real-time transmission protocol, which is also based on References to the prior art of the invention. The real-time transmission protocol method used to estimate the delay is an average delay variation method with a fixed smoothing factor. Although it is simpler than the average delay variation method, the real-time transmission protocol can only provide a rough estimate of the network delay. Other known techniques for estimating playback delay include a dramatic delay difference (S pike) method. This method is described in the article "Adaptive P 1 ay 〇ut Mechanisms for Packetized Audio Applications in Wide-area Networks", Another method that uses the correlation time difference is described in Jesus Pint0 and Kenneth J. Christenseη An An 1 gorithm for P 1 ayout of Packet Voice based on Adaptive Adjustment of Talkspurt Silence Per iodsnt, the extremely long delay detection method And although the correlation time difference detection method increases the difficulty of operation, it is not significantly better than the average delay variation method. Finally, the conventional technique also provides a normalized minimum average method.

1237481 V. Description of the invention (6) (normalized least mean square), this method is described in "An Adaptive Predictor for Media Playout buffering" by Phillip DeLeon and Cormac J. Sreenan. The normalized minimum mean method is a complex calculation Method does not provide significant improvement.

In addition, in the conventional technology, there are many patents concerning the playback of digital information and the monitoring of playback quality. For example, in the US

In Patent 5,815,634, Daum et al. Have proposed a stream synchronization method, which is applied to the recording and playback function of MPEG. In US Patent 6, 259, 677, Jain proposed an instant receiver and a corresponding method for receiving and playing instant packet data. The patents of the above two conventional technologies are attached to the reference materials of the prior art of the present invention. In addition,

Schulman in US Patent 5, 600, 632 proposed a method for monitoring quality in a network using a coherent network analyzer about packet delay, and & Rongchang 3 ~ & 1 and others also in 118? A synchronization prediction method for maintaining and updating the statistical trend of network delay is provided in 3 Shi 0 ^ 6,072,809. The patents of the above two conventional technologies are also attached to the reference materials of the prior art of the present invention. The conventional techniques described above have optimized the playback delay in network statistics', but they have not properly taken into account the duplex characteristics of communication. That is to say, the adjustment of the playback delay does not take into account the half-duplex or full-duplex mode in communication. Therefore, in the case that these different modes have not been considered and implemented, the true playback quality, and the entire communication quality,

Page 10 1237481 V. Description of Invention (7) is still deteriorating. Summary of the Invention: Therefore, the main object of the present invention is to provide a method and a communication device for optimizing the conventional techniques described above. The present invention discloses a two-way four-sense test packet pass-through type that is based on a fully-detected packet communication mode. If it is worse, the nasal playback delay is detected in the delayed signal mode as a half-duplex mode. Playback delay while the packet is passing. A two-way sensor, a playback controller, and a network device to detect the delay of a packet communication mode and a detected packet communication mode to determine the delay mode of the playback buffer. And the calculation of an estimated network delay perturbation delay based on the average is further based on the purpose of providing a two-way perception of the playback delay of packet communication to resolve the perceived dynamic playback method, which is used in duplex mode or half duplex. Working mode to calculate the playback delay of the current packet and the playback of the current packet. When the packet is generalized, the calculated signal mode of the current packet is detected as a full-duplex mode. The dynamic communication device includes a playback slow charge delay estimator and an activity detection. The playback controller is estimated by The network mode is the only full-duplex mode or half-duplex packet playback delay. The previous packet is calculated with an average network delay, and the network delay variation is the current packet. Then, the estimated network delay disturbance of the packet before the current packet is played.

1237481 V. Description of the invention (8) Delay of playback of the previous packet. The playback device includes a playback buffer for receiving and temporarily storing packets, a playback controller determines the playback delay of the packet from the estimated network delay and the packet communication mode, and controls the playback buffer based on the playback delay. A network delay estimator is used to calculate the estimated network delay of the packet and send the estimated network delay to the playback controller, and an activity detector is used to detect the packet communication mode. The packet communication mode can be It is a full-duplex mode and a half-duplex mode. An advantage of the present invention is that delaying the playback of packets with a longer delay in the half-duplex mode can improve the quality of playback. Another advantage of the present invention is that delaying the playback of packets with a short delay in full-duplex mode can speed up the playback response. Another advantage of the present invention is that after the activity detector communicates with the activity detection mode, the playback controller can delay the playback of the packet according to the communication mode. Detailed description of the invention: The two-way perceptual dynamic playback method of the present invention can be combined with a communication device as the best explanation and implementation. The markings are identical in all equations and programs.

Page 12 1237481

Please refer to the figure-structure. The first communication device is = the envelope sent by the system frame of a communication device 30 of the present invention includes a receiver 32 for receiving from the network 10 receivers and 2 receivers, and the second device includes a broadcaster. The slow charger 34 is used to receive a packet from a media wheel ^ parent ^ and delay the data transmission of the packet to delay the playback. The slow charger 34 can absorb the network card out of the dream department "make" The packet can be played very smoothly and continuously in the media. Long output device 4 36 can be-a typical media 刖 ^ > such as an Internet voice protocol player, a speaker,

Net or other device. ㈣… 〇 In addition, I suppress the extension to 3 8 to estimate the network delay of the network 10, an activity detector 40 to detect a packet communication mode, and a playback control 42 to control the playback delay充 器 34. When a media input device 44 is inactive, the activity detector 40 determines whether the packet communication mode is half-duplex mode, and when the media input device 44 is active, it is active. The detector 40 determines whether the packet communication mode is a full-duplex mode. The media input device 44 is a device corresponding to the media output device 36. The media input device 44 can receive data input to the communication device 30 and deliver the packet to a transmitter 46. The transmitter 46 outputs the packet to the network 10. Playback control

The controller 4 2 sets the delay of the packet received in the playback buffer 34 according to the network delay estimated by the network delay estimator 38 and the packet communication mode determined by the activity detector 40. Finally, when the playback delay of a packet expires, the playback buffer 34 will send the packet to the media output device.

Page 13 1237481 V. Description of Invention (ίο) 36 to play. The activity detector 40 detects the communication device 30 is in the half-duplex mode or the full-duplex mode. If the media input device 44 is not receiving information, or the total amount of information is below a threshold (for example, the sound level is almost stationary), the activity detector 40 will detect that the communication device 30 is in a semi-double Work mode. On the contrary, when the amount of information received by the media input device 44 is higher than the threshold value, the activity detector 40 will detect that the communication device 30 is in a full-duplex mode. That is, half-duplex mode is for one-way communication and full-duplex mode is for two-way communication. Other methods for converting half-duplex and full-duplex modes are within the scope of the present invention. The network delay estimator 3 8 and the playback controller 4 2 estimate the network delay and the average network delay variation according to the following equation:

Di = | (\-RJ-(Si -MDi = F x MDh + (1-F) x Di \ =

Page 14 1237481 V. Description of the invention (11) i is a subscript pointing to a current packet; i-1 is a subscript pointing to a previous packet; MD is an average network delay; F is a smoothing coefficient; V is a network The delay variation number; MV is the average network delay variation number; in fact, the network delay estimator 38 will first estimate the network delay of the current packet according to the network statistics of the current packet and the previous packet. The network delay estimator 3 8 then delivers the estimated network delay of the current packet to the playback controller 4 2. The playback controller 4 2 then uses the average network delay to calculate the average network delay variation, smoothing factor, and network delay variation of the current packet. The playback controller 4 2 further calculates the playback delay of the current packet according to the following procedure:

F = FULL _ DUPLEX THEN

= MV £ x FSF PD soil = FF x + (1- FF) x Ji

ELSE

= Called x HSF PD, = HF x PD ^ + (1- HF) x (Equation 2) where M is the packet communication mode, where FULL_DUPLEX and HALF_DUPLEX are valid values;

Page 151237481 V. Description of the invention (12) J is an estimated network delay. FSF is a full-duplex proportionality factor. 'PD is the playback delay.' FF is a full-duplex smoothing factor. Proportional factor '; HF is half-duplex smoothing factor.' The procedure in Equation 2, jealous mode, and then determines the network delay. The 42 first determines the playback delay of the packet communication. To determine the delay perturbation, the number of delay variations of the current packet network, namely, the delay, is calculated. The perturbation needs to depend on the delay of the previous packet in Reference Equation 1. Scale factor. The playback delay must depend on the actual playback delay. The broadcast mode >, a smoothing factor, and the packet communication mode determined by the previous packet are the input controller f dependent on the active sensor 40 and the smoothing factor. , ', Wang Duplex or Half Duplex to select the proportional coefficient to achieve the best performance, μ is better than the full duplex proportional coefficient (Fsn; ^ / duplex proportional coefficient (HSF)) design benefit from the improvement of playback quality. f, if F w people listen to a (meaning that the media input ^ H μ I is in an inactive state in a communication device 30), the increased playback delay can easily improve the broadcast quality L. Conversely, without sacrificing the playback quality The 'Wang Shuanggong' type can reduce the playback delay as much as possible (relative to Lu's FSF) to enable different communication devices to get 30 minutes.

1237481 V. Description of the invention It is important to listen at the same time. It cannot be used in the smoothing mode, which requires a more delayed playback. The relative smoothing factor (13) smooth and synchronous communication. In full-duplex mode, when both parties talk, the shorter playback delay is quite effective for communication. In full-duplex mode, the playback quality and the shorter playback delay are considered. In more detail, the full-duplex smoothing factor F F is longer than the half-duplex HF. This is because the communication device 30 can adjust the playback delay more quickly in the full-duplex mode. As a result, playback relies on true currently estimated network delay disturbances and lower reliance on previously blocked delayed playback records. As a result, the full-duplex model has a faster response. The scale factors HSF and FSF and HF and FF are set and stored in the playback controller 42. In practical applications, the playback device 30 and its components can be completed using conventional electronic circuits, integrated circuits, and related software programs. The logic programs of the playback controller 42 and the network delay estimator 38 and the response and detection thresholds of the activity detector 40 can be fine-tuned and compatible with related communication equipment. These related communication devices include internet voice protocol players, video phones, online games, and other real-time interactive sfl devices used on the internet and wireless networks. ... The process of using the communication device 30 to optimize the delay as described earlier can be summarized into the flowchart in Figure 3, which includes the following steps:

1237481 V. Description of the invention (14) Step 1 0 0: Start. Two or more communication devices communicate with each other through the network 10; Step 1 〇2: The activity detector 40 detects the mode of packet communication as manual or half duplex; Step 1 〇4: Network The delay estimation 38 and the playback controller 42 need to determine the network delay disturbance of the current packet to be received by the receiver 32. Execute the program corresponding to Equation 1. Step 1 06: Playback controller 4 2 calculates the playback delay of the current packet according to the mode detected in step 1 02 and the estimated network delay in step 104. The playback of the current packet is delayed to the playback buffer 34. Then the procedure of Equation 2 is executed. The playback delay of the current packet will be stored in the playback controller 42 so that the next packet can also execute the procedure shown in Figure 3. Step 108: The media output device 36 delays the playback delay according to the playback delay. Play the packet in 34; Step 1 10: Determine whether the communication is complete. If the result is no, return to Step 102. If the result is yes, go to step 1 1 2 · Step 1 1 2: End. In the context of application, the above-mentioned procedures are continuously executed in the individual communication devices 30 interconnected in the network 10. If an individual communication device 30 delays the playback of a received packet according to different duplex modes, the overall communication quality can be greatly improved.

Page 18 1237481

^ Please note that a line of the same procedure in the communication device 30 must be the same as the previous steps, and the components can be implemented as described above. According to the present invention, the above-mentioned playback devices are coordinated with each other. Of course, the procedure implemented in the element with different variations can be in the above and even other and is not limited to this example. Compared with the conventional technology, when the dual-half-duplex mode of the packet transmission mode, the working mode, and the detection mode is extended according to the detection mode, the broadcast = the result of the measurement mode τ 'The present invention uses- Longer delay to delay the playback of Dagger to improve the playback quality, and in full duplex mode, use a shorter delay to delay the playback of the packet to speed up the playback response. Rebroadcast # In the duplex wedge mode, the playback delay of the packet will have a lower dependence on the 2 delay of the previous packet and the dependence on the current network delay disturbance than the communication device described by Bingtian T. Device, which is electrically connected to an activity detector that delays the communication mode of the packet to control the playback of the packet. Any application according to the present invention shall belong to the patent of the present invention. The above description is only for comparison of the present invention. The best patent scope makes equal changes and modified cover scope.

Page 19 1237481 Brief description of the diagram Brief description of the diagram: Figure 1 is a schematic diagram of transmitting sound data in a packet through a network. FIG. 2 is a block diagram of a communication device according to the present invention. FIG. 3 is a flowchart of the operation mode of the communication device of the present invention. Explanation of symbols: 10 Network 12 ^ 46 Transmitter 14, 32 Receiver 20 Sound data 30 Communication device 34 Dial slow charger 36 Media output device 38 Network delay estimator 40 Activity detector 42 Playback controller 44 media input device

Page 20

Claims (1)

1237481 VI. Scope of Patent Application 1. A method for optimizing the playout delay of a packet, the packet is transmitted in a network, and the packet contains the data to be played 'and the network With network delay during packet transmission, the method includes the following steps: (a) detecting a packet communication mode, wherein the packet communication mode includes a full-duplex core and half-duplex (half- duplex) mode; (b) calculating a playback delay of a current packet based on a detected packet communication mode; and (c) delaying playback of the current packet based on the calculated playback delay. 2. The method according to item 1 of the scope of patent application, wherein in step (b), when the packet communication mode is detected as the half-duplex mode, the calculated playback delay in the current packet will be longer than Playback delay when the packet communication mode is detected as the full-duplex mode. 3. As described in step 1 of the scope of patent application: The method described in item 1 further includes the following coefficient of variation for calculating the average network delay of the current packet; and An estimated delay disturbance in the current packet; wherein in step (b), the estimated delay in the current packet disturbs the playback delay of the previous two packets based on the current delay and the playback delay of the previous packet Page 21 1237481 6. The scope of patent application learned. 4. The method as described in item 3 of the scope of patent application, wherein in step (b), when the packet communication mode is detected as the half-duplex mode, the calculated playback delay of the current packet is When the packet communication mode is detected as the full-duplex mode, there is less dependence on the playback delay of the previous packet. 5. The method according to item 3 of the scope of patent application, wherein in step (b), when the packet communication mode is detected as the half-duplex mode, the calculated playback delay of the current packet will be lower than When the packet communication mode is detected as the full-duplex mode, there is more dependence on the estimated delay disturbance in the current packet. 6. The method as described in item 3 of the patent application scope, wherein the estimated delay disturbance in the current packet determined in step (e) will further communicate the packet detected in step (a) A scale factor determined by the model is dependent. 7. According to the method described in item 3 of the scope of the patent application, the playback delay of the current packet calculated in step (b) further determines a delay in the communication mode of the packet detected in step (a). Correlation between smoothing factors0 Page 1212 37481 ^ V plan (voice over online games, and 6. Patent application scope 8. The method described in item 1 of the patent application scope can be located in an Internet protocol), video and other Instant interactive communication system. 9. If you apply for a method that is not described in this article, you can use it as a private network-a small network or a radio transmission connected to a wireless phone. "Send the network, 1 0. — a kind of optimization used for playback The delayed packet is delayed in a network. The communication device is a data communication device. In the packet, the packet includes transmission. The network has a content including: The data is included in the data to be played, and the packet is transmitted. Time Network Play slow charger-play controller, restore = save the packet; mode to determine the packet delay, network delay, and a packet through the play slow charger; ,, and a network based on the playback delay The path delay is estimated to be late and the estimated network-activity detector includes a communication mode of 1 1. If the patent-pending range activity detector detects the stomach, the device is counted by the playback controller, which is used to calculate a prediction. The estimated network delay of the packet is transmitted to the playback controller; and used to detect the packet communication mode, wherein the packet duplex mode and half-duplex mode. The communication device according to item 10, wherein when the packet communication mode is the half-duplex mode, the playback delay of a current packet is longer than that when the packet is active. 1237481 6. The patent application range motion detector detects that the packet communication mode is the full-duplex mode 1 2. The communication device described in item 10 of the patent application range, wherein when the activity detector When it is detected that the packet communication mode is the half-duplex mode, the playback delay calculated by the playback controller for a current packet will be longer than when the activity detector detects that the packet communication mode is the full-duplex mode. Less reliance on the playback delay of a previous packet. 13. The communication device as described in item 10 of the scope of patent application, wherein when the activity detector detects and estimates that the packet communication mode is the half-duplex mode, the playback controller calculates a The playback delay will be more dependent on the estimated network delay of the current packet when the activity detector detects that the packet communication mode is the full-duplex mode. 14. The communication device as described in item 10 of the scope of patent application, the communication device further comprising: a receiver, which allows the playback slow charger to receive packets from the network by the receiver; a media output device, The playback buffer can be allowed to output packets; a media input device is used to receive the packets; and a transmitter can be used by the playback device to send packets to the network. 15. The communication device as described in item 10 of the scope of patent application, wherein the medium Page 24 1237481 6. Scope of patent application The volume output device can be a voice over Internet protocol, a video phone, an online game, and other real-time interactive communication systems. 16. The communication device as described in item 10 of the scope of patent application, wherein the network can be a computer network or a radio transmission network connected to a portable telephone.