CN101887739A - Method and device for media playback synchronization - Google Patents

Abstract

A method for media playback synchronization in the field of electronic technology, which obtains the data of each synchronization reference in media playback in real time; uses the local system clock as a reference to analyze the fluctuation of the data of each synchronization reference; uses the synchronization reference corresponding to the data with the smallest fluctuation as the current synchronization reference, Perform media playback synchronization operations. Correspondingly, an embodiment of the present invention provides a media playing device. The embodiment of the present invention monitors the reliability of several synchronization references in real time, and can determine the synchronization reference according to the actual situation of media playback, realizes the dynamic adjustment of the synchronization reference of media playback, and eliminates the audio and video pauses often caused by PTS and PCR to the greatest extent. , frame loss and other issues, so that the actual playback effect reaches the best smoothness and improves the user experience.

Description

Method and device for media playback synchronization

technical field

The invention relates to the field of electronic technology, in particular to a method and device for synchronizing media playback.

Background technique

Media playback devices generally have the synchronization function of audio and video playback, that is, the picture and sound are played synchronously to provide users with a good viewing experience, especially when playing news programs, lip synchronization is critical to user experience.

At present, the synchronization methods of media playback devices are as follows: audio synchronization, program clock reference (PCR, Program Clock Reference) synchronization, and video synchronization. The audio synchronization, that is, video and subtitles follow the audio, and synchronize with the audio playback time stamp (PTS, Play Time Stamp); the PCR synchronization, that is, a program clock reference provided by the program sending end, audio and video and subtitles with The clock reference is used as a reference for synchronization; the video synchronization, that is, audio and subtitles are synchronized based on the video PTS.

The following uses the example shown in Figure 1 to illustrate the specific synchronization process:

1) First, the media device player specifies a synchronization reference, such as audio, when starting to play a media file. This benchmark can be specified by the developer during development, or specified by the user through the player's interface.

2) Then, the media device player extracts the PTS of the audio from the media data stream according to the specified synchronization reference during playing, and generates a reference clock with the PTS.

3) When the data (video) of the asynchronous reference is to be played, first, the PTS of the data of the asynchronous reference to be played is compared with the reference clock generated in step 2). If the PTS of the data to be played is substantially equal to the reference clock, then play. The substantially equal means that the PTS falls within a range near the reference clock. In this range, it is generally difficult for human eyes to detect out-of-sync conditions. If the played data is earlier than the reference clock, that is, PTS<reference clock interval, it is discarded. On the contrary, if it is later than the reference clock interval, it will wait until the PTS falls within the reference clock interval before playing.

The inventor found in the process of realizing the present invention that the synchronization method of the prior art has at least the following disadvantages:

The synchronization reference is specified when the playback is started, which is specified by the developer or the user interface. Although the user can specify it again during the playback process, after the designation, the player will passively use this synchronization reference, and the specified synchronization reference during the playback process. The synchronization reference may be unstable, or may not exist at all. At this time, the synchronization of each playback data cannot be realized, which may cause audio and video freezes, stuttering, and jumping.

Contents of the invention

Embodiments of the present invention provide a method and device for synchronizing media playback, which dynamically adjusts a media playback synchronization reference according to the actual situation of media playback.

Embodiments of the present invention are achieved through the following technical solutions:

An embodiment of the present invention provides a method for synchronizing media playback, including:

Real-time access to the data of each synchronization benchmark of media playback;

Use the local system clock as a reference to analyze the data fluctuation of each synchronization benchmark;

The synchronization reference corresponding to the data with the smallest fluctuation is used as the current synchronization reference, and the media playback synchronization operation is performed.

An embodiment of the present invention provides a media playback device, including: a media playback unit, an acquisition unit, an analysis unit, and a synchronization reference determination unit;

The media playing unit is used to play media files;

The acquisition unit is used to acquire the data of each synchronization reference of media playback in real time;

The analysis unit is used to analyze the data fluctuation of each synchronization reference with the local system clock as a reference;

The synchronization reference determination unit is used to use the synchronization reference corresponding to the data with the smallest fluctuation as the synchronization reference for the current media playback synchronization operation;

The media playing unit is further configured to perform a media playing synchronization operation based on the synchronization reference determined by the synchronization reference determination unit.

It can be seen from the technical solution provided by the above-mentioned embodiments of the present invention that the embodiment of the present invention monitors the reliability of several synchronization references in real time, and can determine the synchronization reference according to the actual situation of media playback, thereby realizing the dynamic adjustment of the synchronization reference of media playback. Eliminate the audio and video pauses and frame loss problems caused by PTS and PCR to a certain extent, so that the actual playback effect can reach the best smoothness and improve user experience.

Description of drawings

Fig. 1 is prior art audio-video synchronization process;

Fig. 2 is a flow chart of a method for synchronizing media playback in an embodiment of the present invention;

FIG. 3 is a schematic diagram of interval division according to an embodiment of the present invention;

Fig. 4 is a schematic structural diagram of a media player device according to an embodiment of the present invention;

5 is a schematic structural diagram of a media playback device according to another embodiment of the present invention;

6 is a schematic structural diagram of an analyzing unit in a media player device according to an embodiment of the present invention;

7 is a schematic structural diagram of a synchronization reference determination unit in a media playback device according to an embodiment of the present invention;

Fig. 8 is a schematic diagram of cooperation of various units in a media player device according to an embodiment of the present invention to realize media synchronization.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be understood that the described embodiments are only some of the embodiments of the present invention, not all of them. example. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

An embodiment of the present invention provides a method for synchronizing media playback, taking the example shown in Figure 2 as an example, including the following steps:

Step 10: Obtain the data of each synchronization reference of media playback in real time;

Step 11: Taking the local system clock as a reference to analyze the data fluctuation of each synchronization benchmark;

Step 12: Using the synchronization reference corresponding to the data with the smallest fluctuation as the current synchronization reference, perform a media playback synchronization operation.

Wherein, in step 10, the data of each synchronization reference of the media playback includes: one or more of the program clock reference PCR, audio PTS or video PTS, that is, the data that can be used as the synchronization reference of the media playback;

Wherein, in step 11, the analysis of the data fluctuation of each synchronization reference with reference to the local system clock includes:

S110: compare the acquired data of each synchronization reference with a local system clock (STC, SystemTime Clock), and record the deviation value of the data of each synchronization reference;

S111: After the time for obtaining the data of each synchronization reference reaches the specified time, calculate the average value of the deviation value of the recorded data of each synchronization reference for each synchronization reference; The average value of deviation values of the data of each synchronization reference.

S112: respectively counting the number of points where the deviation value of the data of each synchronization reference falls in each interval, and calculating the weighted sum of each synchronization reference according to the number of points where the deviation value falls in each interval and the coefficients of each interval; The average value calculated from the data of the synchronization benchmark is N intervals defined by the center, and N is a natural number greater than zero;

The method of calculating the weighted sum can use the following formula:

Weighted sum = Points in Section 1 × Coefficient in Section 1 + Points in Section 2 x Coefficient in Section 2 + ... + Points in Section N x Coefficient in Section N.

Wherein, the distance from the interval 1 to the center<the distance from the interval 2 to the center<...<the distance from the interval N to the center; the interval 1 coefficient>the interval 2 coefficient > ... > the interval N system. The divided intervals and the coefficients for each interval are the same for various synchronization references.

S113: Determine the magnitude of the fluctuation according to the weighted sum, that is, the fluctuation becomes larger as the weighted sum decreases.

Wherein, in step 12, the synchronization reference corresponding to the data with the smallest fluctuation is used as the current synchronization reference, and the execution of the media playback synchronization operation includes: generating a reference clock according to the determined synchronization reference, and using the reference clock generating unit to generate The reference clock performs media playback synchronization operations.

The embodiment of the present invention monitors the reliability of several synchronization references in real time, determines the synchronization reference according to the actual situation of media playback, realizes the dynamic adjustment of the synchronization reference of media playback, and eliminates audio and video pauses caused by PTS and PCR to the greatest extent. Problems such as frame loss make the actual playback effect reach the best fluency and improve user experience.

In order to further understand the present invention, the following examples illustrate the process of determining the synchronization reference, including the following operations:

1) During media playback, real-time sampling to obtain PCR, audio PTS and video PTS;

2) compare the obtained PCR, audio PTS and video PTS with the local system clock STC respectively, and record the deviation value;

For example, for the obtained audio PTS0, PTS 1, ..., PTSn-1, the recorded deviation value is PTSDELTA0, PTSDELTA1, ..., PTSDELTAn-1; where,

PTSDELTA0=PTS0-STC;

PTSDELTA1=PTS1-STC;

PTSDELTAn-1 = PTSn-1-STC;

3) After the sampling time reaches the set time, for example, after 1 minute, the average value is calculated for all deviations of the data records of each synchronization benchmark;

For example, for deviation values TSDELTA0, PTSDELTA1, ..., PTSDELTAn-1 of the recorded audio PTS, calculate the average value Avg:

Avg=(PTSDELTA0+PTSDELTA1+...+PTSDELTAn-1)/n;

It can be understood that, in this embodiment of the present invention, the timing of calculating the average value is based on the sampling time reaching the specified time, but other methods of judging the timing of calculating the average value are not excluded. For example, the average value can also be calculated when the sampling time reaches the specified number of times.

For PCR and video PTS, the deviation value from the local system clock is calculated in the same way as the calculation deviation value of the audio PTS.

4) take each average value as the center, define 5 intervals, and count the number of sampling points falling in each interval: S1, S2, S3, S4, S5; as shown in Figure 3, it is the interval form defined as the center with Avg;

The intervals defined by PCR, audio PTS, and video PTS have the same form, that is, the time difference between each interval is the same, and the interval system is also the same. The center of the interval is the Avg calculated based on the data of each synchronization reference.

5) Calculate the weighted sum of each synchronization reference according to the number of sampling points (S1, S2, S3, S4, S5) falling in each interval and the coefficients of each interval; set the interval 1 coefficient to be 16 in this example, and fall in the interval The number of sampling points of 1 is S1, the coefficient of interval 2 is 8, the number of sampling points falling in interval 2 is S2, the coefficient of interval 3 is 4, the number of sampling points falling in interval 3 is S3, the coefficient of interval 4 is 2, and the number of sampling points falling in interval 4 is The number of sampling points is S4, the coefficient of interval 5 is 1, and the number of sampling points falling in interval 5 is S5; then the weighted sum calculation formula is:

Weighted sum=S1*16+S2*8+S3*4+S4*2+S5;

6) Determine the magnitude of the fluctuation according to the weighted sum of each synchronization reference. If the weighted sum is large, the sampling points are concentrated, and the fluctuation is considered to be small. In this way, the advantages and disadvantages of PCR and audio/frequency PTS are determined, and the synchronization reference with the smallest fluctuation is used as the synchronization reference of current media playback, and a reference clock is generated according to the determined synchronization reference to perform media playback synchronization operations.

The embodiment of the present invention realizes the dynamic adjustment of the media playback synchronization standard, and eliminates the audio and video pauses and frame loss problems caused by PTS and PCR to the greatest extent, so that the actual playback effect can achieve the best smoothness and improve user experience.

The embodiment of the present invention also provides a media playback device, taking the example shown in FIG. 4 , the media playback device includes: a media playback unit 40, an acquisition unit 41, an analysis unit 42, and a synchronization reference determination unit 43;

The media playing unit 40 is used to play media files;

The acquisition unit 41 is used to obtain the data of each synchronization reference of media playback in real time; the data of each synchronization reference of said media playback includes: program clock reference PCR, audio PTS and video PTS, which can be used as the data of media playback synchronization reference;

The analysis unit 42 is used to analyze fluctuations of data of each synchronization reference with reference to the local system clock;

The synchronization reference determination unit 43 is used to use the synchronization reference corresponding to the data with the smallest fluctuation as the synchronization reference for the current media playback synchronization operation;

The media playing unit 40 is further configured to perform a media playback synchronization operation based on the synchronization reference determined by the synchronization reference determination unit.

As shown in Figure 5, the media player may also include:

The clock monitoring unit 44 is configured to monitor the time when the acquisition unit acquires the data of each synchronization reference, and provide it to the average value calculation subunit as an opportunity to determine the average value calculation.

As shown in Figure 6, the analysis unit 42 may further include:

Deviation recording subunit 421, configured to compare the obtained data of each synchronization reference with the local system clock, and record the deviation value of the data of each synchronization reference;

The average value calculation subunit 422 is configured to calculate the average value for the deviation values of the recorded data of each synchronization reference after the acquisition unit acquires the data of each synchronization reference reaches a specified time:

The statistical subunit 423 is used to count the points where the deviation value of the data of each synchronization reference falls in each interval; the intervals are N intervals defined centering on each of the average values, and N is a natural number greater than zero ;

The weighted sum calculation subunit 424 is configured to calculate the weighted sum of each synchronization reference according to the number of points where the deviation value falls in each interval and the coefficients of each interval, and the fluctuation becomes larger as the weighted sum decreases.

It can be understood that the structure of the analysis unit described in the embodiment of the present invention is only an embodiment listed by the inventor, and other structural divisions are within the protection scope of the present invention on the premise of realizing the current function of the analysis unit.

As shown in FIG. 7, the synchronization reference determining unit 43 may further include:

The synchronization reference determination subunit 431 is used to use the synchronization reference corresponding to the data with the smallest fluctuation as the synchronization reference for the current media playback synchronization operation;

The reference clock generation subunit 432 is configured to generate a reference clock according to the determined synchronization reference.

As shown in FIG. 8 , it is a schematic diagram of media synchronization between the various units of the media player device described in the embodiment of the present invention. The media data is pre-processed, and the pre-processing process is not limited in this embodiment of the present invention. For example, it includes In the demultiplexing process, after the pre-processing, the acquisition unit acquires the data of each synchronization reference, including, for example, PCR, audio PTS and video PTS. The clock monitoring unit monitors the time when the acquisition unit acquires the data of each synchronization reference, and provides the time to the analysis unit. After the acquisition time reaches the specified time, the analysis unit starts to perform the analysis operation, that is, analyzes the data of each synchronization reference with the local system clock as a reference. The fluctuation of the data of the synchronous benchmark, then the synchronous benchmark determining unit knows the pros and cons of each synchronous benchmark according to the analysis result of the analysis unit, and determines the synchronous benchmark corresponding to the data with the smallest fluctuation as the synchronous benchmark for currently executing the media playback synchronous operation, according to The determined synchronization reference calculates a reference clock, and the media playback unit executes a media playback synchronization operation according to the reference clock.

The media playback device described in the embodiment of the present invention monitors the reliability of several synchronization references in real time, can determine the synchronization reference according to the actual situation of media playback, realizes the dynamic adjustment of the media playback synchronization reference, and eliminates the constant band caused by PTS and PCR to the greatest extent. Audio and video pauses, frame loss and other problems, so that the actual playback effect can achieve the best fluency and improve user experience.

In summary, the embodiment of the present invention monitors the reliability of several synchronization references in real time, can determine the synchronization reference according to the actual situation of media playback, realizes the dynamic adjustment of the synchronization reference of media playback, and eliminates the frequent bands caused by PTS and PCR to the greatest extent. Audio and video pauses, frame loss and other problems, so that the actual playback effect can achieve the best fluency and improve user experience.

Those of ordinary skill in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory ( ROM for short), random access memory (RAM for short), magnetic disk, optical disk, etc.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present invention can easily think of changes or Replacement should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (10)

1. A method for media playback synchronization, comprising:

acquiring data of each synchronous reference of media playing in real time;

analyzing the fluctuation of data of each synchronous reference by taking a local system clock as a reference;

and taking the synchronization reference corresponding to the data with the minimum fluctuation as the current synchronization reference to execute the media playing synchronization operation.

2. The method of claim 1, wherein the data for each synchronization reference comprises:

the program clock is referenced to one or more of the PCR, the audio play time stamp, or the video play time stamp.

3. The method of claim 1 or 2, wherein analyzing fluctuations in data of each synchronization reference with reference to a local system clock comprises:

comparing the acquired data of each synchronous reference with a local system clock, and recording the deviation value of the data of each synchronous reference;

after the time for acquiring the data of each synchronization reference reaches a specified time, calculating the average value of the deviation values of the recorded data of each synchronization reference for each synchronization reference;

respectively counting the number of points of the deviation value of the data of each synchronous reference in each interval, and calculating the weighted sum of each synchronous reference according to the number of points of the deviation value in each interval and the coefficient of each interval; the intervals are N intervals defined by taking the calculated average value as a center, and N is a natural number greater than zero;

the fluctuation becomes larger as the weighted sum decreases.

4. The method of claim 3, wherein calculating a weighted sum of synchronization references based on the number of points the deviation value falls in each interval and the interval coefficients comprises:

the weighted sum is 1 point number in a section × 1 coefficient + 2 point number in a section × 2 coefficient + - ·+ N point number in a section × N coefficient in a section.

5. The method of claim 4, wherein the distance of interval 1 to the center < the distance of interval 2 to the center < the distance of interval N to the center; the interval 1 coefficient > the interval 2 coefficient > the interval N system.

6. The method of claim 4, wherein the divided intervals and the coefficients of the intervals are the same for each synchronization reference.

7. A media playback apparatus, comprising: the device comprises a media playing unit, an acquisition unit, an analysis unit and a synchronization reference determination unit;

the media playing unit is used for playing media files;

the acquisition unit is used for acquiring data of each synchronous reference of media playing in real time;

the analysis unit is used for analyzing the fluctuation of the data of each synchronous reference by taking a local system clock as a reference;

the synchronization reference determining unit is used for taking the synchronization reference corresponding to the data with the minimum fluctuation as the synchronization reference for currently executing the media playing synchronization operation;

the media playing unit is further configured to perform media playing synchronization operation according to the synchronization reference determined by the synchronization reference determining unit.

8. The media playback device of claim 7, wherein the analysis unit further comprises:

the deviation recording subunit is used for comparing the acquired data of each synchronous reference with a local system clock and recording the deviation value of the data of each synchronous reference;

an average value calculation operator unit, configured to calculate an average value of deviation values of the recorded data of the respective synchronization references for the respective synchronization references after the time for acquiring the data of the respective synchronization references by the acquisition unit reaches a predetermined time;

the counting subunit is used for counting the number of points of the deviation value of the data of each synchronous reference in each interval; the intervals are N intervals defined by taking the calculated average value as a center, and N is a natural number greater than zero;

and a weighted sum calculating subunit for calculating a weighted sum of the synchronization references based on the number of points at which the deviation value falls in each section and the section coefficients, the fluctuation becoming larger as the weighted sum decreases.

9. The media player of claim 8, further comprising:

and the clock monitoring unit is used for monitoring the time for acquiring the data of each synchronous reference by the acquisition unit and providing the time for the average value operator unit to be used as the time for judging and calculating the average value.

10. The media player of claim 7, 8 or 9, wherein the synchronization reference determination unit further comprises:

a synchronization reference determining subunit, configured to use a synchronization reference corresponding to the data with the minimum fluctuation as a synchronization reference for currently executing a media playing synchronization operation;

and the reference clock generating subunit is used for generating the reference clock according to the determined synchronous reference.

Images