CN101262612A - A system and method for synchronous playing of multimedia file audio and video - Google Patents
A system and method for synchronous playing of multimedia file audio and video Download PDFInfo
- Publication number
- CN101262612A CN101262612A CNA2008100880190A CN200810088019A CN101262612A CN 101262612 A CN101262612 A CN 101262612A CN A2008100880190 A CNA2008100880190 A CN A2008100880190A CN 200810088019 A CN200810088019 A CN 200810088019A CN 101262612 A CN101262612 A CN 101262612A
- Authority
- CN
- China
- Prior art keywords
- video data
- video
- audio
- playing
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 27
- 238000012545 processing Methods 0.000 claims abstract description 37
- 230000005540 biological transmission Effects 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 7
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 239000000047 product Substances 0.000 description 3
- 238000012360 testing method Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
Images
Abstract
The invention provides a system for the synchronous audio and video playing of a multi-media file and a method thereof. The system of the invention can detect the audio playing progress of an audio processing unit by an audio and video synchronous control unit and compare the playing progress with a predetermined value; when the playing progress is quicker than the predetermined value, the system sends a synchronous control command to an audio and video signal shunt unit to direct the signal shunt unit to read the video data of a corresponding frame to output and play the data so as to realize the synchronous audio and video playing. The system of the invention does not require multi-media data to contain a time stamp, thus can effectively reduce the burden of an encoder and a decoder; the system can also be applied to a platform with relatively poor calculating capability due to the small amount of calculation in the process of synchronous control, thereby being widely applicable to a portable mobile device for the synchronous audio and video playing when playing the multi-media data. In addition, the system of the invention has low requirement on a hardware platform which has simple structure and low cost.
Description
Technical field
The present invention relates to the method and system that a kind of multimedia file audio-visual synchronization is play, relating in particular to a kind of playing progress rate with voice data is reference, the method and system of coming synchronization video to play.
Background technology
Along with the development of semi-conductor industry, multimedia technology is IT (Information Technology, information technology) research staff's an important research and development object always.On present personal computer, multiple powerful multimedia solution has been arranged, comprise multimedia broadcast, catch and aspect such as editor.Yet the volume of personal computer is huge relatively, and the switching on and shutting down time is long, can't satisfy the requirement that the consumer enjoys multimedia recreation whenever and wherever possible.Therefore, consumer electronics obtains develop rapidly in recent years, and products such as mobile phone, PMP (Portable Media Player, portable electronic device), DPF emerge in an endless stream.In these products, the technology of the multimedia aspect that people are concerned about most is exactly the synchronous playing that how to realize audio, video data.
For the different multimedia data formats and the CPU of different abilities, the method for synchronous playing is also inequality.At present, much existing simultaneous techniques is all at MPEG (Moving Pictures ExpertsGroup) form, and these methods are more common.But it is higher that the MPEG algorithm has complexity, and consecutive frame is not independent mutually, must comprise the audio-visual characteristics of setting up synchronously such as timestamp information that aim in the code stream.And, use these general method for synchronous to have following shortcoming at the separate data format of other data format, especially each frame of video (as motion jpeg, this form uses in digital camera and monitoring product in a large number):
1. not free stamp the in a lot of data formats is because timestamp can make the burden of encoder increase simultaneously;
2. for the relatively poor platform of operational capability, if its decode procedure is not real-time, then can't be synchronous;
3. in these methods, the Play Control of Voice ﹠ Video all needs timer, also has other hardware resources, and cost is higher, and complexity is also big.
At publication number is CN1791207, open day is on June 21st, 2006, denomination of invention discloses a kind of playing device and method in the Chinese invention patent application of " a kind of playing device and method thereof ", and this specially installs or method is that the time stamp that produces according to audio signal parts comes synchronous image and audio frequency.It learns the broadcast sequential of picture frame in advance by time stamp, so just can promptly finish correction before image is asynchronous with message, effectively improves the quality of audio-visual broadcast.But this patent also is only at the video-audio data that comprises timestamp, and is not suitable for the Synchronous Processing of the video-audio data that does not comprise timestamp.
Summary of the invention
The technical problem to be solved in the present invention provides the method and system that a kind of multimedia file audio-visual synchronization is play, and this method or system stab when not required, under the not real-time situation of decoding, also can realize synchronously, and adapt to the platform of nonidentity operation ability automatically; In addition, little to taking of hardware resource, software complexity is also very low; Can effectively reduce cost.
In order to address the above problem, the invention provides a kind of multimedia file audio-visual synchronization playing system, comprising:
The video-audio signal dividing cell is used for multi-medium data is divided into voice data and video data and output;
Audio treatment unit is used for the voice data of described video-audio signal dividing cell output is decoded and play;
Video processing unit is used for the video data of described video-audio signal dividing cell output is decoded and play;
Also comprise:
Audio-visual synchronous control unit, be used to detect the playing progress rate of described audio treatment unit, and itself and a predetermined value compared, when playing progress rate during greater than this predetermined value, send the Synchronization Control instruction to described video-audio signal dividing cell, indication video-audio signal dividing cell reads the video data of respective frame and exports broadcast.
Further, described audio-visual synchronous control unit comprises voice playing detecting module and order sending module, wherein,
The playing progress rate that described voice playing detecting module is used to the to detect described audio treatment unit sound intermediate frequency playing module line item of going forward side by side, described voice playing detecting module compares playing progress rate and described predetermined value, when playing progress rate during greater than this predetermined value, send control information to described order sending module, the playing progress rate that is write down is deducted described predetermined value and on this difference basis, continue the recording played progress;
Described order sending module is used for after receiving described control information sending described Synchronization Control and instructs video data read module to described video-audio signal dividing cell;
Described video data read module, the feedback information that is used for will whether responding the Synchronization Control instruction after receiving the instruction of described Synchronization Control sends to described order sending module;
Described order sending module also is used for determining according to this feedback information the frame number of next bar Synchronization Control instruction needs reading video data.
Further, described voice playing detecting module comprises the coincidence counter and first comparator, described coincidence counter be used for when audio playing module when an audio frequency buffer module reads a voice data of playing unit, counting is once; Described first comparator is used for the count value and the described predetermined value of comparison coincidence counter,, sends control information to described order sending module, and count value is deducted described predetermined value during greater than predetermined value at synchronometer numerical value.
Further, the counting step-length step of described coincidence counter is obtained by following formula:
step＝rate*Length；
Described predetermined value gate obtains by following formula:
gate＝scale*Fs；
In the formula, Length is the size of voice playing buffer, and Fs is the audio data samples rate, and scale is the time scale of video flowing, and rate is the time scale of video flowing and the ratio of every frame video data reproduction time.
Further, described order sending module comprises a command counter, and this command counter is used for after the control information that receives the transmission of voice playing detecting module, sends Synchronization Control and instructs to the video data read module, and its counting is added 1; Described video data read module receives the count value N+1 of described Synchronization Control instruction back according to command counter, reads the following N+1 video data and the output frequently of present frame and plays; And the feedback information that described command counter receives the transmission of video data read module is when responding the Synchronization Control instruction, its counting of zero clearing.
Further, described order sending module comprises second command counter, response count device and second comparator, wherein,
Described second command counter adds 1 with its count value after being used to receive the control information that the voice playing detecting module sends;
Described response count device is used to receive feedback information that the video data read module sends and its count value is added 1 after for the instruction of response Synchronization Control;
Described second comparator is used for the value of more described second command counter and response count device, when described response count device does not have count value, sends the Synchronization Control that reads the next frame video data and instructs to the video data read module; When described second command counter during than the big M of value of response count device, the Synchronization Control that sends the following M frame video data that reads present frame is instructed to the video data read module; After the video data read module sends the feedback information of response Synchronization Control instruction to described second comparator, the second comparator zero clearing, second command counter and response count device.
Further, this system also comprises a timer, is used for when described multi-medium data does not have voice data, carries out timing according to the reproduction time of the every frame video data in the described multi-medium data, every when reaching regularly, one-frame video data is also play in the decoding of notice video data processing unit.
The method that the present invention also provides a kind of multimedia file audio-visual synchronization to play may further comprise the steps:
Calculate the playing progress rate parameter and the predetermined value of voice data according to the multi-medium data of playing;
Multi-medium data is divided into voice data and video data, decodes and show the first frame video data, enter the Audio Processing flow process simultaneously, decoding and playing audio-fequency data;
Detect the playing progress rate of voice data and the playing progress rate of audio data and whether reach described predetermined value, if reach, the control of video processing unit enters the Video processing flow process, the video data of decoding and demonstration respective frame.
Further, the method whether playing progress rate of described audio data reaches described predetermined value is: when the voice data of unit is play in one of every broadcast, counting once, count value and described predetermined value are compared, if count value is greater than predetermined value, then playing progress rate reaches predetermined value, and the playing progress rate that is write down is deducted described predetermined value, and continues the recording played progress on this difference basis.
Further, described control of video processing unit enters the step of Video processing flow process and is finished by an order sending module, when playing progress rate reaches predetermined value, the order sending module adds 1 with the counting of a command counter, and the transmission Synchronization Control is instructed to the video data read module, described video data read module receives the count value N+1 of described Synchronization Control instruction back according to command counter, reads the following N+1 video data and the output frequently of present frame and plays; And the feedback information that described command counter receives the transmission of video data read module is when responding the Synchronization Control instruction, its counting of zero clearing.
Further, the method that described control of video processing unit enters the Video processing flow process is: when playing progress rate reaches predetermined value, one second command counter is added 1; After the feedback information that the video data read module that receives sends instructs for the response Synchronization Control, one response count device is added 1, the value that compares second command counter and response count device with one second comparator, when described response count device does not have count value, send the Synchronization Control that reads the next frame video data and instruct to the video data read module; When described second command counter during than the big M of value of response count device, the Synchronization Control that sends the following M frame video data that reads present frame is instructed to the video data read module; Described second comparator is after the feedback information that the video data read module that receives sends instructs for the response Synchronization Control, with the count value zero clearing of described command counter and response count device.
The beneficial effect that the present invention produced is, by the playing progress rate that detects audio frequency come that control of video plays synchronously, do not need to comprise in the multi-medium data timestamp, can effectively alleviate the burden of encoder and decoder; Synchronization Control process operand is little, is applicable to the platform that operational capability is relatively poor, the audio-visual synchronization in the time of therefore can being widely used in the portable mobile apparatus play multimedia data; The present invention is very low to the requirement of hardware platform, and hardware platform is simple in structure, and cost is low.
Description of drawings
Fig. 1 is the overall structure block diagram of multimedia file audio-visual synchronization playing system of the present invention;
Fig. 2 is the structured flowchart of audio-visual synchronous control unit in the system of the present invention;
Fig. 3 is the structured flowchart of an embodiment of order sending module in the audio-visual synchronous control unit;
Fig. 4 is the flow chart of the method for multimedia file audio-visual synchronization broadcast of the present invention;
Fig. 5 is the schematic diagram of an embodiment of method sound intermediate frequency handling process of the present invention;
Fig. 6 is a schematic diagram of looking an embodiment of step handling process in the method for the present invention;
Fig. 7 is the flow chart of audio-visual Synchronization Control in the method for the present invention;
Fig. 8 is the flow chart that carries out the audio frequency detecting in the method for the present invention;
Fig. 9 is the flow chart that the order sending module sends an embodiment of video playback Synchronization Control instruction in the method for the present invention;
Figure 10 is the flow chart that the order sending module sends another embodiment of video playback Synchronization Control instruction in the method for the present invention.
Embodiment
The invention will be further described below in conjunction with the drawings and specific embodiments, can be implemented so that those skilled in the art can better understand the present invention also, but illustrated embodiment is not as a limitation of the invention.
The system of multimedia audio-video synchronous playing of the present invention comprises as shown in Figure 1:
Video-audio signal dividing cell 11, the multi-medium data that is used for being stored in file is divided into independently voice data and video data, and video-audio signal dividing cell 11 comprises voice data read module 111 and video data read module 112 again;
Voice data processing unit 12, be used for voice data is handled, comprise audio decoder module 121, audio frequency buffer module 122 and audio playing module 123 again, wherein, audio decoder module 121, be used for the output of voice data read module 111 is decoded, the result leaves in the audio frequency buffer module 122; Audio playing module 123 is responsible for the data in the audio plays buffer module 122;
Video data processing unit 13, be used for video data is handled, comprise video decode module 131, video buffer module 132 and video playback module 133 again, wherein, video decode module 131, be used for the output of video data read module 112 is decoded, the result leaves in the video buffer module 132; Video playback module 133 is used for the data of video buffer module 132 are play.
The internal structure of the above-mentioned functions unit module all internal structure with the corresponding functional unit that is adopted in the existing audio-visual synchronization treatment technology is identical, no longer the processing procedure and the internal structure of each functional unit and module are further set forth at this, and the division of above-mentioned functions module is only for describing the present invention, still can adopt other Module Division method that possesses above-mentioned functions to constitute the present invention, all can adopt same buffer module as audio frequency buffer module 122 and video buffer module 132.The present invention also comprises on the basis of existing technology:
Audio-visual synchronous control unit 14, it comprises a voice playing detecting module 141 and an order sending module 142, voice playing detecting module 141 detects the playing progress rate of audio playing module 123 and playing progress rate is carried out record, voice playing detecting module 141 compares the voice playing progress and a predetermined value of record, when playing progress rate during greater than predetermined value, voice playing detecting module 141 sends control information to order sending module 142, and continues counting after making the playing progress rate value of storing deduct predetermined value; After order sending module 142 receives control information, send the video playback Synchronization Control and instruct the video data that reads respective frame to video data read module 112, the feedback information whether video data read module 112 will respond the Synchronization Control instruction sends to order sending module 142, and order sending module 142 is determined the frame number of next bar video playback Synchronization Control instruction needs reading video data according to feedback information.
Wherein, as shown in Figure 2, voice playing detecting module 141 comprises a coincidence counter 1411 and first comparator 1412, when audio playing module 123 reads the data of a broadcast unit from audio frequency buffer module 122, coincidence counter 1411 is just counted once, and step-length is step; First comparator 1412 is the value and the threshold value gate (being predetermined value) of coincidence counter 1411 relatively, when synchronometer numerical value during greater than threshold value gate, just send control information to order sending module 142, coincidence counter 1411 count values deduct threshold value gate and continue counting.Voice playing detecting module 141 relatively judges whether to send control information to order sending module 142 according to voice data broadcast situation of counting in the coincidence counter 1411 and threshold value, thereby the broadcast of control of video data so just can realize coming by the voice playing progress purpose of synchronization video playing progress rate.
Wherein, as shown in Figure 2, order sending module 142 comprises a command counter 1421, after the control information that receives 141 transmissions of voice playing detecting module, just send the video playback Synchronization Control and instruct to video data read module 112, the count value of command counter 1421 adds 1 simultaneously, if the feedback information that the video data read module that receives 112 sends is the instruction of response Synchronization Control, then the zero clearing command counter 1421, otherwise do not carry out clear operation.If video data read module 112 receives all not responses of the subsynchronous control command of N, after then receiving the Synchronization Control commanded response once more, video data read module 112 is skipped the video data that the N frame reads down the N+1 frame according to the count value N+1 of command counter 1421 from present frame.The command counter 1421 of order in the sending module 142 counted according to the control information and the responsive feedback information that receive, thereby obtains the frame number (the following N+1 frame of present frame) of the video data that next Synchronization Control instruction need read.And then take place when asynchronous at audio, video data, directly read the video data of respective frame according to the playing progress rate of voice data.
In addition, as shown in Figure 3, order sending module 142 can also adopt following manner to constitute: comprise one second command counter 1422 and a response count device 1423, and one second comparator 1424, its count value just adds 1 after the control information that second command counter, 1422 reception voice playing detecting modules 141 send, the feedback information that the video data read module 112 that response count device 1423 receives sends just adds 1 for its count value of response Synchronization Control instruction back, second comparator 1424 compares the count value of second command counter 1422 and response count device 1423, when response count device 1423 no count values, just send the video playback Synchronization Control that reads the next frame video image and instruct to video data read module 112.If the count value of second command counter 1422 than the big M of count value of response count device 1423, is instructed to video data read module 112 just send the Synchronization Control of the following M frame video image that reads present frame.After video data read module 112 feeds back to second comparator, 1424 response Synchronization Control command informations, two counters of second comparator, 1424 zero clearings.
Wherein, voice data read module 111 is that unit reads audio compression data continuously with piece (Block) from source file.
Wherein, 123 of audio frequency buffer module 122 and audio playing module are carried out transfer of data with less broadcast unit (for example 32 sampled points).The reproduction time of supposing a unit data of broadcast is Ta, and the maximum synchronous error of establishing permission is Emax, and then Ta＜Emax gets final product.
Wherein, control command and parameter that video data read module 112 provides according to audio-visual synchronous control unit 14 are that unit reads to the video compression data in the source file with frame (Frame).
Make audio playing module 123 play a time T a who plays the unit voice data, much smaller than the demonstration time T v of video data one frame.Because the demonstration time T v of every frame video data fixes, the artificial setting when playing a time T a who plays the unit voice data and then being playing audio-fequency data.Because this method, the counting that can make coincidence counter just sends control information to order sending module 142 during greater than threshold value gate, and coincidence counter 1411 count values deduct gate.And the count value that coincidence counter 1411 count values deduct behind the gate is counted once more, and therefore counting is on the basis of difference once more, therefore can produce error.Because this error, the frame of video interval error of this method is positive and negative replacing in theory, thus synchronous error can not accumulate, all the time less than Ta.Test shows, when the broadcast unit of voice data hour (for example 32 sampled points), human eye is discovered less than the error less than Ta.In order further to reduce error, artificial setting audio playing module 123 is play a time T a who plays the unit voice data, much smaller than the demonstration time T v of video data one frame.
When concrete calculating,
Ta＝Length/Fs (1)，
In the formula, Length is the size of play-out buffer, and Fs is the audio data samples rate.
Tv has different expression formulas to different multimedia data formats, for example to AVI (AudioVideo Interleaved) file,
Tv＝scale/rate (2)，
In the formula, scale is the time scale of video flowing, and this parameter of rate does not have clear and definite meaning.Illustrate: the frame frequency of archives is 29.96fps, Tv=1/29.96 so, and this is a decimal, because the restriction of machine word length, can loss of accuracy when calculating.If write as Tv=100/2996, when calculating, just do not have decimal and scale=100 in this example, rate=2996 occur.Certainly, also can be write as scale=200, rate=5992, its meaning is the same.This only is the agreement of this form of AVI.Can draw according to above-mentioned formula (1) and formula (2):
Tv/Ta＝(scale*Fs)/(rate*Length) (3)；
Parameter in the formula preferably all is an integer, so that calculate.
Make step=rate*Length, gate=scale*Fs can obtain the step value step of coincidence counter and threshold value (predetermined value) gate.
For there not being voice data, have only the file of video data, can press default setting by audio treatment unit and play quiet data, come the broadcast of control of video data; For the file of voice playing total time less than video playback total time, after voice playing finishes, also can play quietly, finish up to video playback.In addition, a timer also can be set, when the voice data read module is judged received video-audio signal and is not had voice data, do not start the Audio Processing flow process, and start this timer, this timer connects video data read module, carries out timing according to the frame period of video, and the broadcast of video is controlled.
As shown in Figure 4, the method for multimedia file audio-visual synchronization broadcast of the present invention comprises the steps:
Step 41, system initialization is according to the audio, video data calculation of parameter voice playing progress parameter and the predetermined value that show and play;
Step 42 enters the Video processing flow process, decodes and shows the first frame video data, enters the Audio Processing flow process, decodes and plays the voice data that first plays unit;
Step 43 enters audio-visual Synchronization Control flow process, judges whether to send the instruction of video playback Synchronization Control according to voice playing progress situation, if send the instruction of video playback Synchronization Control, execution in step 44; If do not send the instruction of video playback Synchronization Control, execution in step 47;
Step 44 judges whether the Video processing flow process responds this video playback Synchronization Control instruction, if the Video processing flow process responds this video playback Synchronization Control instruction, execution in step 45; If the Video processing flow process does not respond this video playback Synchronization Control instruction, execution in step 46;
Step 45 enters the Video processing flow process, the video data of decoding and demonstration respective frame;
Step 46 is still play the video data of present frame;
Step 47 is hung up the Video processing flow process, no longer carries out the Video processing of next frame.
In step 42, the step of Audio Processing flow process is as shown in Figure 5: voice data read module 111 is that unit reads audio compression data (step 4211) continuously with piece (Block) from multimedia source files, deliver to audio decoder module 121 decode (step 4212), decoded data are stored in audio frequency buffer module 122, and the data in 123 pairs of audio frequency buffer modules 122 of audio playing module are play (step 4213).During broadcast, 123 of audio frequency buffer module 122 and audio playing module are carried out transfer of data with less broadcast unit (for example 32 sampled points).
In step 42, the step of Video processing flow process is as shown in Figure 6: instruction of video playback Synchronization Control and parameter that video data read module 112 provides according to audio-visual synchronous control unit 14, is that unit reads (step 4221) to the video compression data in the multi-medium data source file with frame (Frame), deliver to video decode module 131 decode (step 4222), the result is stored in the video buffer module 132 (step 4223), and the data in 133 pairs of video buffer modules 132 of video playback module are play (step 4224).
In step 43, audio-visual Synchronization Control flow process is as shown in Figure 7: detect the playing progress rate of audio playing module 123 and playing progress rate is write down (step 431), the voice playing progress and a predetermined value of record are compared (step 432), when playing progress rate during greater than predetermined value, the playing progress rate value of order storage deducts predetermined value, and send the video playback Synchronization Control and instruct the video data (step 433) that reads respective frame to video data read module 112, the feedback information whether video data read module 112 will respond the Synchronization Control instruction sends, and determines the frame number (step 434) of next bar Synchronization Control instruction needs reading video data according to this feedback information.
In the above-mentioned steps 43, the audio frequency detecting can be finished by a voice playing detecting module 141.As shown in Figure 8, voice playing detecting module 141 carries out the audio frequency detecting and comprises the steps: when audio playing module 123 reads the voice data of a broadcast unit from audio frequency buffer module 122, one coincidence counter 1411 is just counted once, step-length is playing progress rate parameter s tep (step 81), first comparator 1412 is the count value and the threshold value (being above-mentioned predetermined value) (step 82) of coincidence counter 1411 relatively, when the count value of coincidence counter 1411 during greater than threshold value gate, just send control information to an order sending module 142, coincidence counter 1411 count values deduct gate (step 83) and continue counting.Voice playing detecting module 141 according to the voice data of counting in the coincidence counter 1411 play situation and threshold value relatively judge whether to send control information to ordering sending module, thereby the broadcast of control of video data.So just, can realize coming the purpose of synchronization video playing progress rate by the voice playing progress.
As shown in Figure 9, the step of order sending module 142 transmission video playback Synchronization Control instructions comprises:
Step 91: judge whether order sending module 142 receives the control information that voice playing detecting module 141 sends, if receive, execution in step 92; If do not receive, wait for that voice playing detecting module 141 sends control information;
Step 92: send the video playback Synchronization Control and instruct to video data read module 112, command counter 1421 adds 1 simultaneously;
Step 93: whether the feedback information of judging video data read module 112 transmissions of receiving is the instruction of response Synchronization Control, if, execution in step 94; If not, execution in step 95;
Step 94: video data read module 112 is skipped the video data that the N frame reads down the N+1 frame according to the count value N+1 of command counter 1421 from present frame, zero clearing command counter 1421;
Step 95: command counter 1421 is not carried out clear operation and returns step 91.
In the step 94, if video data read module 112 receives all not responses of the subsynchronous control command of N, after then receiving the Synchronization Control commanded response once more, this moment, the count value of command counter was N+1, and video data read module 112 is skipped the video data that the N frame reads down the N+1 frame according to the counting of command counter 1421 from present frame.The command counter 1421 of order in the sending module 142 counted according to the control information and the responsive feedback information that receive, thereby obtains the frame number (the following N+1 frame of present frame) of the video data that next Synchronization Control instruction need read.And then take place when asynchronous at audio, video data, directly read the video data of respective frame according to the playing progress rate of voice data.
As shown in figure 10, order sending module 142 transmitting control commands can also adopt another kind of scheme, and this scheme comprises the steps:
Step 101: second command counter 1422 judges whether to receive the control information that voice playing detecting module 141 sends, if receive, and execution in step 102; If do not receive, continue to wait for that voice playing detecting module 141 sends control information;
Step 102: the value of second command counter 1422 adds 1;
Step 103: response count device 1423 judges whether the feedback information of video data read module 112 transmissions that receive is the instruction of response Synchronization Control, if, execution in step 104; If not, the feedback information of the response Synchronization Control instruction that video data read module 112 sends is waited in continuation;
Step 104: the value of response count device 1423 adds 1;
Step 105: second comparator 1424 compares the count value of second command counter 1422 and response count device 1423, when response count device 1423 no count values, just send the video playback Synchronization Control that reads the next frame video data and instruct to video data read module 112; If the counting of second command counter 1422 than the big M of counting of response count device 1423, instructs to video data read module 112 just send the video playback Synchronization Control of the following M frame video data that reads present frame;
Step 106: second comparator 1424 judges whether the feedback information of video data read module 112 transmissions that receive is the instruction of response Synchronization Control, if carry out step 107 down; If not, the feedback information of the response Synchronization Control instruction that video data read module 112 sends is waited in continuation;
Step 107: second comparator 1424 zero clearings, second command counter 1422 and the response count device 1423.
The present invention is when play multimedia files, and the principle of audio, video data synchronous playing is as follows:
Audio playing module 123 is play a time T a who plays the unit voice data, much smaller than the demonstration time T v of video data one frame.Because this method, the counting that can make coincidence counter just sends control information to order sending module 142 during greater than threshold value gate, and coincidence counter 1411 count values deduct gate.And the count value that coincidence counter 1411 count values deduct behind the gate is counted once more, and therefore counting is on the basis of difference once more, therefore can produce error.Because this error, the frame of video interval error of this method is positive and negative replacing in theory, so synchronous error can not accumulated, all the time less than Ta, test shows, when the broadcast unit of voice data hour (for example 32 sampled points), human eye is discovered less than the error less than Ta.
During concrete calculating, Ta=Length/Fs, Length are the size of play-out buffer, and Fs is the audio data samples rate; Tv has different expression formulas to different multimedia data formats, and for example to avi file, Tv=scale/rate, scale are the time scale of video flowing, Tv/Ta=(scale*Fs)/(rate*Length).Parameter in the formula all is an integer preferably, is convenient to calculate.Make step=rate*Length; Gate=scale*Fs.
Among the present invention, under the real-time situation of video decode, second command counter 1422 has only 0,1 two kind of value.Under the not real-time situation of video decode, the Video processing flow process does not also respond a synch command, new order has produced again, second command counter 1422 will constantly be counted, till the response of Video processing flow process, if at this moment the value of second command counter 1422 is N, then can pass to video data read module 112 to N, the video data read module is when reading video data from source file, can directly read the following N frame of present frame, send into Video Decoder and decode, thereby realize audio-visual synchronization.
For there not being voice data, have only the file of video data, the Audio Processing flow process can be play quiet data by default setting, comes the broadcast of control of video data; For the file of voice playing total time less than video playback total time, after voice playing finishes, also can play quietly, finish up to video playback.In addition, also can set the time by a timer, when whenever reaching setting-up time, just send Synchronization Control and instruct to the video data read module, order video data read module reads the one-frame video data broadcast of decoding.The time of this setting is generally the demonstration time of every frame video data in the video data.
The above embodiment is the preferred embodiment that proves absolutely that the present invention lifts, and protection scope of the present invention is not limited thereto.Being equal to that those skilled in the art are done on basis of the present invention substitutes or conversion, all within protection scope of the present invention.Protection scope of the present invention is as the criterion with claims.
Claims (11)
1, a kind of multimedia file audio-visual synchronization playing system comprises:
The video-audio signal dividing cell is used for multi-medium data is divided into voice data and video data and output;
Audio treatment unit is used for the voice data of described video-audio signal dividing cell output is decoded and play;
Video processing unit is used for the video data of described video-audio signal dividing cell output is decoded and play;
It is characterized in that, also comprise:
Audio-visual synchronous control unit, be used to detect the playing progress rate of described audio treatment unit, and itself and a predetermined value compared, when playing progress rate during greater than this predetermined value, send the Synchronization Control instruction to described video-audio signal dividing cell, indication video-audio signal dividing cell reads the video data of respective frame and exports broadcast.
2, the system as claimed in claim 1 is characterized in that, described audio-visual synchronous control unit comprises voice playing detecting module and order sending module, wherein,
The playing progress rate that described voice playing detecting module is used to the to detect described audio treatment unit sound intermediate frequency playing module line item of going forward side by side, described voice playing detecting module compares playing progress rate and described predetermined value, when playing progress rate during greater than this predetermined value, send control information to described order sending module, the playing progress rate that is write down is deducted described predetermined value and on this difference basis, continue the recording played progress;
Described order sending module is used for after receiving described control information sending described Synchronization Control and instructs video data read module to described video-audio signal dividing cell;
Described video data read module, the feedback information that is used for will whether responding the Synchronization Control instruction after receiving the instruction of described Synchronization Control sends to described order sending module;
Described order sending module also is used for determining according to this feedback information the frame number of next bar Synchronization Control instruction needs reading video data.
3, system as claimed in claim 2, it is characterized in that, described voice playing detecting module comprises the coincidence counter and first comparator, described coincidence counter be used for when audio playing module when an audio frequency buffer module reads a voice data of playing unit, counting is once; Described first comparator is used for the count value and the described predetermined value of comparison coincidence counter, at synchronometer numerical value during greater than predetermined value, send control information to described order sending module, and count value is deducted described predetermined value, coincidence counter is proceeded counting on the difference basis.
4, system as claimed in claim 3 is characterized in that, the counting step-length step of described coincidence counter is obtained by following formula:
step＝rate*Length；
Described predetermined value gate obtains by following formula:
gate＝scale*Fs；
In the formula, Length is the size of voice playing buffer, and Fs is the audio data samples rate, and scale is the time scale of video flowing, and rate is the time scale of video flowing and the ratio of every frame video data reproduction time.
5, as claim 3 or 4 described systems, it is characterized in that described order sending module comprises a command counter, this command counter is used for after the control information that receives the transmission of voice playing detecting module, send Synchronization Control and instruct, and its counting is added 1 to the video data read module; Described video data read module receives the count value N+1 of described Synchronization Control instruction back according to command counter, reads the following N+1 video data and the output frequently of present frame and plays; And the feedback information that described command counter receives the transmission of video data read module is when responding the Synchronization Control instruction, its counting of zero clearing.
As claim 3 or 4 described systems, it is characterized in that 6, described order sending module comprises second command counter, response count device and second comparator, wherein,
Described second command counter adds 1 with its count value after being used to receive the control information that the voice playing detecting module sends;
Described response count device is used to receive feedback information that the video data read module sends and its count value is added 1 after for the instruction of response Synchronization Control;
Described second comparator is used for the value of more described second command counter and response count device, when described response count device does not have count value, sends the Synchronization Control that reads the next frame video data and instructs to the video data read module; When described second command counter during than the big M of value of response count device, the Synchronization Control that sends the following M frame video data that reads present frame is instructed to the video data read module; After the video data read module sends the feedback information of response Synchronization Control instruction to described second comparator, the second comparator zero clearing, second command counter and response count device.
7, the system as claimed in claim 1, it is characterized in that, this system also comprises a timer, be used for when described multi-medium data does not have voice data, reproduction time according to the every frame video data in the described multi-medium data carries out timing, every when reaching regularly, one-frame video data is also play in the decoding of notice video data processing unit.
8, a kind of method of multimedia file audio-visual synchronization broadcast is characterized in that, may further comprise the steps:
Calculate the playing progress rate parameter and the predetermined value of voice data according to the multi-medium data of playing;
Multi-medium data is divided into voice data and video data, decodes and show the first frame video data, enter the Audio Processing flow process simultaneously, decoding and playing audio-fequency data;
Detect the playing progress rate of voice data and the playing progress rate of audio data and whether reach described predetermined value, if reach, the control of video processing unit enters the Video processing flow process, the video data of decoding and demonstration respective frame.
9, method as claimed in claim 8, it is characterized in that, the method whether playing progress rate of described audio data reaches described predetermined value is: when the voice data of unit is play in one of every broadcast, counting once, count value and described predetermined value are compared, if count value greater than predetermined value, then playing progress rate reaches predetermined value, the playing progress rate that is write down is deducted described predetermined value, and on this difference basis, continue the recording played progress.
10, method as claimed in claim 9, it is characterized in that, described control of video processing unit enters the step of Video processing flow process and is finished by an order sending module, when playing progress rate reaches predetermined value, the order sending module adds 1 with the counting of a command counter, and the transmission Synchronization Control is instructed to the video data read module, described video data read module receives the count value N+1 of described Synchronization Control instruction back according to command counter, reads the following N+1 video data and the output frequently of present frame and plays; And the feedback information that described command counter receives the transmission of video data read module is when responding the Synchronization Control instruction, its counting of zero clearing.
11, method as claimed in claim 9 is characterized in that, the method that described control of video processing unit enters the Video processing flow process is: when playing progress rate reaches predetermined value, one second command counter is added 1; After the feedback information that the video data read module that receives sends instructs for the response Synchronization Control, one response count device is added 1, the value that compares second command counter and response count device with one second comparator, when described response count device does not have count value, send the Synchronization Control that reads the next frame video data and instruct to the video data read module; When described second command counter during than the big M of value of response count device, the Synchronization Control that sends the following M frame video data that reads present frame is instructed to the video data read module; Described second comparator is after the feedback information that the video data read module that receives sends instructs for the response Synchronization Control, with the count value zero clearing of described command counter and response count device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2008100880190A CN100562122C (en) | 2008-03-27 | 2008-03-27 | A kind of multimedia file audio-visual synchronization playing system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2008100880190A CN100562122C (en) | 2008-03-27 | 2008-03-27 | A kind of multimedia file audio-visual synchronization playing system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101262612A true CN101262612A (en) | 2008-09-10 |
| CN100562122C CN100562122C (en) | 2009-11-18 |
Family
ID=39962771
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2008100880190A Active CN100562122C (en) | 2008-03-27 | 2008-03-27 | A kind of multimedia file audio-visual synchronization playing system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100562122C (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102184739A (en) * | 2011-03-08 | 2011-09-14 | 深圳英飞拓科技股份有限公司 | Video play device and method |
| CN102685507A (en) * | 2011-07-12 | 2012-09-19 | 天脉聚源(北京)传媒科技有限公司 | Audio and video synchronization method and system for decoding moving picture experts group-2 (MPEG-2) transport streams |
| CN103281528A (en) * | 2013-04-10 | 2013-09-04 | 深圳康佳通信科技有限公司 | Streaming media decoding method, system and mobile terminal |
| CN103338204A (en) * | 2013-07-05 | 2013-10-02 | 曾德钧 | Audio synchronization output method and system |
| CN103391467A (en) * | 2013-07-16 | 2013-11-13 | 四川省迪特尔电子有限公司 | Method for achieving synchronization of decoding and displaying of audio and video of network set-top box |
| CN104469485A (en) * | 2014-12-10 | 2015-03-25 | 四川长虹电器股份有限公司 | Playing method for people with visual impairment |
| CN104464786A (en) * | 2014-11-21 | 2015-03-25 | 西安诺瓦电子科技有限公司 | Audio frequency controlling device and method |
| CN108965990A (en) * | 2018-07-20 | 2018-12-07 | 广州酷狗计算机科技有限公司 | Control the mobile method and apparatus of pitch line |
| CN113992638A (en) * | 2018-05-02 | 2022-01-28 | 腾讯科技（上海）有限公司 | Synchronous playing method and device of multimedia resources, storage position and electronic device |
-
2008
- 2008-03-27 CN CNB2008100880190A patent/CN100562122C/en active Active
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102184739A (en) * | 2011-03-08 | 2011-09-14 | 深圳英飞拓科技股份有限公司 | Video play device and method |
| CN102685507A (en) * | 2011-07-12 | 2012-09-19 | 天脉聚源(北京)传媒科技有限公司 | Audio and video synchronization method and system for decoding moving picture experts group-2 (MPEG-2) transport streams |
| CN102685507B (en) * | 2011-07-12 | 2014-07-09 | 天脉聚源(北京)传媒科技有限公司 | Audio and video synchronization method and system for decoding moving picture experts group-2 (MPEG-2) transport streams |
| CN103281528A (en) * | 2013-04-10 | 2013-09-04 | 深圳康佳通信科技有限公司 | Streaming media decoding method, system and mobile terminal |
| CN103338204A (en) * | 2013-07-05 | 2013-10-02 | 曾德钧 | Audio synchronization output method and system |
| CN103391467A (en) * | 2013-07-16 | 2013-11-13 | 四川省迪特尔电子有限公司 | Method for achieving synchronization of decoding and displaying of audio and video of network set-top box |
| CN104464786B (en) * | 2014-11-21 | 2018-06-26 | 西安诺瓦电子科技有限公司 | A kind of audio control apparatus and method |
| CN104464786A (en) * | 2014-11-21 | 2015-03-25 | 西安诺瓦电子科技有限公司 | Audio frequency controlling device and method |
| CN104469485A (en) * | 2014-12-10 | 2015-03-25 | 四川长虹电器股份有限公司 | Playing method for people with visual impairment |
| CN113992638A (en) * | 2018-05-02 | 2022-01-28 | 腾讯科技（上海）有限公司 | Synchronous playing method and device of multimedia resources, storage position and electronic device |
| CN113992638B (en) * | 2018-05-02 | 2023-07-14 | 腾讯科技（上海）有限公司 | Synchronous playing method and device for multimedia resources, storage position and electronic device |
| CN108965990A (en) * | 2018-07-20 | 2018-12-07 | 广州酷狗计算机科技有限公司 | Control the mobile method and apparatus of pitch line |
| CN108965990B (en) * | 2018-07-20 | 2020-11-17 | 广州酷狗计算机科技有限公司 | Method and device for controlling movement of sound altitude line |
Also Published As
| Publication number | Publication date |
|---|---|
| CN100562122C (en) | 2009-11-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100562122C (en) | A kind of multimedia file audio-visual synchronization playing system and method | |
| US5668601A (en) | Audio/video decoding system | |
| CN2927556Y (en) | Video and audio re-player, outputting-time converter | |
| CN101303880B (en) | Method and apparatus for recording and playing audio-video document | |
| CN106792154B (en) | Frame skipping synchronization system of video player and control method thereof | |
| US9438876B2 (en) | Method for semantics based trick mode play in video system | |
| WO2020215453A1 (en) | Video recording method and system | |
| CN101710997A (en) | MPEG-2 (Moving Picture Experts Group-2) system based method and system for realizing video and audio synchronization | |
| JP2008311888A5 (en) | ||
| TW200926806A (en) | Content reproduction apparatus, content reproduction method, and content reproduction system | |
| JP4511952B2 (en) | Media playback device | |
| CN1140955A (en) | Timing detection device and method | |
| EP1411522A3 (en) | Determining a scene change point | |
| CN106331820A (en) | Synchronous audio and video processing method and device | |
| US7218841B2 (en) | Method and apparatus for synchronously reproducing audio and video data | |
| JPH08214296A (en) | Decoder and mpeg system decoder | |
| JP4380930B2 (en) | Audio / video decoding / playback apparatus and synchronized playback method thereof | |
| CN1148061C (en) | Image replaying equipment | |
| CN104205860B (en) | Numerical data distributing device and method, numerical data transcriber and method and sync reproduction system | |
| JPH06125363A (en) | Packet communication system | |
| TWI277346B (en) | Processing device switching time axis in video data stream and method thereof | |
| CN100474914C (en) | Method and apparatus for controlling a bit rate of digital video data | |
| US20070058934A1 (en) | Decoding/reproducing apparatus | |
| JP2008054159A (en) | Video-audio multiplexing apparatus | |
| JPH11134804A (en) | Image and audio synchronization system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: LINGTONG TECHNOLOGY CO., LTD. BEIJING SUNPLUS EHUE Free format text: FORMER OWNER: LINGTONG TECHNOLOGY CO., LTD. BEIJING BEIYANG ELECTRONIC TECHNOLOGY CO., LTD. Effective date: 20140414 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20140414 Address after: Science and Industry Park, Taiwan, Hsinchu, China, No. 19 Patentee after: Sunplus Technology Co.,Ltd. Patentee after: GENERALPLUS TECHNOLOGY INC. Patentee after: BEIJING SUNPLUS-EHUE TECH Co.,Ltd. Address before: Science and Industry Park, Taiwan, Hsinchu, China, No. 19 Patentee before: Sunplus Technology Co.,Ltd. Patentee before: GENERALPLUS TECHNOLOGY INC. Patentee before: Beijing Sunnorth Electronic Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20180103 Address after: 063100 No. 9, 41 / 2, North Tower, Linxi long Jiangxi Li machine factory, Guye District, Tangshan City, Hebei Patentee after: Ban Xiuqin Address before: Science and Industry Park, Taiwan, Hsinchu, China, No. 19 Co-patentee before: GENERALPLUS TECHNOLOGY INC. Patentee before: Sunplus Technology Co.,Ltd. Co-patentee before: BEIJING SUNPLUS-EHUE TECH Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220824 Address after: 101d1-7, 1st floor, building 1, No. 6, Shangdi West Road, Haidian District, Beijing 100085 Patentee after: Beijing Dajia Internet Information Technology Co.,Ltd. Address before: 063100 No.9, door 2, 41 / F, North building, Jiangxi Liji factory, linxilong, Guye District, Tangshan City, Hebei Province Patentee before: Ban Xiuqin |
|
| TR01 | Transfer of patent right |