CN101212679B - AV synchronization method and device for switching multi-channel audio streams during playing of AVI files - Google Patents
AV synchronization method and device for switching multi-channel audio streams during playing of AVI files Download PDFInfo
- Publication number
- CN101212679B CN101212679B CN 200610172268 CN200610172268A CN101212679B CN 101212679 B CN101212679 B CN 101212679B CN 200610172268 CN200610172268 CN 200610172268 CN 200610172268 A CN200610172268 A CN 200610172268A CN 101212679 B CN101212679 B CN 101212679B
- Authority
- CN
- China
- Prior art keywords
- audio
- data
- video
- unit
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Television Signal Processing For Recording (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to an AV synchronized method for switching a plurality of audio streams when AVI files are played, which is applied in a system that comprises a switching control unit, a data transmission unit, a data processing unit and a data decode unit. If the audio steams are switched when AVI files are played, the switching control unit sends switching command to the data processing unit and the data decode unit; the data decode unit passes back current decoded video frame numbers; the switching unit obtains a retransmission position and parameters of synchronous playing by calculation of the video frame numbers, index information and file head information, and then sends the retransmission position and the parameters of synchronous playing to the data transmission unit and the data decode unit; after the data decode unit completes the reinitialization of audio and video decoding, the retransmitted audio and video data are decoded, and whether the decoded audio and video data are sent to the a data display unit for synchronous playing is determined by the parameters of synchronous playing. The method of the invention can keep AV synchronized after switching the audio streams when the AVI files comprising Divx video coding data and a plurality of audio data are played.
Description
Technical field
The present invention relates generally to audio frequency and video and plays the field, relates in particular to a kind of avi file synchronous method and apparatus of audio frequency and video (AV) when switching the voice data of playing that is applied to keep in the embedded system containing one road Divx video data encoder, multi-path audio-frequency data.
Background technology
Earlier the relevant technical terms that occurs in the literary composition is made an explanation:
The I frame: the key video sequence frame, the video pictures of lossless compress, the place of all references I frame in this file, what refer to all is frame of video;
Divx: a kind of video code model;
AV:Audio and Video, Voice ﹠ Video;
CBR: fixing code stream;
VBR: variable code stream;
SPHE1001N：Sunplus?Home?Entertainment?1001N；
STB: full name is that Set-Top-Box is a set-top box;
AVI:Audio Video Interleaved, audio, video data interleaving access.
AVI (Audio Video Interleaved (interleaving access)) is the most frequently used a kind of media file format, is used in the application programs such as audio frequency and video seizure, editor, playback more.Generally, an avi file can comprise a plurality of dissimilar Media Streams.Usually the Media Stream that comprises of avi file is for containing a video flowing and an audio stream, but the avi file that contains a video flowing and several audio streams also is legal.
As shown in Figure 1, the avi file structure mainly comprises three parts: avi file head, audio, video data, index information.
Wherein from the avi file head, obtainable information comprises: AVIMAINHEADER (AVI master file head) information, format information, audio frequency fluxion, video flowing time scale and speed index, audio, video data starting position Movie_Offset, the whole big or small Movie_Size of audio, video data, the sample rate of VBR audio stream or the information such as byte rate of CBR audio stream.Wherein, avi file sound intermediate frequency stream is generally two kinds of coded format VBR audio frequency or CBR audio frequency, so according to circumstances resolve relevant parameter.
Wherein audio, video data is partly for being that unit intersects successively and deposits each road audio frequency and video real data with the data block.As shown in Figure 2.
Wherein INDEX is made up of the tabulation of index information piece, each audio or video data block in each index information piece corresponding A VI file, this index information piece is according to the avi file middle pitch, the block of video data form of being staggered is arranged in order and forms tabulation, each index information piece comprises: the data block sign (is a block of video data, the Blank data block that audio data block is still filled, also comprise the sign that belongs to which audio stream for audio data block), whether be key frame sign (being the sign of I frame), be data splitting block mark (being that a plurality of data chunk are combined into a data combination) or forms data piece (data block, it can be a frame, perhaps a plurality of frames), side-play amount dwOffset (every audio/video data piece correspondence is hereof apart from the skew of file header or audio, video data head in the every road of index), and the byte number of diaphone/block of video data.
For next number of video data ordinary circumstance according to the corresponding frame of piece; For the voice data of VBR, the corresponding audio frame of data block for the voice data of CBR, does not have absolute corresponding relation between data block and the audio frame.Have the following situation of minority in avi file: several audio data blocks are combined as a combined data block.
Because the AVI video file does not contain accurate, equally distributed timestamp information.AV for non-handoff procedure is synchronous, mainly be before initializing decoder, from the audio/video flow header, read the time scale parameter and the rate parameter of each stream,, thereby keep AV synchronous according to the decode rate of this two classes parameter difference initialization audio/video decoder.And contain the avi file of one road Divx video data encoder, multi-path audio-frequency data in broadcast, and when switching the voice data of playing, just need recalculate the exact position of audio frequency, video playback, AV is synchronous with assurance.
The present invention is based on embedded system, and decoding work is finished by hardware chip, and Data Transmission Controlling is finished by software, to reduce the operation for embedded system requirement, accelerates decoding speed.But when adopting the hardware decoding, the positional information of transfer of data and the maintenance AV such as positional information of data decode required informational needs synchronously transmit between each unit, and the control of work and coordination are difficult to realize also there is not so a kind of method at present.
Summary of the invention
The technical problem to be solved in the present invention provides AV method for synchronous and the device that switches multichannel voice frequency stream when a kind of avi file is play, when broadcast contains the avi file of one road Divx video data encoder, multi-path audio-frequency data, can keep AV synchronous behind the switch audio stream.
In order to solve the problems of the technologies described above, the invention provides the AV method for synchronous that switches multichannel voice frequency stream when a kind of avi file is play, be applied to comprise the system of switch control unit, data transmission unit, data processing unit and data decode unit, when broadcast contains the avi file of multichannel voice frequency stream, preserve the avi file header earlier, behind user's switch audio stream, carry out following steps:
(a) after switch control unit is received handoff request, send switching command to data processing unit and data decode unit;
(b) the data decode unit returns current decoded video frames number to switch control unit;
(c) switch control unit obtains retransmitting position and synchronous playing parameter according to index information, the file header information calculations of current decoded video frames number and preservation, to retransmit the position and send to data transmission unit, the synchronous playing parameter is sent to the data decode unit, and described synchronous playing parameter is that the audio frequency that calculates is skipped number Skip_audio and video is skipped number Skip_Frame;
(d) data transmission unit retransmits the audio, video data that the position is transmitted avi file again by specifying, and data processing unit receives that road audio stream of isolating video flowing and should switch to behind the audio, video data; Reinitializing of video decode finished in the data decode unit, and carries out the initialization of audio decoder again by the parameter of that road audio stream that will switch to;
(e) after the data decode unit receives the audio, video data of transmission again, the audio/video flow of the new transmission of described data decode unit counterweight begins Skip_audio byte number or frame number and Skip_Frame frame of video of part, be not transferred to data display unit after the decoding, Xie Ma audio, video data is transferred to data display unit again and carries out synchronous playing afterwards, wherein to fixing code streams by using byte number, to variable code streams by using frame number.
Further, above-mentioned audio and video synchronization method also can have following characteristics:
Described data decode unit continues the video data of buffer memory is decoded after receiving switching command, after receiving parameter S kip_audio and Skip_Frame, empties the video data of buffer memory again, carries out reinitializing of video decode.
Further, above-mentioned audio and video synchronization method also can have following characteristics:
Described system is an embedded system, and described data decode unit realizes that with hardware switch control unit, data transmission unit and data processing unit are then carried out corresponding software by CPU and realized.
Further, above-mentioned audio and video synchronization method also can have following characteristics: described step (c) switch control unit is further finished calculating by following steps:
The frame number A that is decoding during (c1) according to the switch audio data flow adds the frame number B of setting, obtains the audio video synchronization point V of this switching
CFrame number, and, determine the index information original position according to the total size of the audio, video data starting position of resolving and its, read index information and preserve;
(c2), the most approaching and less than V with the video frame number of accumulative total from the number of audio, video data index starting position accumulative total frame of video
CThe I frame of frame number is V
CNearest before I frame V
I
(c3) from index information, find V
ICorresponding audio data block A
I, read A
IDeviation post in avi file is put with this position open position of attaching most importance to;
(c4) calculate V
CWith V
IBetween the frame number difference be that video is skipped number Skip_Frame, and play V
CThe time that road audio stream that will switch to audio frequency byte number or the frame number Audio_accum_size and A that should broadcast
IThe audio frequency byte number or the frame number Audio_TotalN_A that add up before and obtain
IDifference, promptly audio frequency is skipped number Skip_audio.
Further, above-mentioned audio and video synchronization method also can have following characteristics: V in the described step (c3)
ICorresponding audio data block A
ISearch in the following manner:
(c31) calculate broadcast V
IAudio frequency byte number or frame number Audio_TotalN_I that road audio stream that before will switch to should be broadcast, formula is:
To fixing code stream: Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * CBR audio frequency byte rate
To variable code stream: Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * VBR audio sample rate/1152
(c32) from the original position of audio, video data, the byte number or the frame number of that road audio stream data piece that will switch to one by one add up, accumulated value is compared with the Audio_TotalN_I that obtains above, if less than Audio_TotalN_I, add up again next time, as in more than or equal to Audio_TotalN_I, carry out next step;
(c33) equal Audio_TotalN_I as accumulated value, then next audio data block of this road audio stream is V
ICorresponding audio data block A
I, greater than Audio_TotalN_I, then this audio data block is V as accumulated value
ICorresponding audio data block A
I, write down A simultaneously
IThe audio frequency byte number or the frame number Audio_TotalN_A that add up before and obtain
I
(c34) from audio data block A
IFind deviation post dwOffset in the corresponding index information piece, this position is and retransmits the position.
Calculate in the described step (c4) and play V
CThe time when audio frequency byte number that should broadcast of that road audio stream that will switch to or frame number Audio_accum_size, only need the V in step (c31) formula
IFrame number make V into
CFrame number get final product.
Further, above-mentioned audio and video synchronization method also can have following characteristics:
Described frame number B span is: 6～12 frames.
Further, above-mentioned audio and video synchronization method also can have following characteristics:
In the step (a), after switch control unit is received handoff request, also send switching command, after described data transmission unit is received switching command, stop transmission immediately former audio/video flow to described data transmission unit; Simultaneously, as comprising three the tunnel and above audio stream data in the avi file, then switch control unit sends in the instruction of data processing unit and data decode unit, and which audio stream what need that indication will switch to is.
The AV synchronizer that switches multichannel voice frequency stream when avi file provided by the invention is play comprises:
Data transmission unit, the audio, video data that is used to read avi file is transferred to data processing unit, and retransmits the audio, video data that the position is transmitted avi file again by specifying after receiving retransmission instructions;
Data processing unit, be used for avi file header and index information are resolved and preserved, finish separating and being transferred to the data decode unit of audio stream and video flowing, after receiving switching command, that road audio stream of from audio, video data, isolating video flowing and should switching to;
Switch control unit, be used for after receiving handoff request, send switching command to data processing unit and data decode unit, index information, file header information calculations according to current decoded video frames number and preservation obtains retransmitting position and synchronous playing parameter then, to retransmit the position and send to data transmission unit, the synchronous playing parameter is sent to the data decode unit, and described synchronous playing parameter is that the audio frequency that calculates is skipped number Skip_audio and video is skipped number Skip_Frame;
The data decode unit comprises audio decoder subelement and video decode subelement; After receiving switching command, return current decoded video frames number to switch control unit, and finish the initialization that reinitializes and carry out audio decoder by the parameter of that road audio stream that will switch to again of video decode; According to the synchronous playing parameter that receives, the audio/video flow of the new transmission of described data decode unit counterweight begins Skip_audio byte number or frame number and Skip_Frame frame of video of part, be not transferred to data display unit after the decoding, Xie Ma audio, video data is transferred to data display unit again and carries out synchronous playing afterwards, wherein to fixing code streams by using byte number, to variable code streams by using frame number.
Further, above-mentioned audio-visual synchronization device also can have following characteristics: described switch control unit further comprises:
The interface subelement is used to finish communicating by letter between switch control unit and other unit;
The information extraction subelement is used for obtaining avi file header and index information from data processing unit, uses for the operator unit.
Control sub unit, be used for after receiving the handoff request of user interface section, send switching command to data decode unit, data processing unit and data decode unit, the instruction that sends to data processing unit and decoding unit should be carried the audio stream sign that will switch to, and obtains current decoded video frames sequence number from the data decode unit and is sent to the operator unit; And after re-transmission position, the audio frequency of receiving the operator unit are skipped number and video and are skipped number of parameters, send retransmission instructions to data transmission unit, carry the re-transmission location parameter, and audio frequency is skipped number and video skip number of parameters and be transferred to the data decode unit.
The operator unit is used for according to current decoded video frames sequence number and avi file header and index information, calculates and retransmits the position, audio frequency skips number Skip_audio and video is skipped number Skip_Frame, is sent to control sub unit.
Further, above-mentioned audio-visual synchronization device also can have following characteristics:
Described data decode unit continues the video data of buffer memory is decoded after receiving switching command, after receiving parameter S kip_audio and Skip_Frame, empties the video data of buffer memory again, carries out reinitializing of video decode.
Further, above-mentioned audio-visual synchronization device also can have following characteristics: described operator unit further comprises with the lower part:
Synchronous points is calculated and index information extracts part, and the frame number A that is decoding during according to the switch audio data flow adds the frame number B of setting, obtains the audio video synchronization point V of this switching
CFrame number, and, determine the index information original position according to the total size of the audio, video data starting position of resolving and its, read index information and preserve;
I frame determining section, from the number of audio, video data index starting position accumulative total frame of video, the most approaching and less than V with the video frame number of accumulative total
CThe I frame of frame number is V
CNearest before I frame V
I
Retransmit the position calculation part, from index information, find V
ICorresponding audio data block A
I, read A
IDeviation post in avi file is put with this position open position of attaching most importance to;
Skip the number calculating section, calculate V
CWith V
IBetween the frame number difference be that video is skipped number Skip_Frame, and play V
CThe time that road audio stream that will switch to audio frequency byte number or the frame number Audio_accum_size and A that should broadcast
IThe audio frequency byte number or the frame number Audio_TotalN_A that add up before and obtain
IDifference, promptly audio frequency is skipped number Skip_audio.
Further, above-mentioned audio-visual synchronization device also can have following characteristics:
Described re-transmission position calculation is partly searched V
ICorresponding audio data block A
IThe time, calculate earlier and play V
IAudio frequency byte number or frame number Audio_TotalN_I that road audio stream that before will switch to should be broadcast, formula is:
To fixing code stream: Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * CBR audio frequency byte rate
To variable code stream: Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * VBR audio sample rate/1152
Again from the original position of audio, video data, the byte number or the frame number of that road audio stream data piece that will switch to one by one add up, accumulated value is compared with the Audio_TotalN_I that obtains above, if less than Audio_TotalN_I, add up again next time, equal Audio_TotalN_I as accumulated value, then next audio data block of this road audio stream is V
ICorresponding audio data block A
I, greater than Audio_TotalN_I, then this audio data block is V as accumulated value
ICorresponding audio data block A
I, write down A simultaneously
IThe audio frequency byte number or the frame number Audio_TotalN_A that add up before and obtain
I
At last, from audio data block A
IFind deviation post dwOffset in the corresponding index information piece, this position is and retransmits the position.
The described number calculating section of skipping calculates broadcast V
CThe time when audio frequency byte number that should broadcast of that road audio stream that will switch to or frame number Audio_accum_size, only need the V in the aforementioned calculation Audio_TotalN_I formula
IFrame number make V into
CFrame number get final product.
Further, above-mentioned audio-visual synchronization device also can have following characteristics:
Described switch control unit also is used for after receiving handoff request, sends switching command to described data transmission unit; After described data transmission unit is received switching command, stop transmission immediately to former audio/video flow; Described switch control unit comprises three the tunnel and during above audio stream data in the avi file of playing, send in the instruction of data processing unit and data decode unit, and which audio stream what need that indication will switch to is.
Further, above-mentioned audio-visual synchronization device also can have following characteristics:
Described device is applied to embedded system, and described data decode unit realizes that with hardware switch control unit, data transmission unit and data processing unit are carried out corresponding software by CPU and realized.
The present invention has guaranteed to play the avi file that contains one road Divx video data encoder, multi-path audio-frequency data, keeps AV synchronous when switching different audio stream data.Further can also keep the continuity of video playback as far as possible.Be particularly useful under the low configuration condition of embedded system, realize that the AV when the avi file voice data switches is synchronous, thereby reduced requirement, can realize former/functions such as vocal accompaniment switching of Karaoke (Karaoke) system's dominant frequency.
Description of drawings
Fig. 1 is the schematic diagram of avi file structure.
Fig. 2 is the structural representation of the audio, video data part among Fig. 1.
Fig. 3 is the structured flowchart of embodiment of the invention system.
Fig. 4 is the process chart of embodiment of the invention method when switching former vocal accompaniment.
Fig. 5 and Fig. 6 are respectively the schematic diagrames of the calculation of parameter of CBR audio stream and VBR audio stream.
Embodiment
The present invention plays to contain in the avi file process of one road Divx video data encoder, multi-path audio-frequency data, keep the synchronous method and system of AV during the switch audio data, below to switch former vocal accompaniment when playing the Karaoke system be example, the present invention is described in detail in conjunction with the accompanying drawings and embodiments.
Because need a period of time to calculate from obtaining switching command transfer period to exercising multichannel voice frequency, video image is static to fix in order to avoid, video council continues to play a period of time.Under the situation of video playback, for the audio ﹠ video after guaranteeing to switch synchronously, a point will selecting audio frequency and video to play simultaneously, i.e. audio video synchronization point (being frame of video) C and corresponding audio data block thereof.This audio video synchronization point should be when switching certain position after the decoded video frames.
According to the requirement of decoding, the decoding of video must be from the I frame in addition, and audio frequency can be from any position, if directly from C point video decode, then mosaic phenomenon can occur.So during the starting position of decoding again after determining to switch, need find audio video synchronization point C (also to use V
CExpression) nearest before I frame (is also used V
IExpression) audio data block of institute correspondence (referring to answer synchronous playing) (below use A
IExpression), transmit again and the avi file data of decoding from the deviation post Goto_offset of this audio data block avi file.For V
CFrame of video before and V
CThe corresponding audio data block (below use A
CExpression) before audio, video data is only decoded and is not play.
Herein transmission again or re-transmission are meant and break order of transmission for the first time, begin the new transmission second time, and the original position of current transmission no longer is original file header.
Fig. 3 is the structured flowchart of embodiment of the invention system, as shown in the figure, comprises data transmission unit, data processing unit, switch control unit, data decode unit, data display unit and user interface section, wherein:
The avi file audio, video data that data transmission unit is used for reading is transferred to data processing unit, and data transmission unit stops transfer of data after receiving the switching command of switch control unit, after receiving the retransmission instructions of this unit, put and restart the transmission tone video data by the heavily transposition of indication in the instruction.Wherein video data adopts coded formats such as Divx3.11, Divx4.x, Divx5.x, and voice data adopts coded formats such as MP1, MP2, MP3.
Data processing unit is used to receive the audio, video data of data transmission unit transmission, avi file header, Index information, audio frequency and video format information are resolved the back to be preserved, isolate the audio, video data that to play and be transferred to the data decode unit, the data of the new transmission of counterweight after receiving switching command, that road audio stream that need from audio, video data, isolate video flowing and should switch to;
Switch control unit further comprises:
The information extraction subelement is used for obtaining from data processing unit the avi file header and the Index information of preservation, uses for the operator unit;
The interface subelement is used to finish communicating by letter between switch control unit and other unit;
Control sub unit, be used for after receiving the handoff request of user interface section, send switching command to data transmission unit, data processing unit and data decode unit, what send to that the instruction indication of data processing unit and decoding unit will switch to is that (indication can identify and realizes which audio stream by carrying the audio stream that will switch to, if certainly having only two-way audio stream can not indicate yet), and obtain current decoded video frames number from the data decode unit and be sent to the operator unit; And after re-transmission position, the audio frequency of receiving the operator unit are skipped number and video and are skipped number of parameters, send retransmission instructions to data transmission unit, comprise the re-transmission location parameter, and audio frequency is skipped number and video skip number of parameters and be transferred to the data decode unit;
The operator unit is used for according to current decoded video frames number and avi file header, Index information, calculates and retransmits the position, audio frequency skips number and video is skipped number, is sent to control sub unit.This subelement can be divided into synchronous points calculating and index information extracts part, I frame determining section, retransmits the position calculation part and skips the number calculating section, the introduction more hereinafter of the function of various piece.
The data decode unit comprises audio decoder subelement and video decode subelement, is respectively applied for the audio, video data of data processing unit transmission is decoded respectively, wherein:
The video decode subelement also is used for after receiving the switching command of switch control unit, and to its passback current video decoded video frames number, and it is data cached to continue decoding; After receiving that video is skipped number, empty buffer memory and reinitialize, the video flowing of the new transmission of counterweight begins the frame of video of this number of part only decodes, and is not transferred to data display unit, outputs to data display unit after the frame of video decoding afterwards;
The audio decoder subelement stops the decoding to voice data after receiving the switching command of switch control unit, reinitialize setting by the audio stream parameter that will switch to; After receiving that audio frequency is skipped number, the audio stream of the new transmission of counterweight begin this number of part audio frame/byte number ("/" expression " or " relation, voice data to CBR is a byte number, voice data to VBR is an audio frame) only decoding, be not transferred to data display unit, output to data display unit after the voice data decoding afterwards.
Data display unit is used for decoded audio, video data is shown, plays, and video can be in the output of TV/VGA equipment, and audio frequency is exported at TV equipment.
User interface section is user's interface operable, and the user can send handoff request by this interface, and wherein this handoff request will comprise the audio stream sign that will switch to when audio stream is arranged more than three the tunnel, and which audio stream what promptly indication will switch to is.
The flow process of carrying out when not switching former the vocal accompaniment may further comprise the steps:
Step 1, STB powers up, and moves the Karaoke Play System automatically, and the user controls this system by broadcast interface;
Step 2, when the user selected to play the song of audio video interleaved, the file header from avi file began to transmit data, as the transmission data that receive are the avi file head, execution in step three, if audio, video data, execution in step five;
Step 3 is resolved the avi file head, preserves the information that wherein comprises;
The avi file header of wherein resolving and preserving comprises: the audio frequency and video fluxion, in order to confirm that several roads audio/video flow is arranged in this file; Video flowing time scale and video flowing speed index; VBR alignment of data value (fixed value is 1152), the sample rate of VBR audio stream or the byte rate of CBR audio stream; The whole big or small Movie_Size of audio, video data starting position Movie_Offset and audio, video data; Video image wide, high, i.e. picture resolution; The Video Decoder type; The Video Decoder palette; Video decoder decodes speed (by video flowing time scale and these two parametric representations of video flowing speed index); Every road audio stream audio decoder type; Every road audio stream audio decoder code stream type (cbr (constant bit rate) or on-fixed code check); Every road audio stream audio decoder decode speed; Sound, Video Decoder begin decode time.
Step 4, the initialization setting operation of decoding according to the information of obtaining from the avi file head;
Mainly comprise following operation: the wide, high of video image is set, i.e. picture resolution; The Video Decoder type is set; The Video Decoder palette is set; Video decoder decodes speed is set; The audio decoder type is set; Audio decoder code stream type (cbr (constant bit rate) or on-fixed code check) is set; Audio decoder decode speed is set; Sound is set, Video Decoder begins decode time.
Wherein avi file only contains one road video data, but can contain multi-path audio-frequency data, so the audio stream parameter of using acquiescence to play during the initialization audio decoder is carried out initialization operation.
Step 5 is separated the audio and video data streams that needs broadcast;
Step 6 is decoded to the audio, video data that separates, and decoded data is outputed to display device and loud speaker is play by the AV interface.
The handling process that this flow process is just conventional.
As shown in Figure 4, contain in the process of avi file of multichannel voice frequency stream in broadcast, the handling process that the user is switched former vocal accompaniment may further comprise the steps:
Audio decoder initialization setting in this step comprises: the audio decoder type is set; Audio decoder code stream type (fixedly frame per second or on-fixed frame per second) is set; Audio decoder decode speed is set; Sound is set, Video Decoder begins decode time.Carrying out these, required parameter is set is originally to obtain from the avi file head.
So far the A/V synchronizing process is finished.Be noted that data decode unit (in audio decoder subelement) also can continue decoding audio data after receiving switching command, receiving the initialization that stops to decode, finishing the audio stream that will switch to after audio frequency is skipped number again.
To explain in step 140 below and how to calculate, ask the schematic diagram of while with reference to the calculation of parameter of (fixedly code stream) audio stream of the CBR shown in Fig. 5 and Fig. 6 and VBR (variable code stream) audio stream." a " or " v " corresponding data block among the figure.This step is further divided into following steps:
Step 1401 is just counted the frame number B that A adds setting in decoded video frames during according to the switch audio data flow, obtains the audio video synchronization point V of this switching
CFrame number, simultaneously determine INDEX information original position according to the audio, video data starting position of resolving and its total size, read INDEX information and preserve;
V
CFrame number be from the audio, video data starting position number of video frames that this frame accumulative total obtains.Frame number B is empirical value or calculates, if B is big, plays and has chattering, if little, just has polyisomenism, and preferable span is: 6～12 frames.This computing is calculated by the synchronous points of operator unit and index information extraction part is finished.
Step 1402 is determined V earlier
CNearest before I frame V
IFrame number;
Present embodiment is from audio, video data index starting position, add up the number of frame of video, preceding all these road audio data blocks of each audio data block, the byte number/frame number of the audio data stream that will switch to respectively, and the deviation post of each audio data block in avi file of each road audio stream; Can once obtain the frame number (promptly being accumulated to the video frame number of this I frame) of each I frame like this, each road audio stream data piece number, byte number/frame number before this I frame, the information of the deviation post of this I frame corresponding audio data block in avi file etc.Frame number is the most approaching and less than V so
CThe I frame of frame number is V
I
Owing to do not have the sequence number of audio data block or block of video data in INDEX information, the number of the accumulative total frame of video that therefore can start anew equals V
CFrame number the time found V
C, and then can determine its nearest I frame before.Perhaps, also can in the accumulative total process, set up the set of each I frame frame number, direct and V
CFrame number relatively, wherein frame number is the most approaching and smaller or equal to V
CThe I frame of frame number be the I frame that will look for.In addition,,, just can directly calculate the frame number of each I frame, by with the comparable V that obtains of quadrat method as long as know the sequence number (as read INDEX information or be fixed value) of first I frame because at interval frame number is fixed between each I frame
IFrame number.
This computing is finished by the I frame determining section of operator unit.
Step 1403 finds V from INDEX information
ICorresponding audio data block A
I, read A
IDeviation post in avi file is put Goto_offset with this position open position of attaching most importance to;
In INDEX, V
ICorresponding audio data block A
ISearch in the following manner:
1) calculates broadcast V
IAudio frequency byte number or frame number Audio_TotalN_I that road audio stream that before will switch to should be broadcast, to the calculating of CBR (fixedly code stream) audio stream is the bytes of audio data number, what calculate for VBR (variable code stream) audio stream is the voice data frame number, and formula is:
To CBR:Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * CBR audio frequency byte rate
To VBR:Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * VBR audio sample rate/1152
Therefore answer the audio data block and the block of video data of synchronous playing always not adjacent, calculate according to the relevant parameter in the avi file head above.
2) from the original position of audio, video data, byte number/the frame number of that road audio stream data piece that will switch to one by one adds up, accumulated value is compared with the Audio_TotalN_I that obtains above, if less than Audio_TotalN_I, add up again next time, as in more than or equal to Audio_TotalN_I, carry out next step;
3) equal Audio_TotalN_I as accumulated value, then next audio data block of this road audio stream is V
ICorresponding audio data block A
I, greater than Audio_TotalN_I, then this audio data block is V as accumulated value
ICorresponding audio data block A
I, write down A simultaneously
IAudio frequency byte number/frame number the Audio_TotalN_A that adds up before and obtain
I
4) from audio data block A
IFind deviation post dwOffset in the corresponding index information piece, this position is and retransmits position Goto_offset.
In another embodiment, also can be after reading in INDEX information, travel through INDEX information earlier, the byte number of each audio data block correspondence of that road audio stream that record will switch to or frame number and deviation post thereof, after calculating Audio_TotalN_I, just can directly relatively obtain retransmitting position Goto_offset like this.
This computing is partly finished by the re-transmission position calculation of operator unit.
Step 1404 calculates V
CWith V
IBetween the frame number difference be that video is skipped number Skip_Frame, and play V
CThe time that road audio stream that will the switch to audio frequency byte number/frame number Audio_accum_size and A that should broadcast
IAudio frequency byte number/frame number the Audio_TotalN_A that adds up before and obtain
IDifference, promptly audio frequency is skipped number Skip_audio;
Skip_Frame just equals V
CWith V
IThe difference of frame number.And the computational methods of Audio_accum_size are identical with Audio_TotalN_I, and formula is as follows:
To CBR:Audio_accum_size=V
CFrame number * (video flowing time scale/video flowing speed index) * CBR audio frequency byte rate
To VBR:Audio_accum_size=V
CFrame number * (video flowing time scale/video flowing speed index) * VBR audio sample rate/1152
Because what the CBR audio stream calculated is the bytes of audio data number, what the VBR audio stream calculated is the voice data frame number, so Audio_accum_size can be by in the middle of a frame in the combined data block of CBR or data block (audio data block of CBR and audio frame are not corresponding relation), is by between the frame frame in the combined data block of VBR.
This computing is finished by the number calculating section of skipping of operator unit.
In sum, mode soft, combination of hardware that the present invention can adopt is play the avi file that contains one road Divx video data encoder, multi-path audio-frequency data, keeps AV synchronous when switching different audio stream data.Guaranteed to hang down under the configuration condition in embedded system, AV was synchronous when the avi file voice data was switched, thereby had reduced the requirement to system's dominant frequency.
In the another embodiment of the present invention,, do not consider the continuity problem play if only need realize synchronously, also can omit with audio frequency skip number and video skip the operation in the relevant flow process of number and install in the unit.
Claims (14)
1. switch the AV method for synchronous of multichannel voice frequency stream when an avi file is play, be applied to comprise the system of switch control unit, data transmission unit, data processing unit and data decode unit, when broadcast contains the avi file of multichannel voice frequency stream, preserve the avi file header earlier, behind user's switch audio stream, it is characterized in that, carry out following steps:
(a) after switch control unit is received handoff request, send switching command to data processing unit and data decode unit;
(b) the data decode unit returns current decoded video frames number to switch control unit;
(c) switch control unit obtains retransmitting position and synchronous playing parameter according to index information, the file header information calculations of current decoded video frames number and preservation, to retransmit the position and send to data transmission unit, the synchronous playing parameter is sent to the data decode unit, and described synchronous playing parameter is that the audio frequency that calculates is skipped number Skip_audio and video is skipped number Skip_Frame;
(d) data transmission unit retransmits the audio, video data that the position is transmitted avi file again by specifying, and data processing unit receives that road audio stream of isolating video flowing and should switch to behind the audio, video data; Reinitializing of video decode finished in the data decode unit, and carries out the initialization of audio decoder again by the parameter of that road audio stream that will switch to;
(e) after the data decode unit receives the audio, video data of transmission again, the audio/video flow of the new transmission of described data decode unit counterweight begins Skip_audio byte number or frame number and Skip_Frame frame of video of part, be not transferred to data display unit after the decoding, Xie Ma audio, video data is transferred to data display unit again and carries out synchronous playing afterwards, wherein to fixing code streams by using byte number, to variable code streams by using frame number.
2. AV method for synchronous as claimed in claim 1 is characterized in that:
Described data decode unit continues the video data of buffer memory is decoded after receiving switching command, after receiving parameter S kip_audio and Skip_Frame, empties the video data of buffer memory again, carries out reinitializing of video decode.
3. AV method for synchronous as claimed in claim 1 is characterized in that:
Described system is an embedded system, and described data decode unit realizes that with hardware switch control unit, data transmission unit and data processing unit are then carried out corresponding software by CPU and realized.
4. AV method for synchronous as claimed in claim 1 is characterized in that: described step (c) switch control unit is further finished calculating by following steps:
The frame number A that is decoding during (c1) according to the switch audio data flow adds the frame number B of setting, obtains the audio video synchronization point V of this switching
CFrame number, and, determine the index information original position according to the total size of the audio, video data starting position of resolving and its, read index information and preserve;
(c2), the most approaching and less than V with the video frame number of accumulative total from the number of audio, video data index starting position accumulative total frame of video
CThe I frame of frame number is V
CNearest before I frame V
I
(c3) from index information, find V
ICorresponding audio data block A
I, read A
IDeviation post in avi file is put with this position open position of attaching most importance to;
(c4) calculate V
CWith V
IBetween the frame number difference be that video is skipped number Skip_Frame, and play V
CThe time that road audio stream that will switch to audio frequency byte number or the frame number Audio_accum_size and A that should broadcast
IThe audio frequency byte number or the frame number Audio_TotalN_A that add up before and obtain
IDifference, promptly audio frequency is skipped number Skip_audio.
5. AV method for synchronous as claimed in claim 4 is characterized in that: V in the described step (c3)
ICorresponding audio data block A
ISearch in the following manner:
(c31) calculate broadcast V
IAudio frequency byte number or frame number Audio_TotalN_I that road audio stream that before will switch to should be broadcast, formula is:
To fixing code stream: Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * CBR audio frequency byte rate
To variable code stream: Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * VBR audio sample rate/1152
(c32) from the original position of audio, video data, the byte number or the frame number of that road audio stream data piece that will switch to one by one add up, accumulated value is compared with the Audio_TotalN_I that obtains above, if less than Audio_TotalN_I, add up again next time, if more than or equal to Audio_TotalN_I, carry out next step;
(c33) equal Audio_TotalN_I as accumulated value, then next audio data block of this road audio stream is V
ICorresponding audio data block A
I, greater than Audio_TotalN_I, then this audio data block is V as accumulated value
ICorresponding audio data block A
I, write down A simultaneously
IThe audio frequency byte number or the frame number Audio_TotalN_A that add up before and obtain
I
(c34) from audio data block A
IFind deviation post dwOffset in the corresponding index information piece, this position is and retransmits the position,
Calculate in the described step (c4) and play V
CThe time when audio frequency byte number that should broadcast of that road audio stream that will switch to or frame number Audio_accum_size, only need the V in step (c31) formula
IFrame number make V into
CFrame number get final product.
6. AV method for synchronous as claimed in claim 3 is characterized in that:
Described frame number B span is: 6～12 frames.
7. AV method for synchronous as claimed in claim 1 is characterized in that:
In the step (a), after switch control unit is received handoff request, also send switching command, after described data transmission unit is received switching command, stop transmission immediately former audio/video flow to described data transmission unit; Simultaneously, as comprising three the tunnel and above audio stream data in the avi file, then switch control unit sends in the instruction of data processing unit and data decode unit, and which audio stream what need that indication will switch to is.
8. switch the AV synchronizer of multichannel voice frequency stream when an avi file is play, it is characterized in that, comprising:
Data transmission unit, the audio, video data that is used to read avi file is transferred to data processing unit, and retransmits the audio, video data that the position is transmitted avi file again by specifying after receiving retransmission instructions;
Data processing unit, be used for avi file header and index information are resolved the back preservation, finish separating and being transferred to the data decode unit of audio stream and video flowing, after receiving switching command, that road audio stream of from audio, video data, isolating video flowing and should switching to;
Switch control unit, be used for after receiving handoff request, send switching command to data processing unit and data decode unit, index information, file header information calculations according to current decoded video frames number and preservation obtains retransmitting position and synchronous playing parameter then, to retransmit the position and send to data transmission unit, the synchronous playing parameter is sent to the data decode unit, and described synchronous playing parameter is that the audio frequency that calculates is skipped number Skip_audio and video is skipped number Skip_Frame;
The data decode unit comprises audio decoder subelement and video decode subelement; After receiving switching command, return current decoded video frames number to switch control unit, and finish the initialization that reinitializes and carry out audio decoder by the parameter of that road audio stream that will switch to again of video decode; According to the synchronous playing parameter that receives, the audio/video flow of the new transmission of described data decode unit counterweight begins Skip_audio byte number or frame number and Skip_Frame frame of video of part, be not transferred to data display unit after the decoding, Xie Ma audio, video data is transferred to data display unit again and carries out synchronous playing afterwards, wherein to fixing code streams by using byte number, to variable code streams by using frame number.
9. AV synchronizer as claimed in claim 8 is characterized in that: described switch control unit further comprises:
The interface subelement is used to finish communicating by letter between switch control unit and other unit;
The information extraction subelement is used for obtaining avi file header and index information from data processing unit, uses for the operator unit
Control sub unit, be used for after receiving the handoff request of user interface section, send switching command to data decode unit, data processing unit and data decode unit, the instruction that sends to data processing unit and decoding unit should be carried the audio stream sign that will switch to, and obtains current decoded video frames sequence number from the data decode unit and is sent to the operator unit; And after re-transmission position, the audio frequency of receiving the operator unit are skipped number and video and are skipped number of parameters, send retransmission instructions to data transmission unit, carry the re-transmission location parameter, and audio frequency is skipped number and video skip number of parameters and be transferred to the data decode unit;
The operator unit is used for according to current decoded video frames sequence number and avi file header and index information, calculates and retransmits the position, audio frequency skips number Skip_audio and video is skipped number Skip_Frame, is sent to control sub unit.
10. AV synchronizer as claimed in claim 8 is characterized in that:
Described data decode unit continues the video data of buffer memory is decoded after receiving switching command, after receiving parameter S kip_audio and Skip_Frame, empties the video data of buffer memory again, carries out reinitializing of video decode.
11. AV synchronizer as claimed in claim 9 is characterized in that: described operator unit further comprises with the lower part:
Synchronous points is calculated and index information extracts part, and the frame number A that is decoding during according to the switch audio data flow adds the frame number B of setting, obtains the audio video synchronization point V of this switching
CFrame number, and, determine the index information original position according to the total size of the audio, video data starting position of resolving and its, read index information and preserve;
I frame determining section, from the number of audio, video data index starting position accumulative total frame of video, the most approaching and less than V with the video frame number of accumulative total
CThe I frame of frame number is V
CNearest before I frame V
I
Retransmit the position calculation part, from index information, find V
ICorresponding audio data block A
I, read A
IDeviation post in avi file is put with this position open position of attaching most importance to;
Skip the number calculating section, calculate V
CWith V
IBetween the frame number difference be that video is skipped number Skip_Frame, and play V
CThe time that road audio stream that will switch to audio frequency byte number or the frame number Audio_accum_size and A that should broadcast
IThe audio frequency byte number or the frame number Audio_TotalN_A that add up before and obtain
IDifference, promptly audio frequency is skipped number Skip_audio.
12. AV synchronizer as claimed in claim 11 is characterized in that:
Described re-transmission position calculation is partly searched V
ICorresponding audio data block A
IThe time, calculate earlier and play V
IAudio frequency byte number or frame number Audio_TotalN_I that road audio stream that before will switch to should be broadcast, formula is:
To fixing code stream: Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * CBR audio frequency byte rate
To variable code stream: Audio_TotalN_I=V
IFrame number * (video flowing time scale/video flowing speed index) * VBR audio sample rate/1152
Again from the original position of audio, video data, the byte number or the frame number of that road audio stream data piece that will switch to one by one add up, accumulated value is compared with the Audio_TotalN_I that obtains above, if less than Audio_TotalN_I, add up again next time, equal Audio_TotalN_I as accumulated value, then next audio data block of this road audio stream is V
ICorresponding audio data block A
I, greater than Audio_TotalN_I, then this audio data block is V as accumulated value
ICorresponding audio data block A
I, write down A simultaneously
IThe audio frequency byte number or the frame number Audio_TotalN_A that add up before and obtain
I
At last, from audio data block A
IFind deviation post dwOffset in the corresponding index information piece, this position is and retransmits the position,
The described number calculating section of skipping calculates broadcast V
CThe time when audio frequency byte number that should broadcast of that road audio stream that will switch to or frame number Audio_accum_size, only need the V in the aforementioned calculation Audio_TotalN_I formula
IFrame number make V into
CFrame number get final product.
13. AV synchronizer as claimed in claim 8 is characterized in that:
Described switch control unit also is used for after receiving handoff request, sends switching command to described data transmission unit; After described data transmission unit is received switching command, stop transmission immediately to former audio/video flow; Described switch control unit comprises three the tunnel and during above audio stream data in the avi file of playing, send in the instruction of data processing unit and data decode unit, and which audio stream what need that indication will switch to is.
14. AV synchronizer as claimed in claim 8 is characterized in that:
Described device is applied to embedded system, and described data decode unit realizes that with hardware switch control unit, data transmission unit and data processing unit are carried out corresponding software by CPU and realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200610172268 CN101212679B (en) | 2006-12-30 | 2006-12-30 | AV synchronization method and device for switching multi-channel audio streams during playing of AVI files |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200610172268 CN101212679B (en) | 2006-12-30 | 2006-12-30 | AV synchronization method and device for switching multi-channel audio streams during playing of AVI files |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101212679A CN101212679A (en) | 2008-07-02 |
| CN101212679B true CN101212679B (en) | 2010-04-14 |
Family
ID=39612282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200610172268 Expired - Fee Related CN101212679B (en) | 2006-12-30 | 2006-12-30 | AV synchronization method and device for switching multi-channel audio streams during playing of AVI files |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101212679B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102098423B (en) * | 2009-12-10 | 2013-03-27 | 北大方正集团有限公司 | Seamless file collection method and system |
| CN102110459B (en) * | 2009-12-24 | 2013-01-16 | Tcl集团股份有限公司 | Playing terminal and multimedia file playing method and device thereof |
| US8681786B2 (en) * | 2010-02-08 | 2014-03-25 | Via Telecom Co., Ltd. | Centralized recording and processing of received packet parameters |
| CN102324097B (en) * | 2011-08-17 | 2013-12-11 | 深圳锐取信息技术股份有限公司 | Method for detecting image change |
| CN102547435B (en) * | 2011-12-16 | 2014-06-25 | Tcl集团股份有限公司 | System and method for playing and processing multimedia file |
| CN102510541B (en) * | 2011-12-30 | 2014-12-17 | Tcl数码科技（深圳）有限责任公司 | Multi-screen interaction video and audio content switching method and media player |
| CN102867525B (en) * | 2012-09-07 | 2016-01-13 | Tcl集团股份有限公司 | A kind of multichannel voice frequency disposal route, audio-frequency playing terminal and apparatus for receiving audio |
| US11176111B2 (en) * | 2013-03-15 | 2021-11-16 | Nuodb, Inc. | Distributed database management system with dynamically split B-tree indexes |
| KR101861941B1 (en) * | 2014-02-10 | 2018-07-02 | 돌비 인터네셔널 에이비 | Embedding encoded audio into transport stream for perfect splicing |
| CN104602115A (en) * | 2015-01-13 | 2015-05-06 | 青岛海信宽带多媒体技术有限公司 | Method and system for switching of network television channels |
| CN106411903B (en) * | 2016-10-08 | 2019-08-23 | Oppo广东移动通信有限公司 | A kind of control method for playing back, device and terminal |
| US10873775B2 (en) * | 2017-06-12 | 2020-12-22 | Netflix, Inc. | Staggered key frame video encoding |
| CN111063376B (en) * | 2019-12-13 | 2021-05-14 | 厦门市美亚柏科信息股份有限公司 | Method, terminal equipment and storage medium for audio and video synchronization in MP4 repairing |
| CN112532991B (en) * | 2020-10-28 | 2023-04-14 | 惠州华阳通用电子有限公司 | Hardware decoding method for RMFF format file |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6512884B1 (en) * | 1998-10-15 | 2003-01-28 | Nec Corporation | Method and apparatus for synchronized play back of audio-video signals |
| FR2866729A1 (en) * | 2004-02-25 | 2005-08-26 | Opencube | Multi-track audiovisual flow processing device, has administration module that synchronizes successive use of synchronized buffers by different active processes according to usage sequence determined by switching process |
| CN1728770A (en) * | 2004-07-28 | 2006-02-01 | 乐金电子（惠州）有限公司 | Device for synchronizing video and audio of Kara-OK system |
| CN1767601A (en) * | 2005-10-21 | 2006-05-03 | 西安交通大学 | Synchronous broadcast controlling method capable of supporting multi-source stream media |
-
2006
- 2006-12-30 CN CN 200610172268 patent/CN101212679B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6512884B1 (en) * | 1998-10-15 | 2003-01-28 | Nec Corporation | Method and apparatus for synchronized play back of audio-video signals |
| FR2866729A1 (en) * | 2004-02-25 | 2005-08-26 | Opencube | Multi-track audiovisual flow processing device, has administration module that synchronizes successive use of synchronized buffers by different active processes according to usage sequence determined by switching process |
| CN1728770A (en) * | 2004-07-28 | 2006-02-01 | 乐金电子（惠州）有限公司 | Device for synchronizing video and audio of Kara-OK system |
| CN1767601A (en) * | 2005-10-21 | 2006-05-03 | 西安交通大学 | Synchronous broadcast controlling method capable of supporting multi-source stream media |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101212679A (en) | 2008-07-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101212679B (en) | AV synchronization method and device for switching multi-channel audio streams during playing of AVI files | |
| JP4481444B2 (en) | Image encoding device | |
| JP3894572B2 (en) | Method for transmitting and receiving compressed television signals | |
| US7610605B2 (en) | Method and apparatus for conversion and distribution of data utilizing trick-play requests and meta-data information | |
| US7274862B2 (en) | Information processing apparatus | |
| US8966103B2 (en) | Methods and system for processing time-based content | |
| CN104618786A (en) | Audio/video synchronization method and device | |
| CN1981492A (en) | Buffer level signaling for rate adaptation in multimedia streaming | |
| CN106792154B (en) | Frame skipping synchronization system of video player and control method thereof | |
| CN102640511A (en) | Method and system for playing video information, and video information content | |
| CN103873888A (en) | Live broadcast method of media files and live broadcast source server | |
| CN101393760B (en) | After-recording apparatus and reproduction apparatus and method | |
| US20060146850A1 (en) | Method and apparatus for synchronizing clocks | |
| CN100536574C (en) | A system and method for quickly playing multimedia information | |
| CN114363648A (en) | Method, equipment and storage medium for audio and video alignment in mixed flow process of live broadcast system | |
| WO2012089670A1 (en) | Method and apparatus for processing a video signal | |
| CN1682206A (en) | Dynamic quality adjustment based on changing streaming constraints | |
| JPH10276407A (en) | Video information providing management method and system therefor | |
| KR20090030779A (en) | A low delay intra-frame coding method using data partition | |
| Westerink et al. | A live intranet distance learning system using MPEG-4 over RTP/RTSP | |
| JPH10271482A (en) | Synchronous reproduction control method and system for coded video | |
| JPH10164556A (en) | Decoder, encoder and video-on-demand system | |
| JP4114868B2 (en) | Multiplexer and multiplexing method | |
| JP3543716B2 (en) | Stream generation method and apparatus | |
| KR0129804B1 (en) | Coding and decoding system capable of the fast search for intra frame |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: BEIJING SUNPLUS EHUE TECHNOLOGY CO., LTD. Free format text: FORMER OWNER: BEIJING BEIYANG ELECTRONIC TECHNOLOGY CO., LTD. Effective date: 20140314 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20140314 Address after: Science and Industry Park, Taiwan, Hsinchu, China, No. 19 Patentee after: Lingyang Science and Technology Co., Ltd. Patentee after: BEIJING SUNPLUS-EHUE TECHNOLOGY CO., LTD. Address before: Science and Industry Park, Taiwan, Hsinchu, China, No. 19 Patentee before: Lingyang Science and Technology Co., Ltd. Patentee before: Beijing Beiyang Electronic Technology Co., Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20100414 Termination date: 20171230 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |