Background technology
from ITU-T (ITU-T for ITU Telecommunication Standardization Sector, International Telecommunications Union's telecommunication standards group) since issue H261 video encoding standard, any compatible H261/263/264, MPEG-1/2/4 and JPEG (Joint Photographic Experts Group, associating picture experts group standard) video CODEC (coder-decoder, coder) all must be achieved as follows about function: estimation, motion compensation, transformation of coefficient, quantize and the entropy coding, these video standards are all in based on same video CODEC abstract model: DPCM/DCT (differential coding/discrete cosine transform) mixed C ODEC model.
Be that prior art is looked the audio frequency broadcast system basic block diagram as shown in Figure 1, the described audio frequency broadcast system of looking comprises: transmit control module 101, input interface 102, audio decoder module 103, video decode module 104, echo interface 105, the first buffering area 106, the second buffering area 107.Described input interface 102, audio decoder module 103, video decode module 104, echo interface 105 are connected respectively at transmit control module 101, described the first buffering area 106 is connected with described input interface 102, described the second buffering area 107 is connected with described video decode module 104, echo interface 107 and the audio decoder module 103 of being connected respectively, and described video decode module 104 also is connected with echo interface 105.Wherein, in prior art, the audio frequency broadcast system of looking of not establishing the second buffering area is arranged also, the echo interface is directly according to the decoded data address of audio decoder and Video Decoder transmission, carries out the presenting of effect of audio frequency and video.
Be the video decode modular structure figure that looks in Fig. 1 in audio frequency broadcast system as described in Figure 2, described video decode module comprises:
Entropy decoder module 1041 is used for the encoding stream of input is carried out the entropy decoding, extracts conversion coefficient, motion vector and the figure information of each macro block;
Firing floor decoding and the module 1042 that reorders are used for the firing floor decoding and reorder, obtain quantizing with conversion after sub-macro block;
Inverse quantization module 1043 is used for described sub-macro block is carried out inverse quantization;
Inverse discrete cosine transformation module 1044 is used for the sub-macro block after described inverse quantization is carried out inverse discrete cosine transformation, generates residual error;
Motion compensating module 1045 is used for motion vector and reference frame image are carried out motion compensated prediction, the generation forecast macro block, and wherein reference frame is the former frame of present frame;
Macro block reconstructed module 1046 is used for described predicted macroblock is added to the macro block that generates reconstruct on residual error, and after complete for whole macro block reconstruct of current frame image, directly generates the present frame decoded picture;
Detection module 1047, complete for detection of whole macro block reconstruct of current frame image after, indicate that the present frame decoded picture generates complete, can be used for output or decoding prediction next time.
In prior art, the video decode module is decoded at every turn and is exported a frame, decoded frame of video raw data (as the YUV view data) are all the frame output buffers (not shown in Fig. 2) that leaves the video decode module in, this is a zone of fully being used and being managed by the video decode module, it only can deposit two field picture raw data of current decoding output, when the video decode module begins new once decoding, can first empty described frame output buffer.That is to say, no matter look the second buffering area 107 or echo interface 105 in audio frequency broadcast system (as shown in Figure 1), when accessing video decoder module 104 and audio decoder module 105, must all copy the data in described frame output buffer to the second buffering area 107 or whole echos complete (for the echo interface).This copy meeting causes all played data all can be copied once more, has wasted greatly nervous cpu resource; And there is no the second buffering area 107 look the sound Play System when the I frame decoding, if the computational resource that CPU distributes is not loose, the situations such as full frame stretching, picture and sound pause, wait usually can appear during displaying video.
Summary of the invention
The purpose of the embodiment of the present invention is to propose a kind of video decoder, usually can occur the problems such as full frame stretching, picture and sound pause, wait when being intended to solve in prior art displaying video.
Embodiments of the invention are achieved in that a kind of video decoder, comprise the entropy decoder module, are used for the encoding stream of input is carried out the entropy decoding, extract conversion coefficient, motion vector and the header of each macro block; Firing floor decoding and the module that reorders are used for the firing floor decoding and reorder, obtain quantizing with conversion after sub-macro block; Inverse quantization module is used for described sub-macro block is carried out inverse quantization; The inverse discrete cosine transformation module is used for the sub-macro block after described inverse quantization is carried out inverse discrete cosine transformation, generates residual error; Motion compensating module is used for motion vector and reference frame image being carried out motion compensated prediction, the generation forecast macro block; The macro block reconstructed module is used for described predicted macroblock is added to the macro block that generates reconstruct on described residual error, and after complete for whole macro block reconstruct of current frame image, directly generates the present frame decoded picture; Detection module, complete for detection of whole macro block reconstruct of current frame image after, indicate that the present frame decoded picture generates complete, can be used for output or decoding prediction next time, wherein, described device also comprises:
The first scheduler module, after being used for realizing the entropy decoding, scheduling CPU enters sleep state;
The second scheduler module, after being used for realizing motion compensated prediction, scheduling CPU enters sleep state.
The purpose of the embodiment of the present invention is to propose a kind of video encoding/decoding method, said method comprising the steps of:
Encoding stream to input carries out the entropy decoding, extracts conversion coefficient, motion vector and the header of each macro block;
The judgement present frame is the I frame;
Scheduling CPU enters sleep state;
Firing floor is decoded and reordered, thereby obtain the sub-macro block after quantification and conversion;
Described sub-macro block is carried out inverse quantization;
Sub-macro block after described inverse quantization is carried out inverse discrete cosine transformation, generate residual error;
Motion vector and reference frame image are carried out motion compensated prediction, the generation forecast macro block;
The judgement present frame is the I frame;
Scheduling CPU enters sleep state;
Described predicted macroblock is added to the macro block that generates reconstruct on described residual error, and after complete for whole macro block reconstruct of current frame image, directly generates the present frame decoded picture;
After whole macro block reconstruct of judgement current frame image are complete, generate present frame frame decoding image.
The purpose of the embodiment of the present invention is to propose a kind of audio frequency broadcast system of looking that comprises described video decoder, and described system comprises transmit control module, input interface, audio decoder module, video decoder, echo interface, the first buffering area; Described input interface, audio decoder module, video decoder, echo interface are connected with transmit control module respectively, and the first buffering area is connected with input interface; Transmit control module is used for giving an order, and chooses file or the chained address, media data position that will play, by the operating system file input/output interface, media data is read in the first buffering area, wherein,
described video decoder also comprises the 3rd output queue buffering area, described video decoder is used for obtaining by input interface the file that will play or the media data of the first buffering area, and carry out video decode, deposit the 3rd output queue buffering area in, and the mutual exclusion lock that is used for the establishment overall situation guarantees the exclusive reference to the output queue of described the 3rd output queue buffering area, described the 3rd output queue buffering area is used for storing the decoded data of described video decode module, and form output queue so that described echo interface and described video decoder can be worked asynchronously,
described audio decoder module also comprises the second output queue buffering area, described audio decoder module is used for obtaining by input interface the file that will play or the media data of the first buffering area, and carry out audio decoder, deposit the second output queue buffering area in, and the mutual exclusion lock that is used for the establishment overall situation guarantees the exclusive reference to the output queue of described the second output queue buffering area, described the second output queue buffering area is used for storing the decoded data of described audio decoder module, and form output queue so that described echo interface and described audio decoder module can be worked asynchronously,
The echo interface is used for by certain sequential, according to the data address after audio decoder of looking of storing in described the second output queue buffering area, the 3rd output queue buffering area, presents depending on the effect of audio frequency.
The purpose of the embodiment of the present invention is to propose a kind of audio frequency broadcast system of looking that comprises described video decoder, and described system comprises transmit control module, input interface, audio decoder module, video decoder, echo interface, the first buffering area; Described input interface, audio decoder module, video decoder, echo interface are connected with transmit control module respectively, and the first buffering area is connected with input interface; Transmit control module is used for giving an order, and chooses file or the chained address, media data position that will play, by the operating system file input/output interface, media data is read in the first buffering area, wherein,
The described audio frequency broadcast system of looking also comprises the 4th buffering area, described the 4th buffering area is connected with described echo interface with described audio decoder module, be used for storing described audio decoder module the output of frame output buffer decoded frame data and form output queue, and be used for providing reference address to described echo interface accessing;
described video decoder also comprises the 3rd output queue buffering area, described video decoder is used for obtaining by input interface the file that will play or the media data of the first buffering area, and carry out video decode, deposit the 3rd output queue buffering area in, and the mutual exclusion lock that is used for the establishment overall situation guarantees the output queue exclusive reference to described the 3rd output queue buffering area, described the 3rd output queue buffering area is used for storing the decoded data of described video decode module, and form output queue so that described echo interface and described video decoder can be worked asynchronously,
Described audio decoder module also is connected with described the 4th buffering area, be used for obtaining by described input interface the file that will play or the media data of the first buffering area, and carry out audio decoder, each decoding output one frame is with the frame output buffer of decoded frame of video raw deposit data in described audio decoder module;
The echo interface is used for by certain sequential, and the effect of looking audio frequency according to the data address after the audio decoder of the data address after the video decode of storing in the 3rd output queue buffering area and the 4th buffer stores presents.
The embodiment of the present invention is by arranging the output frame queue buffer in video decode module, audio decoder module, trade space for time, make (accurate parallel) work asynchronously of echo module and decoder module, specifically, if look the audio frame number certificate in the frame queue buffering, need not wait for decoder output during echo, but directly according to broadcasting speed from the buffer queue echo of fetching data.Grow in the time of can leveling land due to the decoding of video I frame (usually much larger than 40ms), in order to guarantee that this moment, the echo module can not stop less than CPU because distributing, the present invention in video decoding process entropy decoding and motion compensation after by dispatching CPU, guarantee that the echo module can in time be dispatched to CPU, thereby guarantee that the smoothness of playing is level and smooth.Like this, in the time of just making the calculating decode resources that can provide when processor satisfy relatively reluctantly (the every frame of average 36ms when for example the individual event video decode is tested), can eliminate and look audio frequency and play phenomenon unbalanced or card, level and smooth, smooth result of broadcast and audio-visual synchronization more accurately are provided.
Embodiment
In order to make purpose of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, the present invention is further elaborated, for convenience of explanation, only show the part relevant to the embodiment of the present invention.Should be appreciated that the specific embodiment that this place is described, only be used for explaining the present invention, not in order to limit the present invention.
The embodiment of the present invention is by arranging the output frame queue buffer in video decode module, audio decoder module, trade space for time, make (accurate parallel) work asynchronously of echo interface module and decoder module, specifically, if look the audio frame number certificate in the frame queue buffering, need not wait for decoder output during echo, but directly according to broadcasting speed from the buffer queue echo of fetching data.Grow in the time of can leveling land due to the decoding of video I frame (usually much larger than 40ms), in order to guarantee that this moment, the echo module can not stop less than CPU because distributing, the present invention in video decoding process entropy decoding and motion compensation after by dispatching CPU, guarantee that the echo module can in time be dispatched to CPU, thereby guarantee that the smoothness of playing is level and smooth.Like this, in the time of just making the calculating decode resources that can provide when processor satisfy relatively reluctantly (the every frame of average 36ms when for example the individual event video decode is tested), can eliminate and look audio frequency and play phenomenon unbalanced or card, level and smooth, smooth result of broadcast and audio-visual synchronization more accurately are provided.
As shown in Figure 3, look the audio frequency broadcast system structure chart for the embodiment of the present invention.Described system comprises: transmit control module 101, input interface 102, audio decoder module 103, video decode module 104, echo interface 105, the first buffering area 106, wherein audio decoder module 103 comprises that the second output queue buffering area 1031, video decode module 104 comprise the 3rd output queue buffering area 1048.Described input interface 102, audio decoder module 103, video decode module 104, echo interface 105 are connected with transmit control module 101 respectively, and the first buffering area 106 is connected with input interface 102.
Transmit control module 101 is used for giving an order, and chooses file or the chained address, media data position that will play, by operating system file input/output interface (not shown in Fig. 3), media data is read in the first buffering area 106;
audio decoder module 103, be used for obtaining by input interface 102 file that will play or the media data of the first buffering area 106, and carry out audio decoder, deposit the second output queue buffering area 1031 in, and the mutual exclusion lock that is used for the establishment overall situation guarantees the output queue exclusive reference to described the second output queue buffering area 1031, described the second output queue buffering area 1031 is used for storing 103 decoded data of described audio decoder module, and form output queue so that echo interface 105 and audio decoder module 103 asynchronously (accurate parallel) work (specifically, if look the audio frame number certificate in the frame queue buffering, need not wait for 103 outputs of audio frequency decoder module during echo, but directly according to broadcasting speed from the output queue echo of fetching data).
video decode module 104, be used for obtaining by input interface 102 file that will play or the media data of the first buffering area 106, and carry out video decode, deposit the 3rd output queue buffering area 1048 in, and the mutual exclusion lock that is used for the establishment overall situation guarantees the output queue exclusive reference to described the 3rd output queue buffering area 1048, described the 3rd buffering area 1048 is used for storing 104 decoded data of described video decode module, and form output queue so that echo interface 105 and video decode module 1048 asynchronously (accurate parallel) work (specifically, if look the audio frame number certificate in the frame queue buffering, need not wait for 104 outputs of video decode module during echo, but directly according to broadcasting speed from the output queue echo of fetching data).
Echo interface 105 is used for by certain sequential, according to the data address after audio decoder of looking of storing in described the second output queue buffering area 1031 and the 3rd output queue buffering area 1048, presents depending on the effect of audio frequency.
Wherein said echo interface 105 moves at different threads from audio decoder module and video decode module.
Be illustrated in figure 4 as the video decode modular structure figure that looks in Fig. 3 in audio frequency broadcast system.Described video decode module comprises:
Entropy decoder module 1041 is used for the encoding stream of input is carried out the entropy decoding, extracts conversion coefficient, motion vector and the header of each macro block;
The first scheduler module 1049, after being used for realizing the entropy decoding, scheduling CPU enters sleep state can in time be dispatched to CPU to guarantee the echo module, and the wherein said dormant time is less than a cpu clock Tick.;
Firing floor decoding and the module 1042 that reorders are used for the firing floor decoding and reorder, obtain quantizing with conversion after sub-macro block;
Inverse quantization module 1043 is used for described sub-macro block is carried out inverse quantization;
Inverse discrete cosine transformation module 1044 is used for the sub-macro block after described inverse quantization is carried out inverse discrete cosine transformation, generates residual error;
Motion compensating module 1045 is used for motion vector and reference frame image are carried out motion compensated prediction, the generation forecast macro block, and wherein, described reference frame is the former frame of present frame;
The second scheduler module 1040 is used for realizing realizing that scheduling CPU enters sleep state and can in time be dispatched to CPU to guarantee the echo module after motion compensated prediction that the wherein said dormant time is less than a cpu clock Tick;
Macro block reconstructed module 1046 is used for described predicted macroblock is added to the macro block that generates reconstruct on described residual error, and after complete for whole macro block reconstruct of current frame image, directly generates the present frame decoded picture;
Detection module 1047, complete for detection of whole macro block reconstruct of current frame image after, indicate that the present frame decoded picture generates complete, can be used for output or decoding prediction next time.
The embodiment of the present invention is by arranging the output frame queue buffer in video decode module, audio decoder module, trade space for time, make (accurate parallel) work asynchronously of echo interface module and decoder module, specifically, if look the audio frame number certificate in the frame queue buffering, need not wait for decoder output during echo, but directly according to broadcasting speed from the buffer queue echo of fetching data.Grow in the time of can leveling land due to the decoding of video I frame (usually much larger than 40ms), in order to guarantee that this moment, the echo module can not stop less than CPU because distributing, the present invention in video decoding process entropy decoding and motion compensation after by dispatching CPU, guarantee that the echo module can in time be dispatched to CPU, thereby guarantee that the smoothness of playing is level and smooth.Like this, in the time of just making the calculating decode resources that can provide when processor satisfy relatively reluctantly (the every frame of average 36ms when for example the individual event video decode is tested), can eliminate and look audio frequency and play phenomenon unbalanced or card, level and smooth, smooth result of broadcast and audio-visual synchronization more accurately are provided.
Be illustrated in figure 5 as embodiment of the present invention video encoding/decoding method flow chart.Said method comprising the steps of:
S501 carries out the entropy decoding to the encoding stream of inputting, and extracts conversion coefficient, motion vector and the header of each macro block;
S513 judges whether present frame is the I frame, is to enter step S502, otherwise enters step S503;
S502, scheduling CPU enters sleep state (the wherein said dormant time is less than a cpu clock Tick);
S503 decodes and reorders firing floor, thereby obtains the sub-macro block after quantification and conversion;
S504 carries out inverse quantization to described sub-macro block;
S505 carries out inverse discrete cosine transformation to the sub-macro block after described inverse quantization, generates residual error;
S506 carries out motion compensated prediction to motion vector and reference frame image, the generation forecast macro block, and wherein, described reference frame is the former frame of present frame;
S514 judges whether present frame is the I frame, is to enter step S507, otherwise enters step S508;
S507, scheduling CPU advances sleep state (the wherein said dormant time is less than a cpu clock Tick);
S508 is added to described predicted macroblock the macro block that generates reconstruct on described residual error, and after complete for whole macro block reconstruct of current frame image, directly generates present frame frame decoding image;
S509, the whole macro blocks that judge current frame image whether reconstruct complete after, be to enter step S510, otherwise return to step S501;
S510 generates present frame frame decoding image.
grow in the time of can leveling land due to the decoding of video I frame (usually much larger than 40ms), in order to guarantee that this moment, the echo module can not stop less than CPU because distributing, the embodiment of the present invention in video decoding process entropy decoding and motion compensation after by dispatching CPU, guarantee that the echo module can in time be dispatched to CPU, thereby guarantee that the smoothness of playing is level and smooth, like this, in the time of just making the calculating decode resources that can provide when processor satisfy relatively reluctantly (the every frame of average 36ms when for example the individual event video decode is tested), can eliminate and look the phenomenon that the audio frequency broadcast is unbalanced or block, provide level and smooth, smooth result of broadcast and audio-visual synchronization more accurately.
Fig. 6 is that another embodiment of the present invention is looked the audio frequency broadcast system structure chart, the difference of itself and Fig. 3 is, described audio decoder module 103 does not comprise the output queue buffering area, just is provided with the 3rd output queue buffering area 1048 in the video decode module, shown in system comprise:
Transmit control module 101, input interface 102, audio decoder module 103, video decode module 104, echo interface 105, the first buffering area 106, wherein, video decode module 104 comprises the 3rd output queue buffering area 1048.Described input interface 102, audio decoder module 103, video decode module 104, echo interface 105 are connected with transmit control module 101 respectively, the first buffering area 106 is connected with input interface 102, the described audio frequency broadcast system of looking also comprises the 4th buffering area 108, and described the 4th buffering area 108 is connected with described echo interface with described audio decoder module.
Transmit control module 101 is used for giving an order, and chooses file or the chained address, media data position that will play, by operating system file input/output interface (not shown in Fig. 6), media data is read in the first buffering area 106;
Audio decoder module 103, be used for obtaining by input interface 102 file that will play or the media data of the first buffering area 106, and carry out audio decoder, each decoding output one frame is with the frame output buffer (in Fig. 6 not shown) of decoded frame of video raw deposit data in audio decoder module 103;
The 4th buffering area 108, be used for storing described audio decoder module 103 the output of frame output buffer decoded frame data and form output queue, and be used for providing reference address to described echo interface 105 access;
video decode module 104, be used for obtaining by input interface 102 file that will play or the media data of the first buffering area 106, and carry out video decode, deposit the 3rd output queue buffering area 1048 in, and the mutual exclusion lock that is used for the establishment overall situation guarantees the output queue exclusive reference to described the 3rd output queue buffering area 1048, described the 3rd queue buffer 1048 is used for storing the decoded data of described video decode module 104, and form output queue so that echo interface 105 and video decode module 104 asynchronously (accurate parallel) work (specifically, if look the audio frame number certificate in the frame queue buffering, need not wait for 103 outputs of audio frequency decoder module during echo, but directly according to broadcasting speed from the output queue echo of fetching data),
Echo interface 105 is used for by certain sequential, according to the data address after the audio decoder of the data address after the video decode of storing in the 3rd output queue buffering area 1048 and the 4th buffer stores, presents depending on the effect of audio frequency.
Wherein said echo interface 105 moves at different threads from audio decoder module and video decode module.
Described in the internal structure of looking the decoder module in audio frequency broadcast system in described Fig. 6 and operation principle such as embodiment one, do not giving unnecessary details at this.
The embodiment of the present invention is by arranging the output frame queue buffer in video decode module, audio decoder module, trade space for time, make (accurate parallel) work asynchronously of echo interface module and decoder module, specifically, if look the audio frame number certificate in the frame queue buffering, need not wait for decoder output during echo, but directly according to broadcasting speed from the buffer queue echo of fetching data.Grow in the time of can leveling land due to the decoding of video I frame (usually much larger than 40ms), in order to guarantee that this moment, the echo module can not stop less than CPU because distributing, the present invention in video decoding process entropy decoding and motion compensation after by dispatching CPU, guarantee that the echo module can in time be dispatched to CPU, thereby guarantee that the smoothness of playing is level and smooth.Like this, in the time of just making the calculating decode resources that can provide when processor satisfy relatively reluctantly (the every frame of average 36ms when for example the individual event video decode is tested), can eliminate and look audio frequency and play phenomenon unbalanced or card, level and smooth, smooth result of broadcast and audio-visual synchronization more accurately are provided.
Those having ordinary skill in the art will appreciate that, realize that all or part of step in above-described embodiment method can complete by the program command related hardware, described program can be stored in a computer read/write memory medium, and described storage medium can be ROM, RAM, disk, CD etc.
The above is only preferred embodiment of the present invention, not in order to limiting the present invention, all any modifications of doing within the spirit and principles in the present invention, is equal to and replaces and improvement etc., within all should being included in protection scope of the present invention.