Embodiment
In order to make object of the present invention, technical scheme and advantage clearly understand, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.
In the embodiment of the present invention, obtain by the voice data of the file of voice and image combining and video code flow, analog signal is converted to by the voice data of pcm encoder from digital signal again by what obtain, and the audio time stamp of the audio frequency parameter determination voice data returned according to digital to analog converter and determine the video time stamp of the video code flow of being encoded by MJPEG according to the frame per second of video code flow, finally according to audio time stamp and video time stamp decoded video code stream and the video code flow of broadcast decoder.
In order to technical solutions according to the invention are described, be described below by specific embodiment.
embodiment one:
The method flow of displaying video on low side embedded product that Fig. 1 shows that first embodiment of the invention provides, in the present embodiment, resolve by the file of voice and image combining to obtain voice data and the video code flow of this file, and according to the video code flow after the timestamp broadcast decoder of voice data, details are as follows:
In step s 11, obtain voice data and video code flow, and adopt digital to analog converter that this voice data is converted to analog signal from digital signal.
As an alternative embodiment of the invention, at acquisition voice data and video code flow, and the taking a step forward of step adopting digital to analog converter this voice data to be converted to analog signal from digital signal comprises the steps:
Read the file by voice and image combining, and resolve this file to obtain voice data and the video code flow of this file.Is Audio Video Interleaved form (Audio VideoInterleaved as reading by voice and image combining file, AVI) file, then resolve this avi file according to the character code of characterization data block type in this avi file, to obtain voice data and video code flow that this avi file comprises.If the data block type of avi file uses a fourCC to characterize, this fourCC is numbered by the type codes of 2 bytes and the stream of 2 bytes and is formed, and use " db " represents non-compression video frame, " dc " represents compressed video frame, new palette is used in " pc " expression instead, " wb " represents compressed audio, the type characterizing this data block as fourCC " 00wb " is audio data block, and this audio data block is first stream of this avi file.
As an alternative embodiment of the invention, motion jpeg (Motion JPEG should be used by the video code flow of the file of voice and image combining, MJPEG) coded system is encoded, the pulse modulation mode of (Pulse Code Modulation, PCM) of recording should be used to encode by the voice data of the file of voice and image combining.In the present embodiment, by video MJPEG coding, all absolute coding is carried out to each frame video code flow, and there is no interframe encode, thus each the frame video code flow after coding is all key frame, and there is no reference frame and referenced frame, wherein, this key frame is I frame, and the value of four bytes before frame is the length of this frame.Owing to adopting the coded system of full I frame, the management of thus not needing to take exercises when decoding compensation and reference frame, thus can fast decoding, promote decoding speed.On the other hand, by audio frequency pcm encoder, voice data is converted to digital signal from analog signal, owing to not compressing this voice data, thus this voice data of decoding is not needed, the voice data being converted to digital signal is only needed to send into digital-to-analogue (DataAnalog, D/A) transducer, by the voice data of voice data from digital signal converted back into analog signal again.
In the present embodiment, after acquisition voice data and video code flow, by the voice data of acquisition stored in audio buffer district, by the video code flow of acquisition stored in video cache district.Take out from buffer area again when the voice data of acquisition is converted to analog signal from digital signal by needs.
As an alternative embodiment of the invention, the taking a step forward of step of the video code flow of acquisition stored in video cache district is being comprised the steps:
In the video cache district of depositing video code flow, a pre-configured video decode threshold value, for controlling the size of the video code flow that this video cache district deposits.In the present embodiment, according to the actual assignable size of internal memory, a pre-configured video decode threshold value, when video cache district stored in video code flow be greater than this video decode threshold value time, directly abandon the document analysis frame video code flow out from voice and image combining, frame per second due to normal play video code stream is broadcasting 20 multi-frame video frame per second, therefore when broadcasting 20 multi-frame video code stream per second, abandon the result of broadcast that a frame video code flow can not affect whole video code flow once in a while, but effectively can save the Embedded internal memory of low side; When video cache district stored in video code flow be less than this video decode threshold value time, will resolve obtain a frame video code flow stored in this video cache district.Such as, suppose that the upper limit of internal memory is 10K, pre-configured video decode threshold value is 10K, when resolving the frame video code flow obtained by the file of voice and image combining and being greater than 10K, then directly abandons this frame video code flow; When the frame video code flow obtained is not more than 10K, by this frame video code flow stored in video cache district.
In step s 12, the audio time stamp of this voice data is determined according to the audio frequency parameter of digital to analog converter.
In the present embodiment, voice data is converted to from digital signal the sample number (SAMPLES_SEND_ONCE) etc. that audio frequency parameter that analog signal returns has counting (COUNT), sample rate (SAMPLESPERSEC), once transmits, determine the audio time stamp of this voice data according to this audio frequency parameter, this audio time stamp plays the time of this audio frequency for identifying.Suppose that this audio time stamp audioPTS represents, then audioPTS can calculate according to following formula:
audioPTS=COUNT*SAMPLES_SEND_ONCE/SAMPLESPERSEC。
In step s 13, the video time stamp of this video code flow is determined according to the frame per second of the video code flow obtained.
In the present embodiment, the video time stamp of this video code flow is the time of playing this video code flow frame.The timestamp of each frame video code flow determines with reference to the timestamp of previous frame video code flow.Suppose that the frame per second of video code flow obtained is 25f/s, the video time stamp of current video code stream frame is 20ms, then the video time stamp of next video code flow frame should be 20+40=60ms.
In step S14, according to audio time stamp and video time stamp decoded video code stream and the video code flow of broadcast decoder.
In the present embodiment, the voice data of decoded audio buffer area and the video code flow of decoded video buffer area.
Wherein, according to audio time stamp and video time stamp decoded video code stream and the step of the video code flow of broadcast decoder be specially:
Pre-configured one be greater than 0 time threshold, judge whether the absolute value of the difference of video time stamp and audio time stamp is less than this pre-configured time threshold, if the absolute value of the difference of this video time stamp and audio time stamp is less than pre-configured time threshold, illustrate that audio decoder speed and video decode speed are more or less the same, the audio frequency and video play substantially can be synchronous, then continue from video cache district, take out a frame video code flow and carry out decoding and the video code flow of broadcast decoder; If the absolute value of the difference of video time stamp and audio time stamp is greater than pre-configured time threshold, illustrate that the decoding speed of video is greater than the decoding speed of audio frequency, the decoding speed frequently of photopic vision is in other words less than the decoding speed of audio frequency.When the decoding speed of video is greater than the decoding speed of audio frequency, the broadcasting speed of video is faster than the broadcasting speed of audio frequency, the then video code flow in temporary transient video cache district of not decoding, until the absolute value of the difference of video time stamp and audio time stamp is less than or equal to pre-configured time threshold, just continue decoded video code stream, to make the audio-visual synchronization of broadcasting; When the decoding speed of video is less than the decoding speed of audio frequency, the broadcasting speed of video is slower than the broadcasting speed of audio frequency, the video play lags behind the audio frequency of broadcasting, then directly abandon the video code flow frame stored in video cache district, the new video code flow frame obtained of decoding, with the absolute value of the difference of the audio time stamp of the voice data of the video time stamp and current broadcasting that reduce the video code flow frame of current broadcasting, finally reach the object that audio-visual synchronization is play.
In first embodiment of the invention, because the voice data in the file of voice and image combining adopts pcm encoder mode to encode, this pcm encoder does not compress voice data, video code flow in file adopts MJPEG coded system coding, this MJPEG coding carries out absolute coding to each frame video code flow, there is no interframe encode, reference and referenced relation is not had between each frame, therefore during the file of decompress(ion) by voice and image combining, do not need to carry out decompress(ion) to voice data, only need by this voice data by digital to analog converter converted back into analog signal, and during decompressing video code stream, do not need the management of motion compensation and reference frame, greatly reduce decoding complex degree, thus improve the decoding speed of whole file, and MJPEG does not need to have decoded, a whole frame of video just can show, effectively save memory headroom.Obtain the video code flow of being encoded by the voice data of pcm encoder and MJPEG, and the voice data of acquisition is converted to analog signal by digital to analog converter, the video time stamp of the audio time stamp of the audio frequency parameter determination voice data returned according to digital to analog converter again and the frame per second determination video code flow according to the video code flow of being encoded by MJPEG obtained, finally according to audio time stamp and video time stamp decoded video code stream and the video code flow of broadcast decoder.Due to only when the absolute value of the difference of video time stamp and audio time stamp is less than pre-configured time threshold, just the video code flow of the decoding MJPEG coded system coding that adopts complexity lower, therefore can make audio-visual synchronization play.
embodiment two:
The terminal structure of displaying video on low side embedded product that Fig. 2 shows that the embodiment of the present invention provides, for convenience of explanation, illustrate only part related to the present embodiment.
The terminal of displaying video should may be used for the various information processing terminals by wired or wireless network connection server on low side embedded product, such as mobile phone, pocket computer (Pocket PersonalComputer, PPC), palmtop PC, computer, notebook computer, personal digital assistant (PersonalDigital Assistant, PDA) etc., can be run on the software unit in these terminals, the unit that hardware cell or software and hardware combine, also can be integrated in these terminals as independently suspension member or run in the application system of these terminals, wherein:
Analog signal acquiring unit 21 obtains voice data and video code flow, and adopts digital to analog converter that this voice data is converted to analog signal from digital signal.
In the present embodiment, by obtain voice data and video code flow respectively stored in audio buffer district and video cache district.
As an alternative embodiment of the invention, the terminal of displaying video also should comprise on low side embedded product: audio-video document resolution unit 25.
This audio-video document resolution unit 25, for reading the file by voice and image combining, and resolve this file to obtain voice data and the video code flow of this file, wherein, this voice data uses pulse modulation recording pcm encoder, and this video code flow uses motion jpeg coding.
As an alternative embodiment of the invention, the terminal of displaying video also should comprise on low side embedded product: video decode threshold value dispensing unit 26 and video code flow memory cell 27.
This video decode threshold value dispensing unit 26 is for a pre-configured video decode threshold value.
This video code flow memory cell 27 is for judging whether the frame video code flow obtained is greater than this video decode threshold value, when this video code flow is greater than video decode threshold value, abandon this video code flow, when this video code flow is less than or equal to video decode threshold value, by this video code flow stored in video cache district.
In the present embodiment, before the video code flow that will obtain is stored in video cache district, video code flow memory cell 27 first judges whether the video code flow obtained is greater than pre-configured decoding threshold value.
Audio time stamp determining unit 22 determines the audio time stamp of this voice data according to the audio frequency parameter of digital to analog converter.
In the present embodiment, voice data is converted to from digital signal the sample number (SAMPLES_SEND_ONCE) etc. that audio frequency parameter that analog signal returns has counting (COUNT), sample rate (SAMPLESPERSEC), once transmits, determine the audio time stamp of this voice data according to this audio frequency parameter, this audio time stamp plays the time of this audio frequency for identifying.Suppose that this audio time stamp audioPTS represents, then audioPTS can calculate according to following formula:
audioPTS=COUNT*SAMPLES_SEND_ONCE/SAMPLESPERSEC。
Video time stamp determining unit 23 determines the video time stamp of this video code flow according to the frame per second of the video code flow obtained.
Video code flow broadcast unit 24 is according to this audio time stamp and this video time stamp decoded video code stream and the video code flow of broadcast decoder.
As an alternative embodiment of the invention, this video code flow broadcast unit 24 comprises: time threshold judge module 241, video code flow decoder module 242, video code flow not decoder module 243.
This time threshold judge module 241 pre-configured one be greater than 0 time threshold, judge whether the absolute value of the difference of video time stamp and audio time stamp is greater than this time threshold.
Video code flow decoder module 242, when the absolute value of the difference of this video time stamp and audio time stamp is less than or equal to pre-configured time threshold, takes out a frame video code flow and carries out decoding and the video code flow of broadcast decoder from video cache district.
Video code flow not decoder module 243 when the absolute value of the difference of video time stamp and audio time stamp is greater than pre-configured time threshold, the video code flow in video cache district of not decoding.
In the present embodiment, if the absolute value of the difference of video time stamp and audio time stamp is greater than pre-configured time threshold, then illustrate that the decoding speed of video is greater than the decoding speed of audio frequency, the decoding speed frequently of photopic vision is in other words less than the decoding speed of audio frequency.When the decoding speed of video is greater than the decoding speed of audio frequency, the broadcasting speed of video is faster than the broadcasting speed of audio frequency, the then video code flow in temporary transient video cache district of not decoding, until the absolute value of the difference of video time stamp and audio time stamp is less than or equal to pre-configured time threshold, just continue decoded video code stream, to make the audio-visual synchronization of broadcasting; When the decoding speed of video is less than the decoding speed of audio frequency, the broadcasting speed of video is slower than the broadcasting speed of audio frequency, the video play lags behind the audio frequency of broadcasting, then directly abandon the video code flow frame stored in video cache district, the new video code flow frame obtained of decoding, with the absolute value of the difference of the audio time stamp of the voice data of the video time stamp and current broadcasting that reduce the video code flow frame of current broadcasting, finally reach the object that audio-visual synchronization is play.
In second embodiment of the invention, the voice data of resolving due to audio-video document resolution unit 25 uses pcm encoder mode to encode, the video code flow of resolving uses MJPEG coded system coding, therefore the speed of video code flow broadcast unit 24 decoded video code stream can be promoted, in addition, it is wait for or decode or directly abandon that the video time stamp that the audio time stamp determined according to audio time stamp determining unit and 22 video time stamp determining units 23 are determined is determined stored in the video code flow in video cache district, effectively ensure that the synchronous broadcasting of audio frequency and video.
In the embodiment of the present invention, because the voice data in the file of voice and image combining adopts pcm encoder mode to encode, this pcm encoder does not compress voice data, video code flow in file adopts MJPEG coded system coding, this MJPEG coding carries out absolute coding to each frame video code flow, there is no interframe encode, reference and referenced relation is not had between each frame, therefore during the file of decompress(ion) by voice and image combining, do not need to carry out decompress(ion) to voice data, only need by this voice data by digital to analog converter converted back into analog signal, and during decompressing video code stream, do not need the management of motion compensation and reference frame, greatly reduce decoding complex degree, thus improve the decoding speed of whole file, and MJPEG does not need to have decoded, a whole frame of video just can show, effectively save memory headroom.Obtain the video code flow of being encoded by the voice data of pcm encoder and MJPEG, and the voice data of acquisition is converted to analog signal by digital to analog converter, the video time stamp of the audio time stamp of the audio frequency parameter determination voice data returned according to digital to analog converter again and the frame per second determination video code flow according to the video code flow of being encoded by MJPEG obtained, finally according to audio time stamp and video time stamp decoded video code stream and the video code flow of broadcast decoder.Due to only when the absolute value of the difference of video time stamp and audio time stamp is less than pre-configured time threshold, just the video code flow of the decoding MJPEG coded system coding that adopts complexity lower, therefore can make audio-visual synchronization play.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, all any amendments done within the spirit and principles in the present invention, equivalent replacement and improvement etc., all should be included within protection scope of the present invention.