CN104394456B - The method and device of audio/video decoding under a kind of embedded environment - Google Patents
The method and device of audio/video decoding under a kind of embedded environment Download PDFInfo
- Publication number
- CN104394456B CN104394456B CN201410668321.9A CN201410668321A CN104394456B CN 104394456 B CN104394456 B CN 104394456B CN 201410668321 A CN201410668321 A CN 201410668321A CN 104394456 B CN104394456 B CN 104394456B
- Authority
- CN
- China
- Prior art keywords
- video
- transcoding
- audio
- data
- buffering area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4333—Processing operations in response to a pause request
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4398—Processing of audio elementary streams involving reformatting operations of audio signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Abstract
This application discloses a kind of method of audio/video decoding under embedded environment, including：A, audio/video flow data are obtained from the external world；B, the audio/video flow data are decoded；The decoding includes：If the video container that hardware interface decoding is not supported in the audio/video flow data be present, it is the video container that hardware interface is supported to carry out software transcoding to the video container；If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, it is the coded format that hardware interface can be supported to play that software transcoding is carried out to the audio coding；C, soft decoded audio-video frequency content is pushed to hardware interface and carries out audio and video playing.Disclosed herein as well is the device of audio/video decoding under embedded environment.Technical scheme supports more video containers and audio coding on original hardware interface advantage is retained.
Description
Technical field
The application is related to a kind of side of the soft decoding of video under audio video encoding technology field, more particularly to embedded environment
Method and device.
Background technology
In the prior art, the video playback based on bottom chip has two sets of mechanism, respectively by playback interfaces and
Both hardware interfaces of playpump interfaces are realized.
By playpump interfaces carry out video playback, use is more convenient, its advantage be play video when can with regarding
Frequency caching plays out for carrier, without complete video file, using flexible；Its shortcoming is for supporting video container
Support it is less, for video display process exception error processing it is weak the problem of, so as to cause play video process procedures card
Dead and collapse.Playback interfaces are directed to the shortcomings that above-mentioned playpump interfaces and improved.Playback interfaces exist
Many functions are added on the basis of playpump interfaces, for example look for I frames, play MP4 video files etc..But playback is broadcast
Using file as carrier when putting video, more weakness is supported for Streaming Media, secondary development is carried out so as to be unfavorable for being directed to application.
In addition, playback interfaces and playpump interfaces are decoded based on hardware, all in the presence of some popular videos
The situation that container and audio coding are not supported.Moreover, existing playing process is all based on single thread processing, if network condition
It is bad that Caton phenomenon occurs, have a strong impact on Consumer's Experience.
The content of the invention
This application provides a kind of method and device of the soft decoding of video under embedded environment, is retaining hardware interface advantage
On, support more video containers and audio coding.
The method of audio frequency and video transcoding under a kind of embedded environment that the embodiment of the present application provides, it is characterised in that, including with
Lower step:
Audio/video flow data are obtained, transcoding is carried out to the audio/video flow data；The transcoding includes:If the audio frequency and video
The video container that hardware interface decoding is not supported in flow data be present, it is hardware interface to carry out software transcoding to the video container
The video container of support；If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, to the sound
It is the coded format that hardware interface can be supported to play that frequency coding, which carries out software transcoding,.
The method that the embodiment of the present application also provides audio/video decoding under a kind of embedded environment, including：
A, audio/video flow data are obtained from the external world；
B, soft decoding is carried out to the audio/video flow data；The soft decoding includes：If deposited in the audio/video flow data
In the video container that hardware interface decoding is not supported, it is the video that hardware interface is supported that software transcoding is carried out to the video container
Container；If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, the audio coding is carried out
Software transcoding is the coded format that hardware interface can be supported to play；
C, soft decoded audio-video frequency content is pushed to hardware interface and carries out audio and video playing.
It is preferred that step C further comprises：
C1, in the first preset duration, obtain the Presentation Time Stamp in soft decoded audio-video frequency content；If the display
Timestamp does not change in the first preset duration, stops audio and video playing；
C2, the soft decoding of acquisition draw video theory reproduction time T1 and hardware decoding video actual play time T2；Will
T1 times and T2 contrasts, if the absolute value of both differences is more than threshold value set in advance, it is mistake to mark the audio/video flow data
Video, otherwise, marking the audio/video flow data to be can be with normal play video.
It is preferred that step C further comprises：The broadcasting passage that soft decoded audio-video document is pushed to hardware interface delays
After depositing, if this section of spatial cache is not consumed in the second preset duration, the content played in passage caching is emptied.
It is preferred that first preset duration is 500 milliseconds to 1 second.
It is preferred that the step A, step B and step C are realized by an independent thread respectively, and three lines
Cheng Binghang is run.
It is preferred that the step A is realized using receiving thread, the step A includes：
Step 201：The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step
Rapid 205；
Step 202：The agreement of video file is determined, and judges whether video file reads end, if so, return to step
201, otherwise, perform step 204；
Step 204：Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, performing step
206, otherwise perform step 205；
Step 205：Receiving thread dormancy；When file is not run through also, and the writeable capacity of filebuf is more than provisional video
File size, receiving thread wake up, and go to step 201；
Step 206：The video file data equal to provisional video file size is read, is written into filebuf
In；Using soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected,
Draw the numbering of video flowing and audio stream and specific coding information；
Step 207：The writeable capacity of filebuf is subtracted into provisional video file size, as the file buffering after renewal
The writeable capacity in area, is then back to step 201.
It is preferred that the step B is realized using transcoding threads, step B includes：
Step 301：Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, step 302 is performed,
Otherwise step 303 is performed；Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block and together
When be less than the writeable capacity value of transcoding buffering area；
Step 302：Judge whether filebuf is empty, if performing step 303, otherwise performs step 304；
Step 303：Transcoding threads dormancy；If the writeable capacity of transcoding buffering area is more than interim transcoding length, waking up should
Thread return to step 301；
Step 304：The audio frequency and video included less than the video file data of interim transcoding length are read from filebuf to compile
Code data；
Step 305：Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise
Go to step 307；
Step 306：Data after transcoding are write into transcoding buffering area, while the writeable capacity of transcoding buffering area is subtracted into transcoding
File size afterwards, as the writeable capacity of transcoding buffering area after renewal；
Step 307：Filebuf removes the data of interim transcoding length being read, at the same increase filebuf can
Capacity is write, increasing degree is equal to interim transcoding length.It is then back to step 301.
It is preferred that the step C realizes that step C includes using thread is played：
Step 401：Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403；
Step 402：Thread dormancy will be played, if it is determined that transcoding buffering area does not wake up the thread return to step then for sky
401；
Step 403：From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player；It is described to broadcast
Degree of lengthening is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom；
Step 404：Bottom chip carries out audio and video playing to the data after the transcoding that receives；
Step 405：The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
The embodiment of the present application additionally provides a kind of device of audio/video decoding under embedded environment, including receiving module, turns
Code module and playing module；
The receiving module is used to obtain audio/video flow data from the external world；
The transcoding module is used to decode the audio/video flow data；The decoding includes：If the audio frequency and video
The video container that hardware interface decoding is not supported in flow data be present, it is hardware interface to carry out software transcoding to the video container
The video container of support；If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, to the sound
It is the coded format that hardware interface can be supported to play that frequency coding, which carries out software transcoding,；
The playing module is used to soft decoded audio-video frequency content being pushed to hardware interface progress audio and video playing.
It is preferred that the playing module further comprises：
First judging unit, it is aobvious in soft decoded audio-video document content in the first preset duration, obtaining
Show timestamp；If the Presentation Time Stamp does not change in the first preset duration, stop audio and video playing；
Second judging unit, video theory reproduction time T1 is drawn for after audio and video playing is stopped, obtaining soft decoding
And hardware decoding video actual play time T2；T1 times and T2 are contrasted, set in advance if the absolute value of both differences is more than
Fixed threshold value, it is erroneous video to mark the audio/video flow data, otherwise, marks the audio/video flow data as that can be regarded with normal play
Frequently.
It is preferred that the playing module further comprises：
Processing unit is emptied, the broadcasting passage that hardware interface is pushed to for soft decoded audio-video document caches it
Afterwards, if this section of spatial cache is not consumed in the second preset duration, the content played in passage caching is emptied.
It is preferred that first preset duration is 500 milliseconds to 1 second.
It is preferred that the receiving module, transcoding module and playing module are realized by an independent thread respectively, and institute
State three thread parallel operations.
It is preferred that the receiving module is realized using receiving thread, the operational process of the receiving module includes：
Step 201：The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step
Rapid 205；
Step 202：The agreement of video file is determined, and judges whether video file reads end, if so, return to step
201, otherwise, perform step 204；
Step 204：Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, performing step
206, otherwise perform step 205；
Step 205：Receiving thread dormancy；When file is not run through also, and the writeable capacity of filebuf is more than provisional video
File size, receiving thread wake up, and go to step 201；
Step 206：The video file data equal to provisional video file size is read, is written into filebuf
In；Using soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected,
Draw the numbering of video flowing and audio stream and specific coding information；
Step 207：The writeable capacity of filebuf is subtracted into provisional video file size, as the file buffering after renewal
The writeable capacity in area, is then back to step 201.
It is preferred that the transcoding module is realized using transcoding threads, the operational process of transcoding module includes：
Step 301：Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, step 302 is performed,
Otherwise step 303 is performed；Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block and together
When be less than the writeable capacity value of transcoding buffering area；
Step 302：Judge whether filebuf is empty, if performing step 303, otherwise performs step 304；
Step 303：Transcoding threads dormancy；If the writeable capacity of transcoding buffering area is more than interim transcoding length, wakes up and turn
Code line journey return to step 301；
Step 304：The audio frequency and video included less than the video file data of interim transcoding length are read from filebuf to compile
Code data；
Step 305：Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise
Go to step 307；
Step 306：Data after transcoding are write into transcoding buffering area, while the writeable capacity of transcoding buffering area is subtracted into transcoding
File size afterwards, as the writeable capacity of transcoding buffering area after renewal；
Step 307：Filebuf removes the data of interim transcoding length being read, at the same increase filebuf can
Capacity is write, increasing degree is equal to interim transcoding length.It is then back to step 301.
It is preferred that the playing module is realized using thread is played,
The operational process of the playing module includes：
Step 401：Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403；
Step 402：Thread dormancy will be played, if it is determined that transcoding buffering area does not play thread return to step for empty then wake up
401；
Step 403：From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player；It is described to broadcast
Degree of lengthening is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom；
Step 404：Bottom chip carries out audio and video playing to the data after the transcoding that receives；
Step 405：The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
As can be seen from the above technical solutions, the application scheme first step is to the soft solution of audio-video document, the benefit so done
It is：It is the video container that hardware interface is supported that the video container that hardware interface is not supported, which carries out software de-multiplex, and will be hard
It is the coded format that hardware interface can be supported to play that the audio coding that part interface is not supported, which carries out software transcoding,；Second step will pass through
The soft decoded video content of the first step is pushed to hardware interface playing audio-video.Application scheme also passes through video playback mistake
Restoration Mechanism is detected, with a certain degree of error correction audio frequency and video can be allowed to continue to play wrong audio frequency and video.Even if audio-video frequency content exists
Soft solution can not error correction situation, can also program recovery is come, continue to play next video.Application scheme can be well
Extend stream media protocol (stream media protocol includes but is not limited to http, udp, rtsp agreements etc.).In addition, the application is implemented
Example realized using multithreading, can be made more smooth during broadcasting video and stably.Application scheme can protect
Stay in hardware interface advantage, support more video containers and audio coding, while using the automatic error-correcting mechanism of video playback
And Restoration Mechanism, improve and play video playback fluency and robustness.
Brief description of the drawings
Fig. 1 is the structural representation of the device of the soft decoding of video under the embedded environment that the embodiment of the present application provides；
Fig. 2 is the operational process schematic diagram that the receiving module 101 that the embodiment of the present application provides uses receiving thread；
Fig. 3 is operational process schematic diagram of the embodiment of the present application transit code module 103 using transcoding threads；
Fig. 4 is that playing module 105 uses the operational process schematic diagram for playing thread in the embodiment of the present application；
Fig. 5 is that playing module 105 carries out error detection and the handling process schematic diagram recovered；
Fig. 6 is the method flow schematic diagram of audio/video decoding under a kind of embedded environment that the embodiment of the present application provides.
Embodiment
For problems of the prior art, present applicant proposes a kind of solution of the soft decoding of video, wherein wrapping
Include：Soft solution+hardware interface video playback mechanism+video playback error detection Restoration Mechanism.Following examples are with playpump
It is described in detail exemplified by interface.
The structure of the device of the soft decoding of video is as shown in figure 1, receive mould under the embedded environment that the embodiment of the present application provides
Block 101 is used to realize multi-media protocol (various protocols), from external world's acquisition audio/video flow file data and the audio frequency and video are literary
Number of packages evidence is cached in filebuf 102.Transcoding module 103 reads audio-video document data from filebuf 102, right
The video container that hardware interface decoding is not supported carries out software de-multiplex, and the audio coding do not supported hardware interface decoding is carried out
Software transcoding, while by the audio-video document content push transcoding buffering area 104 after soft solution.Playing module 105 be used for from turn
The audio-video document content after transcoding is read in bitstream buffer 105, data-pushing then is carried out into video to hardware interface again broadcasts
Put.Playing module 105 is additionally operable to carry out error detection during playing video file and and mistake recovered.
In order that more smooth and stably during playing video, another embodiment of the application is using multithreading collaboration work
The mode of work realizes receiving module 101, transcoding module 103 and playing module 105.Specifically, it is real with (read) thread is received
Existing receiving module 101, transcoding (transcoder) thread realize transcoding module 103, play (pushdata) _ thread and realize broadcasting
Module 105, the flow that thread specifically cooperates is as shown in Figure 2, Figure 3 and Figure 4.
Fig. 2 is that the receiving module 101 that the embodiment of the present application provides realizes that stream media protocol receives video using read threads
File content.The stream media protocol includes but is not limited to HTTP (HTTP), UDP (UDP,
User Datagram Protocol), real time streaming transport protocol (RTSP, Real Time Streaming Protocol).This
Outside, the embodiment of the present application also supports that audio frequency and video are locally stored.
Receiving module 101 uses the operational process of receiving thread as shown in Fig. 2 including following step in the embodiment of the present application
Suddenly：
Step 201：The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step
Rapid 205.
Step 202：The agreement of video file is determined, and judges whether video file reads end, if so, return to step
201, otherwise, perform step 204.
Step 204：Judge whether the writeable capacity of filebuf is more than provisional video file size (fileTempSize),
If so, performing step 206, step 205 is otherwise performed.FileTemSize is a value fixed.
Step 205：Read thread dormancy.When file is not run through also, and the writeable capacity of filebuf is more than provisional video
File size, read thread wakenings, goes to step 201.
Step 206：The video file data equal to provisional video file size is read, is written into filebuf
In.Using (ffmpeg) soft decoding technique, the video flowing in the video file data that is read and audio stream information are divided
Analysis and detection, draw the numbering of video flowing and audio stream and specific coding information.
Step 207：The writeable capacity of filebuf is subtracted into provisional video file size, as the file buffering after renewal
The writeable capacity in area, is then back to step 201.
The embodiment of the present application transit code module 103 is using the operational process of transcoding threads as shown in figure 3, including following step
Suddenly：
Step 301：Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length (transTempSize), if
It is to perform step 302, otherwise performs step 303.TransTempSize is a value fixed, and inputs and buffers less than or equal to file
The readable capacity in area is while be less than the writeable capacity value of transcoding buffering area.
Step 302：Judge whether filebuf is empty, if performing step 303, otherwise performs step 304.
Step 303：Transcoding threads dormancy.If the writeable capacity of transcoding buffering area is more than interim transcoding length, waking up should
Thread return to step 301.
Step 304：The audio frequency and video included less than the video file data of interim transcoding length are read from filebuf to compile
Code data.
Step 305：Soft decoding is carried out to the audio/video coding data read.If it is successful, step 306 is performed, otherwise
Go to step 307.
In transcoding process, if bottom chip hard decoder directly supports video container, the video volume belonging to video file
Code and audio coding, are demultiplexed to video content.What if the form or audio coding of video file had not been supported, it is right
Audio-video document carries out turning encapsulation, and audio be transcoded into the coded format of hard decoder support, while video file is turned
Encapsulation, and the video file after transcoding be pushed in transcoding buffering area.
Step 306：Data after transcoding are write into transcoding buffering area, while the writeable capacity of transcoding buffering area is subtracted into transcoding
File size afterwards, as the writeable capacity of transcoding buffering area after renewal.
Step 307：Filebuf removes the data of interim transcoding length being read, at the same increase filebuf can
Capacity is write, increasing degree is equal to interim transcoding length.It is then back to step 301.
In transcoding process, video is demultiplexed and the audio to not supporting is decoded and encoded.So doing can
To support the video format of most of main flow and coding.It may further include during soft decoding：Detection and discarding regard
The bad frame of frequency file content, while the synchronizing information of audio frequency and video is adjusted, decoding error can be so detected, and video is carried out
A certain degree of error correction.During decoding, it is irrecoverable if it happens can not to read video flowing and audio stream packet etc.
Mistake, then stop playing video, and state be set to the state for waiting next video playback.
In the embodiment of the present application, the content of the video file after the acquisition software processing of playing module 105, chip bottom is pushed to
Layer plays out.Playing module 105 is using the operational process of broadcasting thread as shown in figure 4, comprising the following steps：
Step 401：Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403.
Step 402：Thread dormancy will be played.If it is determined that transcoding buffering area does not wake up the thread return to step then for sky
401。
Step 403：From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player.
Playout length (playsize) is a fixed value, and playsize is less than buffering area data with existing length after transcoding, together
When playSize be less than bottom can accommodate data.
Step 404：Bottom chip carries out audio and video playing to the data after the transcoding that receives.
Step 405：The increase of transcoding buffering area active volume, increasing degree is playout length.It is then back to step 401.
In the embodiment of the present application, playing module 105 is also equipped with carrying out error detection and extensive during playing video file
The system of answering a pager's call.By soft solution, the specifying information of video file sound intermediate frequency and video has been obtained, according to video concrete composition part, has been adopted
Error detection and recovery are carried out with flow as shown in Figure 5, is specifically comprised the following steps：
Step 501：After soft solution is carried out to video file and obtains audio-video document data, it is pushed to hardware interface and is broadcast
Put, judge whether playing duration is less than preset duration (PTS_WAIT_TIME), its value scope is 500 milliseconds to 1 second, if so, holding
Row step 502, otherwise perform step 510.
Step 502：Judge the composition of the audio-video document, if comprising video flowing, step 503 is performed, if only including audio
Stream, perform step 504.
Step 503：The Presentation Time Stamp (pts) of video render is obtained, then performs step 505.
Step 504：The pts of audio stream broadcasting is obtained, then performs step 505.
Step 505：Judge whether pts is changed, if so, explanation is interior in preset duration (PTS_WAIT_TIME), depending on
Frequency stream broadcasting is normal, without carrying out correction process, return to step 501, otherwise, performs step 506.
Step 506：Audio frequency and video stop playing.
Step 507：Obtain soft decoding and draw video theory reproduction time T1, hardware decoding video actual play time T2.
T1 times and T2 are contrasted, if the absolute value of both differences is more than threshold value set in advance (Max), step 508 is performed, otherwise holds
Row step 509.
Step 508：It is erroneous video to mark the video, and terminates this flow.
Step 509：Marking the video to be can be with normal play video, and terminates this flow.
Step 510：Audio-video document is pushed to after bottom chip broadcasting passage caching, is judged in certain time
(FAILED_TIME) in, whether this section of spatial cache is consumed, if so, performing step 501；Otherwise, step 511 is performed.
Step 511：Empty the content played in passage caching.This is done to exist not in automatic shield video file
Bad frame content can be recovered, continue to play subsequent video content.
Step 512：Statistics obtains bottom chip and plays the count value that the size that passage caches is continuously 0.If video is literary
Part pushes successfully, COUNT clear 0.
Step 513：Judge whether the count value is more than threshold value set in advance (FAILED_COUNT), if so, returning
Step 502；Otherwise, it is soft to solve next section of video flowing and perform above-mentioned flow.
Advantage of this is that can be during video file content is played by soft decoded video, hardware interface
In the case of exception can not play, the situation of mistake generation can be detected in time.
The method that the embodiment of the present application additionally provides audio frequency and video transcoding under a kind of embedded environment, comprises the following steps:
Audio/video flow data are obtained, transcoding is carried out to the audio/video flow data；The transcoding includes：If the audio frequency and video
The video container that hardware interface decoding is not supported in flow data be present, it is hardware interface to carry out software transcoding to the video container
The video container of support；If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, to the sound
It is the coded format that hardware interface can be supported to play that frequency coding, which carries out software transcoding,.
The method that the embodiment of the present application additionally provides audio/video decoding under a kind of embedded environment, as shown in fig. 6, including：
Step 601：Audio/video flow data are obtained from the external world；
Step 602：Soft decoding is carried out to the audio/video flow data；The soft decoding includes：If the audio frequency and video fluxion
According to the middle video container that hardware interface decoding be present and do not support, software transcoding is carried out to the video container and supported for hardware interface
Video container；If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, the audio is compiled
It is the coded format that hardware interface can be supported to play that code, which carries out software transcoding,；
Step 603：Soft decoded audio-video frequency content is pushed to hardware interface and carries out audio and video playing.
According to another embodiment of the application, step C further comprises：
C1, in the first preset duration, obtain the Presentation Time Stamp in soft decoded audio-video frequency content；If the display
Timestamp does not change in the first preset duration, stops audio and video playing；
C2, the soft decoding of acquisition draw video theory reproduction time T1 and hardware decoding video actual play time T2；Will
T1 times and T2 contrasts, if the absolute value of both differences is more than threshold value set in advance, it is mistake to mark the audio/video flow data
Video, otherwise, marking the audio/video flow data to be can be with normal play video.
According to another embodiment of the application, step C further comprises：Soft decoded audio-video document is pushed to hardware and connect
After the broadcasting passage caching of mouth, if this section of spatial cache is not consumed in the second preset duration, broadcasting passage is emptied
Content in caching.
According to another embodiment of the application, first preset duration is 500 milliseconds to 1 second.
According to another embodiment of the application, the step A, step B and step C are real by an independent thread respectively
It is existing, and three thread parallels are run.
According to another embodiment of the application, the step A is realized using receiving thread, as shown in Fig. 2 the step A bags
Include：
Step 201：The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step
Rapid 205；
Step 202：The agreement of video file is determined, and judges whether video file reads end, if so, return to step
201, otherwise, perform step 204；
Step 204：Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, performing step
206, otherwise perform step 205；
Step 205：Receiving thread dormancy；When file is not run through also, and the writeable capacity of filebuf is more than provisional video
File size, receiving thread wake up, and go to step 201；
Step 206：The video file data equal to provisional video file size is read, is written into filebuf
In；Using soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected,
Draw the numbering of video flowing and audio stream and specific coding information；
Step 207：The writeable capacity of filebuf is subtracted into provisional video file size, as the file buffering after renewal
The writeable capacity in area, is then back to step 201.
According to another embodiment of the application, the step B is realized using transcoding threads, as shown in figure 3, step B includes：
Step 301：Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, step 302 is performed,
Otherwise step 303 is performed；Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block and together
When be less than the writeable capacity value of transcoding buffering area；
Step 302：Judge whether filebuf is empty, if performing step 303, otherwise performs step 304；
Step 303：Transcoding threads dormancy；If the writeable capacity of transcoding buffering area is more than interim transcoding length, waking up should
Thread return to step 301；
Step 304：The audio frequency and video included less than the video file data of interim transcoding length are read from filebuf to compile
Code data；
Step 305：Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise
Go to step 307；
Step 306：Data after transcoding are write into transcoding buffering area, while the writeable capacity of transcoding buffering area is subtracted into transcoding
File size afterwards, as the writeable capacity of transcoding buffering area after renewal；
Step 307：Filebuf removes the data of interim transcoding length being read, at the same increase filebuf can
Capacity is write, increasing degree is equal to interim transcoding length.It is then back to step 301.
According to another embodiment of the application, the step C is realized using thread is played.As shown in figure 4, the step C bags
Include：
Step 401：Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403；
Step 402：Thread dormancy will be played, if it is determined that transcoding buffering area does not wake up the thread return to step then for sky
401；
Step 403：From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player；It is described to broadcast
Degree of lengthening is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom；
Step 404：Bottom chip carries out audio and video playing to the data after the transcoding that receives；
Step 405：The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
The foregoing is only the preferred embodiment of the application, not to limit the protection domain of the application, it is all
Within the spirit and principle of technical scheme, any modification, equivalent substitution and improvements done etc., this Shen should be included in
Within the scope of please protecting.
Claims (12)
1. a kind of method of audio/video decoding under embedded environment, it is characterised in that comprise the following steps:
A, audio/video flow data are obtained from the external world；
B, soft decoding is carried out to the audio/video flow data；The soft decoding includes：If exist in the audio/video flow data hard
The video container that part interface decoding is not supported, it is that the video that hardware interface is supported holds that software transcoding is carried out to the video container
Device；If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, the audio coding is carried out soft
Part transcoding is the coded format that hardware interface can be supported to play；
C, soft decoded audio-video frequency content is pushed to hardware interface and carries out audio and video playing；
The step A, step B and step C are realized by an independent thread respectively, and three thread parallels are run；
The step A realizes that the step A includes using receiving thread：
Step 201：The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step
205；
Step 202：The agreement of video file is determined, and judges whether video file reads end, if so, return to step 201,
Otherwise, step 204 is performed；
Step 204：Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, step 206 is performed,
Otherwise step 205 is performed；
Step 205：Receiving thread dormancy；When file is not run through also, and the writeable capacity of filebuf is more than provisional video file
Length, receiving thread wake up, and go to step 201；
Step 206：The video file data equal to provisional video file size is read, is written into filebuf；Adopt
With soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected, is drawn
The numbering and specific coding information of video flowing and audio stream；
Step 207：The writeable capacity of filebuf is subtracted into provisional video file size, can as the filebuf after renewal
Capacity is write, is then back to step 201.
2. according to the method for claim 1, it is characterised in that step C further comprises：
C1, in the first preset duration, obtain the Presentation Time Stamp in soft decoded audio-video frequency content；If the display time
Stamp does not change in the first preset duration, stops audio and video playing；
C2, the soft decoding of acquisition draw video theory reproduction time T1 and hardware decoding video actual play time T2；During by T1
Between and T2 contrasts, if both absolute values of difference are more than threshold value set in advance, it is erroneous video to mark the audio/video flow data,
Otherwise, marking the audio/video flow data to be can be with normal play video.
3. according to the method for claim 2, it is characterised in that step C further comprises：Soft decoded audio-video document
It is pushed to after the broadcasting passage caching of hardware interface, if this section of spatial cache is not consumed in the second preset duration,
Empty the content played in passage caching.
4. according to the method for claim 2, it is characterised in that first preset duration is 500 milliseconds to 1 second.
5. according to the method for claim 1, it is characterised in that the step B realizes that step B includes using transcoding threads：
Step 301：Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, performing step 302, otherwise
Perform step 303；Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block while small
In the writeable capacity value of transcoding buffering area；
Step 302：Judge whether filebuf is empty, if performing step 303, otherwise performs step 304；
Step 303：Transcoding threads dormancy；If the writeable capacity of transcoding buffering area is more than interim transcoding length, the thread is waken up
Return to step 301；
Step 304：The audio/video coding number included less than the video file data of interim transcoding length is read from filebuf
According to；
Step 305：Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise gone to
Step 307；
Step 306：Data after transcoding are write into transcoding buffering area, while after the writeable capacity of transcoding buffering area is subtracted into transcoding
File size, as the writeable capacity of transcoding buffering area after renewal；
Step 307：Filebuf removes the data for the interim transcoding length being read, while increases the writeable appearance of filebuf
Amount, increasing degree are equal to interim transcoding length.It is then back to step 301.
6. according to the method for claim 1, it is characterised in that the step C realizes that step C includes using thread is played：
Step 401：Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403；
Step 402：Thread dormancy will be played, if it is determined that transcoding buffering area does not wake up the thread return to step 401 then for sky；
Step 403：From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player；It is described to play length
Degree is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom；
Step 404：Bottom chip carries out audio and video playing to the data after the transcoding that receives；
Step 405：The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
7. the device of audio/video decoding under a kind of embedded environment, it is characterised in that including receiving module, transcoding module and broadcast
Amplification module；
The receiving module is used to obtain audio/video flow data from the external world；
The transcoding module is used to decode the audio/video flow data；The decoding includes：If the audio frequency and video fluxion
According to the middle video container that hardware interface decoding be present and do not support, software transcoding is carried out to the video container and supported for hardware interface
Video container；If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, the audio is compiled
It is the coded format that hardware interface can be supported to play that code, which carries out software transcoding,；
The playing module is used to soft decoded audio-video frequency content being pushed to hardware interface progress audio and video playing；
The receiving module, transcoding module and playing module are realized by an independent thread respectively, and three threads
Parallel operation；
The receiving module realizes that the operational process of the receiving module includes using receiving thread：
Step 201：The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step
205；
Step 202：The agreement of video file is determined, and judges whether video file reads end, if so, return to step 201,
Otherwise, step 204 is performed；
Step 204：Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, step 206 is performed,
Otherwise step 205 is performed；
Step 205：Receiving thread dormancy；When file is not run through also, and the writeable capacity of filebuf is more than provisional video file
Length, receiving thread wake up, and go to step 201；
Step 206：The video file data equal to provisional video file size is read, is written into filebuf；Adopt
With soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected, is drawn
The numbering and specific coding information of video flowing and audio stream；
Step 207：The writeable capacity of filebuf is subtracted into provisional video file size, can as the filebuf after renewal
Capacity is write, is then back to step 201.
8. device according to claim 7, it is characterised in that the playing module further comprises：
First judging unit, in the first preset duration, when obtaining the display in soft decoded audio-video document content
Between stab；If the Presentation Time Stamp does not change in the first preset duration, stop audio and video playing；
Second judging unit, for after audio and video playing is stopped, obtaining soft decoding draw video theory reproduction time T1 and
Hardware decoding video actual play time T2；T1 times and T2 are contrasted, if the absolute value of both differences is more than set in advance
Threshold value, it is erroneous video to mark the audio/video flow data, and otherwise, marking the audio/video flow data to be can be with normal play video.
9. device according to claim 8, it is characterised in that the playing module further comprises：
Processing unit is emptied, after being pushed to the broadcasting passage caching of hardware interface for soft decoded audio-video document, if
This section of spatial cache is not consumed in the second preset duration, then empties the content played in passage caching.
10. according to the method for claim 7, it is characterised in that first preset duration is 500 milliseconds to 1 second.
11. device according to claim 7, it is characterised in that the transcoding module is using transcoding threads realization, transcoding mould
The operational process of block includes：
Step 301：Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, performing step 302, otherwise
Perform step 303；Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block while small
In the writeable capacity value of transcoding buffering area；
Step 302：Judge whether filebuf is empty, if performing step 303, otherwise performs step 304；
Step 303：Transcoding threads dormancy；If the writeable capacity of transcoding buffering area is more than interim transcoding length, transcoding line is waken up
Journey return to step 301；
Step 304：The audio/video coding number included less than the video file data of interim transcoding length is read from filebuf
According to；
Step 305：Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise gone to
Step 307；
Step 306：Data after transcoding are write into transcoding buffering area, while after the writeable capacity of transcoding buffering area is subtracted into transcoding
File size, as the writeable capacity of transcoding buffering area after renewal；
Step 307：Filebuf removes the data for the interim transcoding length being read, while increases the writeable appearance of filebuf
Amount, increasing degree are equal to interim transcoding length.It is then back to step 301.
12. device according to claim 7, it is characterised in that the playing module, which uses, plays thread realization,
The operational process of the playing module includes：
Step 401：Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403；
Step 402：Thread dormancy will be played, if it is determined that transcoding buffering area does not play thread return to step 401 for empty then wake up；
Step 403：From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player；It is described to play length
Degree is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom；
Step 404：Bottom chip carries out audio and video playing to the data after the transcoding that receives；
Step 405：The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410668321.9A CN104394456B (en) | 2014-11-20 | 2014-11-20 | The method and device of audio/video decoding under a kind of embedded environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410668321.9A CN104394456B (en) | 2014-11-20 | 2014-11-20 | The method and device of audio/video decoding under a kind of embedded environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104394456A CN104394456A (en) | 2015-03-04 |
| CN104394456B true CN104394456B (en) | 2017-12-01 |
Family
ID=52612284
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410668321.9A Active CN104394456B (en) | 2014-11-20 | 2014-11-20 | The method and device of audio/video decoding under a kind of embedded environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104394456B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106384596A (en) * | 2016-09-22 | 2017-02-08 | 努比亚技术有限公司 | Audio data processing method and terminal |
| CN106658117A (en) * | 2016-12-30 | 2017-05-10 | 百度在线网络技术（北京）有限公司 | Method and device for processing audio/video data |
| CN106792124A (en) * | 2016-12-30 | 2017-05-31 | 合网络技术（北京）有限公司 | Multimedia resource decodes player method and device |
| CN106604115B (en) * | 2016-12-30 | 2020-08-14 | 深圳Tcl新技术有限公司 | Video playing control device and method |
| CN108683946B (en) * | 2018-05-23 | 2020-10-27 | 成都智达万应科技有限公司 | Method for realizing online video education based on face recognition and cache mechanism |
| US11368745B2 (en) | 2018-07-05 | 2022-06-21 | Dolby International Ab | Processing media data structures |
| CN110858827B (en) * | 2018-08-22 | 2023-03-24 | 浙江宇视科技有限公司 | Broadcast starting acceleration method and device and computer readable storage medium |
| CN110858829B (en) * | 2018-08-22 | 2023-02-07 | 杭州海康威视数字技术股份有限公司 | Video processing method, device and system, analysis server and transcoding server |
| CN109088887A (en) * | 2018-09-29 | 2018-12-25 | 北京金山云网络技术有限公司 | A kind of decoded method and device of Streaming Media |
| CN109257646B (en) | 2018-10-15 | 2020-09-22 | Oppo广东移动通信有限公司 | Video processing method and device, electronic equipment and computer readable medium |
| CN109151574B (en) | 2018-10-15 | 2020-03-24 | Oppo广东移动通信有限公司 | Video processing method, video processing device, electronic equipment and storage medium |
| CN111147926B (en) * | 2018-11-02 | 2022-05-06 | 杭州海康威视数字技术股份有限公司 | Data transcoding method and device |
| CN110351600A (en) * | 2019-01-21 | 2019-10-18 | 福州友宝电子科技有限公司 | A kind of method and device playing video |
| CN114760495A (en) * | 2022-03-21 | 2022-07-15 | 南京芯驰半导体科技有限公司 | Video hardware decoding method and system based on video processing unit |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101951517A (en) * | 2010-09-27 | 2011-01-19 | 深圳市融创天下科技发展有限公司 | Method, system and terminal equipment for decoding and playing video |
| CN102811383A (en) * | 2012-07-06 | 2012-12-05 | 深圳市同洲电子股份有限公司 | Video file playing method and device based on set top box |
| CN103024535A (en) * | 2012-12-25 | 2013-04-03 | 北京奇虎科技有限公司 | Video playing method, server and video playing system |
| CN103747317A (en) * | 2013-12-27 | 2014-04-23 | 高新兴科技集团股份有限公司 | A smooth playing control method of a play database in an instable network |
-
2014
- 2014-11-20 CN CN201410668321.9A patent/CN104394456B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101951517A (en) * | 2010-09-27 | 2011-01-19 | 深圳市融创天下科技发展有限公司 | Method, system and terminal equipment for decoding and playing video |
| CN102811383A (en) * | 2012-07-06 | 2012-12-05 | 深圳市同洲电子股份有限公司 | Video file playing method and device based on set top box |
| CN103024535A (en) * | 2012-12-25 | 2013-04-03 | 北京奇虎科技有限公司 | Video playing method, server and video playing system |
| CN103747317A (en) * | 2013-12-27 | 2014-04-23 | 高新兴科技集团股份有限公司 | A smooth playing control method of a play database in an instable network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104394456A (en) | 2015-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104394456B (en) | The method and device of audio/video decoding under a kind of embedded environment | |
| KR101107815B1 (en) | Media stream recording into a reception hint track of a multimedia container file | |
| CN103338386B (en) | Based on the audio and video synchronization method simplifying timestamp | |
| WO2007074361A3 (en) | Tune-in time reduction | |
| WO2007120261A3 (en) | Expedited digital signal decoding | |
| CN101193311B (en) | Audio and video data synchronization method in P2P system | |
| CN101427579A (en) | Time-shifted presentation of media streams | |
| CN103686315A (en) | Synchronous audio and video playing method and device | |
| TW200644639A (en) | Encoding apparatus and method, and decoding apparatus and method | |
| JP2012532568A5 (en) | ||
| MX2009006663A (en) | Method and apparatus for store and replay functions in a digital radio broadcasting receiver. | |
| CN102264004A (en) | Method and device for preventing deficiency of key frame from causing seek incapability | |
| WO2007090988A3 (en) | Method and device for the hierarchical coding of a source audio signal and corresponding decoding method and device, programs and signal | |
| EP1195996A3 (en) | Apparatus, method and computer program product for decoding and reproducing moving images, time control method and multimedia information receiving apparatus | |
| CN101442675A (en) | Multimedia play terminal and play method | |
| TW200623881A (en) | Method of decoding digital video and digital video decoder system thereof | |
| WO2009137972A1 (en) | A method and system for transmitting video-audio in same stream and the corresponding receiving method and device | |
| CN104853132A (en) | Delay video recording method and system | |
| CN106878290B (en) | Audio and video playing method and device | |
| EP1589770A3 (en) | Apparatus and method for decoding a moving picture sequence | |
| CN104023260B (en) | Hardware decoding realization method, device and player | |
| WO2007125458A3 (en) | Method and apparatus for encoding/transcoding and decoding | |
| CN105657448B (en) | A kind of retransmission method, the apparatus and system of encoded video stream | |
| JP2009171294A5 (en) | ||
| CN101102485A (en) | Audio and video synchronization device and method of mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |