CN104394456B - The method and device of audio/video decoding under a kind of embedded environment - Google Patents

The method and device of audio/video decoding under a kind of embedded environment Download PDF

Info

Publication number
CN104394456B
CN104394456B CN201410668321.9A CN201410668321A CN104394456B CN 104394456 B CN104394456 B CN 104394456B CN 201410668321 A CN201410668321 A CN 201410668321A CN 104394456 B CN104394456 B CN 104394456B
Authority
CN
China
Prior art keywords
video
transcoding
audio
data
buffering area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201410668321.9A
Other languages
Chinese (zh)
Other versions
CN104394456A (en
Inventor
袁嘉晟
梁文森
陈旭孟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujian Star Net eVideo Information Systems Co Ltd
Original Assignee
Fujian Star Net eVideo Information Systems Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujian Star Net eVideo Information Systems Co Ltd filed Critical Fujian Star Net eVideo Information Systems Co Ltd
Priority to CN201410668321.9A priority Critical patent/CN104394456B/en
Publication of CN104394456A publication Critical patent/CN104394456A/en
Application granted granted Critical
Publication of CN104394456B publication Critical patent/CN104394456B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331Caching operations, e.g. of an advertisement for later insertion during playback
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4333Processing operations in response to a pause request
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439Processing of audio elementary streams
    • H04N21/4398Processing of audio elementary streams involving reformatting operations of audio signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/472End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/8547Content authoring involving timestamps for synchronizing content

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

This application discloses a kind of method of audio/video decoding under embedded environment, including:A, audio/video flow data are obtained from the external world;B, the audio/video flow data are decoded;The decoding includes:If the video container that hardware interface decoding is not supported in the audio/video flow data be present, it is the video container that hardware interface is supported to carry out software transcoding to the video container;If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, it is the coded format that hardware interface can be supported to play that software transcoding is carried out to the audio coding;C, soft decoded audio-video frequency content is pushed to hardware interface and carries out audio and video playing.Disclosed herein as well is the device of audio/video decoding under embedded environment.Technical scheme supports more video containers and audio coding on original hardware interface advantage is retained.

Description

The method and device of audio/video decoding under a kind of embedded environment
Technical field
The application is related to a kind of side of the soft decoding of video under audio video encoding technology field, more particularly to embedded environment Method and device.
Background technology
In the prior art, the video playback based on bottom chip has two sets of mechanism, respectively by playback interfaces and Both hardware interfaces of playpump interfaces are realized.
By playpump interfaces carry out video playback, use is more convenient, its advantage be play video when can with regarding Frequency caching plays out for carrier, without complete video file, using flexible;Its shortcoming is for supporting video container Support it is less, for video display process exception error processing it is weak the problem of, so as to cause play video process procedures card Dead and collapse.Playback interfaces are directed to the shortcomings that above-mentioned playpump interfaces and improved.Playback interfaces exist Many functions are added on the basis of playpump interfaces, for example look for I frames, play MP4 video files etc..But playback is broadcast Using file as carrier when putting video, more weakness is supported for Streaming Media, secondary development is carried out so as to be unfavorable for being directed to application.
In addition, playback interfaces and playpump interfaces are decoded based on hardware, all in the presence of some popular videos The situation that container and audio coding are not supported.Moreover, existing playing process is all based on single thread processing, if network condition It is bad that Caton phenomenon occurs, have a strong impact on Consumer's Experience.
The content of the invention
This application provides a kind of method and device of the soft decoding of video under embedded environment, is retaining hardware interface advantage On, support more video containers and audio coding.
The method of audio frequency and video transcoding under a kind of embedded environment that the embodiment of the present application provides, it is characterised in that, including with Lower step:
Audio/video flow data are obtained, transcoding is carried out to the audio/video flow data;The transcoding includes:If the audio frequency and video The video container that hardware interface decoding is not supported in flow data be present, it is hardware interface to carry out software transcoding to the video container The video container of support;If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, to the sound It is the coded format that hardware interface can be supported to play that frequency coding, which carries out software transcoding,.
The method that the embodiment of the present application also provides audio/video decoding under a kind of embedded environment, including:
A, audio/video flow data are obtained from the external world;
B, soft decoding is carried out to the audio/video flow data;The soft decoding includes:If deposited in the audio/video flow data In the video container that hardware interface decoding is not supported, it is the video that hardware interface is supported that software transcoding is carried out to the video container Container;If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, the audio coding is carried out Software transcoding is the coded format that hardware interface can be supported to play;
C, soft decoded audio-video frequency content is pushed to hardware interface and carries out audio and video playing.
It is preferred that step C further comprises:
C1, in the first preset duration, obtain the Presentation Time Stamp in soft decoded audio-video frequency content;If the display Timestamp does not change in the first preset duration, stops audio and video playing;
C2, the soft decoding of acquisition draw video theory reproduction time T1 and hardware decoding video actual play time T2;Will T1 times and T2 contrasts, if the absolute value of both differences is more than threshold value set in advance, it is mistake to mark the audio/video flow data Video, otherwise, marking the audio/video flow data to be can be with normal play video.
It is preferred that step C further comprises:The broadcasting passage that soft decoded audio-video document is pushed to hardware interface delays After depositing, if this section of spatial cache is not consumed in the second preset duration, the content played in passage caching is emptied.
It is preferred that first preset duration is 500 milliseconds to 1 second.
It is preferred that the step A, step B and step C are realized by an independent thread respectively, and three lines Cheng Binghang is run.
It is preferred that the step A is realized using receiving thread, the step A includes:
Step 201:The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step Rapid 205;
Step 202:The agreement of video file is determined, and judges whether video file reads end, if so, return to step 201, otherwise, perform step 204;
Step 204:Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, performing step 206, otherwise perform step 205;
Step 205:Receiving thread dormancy;When file is not run through also, and the writeable capacity of filebuf is more than provisional video File size, receiving thread wake up, and go to step 201;
Step 206:The video file data equal to provisional video file size is read, is written into filebuf In;Using soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected, Draw the numbering of video flowing and audio stream and specific coding information;
Step 207:The writeable capacity of filebuf is subtracted into provisional video file size, as the file buffering after renewal The writeable capacity in area, is then back to step 201.
It is preferred that the step B is realized using transcoding threads, step B includes:
Step 301:Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, step 302 is performed, Otherwise step 303 is performed;Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block and together When be less than the writeable capacity value of transcoding buffering area;
Step 302:Judge whether filebuf is empty, if performing step 303, otherwise performs step 304;
Step 303:Transcoding threads dormancy;If the writeable capacity of transcoding buffering area is more than interim transcoding length, waking up should Thread return to step 301;
Step 304:The audio frequency and video included less than the video file data of interim transcoding length are read from filebuf to compile Code data;
Step 305:Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise Go to step 307;
Step 306:Data after transcoding are write into transcoding buffering area, while the writeable capacity of transcoding buffering area is subtracted into transcoding File size afterwards, as the writeable capacity of transcoding buffering area after renewal;
Step 307:Filebuf removes the data of interim transcoding length being read, at the same increase filebuf can Capacity is write, increasing degree is equal to interim transcoding length.It is then back to step 301.
It is preferred that the step C realizes that step C includes using thread is played:
Step 401:Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403;
Step 402:Thread dormancy will be played, if it is determined that transcoding buffering area does not wake up the thread return to step then for sky 401;
Step 403:From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player;It is described to broadcast Degree of lengthening is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom;
Step 404:Bottom chip carries out audio and video playing to the data after the transcoding that receives;
Step 405:The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
The embodiment of the present application additionally provides a kind of device of audio/video decoding under embedded environment, including receiving module, turns Code module and playing module;
The receiving module is used to obtain audio/video flow data from the external world;
The transcoding module is used to decode the audio/video flow data;The decoding includes:If the audio frequency and video The video container that hardware interface decoding is not supported in flow data be present, it is hardware interface to carry out software transcoding to the video container The video container of support;If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, to the sound It is the coded format that hardware interface can be supported to play that frequency coding, which carries out software transcoding,;
The playing module is used to soft decoded audio-video frequency content being pushed to hardware interface progress audio and video playing.
It is preferred that the playing module further comprises:
First judging unit, it is aobvious in soft decoded audio-video document content in the first preset duration, obtaining Show timestamp;If the Presentation Time Stamp does not change in the first preset duration, stop audio and video playing;
Second judging unit, video theory reproduction time T1 is drawn for after audio and video playing is stopped, obtaining soft decoding And hardware decoding video actual play time T2;T1 times and T2 are contrasted, set in advance if the absolute value of both differences is more than Fixed threshold value, it is erroneous video to mark the audio/video flow data, otherwise, marks the audio/video flow data as that can be regarded with normal play Frequently.
It is preferred that the playing module further comprises:
Processing unit is emptied, the broadcasting passage that hardware interface is pushed to for soft decoded audio-video document caches it Afterwards, if this section of spatial cache is not consumed in the second preset duration, the content played in passage caching is emptied.
It is preferred that first preset duration is 500 milliseconds to 1 second.
It is preferred that the receiving module, transcoding module and playing module are realized by an independent thread respectively, and institute State three thread parallel operations.
It is preferred that the receiving module is realized using receiving thread, the operational process of the receiving module includes:
Step 201:The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step Rapid 205;
Step 202:The agreement of video file is determined, and judges whether video file reads end, if so, return to step 201, otherwise, perform step 204;
Step 204:Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, performing step 206, otherwise perform step 205;
Step 205:Receiving thread dormancy;When file is not run through also, and the writeable capacity of filebuf is more than provisional video File size, receiving thread wake up, and go to step 201;
Step 206:The video file data equal to provisional video file size is read, is written into filebuf In;Using soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected, Draw the numbering of video flowing and audio stream and specific coding information;
Step 207:The writeable capacity of filebuf is subtracted into provisional video file size, as the file buffering after renewal The writeable capacity in area, is then back to step 201.
It is preferred that the transcoding module is realized using transcoding threads, the operational process of transcoding module includes:
Step 301:Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, step 302 is performed, Otherwise step 303 is performed;Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block and together When be less than the writeable capacity value of transcoding buffering area;
Step 302:Judge whether filebuf is empty, if performing step 303, otherwise performs step 304;
Step 303:Transcoding threads dormancy;If the writeable capacity of transcoding buffering area is more than interim transcoding length, wakes up and turn Code line journey return to step 301;
Step 304:The audio frequency and video included less than the video file data of interim transcoding length are read from filebuf to compile Code data;
Step 305:Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise Go to step 307;
Step 306:Data after transcoding are write into transcoding buffering area, while the writeable capacity of transcoding buffering area is subtracted into transcoding File size afterwards, as the writeable capacity of transcoding buffering area after renewal;
Step 307:Filebuf removes the data of interim transcoding length being read, at the same increase filebuf can Capacity is write, increasing degree is equal to interim transcoding length.It is then back to step 301.
It is preferred that the playing module is realized using thread is played,
The operational process of the playing module includes:
Step 401:Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403;
Step 402:Thread dormancy will be played, if it is determined that transcoding buffering area does not play thread return to step for empty then wake up 401;
Step 403:From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player;It is described to broadcast Degree of lengthening is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom;
Step 404:Bottom chip carries out audio and video playing to the data after the transcoding that receives;
Step 405:The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
As can be seen from the above technical solutions, the application scheme first step is to the soft solution of audio-video document, the benefit so done It is:It is the video container that hardware interface is supported that the video container that hardware interface is not supported, which carries out software de-multiplex, and will be hard It is the coded format that hardware interface can be supported to play that the audio coding that part interface is not supported, which carries out software transcoding,;Second step will pass through The soft decoded video content of the first step is pushed to hardware interface playing audio-video.Application scheme also passes through video playback mistake Restoration Mechanism is detected, with a certain degree of error correction audio frequency and video can be allowed to continue to play wrong audio frequency and video.Even if audio-video frequency content exists Soft solution can not error correction situation, can also program recovery is come, continue to play next video.Application scheme can be well Extend stream media protocol (stream media protocol includes but is not limited to http, udp, rtsp agreements etc.).In addition, the application is implemented Example realized using multithreading, can be made more smooth during broadcasting video and stably.Application scheme can protect Stay in hardware interface advantage, support more video containers and audio coding, while using the automatic error-correcting mechanism of video playback And Restoration Mechanism, improve and play video playback fluency and robustness.
Brief description of the drawings
Fig. 1 is the structural representation of the device of the soft decoding of video under the embedded environment that the embodiment of the present application provides;
Fig. 2 is the operational process schematic diagram that the receiving module 101 that the embodiment of the present application provides uses receiving thread;
Fig. 3 is operational process schematic diagram of the embodiment of the present application transit code module 103 using transcoding threads;
Fig. 4 is that playing module 105 uses the operational process schematic diagram for playing thread in the embodiment of the present application;
Fig. 5 is that playing module 105 carries out error detection and the handling process schematic diagram recovered;
Fig. 6 is the method flow schematic diagram of audio/video decoding under a kind of embedded environment that the embodiment of the present application provides.
Embodiment
For problems of the prior art, present applicant proposes a kind of solution of the soft decoding of video, wherein wrapping Include:Soft solution+hardware interface video playback mechanism+video playback error detection Restoration Mechanism.Following examples are with playpump It is described in detail exemplified by interface.
The structure of the device of the soft decoding of video is as shown in figure 1, receive mould under the embedded environment that the embodiment of the present application provides Block 101 is used to realize multi-media protocol (various protocols), from external world's acquisition audio/video flow file data and the audio frequency and video are literary Number of packages evidence is cached in filebuf 102.Transcoding module 103 reads audio-video document data from filebuf 102, right The video container that hardware interface decoding is not supported carries out software de-multiplex, and the audio coding do not supported hardware interface decoding is carried out Software transcoding, while by the audio-video document content push transcoding buffering area 104 after soft solution.Playing module 105 be used for from turn The audio-video document content after transcoding is read in bitstream buffer 105, data-pushing then is carried out into video to hardware interface again broadcasts Put.Playing module 105 is additionally operable to carry out error detection during playing video file and and mistake recovered.
In order that more smooth and stably during playing video, another embodiment of the application is using multithreading collaboration work The mode of work realizes receiving module 101, transcoding module 103 and playing module 105.Specifically, it is real with (read) thread is received Existing receiving module 101, transcoding (transcoder) thread realize transcoding module 103, play (pushdata) _ thread and realize broadcasting Module 105, the flow that thread specifically cooperates is as shown in Figure 2, Figure 3 and Figure 4.
Fig. 2 is that the receiving module 101 that the embodiment of the present application provides realizes that stream media protocol receives video using read threads File content.The stream media protocol includes but is not limited to HTTP (HTTP), UDP (UDP, User Datagram Protocol), real time streaming transport protocol (RTSP, Real Time Streaming Protocol).This Outside, the embodiment of the present application also supports that audio frequency and video are locally stored.
Receiving module 101 uses the operational process of receiving thread as shown in Fig. 2 including following step in the embodiment of the present application Suddenly:
Step 201:The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step Rapid 205.
Step 202:The agreement of video file is determined, and judges whether video file reads end, if so, return to step 201, otherwise, perform step 204.
Step 204:Judge whether the writeable capacity of filebuf is more than provisional video file size (fileTempSize), If so, performing step 206, step 205 is otherwise performed.FileTemSize is a value fixed.
Step 205:Read thread dormancy.When file is not run through also, and the writeable capacity of filebuf is more than provisional video File size, read thread wakenings, goes to step 201.
Step 206:The video file data equal to provisional video file size is read, is written into filebuf In.Using (ffmpeg) soft decoding technique, the video flowing in the video file data that is read and audio stream information are divided Analysis and detection, draw the numbering of video flowing and audio stream and specific coding information.
Step 207:The writeable capacity of filebuf is subtracted into provisional video file size, as the file buffering after renewal The writeable capacity in area, is then back to step 201.
The embodiment of the present application transit code module 103 is using the operational process of transcoding threads as shown in figure 3, including following step Suddenly:
Step 301:Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length (transTempSize), if It is to perform step 302, otherwise performs step 303.TransTempSize is a value fixed, and inputs and buffers less than or equal to file The readable capacity in area is while be less than the writeable capacity value of transcoding buffering area.
Step 302:Judge whether filebuf is empty, if performing step 303, otherwise performs step 304.
Step 303:Transcoding threads dormancy.If the writeable capacity of transcoding buffering area is more than interim transcoding length, waking up should Thread return to step 301.
Step 304:The audio frequency and video included less than the video file data of interim transcoding length are read from filebuf to compile Code data.
Step 305:Soft decoding is carried out to the audio/video coding data read.If it is successful, step 306 is performed, otherwise Go to step 307.
In transcoding process, if bottom chip hard decoder directly supports video container, the video volume belonging to video file Code and audio coding, are demultiplexed to video content.What if the form or audio coding of video file had not been supported, it is right Audio-video document carries out turning encapsulation, and audio be transcoded into the coded format of hard decoder support, while video file is turned Encapsulation, and the video file after transcoding be pushed in transcoding buffering area.
Step 306:Data after transcoding are write into transcoding buffering area, while the writeable capacity of transcoding buffering area is subtracted into transcoding File size afterwards, as the writeable capacity of transcoding buffering area after renewal.
Step 307:Filebuf removes the data of interim transcoding length being read, at the same increase filebuf can Capacity is write, increasing degree is equal to interim transcoding length.It is then back to step 301.
In transcoding process, video is demultiplexed and the audio to not supporting is decoded and encoded.So doing can To support the video format of most of main flow and coding.It may further include during soft decoding:Detection and discarding regard The bad frame of frequency file content, while the synchronizing information of audio frequency and video is adjusted, decoding error can be so detected, and video is carried out A certain degree of error correction.During decoding, it is irrecoverable if it happens can not to read video flowing and audio stream packet etc. Mistake, then stop playing video, and state be set to the state for waiting next video playback.
In the embodiment of the present application, the content of the video file after the acquisition software processing of playing module 105, chip bottom is pushed to Layer plays out.Playing module 105 is using the operational process of broadcasting thread as shown in figure 4, comprising the following steps:
Step 401:Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403.
Step 402:Thread dormancy will be played.If it is determined that transcoding buffering area does not wake up the thread return to step then for sky 401。
Step 403:From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player.
Playout length (playsize) is a fixed value, and playsize is less than buffering area data with existing length after transcoding, together When playSize be less than bottom can accommodate data.
Step 404:Bottom chip carries out audio and video playing to the data after the transcoding that receives.
Step 405:The increase of transcoding buffering area active volume, increasing degree is playout length.It is then back to step 401.
In the embodiment of the present application, playing module 105 is also equipped with carrying out error detection and extensive during playing video file The system of answering a pager's call.By soft solution, the specifying information of video file sound intermediate frequency and video has been obtained, according to video concrete composition part, has been adopted Error detection and recovery are carried out with flow as shown in Figure 5, is specifically comprised the following steps:
Step 501:After soft solution is carried out to video file and obtains audio-video document data, it is pushed to hardware interface and is broadcast Put, judge whether playing duration is less than preset duration (PTS_WAIT_TIME), its value scope is 500 milliseconds to 1 second, if so, holding Row step 502, otherwise perform step 510.
Step 502:Judge the composition of the audio-video document, if comprising video flowing, step 503 is performed, if only including audio Stream, perform step 504.
Step 503:The Presentation Time Stamp (pts) of video render is obtained, then performs step 505.
Step 504:The pts of audio stream broadcasting is obtained, then performs step 505.
Step 505:Judge whether pts is changed, if so, explanation is interior in preset duration (PTS_WAIT_TIME), depending on Frequency stream broadcasting is normal, without carrying out correction process, return to step 501, otherwise, performs step 506.
Step 506:Audio frequency and video stop playing.
Step 507:Obtain soft decoding and draw video theory reproduction time T1, hardware decoding video actual play time T2. T1 times and T2 are contrasted, if the absolute value of both differences is more than threshold value set in advance (Max), step 508 is performed, otherwise holds Row step 509.
Step 508:It is erroneous video to mark the video, and terminates this flow.
Step 509:Marking the video to be can be with normal play video, and terminates this flow.
Step 510:Audio-video document is pushed to after bottom chip broadcasting passage caching, is judged in certain time (FAILED_TIME) in, whether this section of spatial cache is consumed, if so, performing step 501;Otherwise, step 511 is performed.
Step 511:Empty the content played in passage caching.This is done to exist not in automatic shield video file Bad frame content can be recovered, continue to play subsequent video content.
Step 512:Statistics obtains bottom chip and plays the count value that the size that passage caches is continuously 0.If video is literary Part pushes successfully, COUNT clear 0.
Step 513:Judge whether the count value is more than threshold value set in advance (FAILED_COUNT), if so, returning Step 502;Otherwise, it is soft to solve next section of video flowing and perform above-mentioned flow.
Advantage of this is that can be during video file content is played by soft decoded video, hardware interface In the case of exception can not play, the situation of mistake generation can be detected in time.
The method that the embodiment of the present application additionally provides audio frequency and video transcoding under a kind of embedded environment, comprises the following steps:
Audio/video flow data are obtained, transcoding is carried out to the audio/video flow data;The transcoding includes:If the audio frequency and video The video container that hardware interface decoding is not supported in flow data be present, it is hardware interface to carry out software transcoding to the video container The video container of support;If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, to the sound It is the coded format that hardware interface can be supported to play that frequency coding, which carries out software transcoding,.
The method that the embodiment of the present application additionally provides audio/video decoding under a kind of embedded environment, as shown in fig. 6, including:
Step 601:Audio/video flow data are obtained from the external world;
Step 602:Soft decoding is carried out to the audio/video flow data;The soft decoding includes:If the audio frequency and video fluxion According to the middle video container that hardware interface decoding be present and do not support, software transcoding is carried out to the video container and supported for hardware interface Video container;If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, the audio is compiled It is the coded format that hardware interface can be supported to play that code, which carries out software transcoding,;
Step 603:Soft decoded audio-video frequency content is pushed to hardware interface and carries out audio and video playing.
According to another embodiment of the application, step C further comprises:
C1, in the first preset duration, obtain the Presentation Time Stamp in soft decoded audio-video frequency content;If the display Timestamp does not change in the first preset duration, stops audio and video playing;
C2, the soft decoding of acquisition draw video theory reproduction time T1 and hardware decoding video actual play time T2;Will T1 times and T2 contrasts, if the absolute value of both differences is more than threshold value set in advance, it is mistake to mark the audio/video flow data Video, otherwise, marking the audio/video flow data to be can be with normal play video.
According to another embodiment of the application, step C further comprises:Soft decoded audio-video document is pushed to hardware and connect After the broadcasting passage caching of mouth, if this section of spatial cache is not consumed in the second preset duration, broadcasting passage is emptied Content in caching.
According to another embodiment of the application, first preset duration is 500 milliseconds to 1 second.
According to another embodiment of the application, the step A, step B and step C are real by an independent thread respectively It is existing, and three thread parallels are run.
According to another embodiment of the application, the step A is realized using receiving thread, as shown in Fig. 2 the step A bags Include:
Step 201:The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step Rapid 205;
Step 202:The agreement of video file is determined, and judges whether video file reads end, if so, return to step 201, otherwise, perform step 204;
Step 204:Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, performing step 206, otherwise perform step 205;
Step 205:Receiving thread dormancy;When file is not run through also, and the writeable capacity of filebuf is more than provisional video File size, receiving thread wake up, and go to step 201;
Step 206:The video file data equal to provisional video file size is read, is written into filebuf In;Using soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected, Draw the numbering of video flowing and audio stream and specific coding information;
Step 207:The writeable capacity of filebuf is subtracted into provisional video file size, as the file buffering after renewal The writeable capacity in area, is then back to step 201.
According to another embodiment of the application, the step B is realized using transcoding threads, as shown in figure 3, step B includes:
Step 301:Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, step 302 is performed, Otherwise step 303 is performed;Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block and together When be less than the writeable capacity value of transcoding buffering area;
Step 302:Judge whether filebuf is empty, if performing step 303, otherwise performs step 304;
Step 303:Transcoding threads dormancy;If the writeable capacity of transcoding buffering area is more than interim transcoding length, waking up should Thread return to step 301;
Step 304:The audio frequency and video included less than the video file data of interim transcoding length are read from filebuf to compile Code data;
Step 305:Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise Go to step 307;
Step 306:Data after transcoding are write into transcoding buffering area, while the writeable capacity of transcoding buffering area is subtracted into transcoding File size afterwards, as the writeable capacity of transcoding buffering area after renewal;
Step 307:Filebuf removes the data of interim transcoding length being read, at the same increase filebuf can Capacity is write, increasing degree is equal to interim transcoding length.It is then back to step 301.
According to another embodiment of the application, the step C is realized using thread is played.As shown in figure 4, the step C bags Include:
Step 401:Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403;
Step 402:Thread dormancy will be played, if it is determined that transcoding buffering area does not wake up the thread return to step then for sky 401;
Step 403:From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player;It is described to broadcast Degree of lengthening is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom;
Step 404:Bottom chip carries out audio and video playing to the data after the transcoding that receives;
Step 405:The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
The foregoing is only the preferred embodiment of the application, not to limit the protection domain of the application, it is all Within the spirit and principle of technical scheme, any modification, equivalent substitution and improvements done etc., this Shen should be included in Within the scope of please protecting.

Claims (12)

1. a kind of method of audio/video decoding under embedded environment, it is characterised in that comprise the following steps:
A, audio/video flow data are obtained from the external world;
B, soft decoding is carried out to the audio/video flow data;The soft decoding includes:If exist in the audio/video flow data hard The video container that part interface decoding is not supported, it is that the video that hardware interface is supported holds that software transcoding is carried out to the video container Device;If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, the audio coding is carried out soft Part transcoding is the coded format that hardware interface can be supported to play;
C, soft decoded audio-video frequency content is pushed to hardware interface and carries out audio and video playing;
The step A, step B and step C are realized by an independent thread respectively, and three thread parallels are run;
The step A realizes that the step A includes using receiving thread:
Step 201:The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step 205;
Step 202:The agreement of video file is determined, and judges whether video file reads end, if so, return to step 201, Otherwise, step 204 is performed;
Step 204:Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, step 206 is performed, Otherwise step 205 is performed;
Step 205:Receiving thread dormancy;When file is not run through also, and the writeable capacity of filebuf is more than provisional video file Length, receiving thread wake up, and go to step 201;
Step 206:The video file data equal to provisional video file size is read, is written into filebuf;Adopt With soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected, is drawn The numbering and specific coding information of video flowing and audio stream;
Step 207:The writeable capacity of filebuf is subtracted into provisional video file size, can as the filebuf after renewal Capacity is write, is then back to step 201.
2. according to the method for claim 1, it is characterised in that step C further comprises:
C1, in the first preset duration, obtain the Presentation Time Stamp in soft decoded audio-video frequency content;If the display time Stamp does not change in the first preset duration, stops audio and video playing;
C2, the soft decoding of acquisition draw video theory reproduction time T1 and hardware decoding video actual play time T2;During by T1 Between and T2 contrasts, if both absolute values of difference are more than threshold value set in advance, it is erroneous video to mark the audio/video flow data, Otherwise, marking the audio/video flow data to be can be with normal play video.
3. according to the method for claim 2, it is characterised in that step C further comprises:Soft decoded audio-video document It is pushed to after the broadcasting passage caching of hardware interface, if this section of spatial cache is not consumed in the second preset duration, Empty the content played in passage caching.
4. according to the method for claim 2, it is characterised in that first preset duration is 500 milliseconds to 1 second.
5. according to the method for claim 1, it is characterised in that the step B realizes that step B includes using transcoding threads:
Step 301:Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, performing step 302, otherwise Perform step 303;Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block while small In the writeable capacity value of transcoding buffering area;
Step 302:Judge whether filebuf is empty, if performing step 303, otherwise performs step 304;
Step 303:Transcoding threads dormancy;If the writeable capacity of transcoding buffering area is more than interim transcoding length, the thread is waken up Return to step 301;
Step 304:The audio/video coding number included less than the video file data of interim transcoding length is read from filebuf According to;
Step 305:Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise gone to Step 307;
Step 306:Data after transcoding are write into transcoding buffering area, while after the writeable capacity of transcoding buffering area is subtracted into transcoding File size, as the writeable capacity of transcoding buffering area after renewal;
Step 307:Filebuf removes the data for the interim transcoding length being read, while increases the writeable appearance of filebuf Amount, increasing degree are equal to interim transcoding length.It is then back to step 301.
6. according to the method for claim 1, it is characterised in that the step C realizes that step C includes using thread is played:
Step 401:Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403;
Step 402:Thread dormancy will be played, if it is determined that transcoding buffering area does not wake up the thread return to step 401 then for sky;
Step 403:From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player;It is described to play length Degree is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom;
Step 404:Bottom chip carries out audio and video playing to the data after the transcoding that receives;
Step 405:The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
7. the device of audio/video decoding under a kind of embedded environment, it is characterised in that including receiving module, transcoding module and broadcast Amplification module;
The receiving module is used to obtain audio/video flow data from the external world;
The transcoding module is used to decode the audio/video flow data;The decoding includes:If the audio frequency and video fluxion According to the middle video container that hardware interface decoding be present and do not support, software transcoding is carried out to the video container and supported for hardware interface Video container;If the audio coding that hardware interface decoding is not supported in the audio/video flow data be present, the audio is compiled It is the coded format that hardware interface can be supported to play that code, which carries out software transcoding,;
The playing module is used to soft decoded audio-video frequency content being pushed to hardware interface progress audio and video playing;
The receiving module, transcoding module and playing module are realized by an independent thread respectively, and three threads Parallel operation;
The receiving module realizes that the operational process of the receiving module includes using receiving thread:
Step 201:The url address resolution that video file be present is detected whether, if so, performing step 202, otherwise goes to step 205;
Step 202:The agreement of video file is determined, and judges whether video file reads end, if so, return to step 201, Otherwise, step 204 is performed;
Step 204:Judge whether the writeable capacity of filebuf is more than provisional video file size, if so, step 206 is performed, Otherwise step 205 is performed;
Step 205:Receiving thread dormancy;When file is not run through also, and the writeable capacity of filebuf is more than provisional video file Length, receiving thread wake up, and go to step 201;
Step 206:The video file data equal to provisional video file size is read, is written into filebuf;Adopt With soft decoding technique, the video flowing in the video file data that is read and audio stream information are analyzed and detected, is drawn The numbering and specific coding information of video flowing and audio stream;
Step 207:The writeable capacity of filebuf is subtracted into provisional video file size, can as the filebuf after renewal Capacity is write, is then back to step 201.
8. device according to claim 7, it is characterised in that the playing module further comprises:
First judging unit, in the first preset duration, when obtaining the display in soft decoded audio-video document content Between stab;If the Presentation Time Stamp does not change in the first preset duration, stop audio and video playing;
Second judging unit, for after audio and video playing is stopped, obtaining soft decoding draw video theory reproduction time T1 and Hardware decoding video actual play time T2;T1 times and T2 are contrasted, if the absolute value of both differences is more than set in advance Threshold value, it is erroneous video to mark the audio/video flow data, and otherwise, marking the audio/video flow data to be can be with normal play video.
9. device according to claim 8, it is characterised in that the playing module further comprises:
Processing unit is emptied, after being pushed to the broadcasting passage caching of hardware interface for soft decoded audio-video document, if This section of spatial cache is not consumed in the second preset duration, then empties the content played in passage caching.
10. according to the method for claim 7, it is characterised in that first preset duration is 500 milliseconds to 1 second.
11. device according to claim 7, it is characterised in that the transcoding module is using transcoding threads realization, transcoding mould The operational process of block includes:
Step 301:Judge whether the writeable capacity of transcoding buffering area is more than interim transcoding length, if so, performing step 302, otherwise Perform step 303;Interim transcoding length is a value fixed, less than or equal to the readable capacity in file input block while small In the writeable capacity value of transcoding buffering area;
Step 302:Judge whether filebuf is empty, if performing step 303, otherwise performs step 304;
Step 303:Transcoding threads dormancy;If the writeable capacity of transcoding buffering area is more than interim transcoding length, transcoding line is waken up Journey return to step 301;
Step 304:The audio/video coding number included less than the video file data of interim transcoding length is read from filebuf According to;
Step 305:Soft decoding is carried out to the audio/video coding data read, if it is successful, performing step 306, otherwise gone to Step 307;
Step 306:Data after transcoding are write into transcoding buffering area, while after the writeable capacity of transcoding buffering area is subtracted into transcoding File size, as the writeable capacity of transcoding buffering area after renewal;
Step 307:Filebuf removes the data for the interim transcoding length being read, while increases the writeable appearance of filebuf Amount, increasing degree are equal to interim transcoding length.It is then back to step 301.
12. device according to claim 7, it is characterised in that the playing module, which uses, plays thread realization,
The operational process of the playing module includes:
Step 401:Judge whether transcoding buffering area is empty, if performing step 402, otherwise performs step 403;
Step 402:Thread dormancy will be played, if it is determined that transcoding buffering area does not play thread return to step 401 for empty then wake up;
Step 403:From the data-pushing after the transcoding of transcoding buffering area reading playout length to bottom player;It is described to play length Degree is less than buffering area data with existing length after transcoding, while can accommodate data less than bottom;
Step 404:Bottom chip carries out audio and video playing to the data after the transcoding that receives;
Step 405:The increase of transcoding buffering area active volume, increasing degree is playout length, is then back to step 401.
CN201410668321.9A 2014-11-20 2014-11-20 The method and device of audio/video decoding under a kind of embedded environment Active CN104394456B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410668321.9A CN104394456B (en) 2014-11-20 2014-11-20 The method and device of audio/video decoding under a kind of embedded environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410668321.9A CN104394456B (en) 2014-11-20 2014-11-20 The method and device of audio/video decoding under a kind of embedded environment

Publications (2)

Publication Number Publication Date
CN104394456A CN104394456A (en) 2015-03-04
CN104394456B true CN104394456B (en) 2017-12-01

Family

ID=52612284

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410668321.9A Active CN104394456B (en) 2014-11-20 2014-11-20 The method and device of audio/video decoding under a kind of embedded environment

Country Status (1)

Country Link
CN (1) CN104394456B (en)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106384596A (en) * 2016-09-22 2017-02-08 努比亚技术有限公司 Audio data processing method and terminal
CN106658117A (en) * 2016-12-30 2017-05-10 百度在线网络技术(北京)有限公司 Method and device for processing audio/video data
CN106604115B (en) * 2016-12-30 2020-08-14 深圳Tcl新技术有限公司 Video playing control device and method
CN106792124A (en) * 2016-12-30 2017-05-31 合网络技术(北京)有限公司 Multimedia resource decodes player method and device
CN108683946B (en) * 2018-05-23 2020-10-27 成都智达万应科技有限公司 Method for realizing online video education based on face recognition and cache mechanism
EP3818722A1 (en) * 2018-07-05 2021-05-12 Dolby International AB Processing media data structures
CN110858827B (en) * 2018-08-22 2023-03-24 浙江宇视科技有限公司 Broadcast starting acceleration method and device and computer readable storage medium
CN110858829B (en) * 2018-08-22 2023-02-07 杭州海康威视数字技术股份有限公司 Video processing method, device and system, analysis server and transcoding server
CN109088887A (en) * 2018-09-29 2018-12-25 北京金山云网络技术有限公司 A kind of decoded method and device of Streaming Media
CN109151574B (en) 2018-10-15 2020-03-24 Oppo广东移动通信有限公司 Video processing method, video processing device, electronic equipment and storage medium
CN109257646B (en) * 2018-10-15 2020-09-22 Oppo广东移动通信有限公司 Video processing method and device, electronic equipment and computer readable medium
CN111147926B (en) * 2018-11-02 2022-05-06 杭州海康威视数字技术股份有限公司 Data transcoding method and device
CN110351600A (en) * 2019-01-21 2019-10-18 福州友宝电子科技有限公司 A kind of method and device playing video
CN114760495B (en) * 2022-03-21 2024-07-16 南京芯驰半导体科技有限公司 Video hardware decoding method and system based on video processing unit
CN114866836A (en) * 2022-04-12 2022-08-05 北京小米移动软件有限公司 Video processing method and device, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101951517A (en) * 2010-09-27 2011-01-19 深圳市融创天下科技发展有限公司 Method, system and terminal equipment for decoding and playing video
CN102811383A (en) * 2012-07-06 2012-12-05 深圳市同洲电子股份有限公司 Video file playing method and device based on set top box
CN103024535A (en) * 2012-12-25 2013-04-03 北京奇虎科技有限公司 Video playing method, server and video playing system
CN103747317A (en) * 2013-12-27 2014-04-23 高新兴科技集团股份有限公司 A smooth playing control method of a play database in an instable network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101951517A (en) * 2010-09-27 2011-01-19 深圳市融创天下科技发展有限公司 Method, system and terminal equipment for decoding and playing video
CN102811383A (en) * 2012-07-06 2012-12-05 深圳市同洲电子股份有限公司 Video file playing method and device based on set top box
CN103024535A (en) * 2012-12-25 2013-04-03 北京奇虎科技有限公司 Video playing method, server and video playing system
CN103747317A (en) * 2013-12-27 2014-04-23 高新兴科技集团股份有限公司 A smooth playing control method of a play database in an instable network

Also Published As

Publication number Publication date
CN104394456A (en) 2015-03-04

Similar Documents

Publication Publication Date Title
CN104394456B (en) The method and device of audio/video decoding under a kind of embedded environment
KR101107815B1 (en) Media stream recording into a reception hint track of a multimedia container file
CN103338386B (en) Based on the audio and video synchronization method simplifying timestamp
WO2007120261A3 (en) Expedited digital signal decoding
CN101193311B (en) Audio and video data synchronization method in P2P system
CN103686315A (en) Synchronous audio and video playing method and device
JP2012532568A5 (en)
CN102264004A (en) Method and device for preventing deficiency of key frame from causing seek incapability
WO2007090988A3 (en) Method and device for the hierarchical coding of a source audio signal and corresponding decoding method and device, programs and signal
EP1195996A3 (en) Apparatus, method and computer program product for decoding and reproducing moving images, time control method and multimedia information receiving apparatus
CN101442675A (en) Multimedia play terminal and play method
JP2004534484A5 (en)
TW200623881A (en) Method of decoding digital video and digital video decoder system thereof
CN104853132A (en) Delay video recording method and system
CN102254560B (en) Audio processing method in mobile digital television recording
CN106878290B (en) Audio and video playing method and device
EP1589770A3 (en) Apparatus and method for decoding a moving picture sequence
CN104023260B (en) Hardware decoding realization method, device and player
WO2007125458A3 (en) Method and apparatus for encoding/transcoding and decoding
JP2006319956A5 (en)
CN105657448B (en) A kind of retransmission method, the apparatus and system of encoded video stream
JP2009171294A5 (en)
CN101102485A (en) Audio and video synchronization device and method of mobile terminal
TW200625953A (en) Video reproducing apparatus
WO2009143260A3 (en) Apparatus and method for recording video data

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant