CN106878290B - Audio and video playing method and device - Google Patents
Audio and video playing method and device Download PDFInfo
- Publication number
- CN106878290B CN106878290B CN201710047508.0A CN201710047508A CN106878290B CN 106878290 B CN106878290 B CN 106878290B CN 201710047508 A CN201710047508 A CN 201710047508A CN 106878290 B CN106878290 B CN 106878290B
- Authority
- CN
- China
- Prior art keywords
- format
- multimedia data
- decoding
- coding
- audio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4398—Processing of audio elementary streams involving reformatting operations of audio signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/485—End-user interface for client configuration
- H04N21/4858—End-user interface for client configuration for modifying screen layout parameters, e.g. fonts, size of the windows
Abstract
The invention discloses a method and a device for playing audio and video, wherein the method comprises the following steps: adding a description index of format information into the coding and decoding information of each coded data packet; detecting the coding and decoding formats of the currently received multimedia data packets in real time in the playing process; when the coding and decoding format of the currently received multimedia data packet is different from the coding and decoding format of the multimedia data packet being played, waiting for the completion of the playing of the multimedia data in the former coding and decoding format, and simultaneously preparing a decoder in a new coding format, decoding and outputting the currently received multimedia data packet; and rendering the decoded multimedia data and outputting the rendered multimedia data for playing. The invention can realize seamless continuous playing during audio/video stream switching and improve user experience.
Description
Technical Field
The invention relates to the technical field of audio and video processing, in particular to an audio and video playing method and device.
Background
At present, a common video player only judges the type of a decoder of a multimedia stream and initializes the playing when the playing is started, and cannot judge whether the coding format of the data is changed and timely change the decoder to continue the playing in the playing process. Once the encoding format changes during the playing process, for example, the same movie is switched from h.264 format to h.265 format, or the audio is switched from normal audio to dolby audio, the player needs to be shut down and a new playing flow needs to be restarted, which causes playing interruption and poor user experience.
Disclosure of Invention
The invention mainly aims to provide an audio and video playing method, and aims to solve the technical problem that playing is interrupted due to the fact that a player needs to be closed and a new playing process needs to be restarted in the prior art.
In order to achieve the above object, the present invention provides an audio/video playing method, which comprises the following steps:
adding format information description index into the coding and decoding information of each multimedia data packet;
detecting the coding and decoding format of the currently received multimedia data packet in the playing process;
when the coding and decoding format of the currently received multimedia data packet is different from the coding and decoding format of the multimedia data packet being played, waiting for the completion of the playing of the multimedia data in the former coding and decoding format, starting a decoder in the new coding format, and decoding and outputting the currently received multimedia data packet;
and rendering the decoded multimedia data and outputting the rendered multimedia data for playing.
Preferably, when the codec format of the currently received multimedia data packet is different from the codec format of the multimedia data packet being played, the step of waiting for the completion of the playing of the multimedia data in the previous codec format and enabling the decoder in the new codec format to decode and output the currently received multimedia data packet includes:
after a multimedia data packet is obtained, whether the coding and decoding information of the multimedia data packet is consistent with the coding and decoding information of the previous multimedia data packet is detected, if not, after the multimedia data in the previous coding and decoding format is played, the adaptive decoder is switched according to the format information description index in the coding and decoding information, and the currently received multimedia data packet is decoded and output by the decoder.
Preferably, the rendering the decoded data and outputting to a playing device for playing includes:
and when detecting that the data format and the picture size of the current video frame are inconsistent with those of the previous video frame, initializing the mapping data format of the renderer to match the coding and decoding format and size of the current frame.
Preferably, the rendering the decoded data and outputting to a playing device for playing includes:
and when the encoding and decoding format of the current audio data frame is detected to be inconsistent with the previous frame, creating an audio output device matched with the current audio data frame.
Preferably, the multimedia data packets include h.264 format, h.265 format audio-video data packets and dolby audio data packets.
The invention also provides an audio and video playing device, which comprises:
the audio and video data description module is used for adding a format information description index into the coding and decoding information of each multimedia data packet;
the detection module is used for detecting the coding and decoding formats of the currently received multimedia data packets in the playing process;
the audio/video decoding module is used for waiting for the completion of the playing of the multimedia data in the former coding and decoding format when the coding and decoding format of the currently received multimedia data packet is different from the coding and decoding format of the multimedia data packet being played, starting a decoder in the new coding format, and decoding and outputting the currently received multimedia data packet;
and the audio/video rendering module is used for rendering the decoded multimedia data and outputting the rendered multimedia data for playing.
Preferably, the audio/video decoding module is configured to:
after a multimedia data packet is obtained, whether the coding and decoding information of the multimedia data packet is consistent with the coding and decoding information of the previous multimedia data packet is detected, if not, after the multimedia data in the previous coding and decoding format is played, the adaptive decoder is switched according to the format information description index in the coding and decoding information, and the currently received multimedia data packet is decoded and output by the decoder.
Preferably, the audio/video rendering module is configured to:
and when detecting that the data format and the picture size of the current video frame are inconsistent with those of the previous video frame, initializing the mapping data format of the renderer to match the current frame.
Preferably, the audio/video rendering module is configured to: and when the encoding and decoding format of the audio data frame is detected to be inconsistent with the previous frame, creating an audio output device matched with the current audio data frame.
Preferably, the multimedia data packets include h.264 format, h.265 format audio-video data packets and dolby audio data packets.
The audio and video playing method provided by the invention can detect the change of the coding format of the multimedia stream in the playing process, change the coding format into the corresponding format decoder, and change the audio and video renderer with the corresponding format, so that seamless continuous playing can be realized without quitting playing.
Drawings
In order to clearly understand the technical solution of the present invention, the drawings used in describing the embodiments of the present invention will be briefly described below. It is obvious that these drawings are only some embodiments of the invention, and that a person skilled in the art will be able to obtain further drawings without the inventive step.
Fig. 1 is a schematic flow chart of steps of an audio/video playing method according to the present invention;
fig. 2 is a schematic structural diagram of an audio/video playing device according to an embodiment of the present invention.
Detailed Description
The objects, features and advantages of the present invention will be further explained with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides an audio and video playing method, and referring to fig. 1, in an embodiment, the audio and video playing method includes the following steps:
step S10, adding the description index of the format information into the coding and decoding information of each multimedia data packet;
in order to detect the codec format of the current multimedia stream data, when the multimedia stream is separated (for example, audio data packets are separated from video data packets), a description index of format information is added to each coded data packet to point to codec related information corresponding to the current data, so that it is ensured that the decoding information corresponding to the data can be read from the received data packet.
Step S20, detecting the coding and decoding format of the multimedia data packet received currently in real time during the playing process; during the playing process, it is necessary to be able to detect the codec format of the currently received multimedia data stream in time, rather than performing a check only once at the beginning of the playing.
Step S30, when the coding and decoding format of the currently received multimedia data packet is different from the coding and decoding format of the multimedia data packet being played, waiting for the completion of the playing of the multimedia data in the former coding and decoding format, starting a decoder in the new coding format, and decoding and outputting the currently received multimedia data packet; when a new-format multimedia data packet is detected, waiting for the completion of the playing of the data in the former encoding and decoding format, and simultaneously preparing a decoder in the new encoding format to decode the data in the new format.
In the embodiment of the invention, the data output by the decoder is also added with the format information, so that the decoding process and the rendering part can process the data relatively independently.
Step S40, rendering the decoded multimedia data and outputting for playing.
The decoded data format may not be consistent with the previous decoded data format, and in order to achieve the effect of continuous playing without perception by the user, the problem of seamlessly outputting different formats to the display device and the audio device needs to be solved. When the change of the coding format of the multimedia stream is detected, the invention renders the coded multimedia stream in the aspects of picture size and the like so as to match the current multimedia stream with the multimedia data in the former coding and decoding format and realize seamless continuous playing.
The audio and video playing method provided by the invention can detect the change of the coding format of the multimedia stream in the playing process, change the coding format into the corresponding format decoder, and change the audio and video renderer with the corresponding format, so that seamless continuous playing can be realized without quitting playing.
In a preferred embodiment of the present invention, the step 30 may include:
after the multimedia data packet is obtained, whether the coding and decoding information of the multimedia data packet is consistent with the coding and decoding information of the previous multimedia data packet is detected, if not, the index is described according to the format information in the coding and decoding information, and the adaptive decoder is switched to perform decoding output.
In one embodiment, the step S40 includes:
when the data format and the picture size of the current video frame are detected to be inconsistent with those of the previous video frame, the charting data format of the renderer is initialized, and the coding and decoding formats of the current frame are matched. For the video renderer module, after the decoded data frame is read, whether the data format and the picture size of the previous video frame are consistent with the current frame or not is checked, if the data format and the picture size of the previous video frame are different from the current frame, the mapping data format of the renderer needs to be initialized again, the coding and decoding format and the size of the current frame are matched, and correct connection with the previous frame is guaranteed.
In an embodiment, the step S40 further includes:
and when the coding and decoding formats of the current audio data frame are detected to be inconsistent with those of the previous frame, the audio output device is matched with the current audio data frame. And the audio renderer part needs to re-create an output device when detecting that the coding and decoding formats of the audio data frames are inconsistent with those of the previous audio data frames, so that the sampling rate and the channel number of the output device can be matched with the new audio data frames, and the correct connection of the previous section of audio playing is ensured.
Through the process, seamless continuous playing can be smoothly realized for the change of the data stream format in the playing process without restarting playing.
It should be noted that, in the embodiment of the present invention, the multimedia data packet may be an audio/video data packet in an h.264 format or an h.265 format; and common audio data packets and dolby audio data packets, and audio and video data packets in other formats.
The present invention further provides an audio/video playing device, as shown in fig. 2, in an embodiment, the device includes:
the audio and video data description module 10 is configured to add a description index of format information of the format information to the coding and decoding information of each coded data packet;
in order to detect the codec format of the current multimedia stream data, when the multimedia stream is separated (for example, the audio data packet is separated from the video data packet), the audio/video data description module 10 adds a description index of format information to each coded data packet, and points to the codec related information corresponding to the current data, so as to ensure that the decoding information corresponding to the data can be read from the received data packet.
A detecting module 20, configured to detect, in real time, an encoding/decoding format of a currently received multimedia data packet during a playing process;
during the playing process, the detection module 20 detects the codec format of the currently received multimedia data stream in real time, instead of checking only once when the playing starts.
The audio/video decoding module 30 is configured to wait for the multimedia data in the previous codec format to be played when the codec format of the currently received multimedia data packet is different from the codec format of the currently played multimedia data packet, and start a decoder in a new codec format to decode and output the currently received multimedia data packet;
when the audio/video decoding module 30 detects a multimedia data packet with a new format, it waits for the completion of the playing of the data in the previous encoding and decoding format, and prepares a decoder with the new encoding format to decode the data in the new format. In the embodiment of the present invention, the data output by the audio/video decoding module 30 is also appended with format information, so that the decoding process and the rendering part can process data relatively independently.
And an audio/video rendering module 40, configured to render the decoded multimedia data and output the rendered multimedia data for playing.
The decoded data format may not be consistent with the previous decoded data format, and in order to achieve the effect of continuous playing without perception by the user, the problem of seamlessly outputting different formats to the display device and the audio device needs to be solved.
In the present invention, when detecting the change of the encoding format of the multimedia stream, the audio/video rendering module 40 renders the encoded multimedia stream in the aspects of picture size and the like to match the current multimedia stream with the multimedia data in the previous encoding and decoding format, thereby realizing seamless continuous playing.
In one embodiment, audio/video decoding module 30 is configured to:
after acquiring the multimedia data packet, detecting whether the coding and decoding information of the current multimedia data packet is consistent with the coding and decoding information of the last multimedia data packet, if not, after the multimedia data in the former coding and decoding format is played, switching to an adaptive decoder according to the format information description index in the coding and decoding information, and decoding and outputting the currently received multimedia data packet by adopting the decoder.
In an embodiment, the audio/video rendering module 30 is configured to:
and when detecting that the data format and the picture size of the current video frame are inconsistent with those of the previous video frame, initializing the mapping data format of the renderer to match the coding and decoding format of the current frame.
In an embodiment, the audio/video rendering module 40 is configured to: when it is detected that the codec format of the audio data frame does not correspond to the previous one, an audio output device matching the new audio data frame is created.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (6)
1. A method for playing audio and video is characterized by comprising the following steps:
adding a description index of format information into the coding and decoding information of each multimedia data packet;
detecting the coding and decoding format of the currently received multimedia data packet in the playing process;
when the coding and decoding format of the currently received multimedia data packet is different from the coding and decoding format of the multimedia data packet being played, waiting for the completion of the playing of the multimedia data in the former coding and decoding format, starting a decoder in the new coding format, and decoding and outputting the currently received multimedia data packet;
rendering the decoded multimedia data and outputting the rendered multimedia data for playing; the decoded multimedia data includes format information; the format information is used for mutually and independently carrying out a decoding process of a decoder and a rendering part of a renderer;
wherein, the step of rendering and outputting the decoded multimedia data for playing comprises:
when detecting that the data format and the picture size of the current video frame are inconsistent with those of the previous video frame, initializing the chartlet data format of a renderer to match the coding and decoding format of the current frame;
when the encoding and decoding format of the current audio data frame is detected to be inconsistent with the encoding and decoding format of the previous frame, an audio output device matched with the current audio data frame is created; the audio output device matched with the current audio data frame is an output device which is matched with the current audio data frame by the sampling rate and the number of channels and is used for connecting the current audio data frame with the previous frame.
2. The method for playing audio/video according to claim 1, wherein when the codec format of the currently received multimedia data packet is different from the codec format of the currently played multimedia data packet, waiting for the completion of playing the multimedia data in the former codec format and enabling a decoder in the new codec format, and the step of decoding and outputting the currently received multimedia data packet comprises:
after a multimedia data packet is obtained, whether the coding and decoding information of the multimedia data packet is consistent with the coding and decoding information of the previous multimedia data packet is detected;
if not, after the multimedia data in the former coding and decoding format is played, switching to an adaptive decoder according to the format information description index in the coding and decoding information, and decoding and outputting the currently received multimedia data packet by adopting the decoder.
3. The method for playing audio/video according to claim 1, wherein the multimedia data packets include h.264 format, h.265 format audio/video data packets, and dolby audio data packets.
4. An audio/video playing device, comprising:
the audio and video data description module is used for adding the description index of the format information into the coding and decoding information of each multimedia data packet;
the detection module is used for detecting the coding and decoding formats of the currently received multimedia data packets in real time in the playing process;
the audio/video decoding module is used for waiting for the completion of the playing of the multimedia data in the former coding and decoding format when the coding and decoding format of the currently received multimedia data packet is different from the coding and decoding format of the multimedia data packet being played, starting a decoder in the new coding format, and decoding and outputting the currently received multimedia data packet;
the audio/video rendering module is used for rendering the decoded multimedia data and outputting the multimedia data for playing; the decoded multimedia data includes format information; the format information is used for mutually and independently carrying out a decoding process of a decoder and a rendering part of a renderer;
the audio/video rendering module is used for initializing a mapping data format of the renderer to match the coding and decoding format and size of the current frame when detecting that the data format and the picture size of the current video frame are inconsistent with those of the previous video frame;
the audio/video rendering module is further used for creating an audio output device matched with the current audio data frame when the fact that the coding and decoding format of the current audio data frame is inconsistent with the coding and decoding format of the previous frame is detected; the audio output device matched with the current audio data frame is an output device which is matched with the current audio data frame by the sampling rate and the number of channels and is used for connecting the current audio data frame with the previous frame.
5. The playback apparatus of audio/video according to claim 4, wherein the audio/video decoding module is configured to:
after a multimedia data packet is obtained, whether the coding and decoding information of the multimedia data packet is consistent with the coding and decoding information of the previous multimedia data packet is detected, if not, after the multimedia data in the previous coding and decoding format is played, the adaptive decoder is switched according to the format information description index in the coding and decoding information, and the currently received multimedia data packet is decoded and output by the decoder.
6. The audio-video playback device according to claim 4, wherein the multimedia data packets include audio-video data packets in h.264 format, h.265 format, and dolby audio data packets.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710047508.0A CN106878290B (en) | 2017-01-22 | 2017-01-22 | Audio and video playing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710047508.0A CN106878290B (en) | 2017-01-22 | 2017-01-22 | Audio and video playing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106878290A CN106878290A (en) | 2017-06-20 |
| CN106878290B true CN106878290B (en) | 2021-02-12 |
Family
ID=59159169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710047508.0A Active CN106878290B (en) | 2017-01-22 | 2017-01-22 | Audio and video playing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106878290B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107231578A (en) * | 2017-08-04 | 2017-10-03 | 四川长虹电器股份有限公司 | The system and method that video file is quickly played |
| CN110536096A (en) * | 2018-05-25 | 2019-12-03 | 视联动力信息技术股份有限公司 | A kind of decoded method and apparatus of data |
| CN110753265B (en) * | 2019-10-28 | 2022-04-19 | 北京奇艺世纪科技有限公司 | Data processing method and device and electronic equipment |
| CN113676769B (en) * | 2021-08-23 | 2023-03-31 | 北京字跳网络技术有限公司 | Video decoding method, device and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9203685B1 (en) * | 2009-06-01 | 2015-12-01 | Sony Computer Entertainment America Llc | Qualified video delivery methods |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106060628A (en) * | 2016-06-28 | 2016-10-26 | 暴风集团股份有限公司 | DirectShow-based method and system supporting variable coding |
-
2017
- 2017-01-22 CN CN201710047508.0A patent/CN106878290B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9203685B1 (en) * | 2009-06-01 | 2015-12-01 | Sony Computer Entertainment America Llc | Qualified video delivery methods |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106878290A (en) | 2017-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI502977B (en) | Audio/video playing device, audio/video processing device, systems, and method thereof | |
| CN106878290B (en) | Audio and video playing method and device | |
| CN109168078B (en) | Video definition switching method and device | |
| CN111479165B (en) | Seamless switching method and device for software and hardware decoding resolution and storage medium | |
| US20150062353A1 (en) | Audio video playback synchronization for encoded media | |
| CN102264004B (en) | Method and device for preventing deficiency of key frame from causing seek incapability | |
| CN105359544B (en) | Special play-back in digital video frequency flow transmission | |
| WO2018010662A1 (en) | Video file transcoding method and device, and storage medium | |
| WO2020151398A1 (en) | Method，device and storage medium for seamlessly switching hardware decoding dynamic resolution, | |
| WO2019170073A1 (en) | Media playback | |
| CN110858827B (en) | Broadcast starting acceleration method and device and computer readable storage medium | |
| EP3142381B1 (en) | Network video playing method and device | |
| WO2017096885A1 (en) | Video playing method and device | |
| CN104394456A (en) | Method and device for decoding audios and videos in embedded environment | |
| US10819951B2 (en) | Recording video from a bitstream | |
| CN109862409A (en) | Video decoding, playback method, device, system, terminal and storage medium | |
| KR20090006014A (en) | Method and apparatus for signaling and decoding avs1-p2 bitstreams of different versions | |
| CN103841451B (en) | Multi-medium play method, device and terminal | |
| CN106060628A (en) | DirectShow-based method and system supporting variable coding | |
| CN101668206A (en) | Multichannel video decoding display method and system based on H.264 | |
| US20120328017A1 (en) | Video decoder and video decoding method | |
| TWI700918B (en) | Coding of video and audio with initialization fragments | |
| KR101970787B1 (en) | Video decoding apparatus and method based on android platform using dual memory | |
| CN113055744A (en) | Video decoding method and device | |
| CN104811782A (en) | Audio video code stream synchronization decoder and decoding method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |