WO2014183566A1

WO2014183566A1 - Picture playback method and apparatus, computer storage medium, and set top box

Info

Publication number: WO2014183566A1
Application number: PCT/CN2014/076737
Authority: WO
Inventors: 陈勇; 王峰
Original assignee: 中兴通讯股份有限公司
Priority date: 2013-11-20
Filing date: 2014-05-04
Publication date: 2014-11-20
Also published as: CN104661059A

Abstract

The present invention provides a picture playback method and apparatus, a computer storage medium, and a set top box. The picture playback method comprises: when it is detected that a program stream of a current channel is played, obtaining an occupation condition of a hardware decoder resource; and in a first time period when the hardware decoder resource is fully occupied, if a picture playback instruction of the current program stream is received and the original hardware decoder resource is still occupied, processing the program stream, obtaining a picture, and playing the picture.

Description

Method, device and calculation for picture playing *^Storage medium and set top box

The present invention relates to the field of multimedia technologies, and in particular, to a method and device for playing a picture, a computer storage medium, and a set top box. Background technique

Streaming refers to the form of media that streams audio, video, and multimedia files over a network. Relative to the form of network play after downloading, the typical feature of streaming media is that compressed audio and video information is compressed and placed on a web server, and the user can watch while downloading without waiting for the entire file to be downloaded.

Hardware decoding refers to decoding high-definition video through the video acceleration function of the graphics card. Therefore, hardware decoding can release the central processing unit (CPU) from the heavy video decoding operation, so that the computer has the ability to smoothly play high-definition video. The graphics processing unit (GPU) of the graphics card/VPU (Video Processing Unit) is more suitable than the CPU for hardware decoding such large amount of data and low difficulty. In general, the video decoding work is separated from the processor and processed by the graphics card, which is called hardware decoding.

FFmpeg is an open source, free, cross-platform video and audio streaming solution that is free software and uses the General Public License (LGPL, Lesser General Public License) or the General Public License (GPL). FFmpeg provides a complete solution for recording, converting and streaming audio and video, including a very advanced audio/video codec library (eg libavcodec), and many codecs in libavcodec for high portability and codec quality. (codec) is developed from scratch.

The following technical problems exist in the prior art: When the hardware decoder resource is found to have been completely After the occupation, continuing the hardware decoding will overburden the CPU, which will cause the video to be jammed, so that the user cannot select the HD program to play. Summary of the invention

In view of this, the embodiment of the present invention is to provide a method, a device, a computer storage medium, and a set top box for playing a picture, which can solve the problem that after the hardware decoder resource is completely occupied, hardware decoding is performed to make the CPU overburden, and the video is emphasized. Caton, so users can't choose the problem of HD program playback.

To achieve the above objective, the technical solution of the embodiment of the present invention is implemented as follows:

The embodiment of the invention provides a picture playing method, and the method includes:

Obtaining the occupation of the hardware decoder resource when the program stream of the current channel is detected for playing;

If the hardware decoder resource is in the first occupied moment, if the picture playback instruction of the current program stream is received, and the original hardware decoder resource is still occupied, the program stream is processed to obtain a picture. And play the picture.

The method further includes: when the hardware decoder resource is in the first time that all the resources are occupied, the method further includes:

The prompt playback capability is insufficient.

Wherein, when the picture is played, the method further includes:

Apply for hardware decoder resources periodically;

If an idle hardware decoder resource is applied, the picture playback is stopped, and the current program stream is hardware decoded using the idle hardware decoder resource, and the decoded program stream is played from the first time.

The process of processing the program stream, obtaining a picture, and playing the picture includes: drawing a frame of the program stream and generating a picture process, obtaining a picture, and playing the picture.

The performing, by performing frame drawing on the program stream and generating a picture process, obtaining a picture, and Play the picture, including:

The program stream obtained from the streaming server is stored in the buffer list;

Looping the frame from the buffer in the buffer list;

Converting each frame of the framed data into a picture and storing it in the picture link table through the picture path;

Looping through the pictures in the picture linked list.

The image path is stored in the image linked list, and further includes:

Delete the fetch buffer of the fetched frame data.

After the loop playing the picture in the picture linked list, the method further includes:

After the predetermined time, the picture path node and the played picture are deleted.

Correspondingly, in order to solve the above technical problem, an embodiment of the present invention further provides a picture playing device, where the device includes:

The acquiring module is configured to acquire the occupation of the hardware decoder resource when the program stream of the current channel is detected to be played;

The first processing module is configured to: when the hardware decoder resource is in the first occupied moment, if the picture play instruction of the current program stream is received, and the original hardware decoder resource is still occupied, the program is The stream is processed, the picture is taken, and the picture is played.

The device further includes:

The prompt module is configured to indicate that the playback capability is insufficient.

The device further includes:

The application module is configured to periodically apply for hardware decoder resources;

a second processing module, configured to stop picture playback if an idle hardware decoder resource is applied, and perform hardware decoding on the current program stream by using the idle hardware decoder resource, and play decoding from the first time After the program stream.

Preferably, the first processing module includes:

a first processing submodule configured to perform frame drawing on the program stream and generate image processing, Get the picture and play the picture.

Preferably, the first processing submodule includes:

a storage unit configured to store the program stream obtained from the streaming server in a buffer list;

The frame drawing unit is configured to: cyclically draw a frame from the buffer in the buffer linked list; the image generating unit is configured to convert each frame of the framed data into a picture, and store the image linked list through the picture path;

The picture playing unit is configured to loop the pictures in the picture linked list.

Preferably, the first processing submodule further includes:

The first deleting unit is configured to delete the fetch buffer of the fetched frame data.

Preferably, the first processing submodule further includes:

The second deleting unit is configured to delete the picture path node and the played picture after a predetermined time.

The embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is used to execute the picture playing method described above.

The embodiment of the present invention further provides a set top box, wherein the set top box includes the picture playing device as described above, and the set top box includes:

The method, the device, the computer storage medium and the set top box provided by the embodiment of the present invention detect whether the hardware decoder resource is completely occupied when the current channel is played, and if the hardware decoder resource is completely occupied, Select the picture broadcast of the current program stream The instruction is placed, and the service of the originally occupied hardware decoder resource is not stopped, and the picture stream is processed to obtain a picture and play the picture. In this way, when the hardware decoder resource has no resources, the program stream is processed to perform picture playback, the program stream is buffered, the utilization of the hardware decoder resources is reduced, the CPU occupancy rate is reduced, the CPU load is reduced, and the user experience is improved. DRAWINGS

1 is a schematic diagram of basic steps of a picture playing method according to an embodiment of the present invention; FIG. 2 is a flowchart of overall processing of a picture playing according to an embodiment of the present invention;

FIG. 3 is a flowchart of a frame drawing and a picture playing according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of the overall structure of a picture playing device according to an embodiment of the present invention. detailed description

The technical problems, the technical solutions, and the advantages of the present invention will become more apparent from the following detailed description.

The method, device, computer storage medium and set top box provided by the embodiment of the invention, after the hardware decoder resource is completely occupied, the program stream is processed by software decoding without adding a new hardware decoder, thereby reducing the burden on the CPU. .

As shown in FIG. 1 , the picture playing method provided by the embodiment of the present invention mainly includes: Step 101: When detecting a program stream of a current channel for playing, acquiring a occupation condition of a hardware decoder resource;

The program stream of the current channel refers to that the user currently selects a high-definition program to play.

Step 102: When the hardware decoder resource is in the first occupied moment, if the picture play instruction of the current program stream is received, and the original hardware decoder resource is still occupied, the program stream is processed. Get the picture and play the picture.

Among them, the processing of the program stream can be realized by the optimized FFmpeg.

Here, if the program stream is processed directly without user selection, it will affect the user. Effect. Therefore, in the picture playing method of the embodiment of the present invention, when the hardware decoder resource is in the first occupied moment, the method further includes:

The prompt playback capability is insufficient.

The lack of the playback capability means that there is no idle hardware decoder resource.

Specifically, the prompting manner can facilitate the user to perform an autonomous selection operation.

That is to say, when the user selects the high-definition program, the hardware decoder resource is at the first moment of being occupied, that is, there is no idle hardware decoder resource, in order to facilitate the user to understand the current CPU occupancy status, the playback capability is performed. Insufficient reminder.

The picture is played to ease the CPU's share, but in order to improve the playback quality, the picture cannot be played all the time. Therefore, in the picture playing method according to the embodiment of the present invention, in the process of playing the picture, the following processing is further included:

Apply for hardware decoder resources periodically;

The first moment refers to a moment when the hardware decoder has no resources and starts to play the picture. When the resources of the hardware decoder are all occupied, the picture is played to alleviate the CPU pressure, but the picture cannot be played all the time, and the program stream is played by periodically applying the hardware decoder resources. This will help improve the user experience.

As shown in FIG. 2, FIG. 2 is a flowchart of overall processing of picture playback according to an embodiment of the present invention. In this embodiment, the set-top box side can provide a hardware decoder resource management module, and has the application/release/query capability of the hardware decoder resource.

Specifically, the overall process mainly includes the following steps:

Step 201: detecting a program stream of the current channel for playing;

The program stream of the current channel refers to a program in which the current user selects high definition. Step 202: Obtain the occupation status of the hardware decoder resource, and determine whether the hardware decoder resource is completely occupied, if yes, go to step 203; if no, go to step 209;

Step 203: pop-up prompt box, indicating that the playing ability is insufficient; then performing step 204;

Specifically, the electronic program guide (EPG, Electronic Program Guide) can be notified to pop up a prompt box, indicating that the playback capability is insufficient.

Here, the user is prompted to select whether or not to perform the picture playback of the current channel by popping up a prompt box.

Step 204: Determine whether the picture needs to be played; if necessary, go to step 205; if not, go to step 208;

Step 205: Using FFmpeg to frame the program stream and generate picture processing, and perform picture playback; then perform step 206;

That is to say, when the user does not choose to stop the service that originally occupied the hardware decoder, and selects to play the picture of the current channel, the set-top box background starts to use the FFmpeg to extract the frame and the picture generation operation from the high-definition code stream transmitted from the network. And start playing the picture.

Step 206: During the process of playing the picture, periodically request whether the hardware decoder resource exists, if yes, go to step 207; otherwise, continue to step 205;

Step 207: Stop the picture playing, and perform hardware decoding on the current program stream by using the idle hardware decoder resource, and play the decoded program stream from the first time;

Here, in the process of playing a picture, the time can be set by the timer, and the resource management module applies for the hardware decoder resource. If the hardware decoder resource can be applied, the FFmpeg frame drawing and the picture playing operation are stopped, and the current operation is started. The program stream is hardware decoded.

Here, the first moment may be a moment when the hardware decoder has no resources and starts to play the picture. Step 208: Exit the prompt box, and end the processing flow;

Step 209: Continue to perform normal HD playback on the program stream of the current channel.

In a picture playing method according to still another embodiment of the present invention, the program stream is processed and obtained The steps to get a picture, and play the picture, mainly include:

Step 102a: Perform frame drawing on the program stream and generate picture processing, obtain a picture, and play the picture.

The step 102a specifically includes:

Step 102al: storing the program stream obtained from the streaming media server into the buffer linked list; Step 102a2: Looping the frame from the buffer in the buffer linked list;

Step 102a3: Convert each frame of the framed data into a picture, store the picture in the picture link table, and delete the fetch buffer of the fetched frame data;

Step 102a4: Looping the picture in the picture linked list, and deleting the picture path node and the played picture after a predetermined time.

The program stream to be buffered is first stored in the corresponding buffer linked list by step 102al, and then cyclically framed by step 102a2, and the framed data is converted into a picture according to step 102a3, stored in the picture linked list, and then The picture is played back in step 102a4, so that the program stream to be buffered is converted into a picture by the conversion of the frame and the picture, and can be realized by software decoding, without adding a hardware decoder, which can reduce the CPU occupation rate and improve Practicality.

As shown in FIG. 3, FIG. 3 is a flowchart of a frame drawing and a picture playing according to an embodiment of the present invention. The specific implementation process of processing the program stream by using FFmpeg, obtaining a picture, and playing the picture provided by the embodiment of the present invention mainly includes the following steps:

Step 301: Receive a media stream from a streaming server.

Step 302: The media stream is stored in a buffer linked list according to a size of 188*200. Here, each block size in the buffer linked list may be 188*200.

Step 303: Query whether the buffer linked list is empty; if not, proceed to step 304; if it is empty, continue to step 302;

Step 304: Looping a buffer from the buffer list to perform frame drawing;

Step 305: Convert each frame of data after the frame is converted into a picture, and delete the data in the buffer list. The buffer that has been processed;

Here, the picture can be stored locally in the set top box.

Step 306: Generate a picture linked list of the converted picture; then perform step 307;

Step 307: Query whether the picture path list is empty, if not empty, go to step 308; if it is empty, continue to step 306;

Step 308: The loop takes the picture path from the picture linked list to the page, and deletes the picture path node and the displayed picture itself after 5 seconds.

The embodiment of the present invention further describes a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is used to execute the picture playing method shown in FIG. 1 in the embodiment of the present invention.

As shown in FIG. 4, the embodiment of the present invention further provides a picture playing device, where the device includes:

The obtaining module 41 is configured to: when the program stream of the current channel is detected to be played, acquire the occupation status of the hardware decoder resource;

The first processing module 42 is configured to: when the hardware decoder resource is in the first occupied moment, if the picture play instruction of the current program stream is received, and the original hardware decoder resource is still occupied, The program stream is processed, the picture is obtained, and the picture is played.

Among them, the processing of the program stream can be realized by the software FFmpeg. Of course, the above software processing is only an example, and any software that processes the program stream can be applied to this, and is not here - for example.

In the embodiment of the present invention, when the acquisition module 41 detects that the program stream belongs to play, the remaining condition of the hardware decoder resource is obtained, so that when the hardware decoder resources are all occupied, the first processing module 42 needs to buffer the current. The program stream is processed by the corresponding picture, so that the program stream can be decoded by the software, the program stream is turned into a picture, the burden of the CPU is reduced, and the buffering of the picture is played, the phenomenon of the card is alleviated, and the comfort of the user is improved. As shown in FIG. 4, in the picture playing device of the above embodiment of the present invention, the device further includes:

The prompt module 43 is configured to indicate that the playback capability is insufficient.

The prompting module 43 can be an EPG.

In a picture playing device according to still another embodiment of the present invention, the device further includes:

The application module 44 is configured to periodically apply for a hardware decoder resource;

Among them, the hardware decoder resource can be applied by setting a certain time by the timer. The second processing module 45 is configured to stop the picture play if the idle hardware decoder resource is applied, and perform hardware decoding on the current program stream by using the idle hardware decoder resource, and play from the first time The decoded program stream.

The obtaining module 41, the first processing module 42, the application module 44, and the second processing module 45 may be fused together as a hardware decoder resource management module, having the application/release/query capability of the hardware decoder resource.

In the embodiment of the present invention, in the process of software decoding and processing the program stream, if the user needs to play the video in high definition, the application module 44 needs to perform real-time occupancy rate inquiry on the hardware decoder, when there is idle hardware decoder resource, The second processing module 45 performs processing decoding to restore normal HD video playback, providing convenience for the user.

In the picture playing device of the embodiment of the present invention, the first processing module 42 includes: a first processing submodule 421 configured to perform frame drawing on the program stream and generate image processing, obtain a picture, and play the picture. .

In the picture playing device of the embodiment of the present invention, the first processing submodule 421 includes:

The storage unit 4210 is configured to store the program stream obtained from the streaming media server in a buffer list;

The frame drawing unit 4211 is configured to: cyclically draw a frame from a buffer in the buffer linked list; The picture generating unit 4212 is configured to convert each frame of the framed data into a picture, and store the picture into the picture linked list via the picture path;

The picture playing unit 4213 is configured to cyclically play the pictures in the picture linked list.

In the picture playing device of the embodiment of the present invention, the first processing submodule 421 further includes:

The first deleting unit 4214 is configured to delete the fetch buffer of the fetched frame data.

The second deleting unit 4215 is configured to delete the picture path node and the played picture after a predetermined time.

In an actual application, the picture playing device may be applied to the field of multimedia technology; the acquiring module 41, the first processing module 42, the application module 44, and the second processing module 45 may each belong to the picture playing device or the picture playing device. The device is implemented by a central processing unit, a digital signal processor (DSP) or a Field Programmable Gate Array (FPGA).

In the picture playing device of the embodiment of the present invention, when the acquiring module 41 detects that the program stream belongs to play, the remaining condition of the hardware decoder resource is acquired, so that when the hardware decoder resources are all occupied, the prompting module 43 prompts Then, the first processing module 42 performs corresponding picture processing on the program stream that needs to be buffered, so that the program stream can be decoded by software decoding, and the data in the storage unit 4210 is framed by the frame drawing unit 4211. The picture generation unit 4212 changes the data of the extracted frame to a picture, and the picture playback unit 4213 displays the picture to the page play, thereby reducing the burden on the CPU. At the same time, in order to meet the needs of the user to play video in high definition, the application module 44 needs to perform real-time occupancy inquiry on the hardware decoder, and when there is an idle hardware decoder, the second processing module 45 performs decoding, that is, returns to normal. The playback of HD video is convenient for users. All of the above embodiments of the picture playback device and their beneficial effects are applicable to the set top box. Correspondingly, the embodiment of the present invention further provides a set top box, including the picture playing device as described above. As shown in FIG. 4, the set top box includes:

The set top box further includes:

The second processing module 45 is configured to stop the picture play if the idle hardware decoder resource is applied, and perform hardware decoding on the current program stream by using the idle hardware decoder resource, and play from the first time The decoded program stream.

Preferably, the first processing module 42 includes:

The first processing sub-module 421 is configured to perform frame extraction on the program stream and generate a picture process, obtain a picture, and play the picture.

Preferably, the first processing submodule 421 includes:

The frame drawing unit 4211 is configured to cyclically draw a frame from the buffer in the buffer list; the image generating unit 4212 is configured to convert each frame of the framed data into a picture, and deposit the picture through the picture path. Link list

The picture playing unit 4213 is configured to cyclically play the pictures in the picture linked list. Preferably, the first processing submodule 421 further includes:

Preferably, the first processing submodule 421 further includes:

The above is a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It should be considered as the scope of protection of the present invention. Industrial applicability

In the embodiment of the present invention, when the program stream of the current channel is detected to be played, the occupation of the hardware decoder resource is obtained; if the hardware decoder resource is in the first occupied moment, if the current program stream is received, When the instruction is played and the original hardware decoder resource is still occupied, the program stream is processed to obtain a picture and play the picture. In this way, after the hardware decoder resources are completely occupied, the program stream is processed by software decoding without adding a new hardware decoder, thereby reducing the burden on the CPU.

Claims

claims

1. A picture playing method, the method includes:

When the program stream of the current channel is detected and played, the occupancy status of the hardware decoder resources is obtained;

At the first moment when the hardware decoder resources are all occupied, if the picture playback instruction of the current program stream is received and the original hardware decoder resources are still occupied, the program stream is processed to obtain the picture, and play the picture.

2. The method according to claim 1, wherein when the hardware decoder resources are all occupied at the first moment, the method further includes:

It indicates insufficient playback capability.

3. The method according to claim 1, wherein when playing the picture, the method further includes:

Apply for hardware decoder resources regularly;

If idle hardware decoder resources are applied for, picture playback is stopped, and the idle hardware decoder resources are used to hardware decode the current program stream, and the decoded program stream is played from the first moment.

4. The method according to claim 1, characterized in that processing the program stream, obtaining pictures, and playing the pictures includes:

Perform frame extraction and picture generation processing on the program stream to obtain pictures, and play the pictures.

5. The method according to claim 4, characterized in that: performing frame extraction and picture generation processing on the program stream, obtaining pictures, and playing the pictures, including:

Store the program stream obtained from the streaming media server in the buffer linked list;

Cyclically extract frames from the buffer in the buffer linked list;

Convert each frame of data extracted into a picture and store it in the picture linked list via the picture path; play the pictures in the picture linked list in a loop.

6. The method according to claim 5, wherein after the picture link list is stored via the picture path, the method further includes:

Delete the fetch buffer of fetched frame data.

7. The method according to claim 5, wherein after the pictures in the picture linked list are played in a loop, the method further includes:

After a predetermined time, the picture path node and the played picture are deleted.

8. A picture playback device, the device includes:

The acquisition module is configured to acquire the occupancy status of hardware decoder resources when detecting the program stream of the current channel for playback;

The first processing module is configured to: at the first moment when the hardware decoder resources are all occupied, if a picture playback instruction of the current program stream is received and the original hardware decoder resources are still occupied, the program The stream is processed, the picture is obtained, and the picture is played.

9. The device according to claim 8, wherein the device further includes:

Prompt module, configured to prompt insufficient playback capabilities.

10. The device according to claim 8, wherein the device further includes:

The application module is configured to apply for hardware decoder resources regularly;

The second processing module is configured to stop picture playback when applying for idle hardware decoder resources, and use the idle hardware decoder resources to perform hardware decoding of the current program stream, and play the decoded video from the first moment. The subsequent program stream.

11. The picture playing device according to claim 8, wherein the first processing module includes:

The first processing submodule is configured to perform frame extraction and picture generation processing on the program stream, obtain pictures, and play the pictures.

12. The device according to claim 11, wherein the first processing sub-module includes: a storage unit configured to store the program stream obtained from the streaming media server in a buffer linked list middle;

The frame extraction unit is configured to cyclically extract frames from the buffer in the buffer linked list; the picture generation unit is configured to convert each frame of data extracted from the frame into a picture, and store it in the picture linked list through the picture path. ;

The picture playback unit is configured to play the pictures in the picture linked list in a loop.

13. The device according to claim 12, wherein the first processing sub-module further includes:

The first deletion unit is configured to delete the fetch buffer of fetched frame data.

14. The device according to claim 12, characterized in that the first processing sub-module further includes:

The second deletion unit is configured to delete the picture path node and the played picture after a predetermined time.

15. A set-top box, the set-top box including the picture playback device as claimed in any one of claims 8 to 14; the set-top box including:

16. A computer storage medium, the computer storage medium stores computer-executable instructions, and the computer-executable instructions are configured to execute the method described in any one of claims 1 to 7.