CN111385628A

CN111385628A - Method and device for realizing picture-in-picture and set top box

Info

Publication number: CN111385628A
Application number: CN201811633998.3A
Authority: CN
Inventors: 宋玉林; 朱兴昌; 张维
Original assignee: ZTE Corp
Current assignee: ZTE Corp
Priority date: 2018-12-29
Filing date: 2018-12-29
Publication date: 2020-07-07
Also published as: WO2020134592A1

Abstract

The embodiment of the invention discloses a method, a device and a set-top box for realizing picture-in-picture, wherein the method comprises the following steps: receiving a request to open a picture-in-picture; and according to the request, displaying a suspended picture-in-picture window independent of the current application of the set top box on the window of the current application of the set top box. In the embodiment of the invention, the suspended picture-in-picture window is separated from the current application of the set top box, so that the user experience is effectively improved.

Description

Method and device for realizing picture-in-picture and set top box

Technical Field

The embodiment of the invention relates to a method, a device and a set top box for realizing picture-in-picture.

Background

In an existing Set-Top Box (Set Top Box, abbreviated as STB), there are two common scenes for playing Picture-in-Picture (PIP):

(1) the picture-in-picture of the traditional non-intelligent set top box adopts two paths of code stream playing, wherein one path is large video channel playing, and the other path is small window channel playing;

(2) on the intelligent set-top box, because the application provides the presentation and the playing of the video programs, when the picture-in-picture service is realized, the video channels are played and presented by digging holes for small video picture pages.

The two scenes are associated with the application of the set top box, and if a user switches the application or the video application switches pages, the current picture-in-picture video playing is necessarily stopped; picture-in-picture video playback is triggered by set-top applications, and the current video channel playback cannot be played away from these set-top applications.

With the increasing number of third-party value-added applications of the set-top box, the third-party value-added applications except the video applications cannot acquire channel information, so that a picture-in-picture service cannot be provided on the third-party value-added applications.

The above situation causes that the user cannot pay attention to the playing of a certain channel at any time when operating the set-top box application.

Disclosure of Invention

The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

The embodiment of the invention provides a picture-in-picture realization method, a picture-in-picture realization device and a set top box.

The embodiment of the invention provides a method for realizing picture-in-picture, which comprises the following steps:

receiving a request to open a picture-in-picture;

and according to the request, displaying a suspended picture-in-picture window independent of the current application of the set top box on the window of the current application of the set top box.

The embodiment of the present invention further provides a device for implementing picture-in-picture, including:

a receiving unit for receiving a request to open a picture-in-picture;

and the display unit is used for displaying the suspended picture-in-picture window independent of the current application of the set top box on the window of the current application of the set top box according to the request.

An embodiment of the present invention further provides a set top box, including: memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of achieving picture-in-picture when executing the program.

The embodiment of the invention also provides a computer-readable storage medium, which stores computer-executable instructions for executing the method for realizing picture-in-picture.

The embodiment of the invention comprises the following steps: receiving a request to open a picture-in-picture; and according to the request, displaying a suspended picture-in-picture window independent of the current application of the set top box on the window of the current application of the set top box. In the embodiment of the invention, the suspended picture-in-picture window is separated from the current application of the set top box, so that the user experience is effectively improved.

Other aspects will be apparent upon reading and understanding the attached drawings and detailed description.

Drawings

FIG. 1 is a schematic diagram of a system architecture of a set-top box;

fig. 2 is a first scenario for implementing picture-in-picture;

fig. 3 is a second scenario for implementing picture-in-picture;

fig. 4 is a flowchart of a method of implementing picture-in-picture according to an embodiment of the present invention;

FIG. 5 is a diagram illustrating a layer of a set-top box according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of viewing a floating video channel on a set-top application in accordance with an embodiment of the present invention;

FIG. 7 is a flowchart of displaying the floating PIP window through the integration layer according to an embodiment of the present invention;

fig. 8 is a flowchart of determining which layer the content to be displayed is displayed on according to the layer identifier according to the embodiment of the present invention;

fig. 9 is a flowchart of adjusting pip window attributes (content includes video) according to an embodiment of the present invention;

fig. 10 is a flow chart of adjusting the attributes of a pip window (content includes pages) according to an embodiment of the present invention;

fig. 11 is a flow chart of closing a picture-in-picture according to an embodiment of the present invention;

fig. 12 is a flowchart of switching from a picture-in-picture window to a currently active application window of a set-top box according to an embodiment of the present invention;

FIG. 13 is a flow chart of video switching according to an embodiment of the present invention;

FIG. 14 is a block diagram of a set-top box according to an embodiment of the invention;

fig. 15 is a flowchart illustrating a video playing or interactive display starting process according to an embodiment of the present invention;

FIG. 16 is a flowchart of creating a synthesized layer according to an embodiment of the present invention;

fig. 17 is a flow chart of a picture-in-picture video playback process according to an embodiment of the present invention;

fig. 18 is a picture-in-picture video information generation and query flow diagram of an embodiment of the present invention;

FIG. 19 is a flowchart illustrating a change of an attribute of a video playback window according to an embodiment of the present invention;

FIG. 20 is a flowchart illustrating the change of properties of an interactive display window according to an embodiment of the present invention;

fig. 21 is a flow chart of video switching for video playing according to an embodiment of the present invention;

fig. 22 is a flow chart of closing pip video playback according to an embodiment of the present invention;

fig. 23 is a flow chart of closed pip interaction display according to an embodiment of the present invention;

FIG. 24 is a flowchart of an embodiment of the present invention for switching from a picture-in-picture layer to a currently active application window layer;

FIG. 25 is a timing diagram of a floating PIP playback video according to an embodiment of the present invention;

FIG. 26 is a timing diagram illustrating a floating PIP interaction of an example of the present invention;

fig. 27 is a schematic diagram of a pd-implementing apparatus according to an embodiment of the present invention;

fig. 28 is a schematic diagram of a set-top box according to an embodiment of the present invention.

Detailed Description

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.

As shown in fig. 1, the set-top box system architecture comprises: a remote control 11, a set-top box 12 and a television 13. The remote controller 11 receives a key instruction of a user, converts the key instruction into a corresponding request, and sends the request to the set-top box 12, and the set-top box 12 displays a corresponding image through the television 13 according to the received request. The set-top box 12 may also have buttons, and the set-top box 12 may also perform corresponding processing directly according to the received key information.

The picture layer of the set-top box is generally divided into two picture layers, namely a video picture layer and a page picture layer, and the picture-in-picture of the set-top box plays the following two scenes:

(1) the first scene is shown in fig. 2, two paths of code streams are injected into a video layer, one path is a full-screen video stream, and the other path is a small-window video stream; in the scene, two paths of code streams are realized in the video layer, and the video stream in the video layer is played, so that the position can only be fixed and cannot be reduced or enlarged; if the user performs the page switching operation on the set-top box, the playing of the two paths of code streams is stopped, and the user is influenced to watch the picture-in-picture playing.

(2) The second scenario is shown in fig. 3, that is, a channel is clicked on the left side of a channel list preview window, the widget of the channel appears in a fixed area of a screen, and different videos are played on the right widget along with the selection of different channels on the left side. The small window video stream is in the video layer, the channel list preview window is in the page layer, and in order to see the video of the small window, the page layer must dig out a small window page frame (i.e. make the page frame transparent) with the same size as the small window video, so that the picture-in-picture video below can be seen, thus achieving the effect of picture-in-picture small window playing. If the user performs the page switching operation on the set-top box, the playing of the small window code stream is stopped, and the user is influenced to watch the picture-in-picture playing.

The picture-in-picture playing of the two scenes is associated with the current application of the set top box, and the picture-in-picture playing is influenced as long as the current page is left to operate other functions; the picture-in-picture playing window cannot be moved, and the position and the size of the played picture-in-picture window are fixed, so that the picture-in-picture playing is inconvenient for a user to watch when the user operates the set top box application.

Another scenario is that a third-party value-added application (such as a game) cannot watch the current pip channel playing.

The embodiment of the invention provides a method for realizing picture-in-picture, which realizes that a suspended picture-in-picture window can be independent of any application through suspended playing of videos or pages, so that a picture-in-picture effect can be presented in any state of a set top box under any application.

As shown in fig. 4, the method for implementing picture-in-picture according to the embodiment of the present invention is applied to a set top box, and includes:

step 201, a request for opening a picture-in-picture is received.

Step 202, according to the request, displaying a floating pip window independent of the current application of the set top box on the window of the current application of the set top box.

In the embodiment of the invention, the suspended picture-in-picture window is separated from the current application of the set top box, so that the user experience is effectively improved.

In the embodiment of the present invention, the floating pip window is independent of the current application of the set top box, which means that the floating pip window has no association with the current application of the set top box. That is, the floating pip window is not changed or closed when the set-top box currently applies a switch. "floating" means that the pip window is always displayed on the window currently applied by the set-top box and is not covered when the current application of the set-top box is switched.

It should be noted that, in the embodiment of the present invention, the floating pd window also has a corresponding application, such as a video application or a page application displayed in the floating pd window, which may be referred to as a floating pd application.

In one embodiment, the presenting a floating pip window independent of the set top box's current application includes:

creating a comprehensive layer on the top layer of the set top box layer, and displaying the suspended picture-in-picture window through the comprehensive layer;

the comprehensive layer is used for performing at least one of video playing and interactive display.

In the embodiment of the invention, a new layer is created, which is called a comprehensive layer and can also be called a picture-in-picture layer, and can be used for playing videos and carrying out non-video interactive services, wherein the interactive services comprise advertisements, system message display and the like; the picture layer can display videos and pages, videos are rendered or interacted on the picture layer, so that picture-in-picture and set top box application separation is achieved, meanwhile, a picture-in-picture window can be moved and zoomed, and user experience is improved better. The comprehensive layer has the following characteristics:

1. based on the current situation that two image layers of the existing set top box are separated, the comprehensive image layer inherits the characteristics of the video image layer and the page image layer, the video playing and page displaying effects can be achieved, the comprehensive image layer renders a video on the basis of the characteristics of the page image layer, and the effects that the characteristics of the page and the video can be played are achieved.

2. The comprehensive layer is independent of a video layer and a page layer of the set top box, and all applications of the set top box are rendered on the video layer and the page layer, so that the video and interactive information rendered in the comprehensive layer cannot be influenced on the operation of the applications of the set top box.

3. Because the page characteristics of the comprehensive layer can be moved and zoomed, the video on the page can also be moved and zoomed, and the comprehensive layer can be conveniently used for moving to a proper position for previewing at any time.

4. The layers in the set-top box are generally the lowest layer of a video layer and the next upper layer of a page layer. The integrated layer created in the embodiment of the invention has higher layer level than the video layer and the page layer. The comprehensive layer can render information such as videos and pages, and due to the fact that the information is displayed on the top of the set top box, the videos and the interactive information displayed in the comprehensive layer can be previewed in real time regardless of switching of the following applications.

Referring to fig. 5, the layers of the set top box according to the embodiment of the present invention are divided into three layers, where the first layer is a video layer, the second layer is a page layer, and the third layer is a comprehensive layer, where the comprehensive layer is higher in level than the video layer and the page layer, and is used for displaying pip video and interactive display. The information displayed in the comprehensive layer is not influenced by the video layer and the page layer, is suspended on the top, and can be used for scaling the size, the position and the like of the video in the comprehensive layer.

Referring to fig. 6, a user may view a floating video channel playback on top of any set-top box application. Fig. 6 shows that in the third-party application (game), when the user plays the game, the floating pip window can be opened through the remote controller to track the channel playing information of interest.

In the embodiment of the invention, the suspended picture-in-picture window is not associated with the current application of the set top box, and the suspended picture-in-picture window is not influenced no matter how the current application of the set top box is switched, so that the condition that the channel playing is closed due to the page switching of the set top box is avoided; the suspended picture-in-picture can be played or displayed on a page on the third-party value-added application, and in any state, a picture-in-picture (PIP) button of the remote controller can be used for opening a suspended picture-in-picture window to browse and play channels, so that the function of displaying the picture-in-picture under the third-party value-added application is realized.

As shown in fig. 7, in an embodiment, the displaying the floating pip window through the integration layer includes:

step 301, when the content of the floating pip window is determined to be a video, acquiring video information corresponding to the video.

And step 302, rendering and playing the video on the comprehensive layer according to the video information.

The obtaining of the video information corresponding to the video may include:

and acquiring video information when the last suspended picture-in-picture window exits, or acquiring video information corresponding to the video according to a preset strategy.

For example, the video information may be channel information, window position information, window size information, etc., and the video playing may be continued according to the information when the last floating pip window exits. Video playing can also be performed according to preset strategies, such as a specified channel, a position and a size of a window, and the like.

In one embodiment, the step 202 includes:

and when the content of the suspended picture-in-picture window is determined to be a page, rendering interactive display is carried out on the comprehensive layer according to the page.

The page may be an advertisement, a system message, or the like, or may be a game screen that needs to be interacted with.

In the embodiment of the present invention, the set top box has three layers, and which layer the content to be displayed is displayed on can be determined according to the layer identifier, as shown in fig. 8, the method includes the following steps:

step 401, determining a comprehensive layer identifier (i.e. a comprehensive layer ID) according to the request for opening the pip;

step 402, determining a layer of the display content as the integrated layer according to the integrated layer identifier.

When the set top box is used for rendering videos or pages, the set top box is provided with a special rendering module for rendering the video and non-video page functions;

the rendering module of the set top box renders the video to the video layer by default, the size of a video playing window after rendering cannot be scaled, and the position of the video playing window cannot be moved.

The set top box rendering module renders the default page to the page layer, and the page window can move randomly after rendering.

In the embodiment of the invention, a unique identification mark (called a comprehensive layer identification) is created for the comprehensive layer, and is different from a video layer and a page layer.

When a video needs to be rendered on the comprehensive layer, the video playing application transmits the comprehensive layer identifier to the rendering module, the rendering module finds the comprehensive layer according to the comprehensive layer identifier, and the video is rendered on the comprehensive layer;

when a page needs to be rendered on the comprehensive layer, the interactive application transmits the comprehensive layer identifier to the rendering module, the rendering module finds the comprehensive layer according to the comprehensive layer identifier, and the page is rendered on the comprehensive layer;

the rendering module can render the page and the video which need to be displayed to the comprehensive layer by fusing the comprehensive layer identification, the comprehensive layer can be zoomed and moved at will due to the characteristic of the page layer, and the rendered video on the comprehensive layer can also be zoomed and moved along with all the video.

In an embodiment, the attributes of the floating pip window are adjusted in accordance with a received request to adjust pip window attributes.

Wherein the adjusting the attribute of the floating pip window includes at least one of:

adjusting the size of the suspended picture-in-picture window, and adjusting the position of the suspended picture-in-picture window.

In an embodiment, when the attribute of the pip window needs to be adjusted, as shown in fig. 9, when the content of the floating pip window is a video, the method includes:

step 501, adjusting the attribute of a video playing window according to a received request for adjusting the attribute of a picture-in-picture window; and

step 502, adjusting the window attribute of the comprehensive layer.

The comprehensive layer has page characteristics, so that the size and the position of the comprehensive layer can be adjusted, and the size and the position of the video played in the picture-in-picture can be adjusted.

In an embodiment, when the pip window attribute needs to be adjusted, as shown in fig. 10, the content of the floating pip window includes a page, the method further includes:

step 601, adjusting the attribute of the interactive display window according to the received request for adjusting the attribute of the picture-in-picture window, and,

step 602, adjusting the window attribute of the synthesis layer.

Because the comprehensive layer has page characteristics, the size and the position can be adjusted, so that the size and the position of the page displayed in the picture-in-picture can be adjusted.

In the embodiment of the invention, the suspended picture-in-picture window can be randomly moved at any position on the picture of the set-top box, so that a user can browse contents at different positions of the interface of the set-top box and can preview the suspended picture-in-picture image at the same time. Moreover, the suspended pip window can be randomly enlarged and reduced on the picture of the set-top box, which also facilitates the user to achieve the following effects: when the floating video channel is previewed and played, if the set top box application needs to be focused, the floating picture-in-picture window can be reduced; the floating pip window may be enlarged if focus is needed on the floating pip.

When the pip needs to be closed, in an embodiment, the method further comprises: closing the floating pip window according to a received request to close pip.

As shown in fig. 11, the following steps may be included:

step 701, stopping displaying content on the comprehensive layer according to a received request for closing the picture-in-picture;

and step 702, releasing the comprehensive layer.

And when the content is a video, stopping playing the video on the comprehensive layer. And when the content is a page, stopping performing interactive display on the comprehensive layer.

In one embodiment, when it is necessary to switch from the picture-in-picture window to the currently active application window of the set-top box, as shown in fig. 12, the method includes the following steps:

step 801, according to a received request for switching from a picture-in-picture window to a current active application window of the set top box, transmitting the content displayed on the comprehensive layer to the current active application window for displaying;

and 802, stopping displaying the content on the comprehensive layer, and releasing the comprehensive layer.

In step 801, the content displayed on the integrated layer is transferred to the current active application window for displaying, and step 802 may be performed simultaneously.

Step 802 closes the pip flow.

In an embodiment, when the content includes a video, and a video switching is required (for example, changing a channel), as shown in fig. 13, the method includes the following steps:

step 901, acquiring target video information according to a received video switching request;

and step 902, according to the target video information, playing a target video corresponding to the target video information through the suspended picture-in-picture window.

In step 902, a target video corresponding to the target video information is rendered and played on the integrated layer.

In this case, the currently playing video of the picture-in-picture can be switched to the target video to be played.

As shown in fig. 14, the set-top box according to the embodiment of the present invention may include the following modules:

a general control module 1001 for receiving an instruction (such as a key instruction) from the STB system to use the pip function;

the layer management module 1002 is configured to create a set top box integrated layer for performing a picture-in-picture function;

a play control management module 1003, configured to perform floating picture-in-picture play control management;

the video information management module 1004 is used for suspending functions of storing, inquiring, memorizing and the like of the picture-in-picture video information;

a video playing module 1005, configured to suspend the functions of picture-in-picture video playing;

the interactive module 1007 is used for controlling, managing and displaying the interactive service in the floating picture-in-picture;

and a rendering module 1006, configured to render videos, pages and the like onto the layer.

Referring now to fig. 14, an implementation of an embodiment of the present invention will be described.

Video playing or interactive display starting process:

referring to fig. 15, the method includes the steps of:

step 1101, the STB system is normally used, a user presses a picture-in-picture shortcut key of a remote controller, the master control module receives the key processing, the master control module judges that a suspended picture-in-picture function needs to be opened, and notifies the layer management module;

step 1102, the layer management module creates a comprehensive layer, the layer inherits all characteristics of the video layer and the page layer, renders a video on the characteristics of the page layer, and has the advantages of both the page layer and the video layer, the comprehensive layer is higher than the video layer and the page layer, and the comprehensive layer is used for video playing and interactive (such as advertisement and message) display. The comprehensive layer display information is not influenced by the application of the set top box, and window size scaling and position movement can be performed;

step 1103, the master control module judges whether the video is played, informs the broadcast control management module to trigger the broadcast, and executes the next step; otherwise go to step 1108;

step 1104, the broadcast control management module goes to the video information management module to inquire the played video information;

in step 1105, the video information management module obtains the video information played last time (i.e. the video information when the last picture-in-picture exits) according to the stored picture-in-picture video information, and if there is no video information played last time, finds out a suitable video information according to a preset policy and returns it to the broadcast control management module.

In step 1106, the playing control management module sends the obtained video information to the video playing module for playing.

Step 1107, the video playing module sends a message to the rendering module and carries the comprehensive layer identifier, and the rendering module plays the rendered video in the comprehensive layer according to the comprehensive layer identifier.

Step 1108, if the processing is interactive, the general control module informs the interactive module to process.

Step 1109, the interactive module processes the information which needs to be displayed by the rendering module through the sending information and brings the comprehensive layer identification, and the rendering module renders and displays the comprehensive layer according to the comprehensive layer identification.

As shown in fig. 16, the process of creating the synthesis layer includes the following steps;

step 1201, the layer management module receives a command for creating a comprehensive layer sent by the master control module;

step 1202, the layer management module creates a comprehensive layer, which inherits all the characteristics of the video layer and the page layer, renders a video on the characteristics of the page layer, has the advantages of both the page layer and the video layer, and can simultaneously display a page and a video on the layer;

step 1203, the layer management module sets the layer of the comprehensive layer to be higher than the page layer and the video layer;

and step 1204, the layer management module returns information corresponding to the comprehensive layer to the master control module.

As shown in fig. 17, the process flow of playing the pip video includes the following steps:

step 1301, the playing control management module receives a playing request of the master control module, wherein the playing request carries the comprehensive layer identifier;

step 1302, the broadcast control management module firstly queries corresponding video information from the video information management module;

step 1303, the broadcast control management module sends the inquired video information and the comprehensive layer identifier to the video playing module, and instructs the video playing module to play;

in step 1304, the video playing module starts playing and sends the video and the comprehensive layer identifier to the rendering module;

step 1305, the rendering module renders and plays the video on the comprehensive layer.

As shown in fig. 18, the pip video information generation and query process includes:

1401, the STB successfully logs in the platform, and the video information management module acquires the video resource played in the picture-in-picture;

step 1402, the video information management module stores the video information played in the picture-in-picture;

step 1403, the video information management module receives the video information to be played;

step 1404, the video information management module queries the video information played last time;

step 1405, the video information module returns the queried video information.

(II) video playing window attribute changing process: (wherein the window attribute change mainly includes up, down, left and right movement of the window position, scaling of the window size, etc.)

Referring to fig. 19, the method includes the steps of:

step 1501, the master control module receives a window attribute change request, and the request can be realized by pressing a picture-in-picture shortcut key of a remote controller by a user;

step 1502, the master control module sends a video window attribute change request to the broadcast control management module;

step 1503, the playing control management module forwards the window attribute change information to the video playing module;

step 1504, the video playing module forwards the window attribute change information to the rendering module;

step 1505, the rendering module completes processing;

step 1506, the master control module sends an attribute change request to the layer management module;

step 1507, the layer management module performs attribute change operation according to the integrated layer;

in step 1508, the layer management module completes the processing of the window attribute change.

(III) interactive display window attribute change process: (wherein the window attribute change mainly includes up, down, left and right movement of the window position, scaling of the window size, etc.)

Referring to fig. 20, the method includes the steps of:

step 1601, the master control module receives a request for changing the window attribute, and the request can be realized by pressing a picture-in-picture shortcut key of a remote controller by a user;

step 1602, the master control module sends an interactive window attribute change request to the interactive module;

step 1603, the interactive module sends the attribute change information of the interactive window to the rendering module;

step 1604, the rendering module completes processing;

1605, the master control module sends a window attribute change request to the layer management module;

step 1606, the layer management module performs attribute change operation according to the integrated layer;

step 1607, the layer management module completes the processing of the window attribute change.

(IV) video playing and video switching process:

referring to fig. 21, the method includes the steps of:

step 1701, the master control module receives a video switching request, and the request can be realized by pressing a picture-in-picture shortcut key of a remote controller by a user;

1702, sending a video switching request to a broadcast control management module by a master control module;

step 1703, the broadcast control management module queries corresponding video information from the video information management module;

step 1704, the video information management module returns corresponding video information;

step 1705, the broadcast control management module notifies the video playing module to play a new video (i.e. a target video);

step 1706, the video playing module sends the new video and the integrated layer identifier to the rendering module;

step 1707, the rendering module renders and plays the new video.

(V) closing the picture-in-picture video playing flow:

referring to fig. 22, the method includes the steps of:

step 1801, the master control module receives a request for closing the picture-in-picture video, and the request can be realized by pressing a picture-in-picture shortcut key of a remote controller by a user;

step 1802, the master control module sends a play stopping command to the play control management module;

step 1803, the playing control management module sends a playing stop command to the video playing module;

step 1804, the video playing module stops playing and sends a playing stopping command to the rendering module;

step 1805, the rendering module stops rendering and playing the comprehensive layer;

step 1806, the master control module sends a comprehensive layer closing request to the layer management module;

step 1807, the layer management module releases the integrated layer.

And (VI) closing the picture-in-picture interactive display process:

referring to fig. 23, the method includes the steps of:

1901, the master control module receives a request for closing the picture-in-picture interactive display, which can be realized by pressing a picture-in-picture shortcut key of the remote controller by the user;

step 1902, the master control module sends a command of stopping the interactive display to the interactive module;

1903, the interactive display module stops the interactive display and sends an interactive display command to the rendering module;

1904, stopping the rendering module from interactively displaying on the comprehensive layer;

step 1905, the general control module sends a comprehensive layer closing request to the layer management module;

step 1906, the layer management module releases the integrated layer.

Switching from the picture-in-picture layer to the current active application window layer:

referring to fig. 24, the method includes the steps of:

step 2001, the master control module receives a key request for switching the picture-in-picture window to the current active application window of the set-top box;

step 2002, the master control module judges whether the current picture-in-picture window is video playing or interactive display, if the current picture-in-picture window is video playing, the step 2003 is carried out, and if the current picture-in-picture window is interactive display, the step 2004 is carried out;

in step 2003, the general control module processes the pip video playing closing process, which is the same as the process shown in fig. 22.

The master control module transmits the video information to the current active application window for display, and the current active application window of the set top box plays the video information played in the picture-in-picture mode, so that the effect of switching to the current active application window (such as a video layer) from the suspension comprehensive picture layer by one key is realized, and the user experience is facilitated.

In step 2004, the general control module processes the pip interactive display closing process, which is the same as the process shown in fig. 23.

The master control module transmits the interactive information to the current active application window to display (such as a page layer), and the current active application window of the set top box displays the picture-in-picture interactive information, so that the effect of switching from the suspension layer to the current active application window by one key is realized, and the user experience is facilitated.

The following description is given with reference to some application examples.

As shown in fig. 25, the video timing chart for floating pip playback includes:

video playing process:

step 2101, pressing the picture-in-picture shortcut key of the remote controller by the user to show that the picture-in-picture is opened for playing;

step 2102, the master control module receives the key processing, judges that the suspended picture-in-picture function needs to be opened, and informs the layer management module;

step 2103, the layer management module creates a comprehensive layer and returns a creation completion message to the master control module;

step 2104, the master control module judges that the video is played, informs the broadcast control management module to play the video, and informs the broadcast control management module to synthesize the layer identifier;

step 2105, the broadcast control management module sends the video information played by the query video information management module;

step 2106, the video information management module returns video information to the broadcast control management module;

step 2107, the broadcast control management module sends a video playing request to the video playing module;

step 2108, the video playing module sends the video and the comprehensive layer identifier to the rendering module;

step 2109, the rendering module plays a rendered video in the integrated layer according to the integrated layer identifier;

step 2110, the video playing module sends a video playing success response to the playing control management module;

step 2111, the broadcast control management module sends a video broadcast success response to the master control module.

Adjusting the attribute flow of the picture-in-picture window:

step 2112, the user presses the shortcut key of picture-in-picture of the remote controller to indicate to adjust the window attribute of picture-in-picture;

step 2113, the master control module receives the key processing, judges the window attribute to be adjusted and informs the broadcast control management module;

step 2114, the playing control management module informs the video playing module of the change of the processing window attribute;

step 2115, the video playing module informs the rendering module of the change of the processing window attribute;

step 2116, the rendering module finishes processing the change of the window attribute, and returns a notification to the video playing module to adjust the video playing response;

step 2117, the video playing module returns a notification to the playing control management module to adjust the video playing response;

step 2118, the broadcast control management module returns a notification to the master control module to adjust the video broadcast response;

step 2119, the master control module informs the layer management module of processing the window attribute change;

step 2110, the layer management module finishes processing layer attribute change and returns a response of adjusting layer attributes to the master control module.

Video playing and video switching process:

step 2121, pressing a picture-in-picture shortcut key of a remote controller by a user to indicate that video switching is carried out;

2122, the master control module receives the key processing, judges that video switching is needed and informs the broadcast control management module;

step 2123, the broadcast control management module queries corresponding video information from the video information management module;

step 2124, the video information management module returns corresponding video information;

step 2125, the broadcast control management module informs the video playing module to play a new video;

step 2126, the video playing module sends the new video and the comprehensive layer identifier to the rendering module;

step 2127, the rendering module plays and renders a new video in the comprehensive layer according to the comprehensive layer identifier;

step 2128, the video playing module sends a new video playing success response to the playing control management module;

and step 2129, the broadcast control management module sends a video switching success response to the master control module.

Closing the picture-in-picture video playing flow:

step 2130, pressing a picture-in-picture shortcut key of the remote controller by a user to show that picture-in-picture video playing is closed;

2131, the master control module receives the key processing, judges that the suspended picture-in-picture function needs to be closed, and sends a picture-in-picture video playing closing request to the playing control management module;

step 2132, the playing control management module triggers a playing stop request to the video playing module;

step 2133, the video playing module stops playing and sends a playing stopping request to the rendering module;

step 2134, the rendering module stops playing and sends a play stop response to the video playing module;

step 2135, the video playing module sends a play stop response to the play control management module;

step 2136, the broadcast control management module releases the broadcast control resources;

step 2137, the playing control management module sends a playing stop response to the master control module;

step 2138, the master control module triggers a request for releasing the comprehensive layer to the layer management module;

step 2139, the layer management module releases the comprehensive layer;

step 2140, the layer management module sends a release layer response to the master control module.

Switching the picture-in-picture layer to the current active application window layer flow:

2141, pressing a shortcut key of picture-in-picture of the remote controller by the user to indicate that the picture-in-picture layer is switched to the current active application window layer;

step 2142, the master control module executes the above-mentioned closed picture-in-picture video playing process;

step 2143, the master control module transmits the video information to the current active application window for playing.

As shown in fig. 26, a timing diagram for floating pip interaction is shown, which includes:

and (3) an interactive display flow:

step 2201, pressing a shortcut key of a picture-in-picture of a remote controller by a user to show that the picture-in-picture interactive display is opened;

step 2202, the master control module receives the key processing, judges that the suspended picture-in-picture function needs to be opened, and informs the layer management module;

step 2203, the layer management module creates a comprehensive layer and returns a creation completion message to the master control module;

step 2204, the master control module judges that the display is interactive display, informs the interactive module and informs the interactive module of the comprehensive layer identifier;

2205, processing the information to be displayed by the rendering module through the interactive module and bringing the comprehensive layer identification;

step 2206, the rendering module renders and displays the comprehensive layer according to the comprehensive layer identifier;

step 2207, the interactive module sends a successful interactive response to the master control module.

Adjusting the attribute flow of the picture-in-picture window:

step 2208, pressing the shortcut key of picture-in-picture of the remote controller by the user to adjust the window attribute of the picture-in-picture;

step 2209, the master control module receives the key processing, judges that the window attribute needs to be adjusted and informs the interactive module;

step 2210, the interaction module notifies the rendering module of a change in the property of the processing window;

2211, the rendering module finishes processing the change of the window attribute and returns a notification to the interaction module to adjust the interaction display response;

2212, the interactive module returns a notification to the master control module to adjust the interactive display response;

2213, the master control module informs the layer management module of processing the window attribute change;

and 2214, the layer management module finishes the layer attribute change processing and returns a response for adjusting the layer attributes to the master control module.

Closing the picture-in-picture interaction process:

step 2215, pressing the shortcut key of picture-in-picture of the remote controller by the user to show that the interactive display of picture-in-picture is closed;

2216, the master control module receives the key processing, judges that the suspended picture-in-picture function needs to be closed, and sends a picture-in-picture closing interactive display request to the interactive module;

step 2217, the interactive module stops interactive display and sends a request for stopping interactive display to the rendering module;

step 2218, the rendering module stops interactive display and sends a response of stopping interactive display to the interactive module;

2219, the interactive module releases the interactive resources;

step 2220, the interactive module sends a response to the master control module to stop the interactive display;

step 2221, the master control module triggers a request for releasing the comprehensive layer to the layer management module;

step 2222, the layer management module releases the integrated layer;

step 2223, the layer management module sends a layer release response to the master control module.

step 2224, the user presses the pip shortcut key of the remote controller to indicate that the pip layer is switched to the current active application window layer;

step 2225, the master control module executes the above-mentioned picture-in-picture closing interaction process;

step 2226, the master control module transmits the interactive information to the current active application window for playing.

In summary, the floating pip provided by the embodiment of the present invention implements video playing and interactive display, so that when a user uses various applications of the set top box, the user can use the pip function without being affected, and can freely zoom the size of the pip window and move the position of the pip window, thereby effectively improving user experience. Therefore, the application has wide significance in the field of set top boxes.

As shown in fig. 27, an embodiment of the present invention further provides a device for implementing picture-in-picture, which is applied to a set top box, and includes:

a receiving unit 2301 for receiving a request to open a picture-in-picture;

a presentation unit 2302 is configured to present a floating pip window independent of a current application of the set top box on a window currently applied by the set top box according to the request.

In one embodiment, the floating pip window has no relationship with the current application of the set top box and is always displayed on the window of the current application of the set top box.

In an embodiment, the presentation unit 2302 is configured to:

In an embodiment, the presentation unit 2302 is configured to sample at least one of the following to present the floating pd window through the integration layer:

when the content of the suspended picture-in-picture window is determined to be a video, video information corresponding to the video is obtained, and the video is rendered and played on the comprehensive layer according to the video information;

In one embodiment, the presentation unit 2302 is used for

In an embodiment, the presentation unit 2302 is further configured to, after receiving a request for opening a pip, determine an integrated layer identifier according to the request for opening the pip;

and determining the layer of the display content as the comprehensive layer according to the comprehensive layer identifier.

In one embodiment, the apparatus further comprises:

and the adjusting unit is used for adjusting the attribute of the suspended picture-in-picture window according to the received request for adjusting the attribute of the picture-in-picture window.

In an embodiment, the adjusting the property of the floating pip window includes at least one of:

In one embodiment, the apparatus further comprises:

and the closing unit is used for closing the floating picture-in-picture window according to the received request for closing the picture-in-picture.

In an embodiment, the closing unit is configured to:

and stopping displaying the content on the comprehensive layer and releasing the comprehensive layer.

In one embodiment, the apparatus further comprises:

and the first switching unit is used for transmitting the content displayed on the comprehensive layer to the current active application window of the set top box for displaying according to the received request for switching from the picture-in-picture window to the current active application window of the set top box, stopping displaying the content on the comprehensive layer and releasing the comprehensive layer.

In one embodiment, the apparatus further comprises:

the second switching unit is used for acquiring target video information according to a received video switching request when the content of the suspended picture-in-picture window comprises videos;

and according to the target video information, playing a target video corresponding to the target video information through the suspended picture-in-picture window.

As shown in fig. 28, an embodiment of the present invention further provides a set top box, including: a memory 2401, a processor 2402 and a computer program 2403 stored on the memory 2401 and executable on the processor 2402, the processor 2402 implementing the above-described method of implementing picture-in-picture when executing the program.

In this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

Claims

1. A method of implementing a picture-in-picture, comprising:

receiving a request to open a picture-in-picture;

2. The method of claim 1,

the suspended picture-in-picture window has no association relation with the current application of the set top box and is always displayed on the window of the current application of the set top box.

3. The method of claim 1, wherein the presenting the floating pip window independent of a current application of the set-top box comprises:

4. The method of claim 3, wherein said revealing the floating pip window by the integration layer includes at least one of:

5. The method of claim 4, wherein the obtaining video information corresponding to the video comprises:

6. The method of claim 3, wherein after receiving the request to open the picture-in-picture, the method further comprises:

determining a comprehensive layer identifier according to the request for opening the picture-in-picture;

7. The method of claim 1, wherein the method further comprises:

and adjusting the attribute of the suspended picture-in-picture window according to the received request for adjusting the attribute of the picture-in-picture window.

8. The method of claim 7, wherein the adjusting the property of the floating pip window comprises at least one of:

9. The method of claim 1, wherein the method further comprises:

closing the floating pip window according to a received request to close pip.

10. The method of claim 9, wherein the closing the floating pip window comprises:

11. The method of claim 1, wherein the method further comprises:

according to a received request for switching from a picture-in-picture window to a current active application window of the set top box, transmitting the content displayed on the comprehensive layer to the current active application window for displaying, stopping displaying the content on the comprehensive layer, and releasing the comprehensive layer.

12. The method of claim 1, wherein when the content of the floating pip window includes video, the method further comprises:

acquiring target video information according to a received video switching request;

13. An apparatus for implementing picture-in-picture, comprising:

a receiving unit for receiving a request to open a picture-in-picture;

14. A set-top box comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of implementing picture-in-picture according to any of claims 1 to 12 when executing the program.

15. A computer-readable storage medium storing computer-executable instructions for performing the method of implementing picture-in-picture according to any one of claims 1 to 12.