WO2024041672A1

WO2024041672A1 - Iptv service-based vr panoramic video playback method and system

Info

Publication number: WO2024041672A1
Application number: PCT/CN2023/125602
Authority: WO
Inventors: 姜超; 熊伟成; 李明
Original assignee: 烽火通信科技股份有限公司
Priority date: 2022-08-22
Filing date: 2023-10-20
Publication date: 2024-02-29
Also published as: CN115379277B; CN115379277A; CN116634207A

Abstract

The application relates to an IPTV service-based VR panoramic video playback method and system. The invention comprises an IPTV service platform and an IPTV terminal; the IPTV service platform comprises IPTV content operations, IPTV service navigation, and IPTV media delivery; the IPTV terminal comprises an IPTV terminal panoramic media processing system, a display device, and an external device; the IPTV terminal panoramic media processing system comprises a browser subsystem, a streaming media subsystem, a graphics rendering subsystem, a web page presentation window module, a VR video presentation window module and a synthesis module. An IPTV application decides, according to an extended JS interface called by the IPTV service navigation, whether to start a local panoramic playback application of the IPTV terminal. When the local panoramic playback application of the IPTV terminal is started, the local application calls a playback interface, creates a 3D model, renders a decoded video image to a 3D model and, when necessary, displays a corresponding UI interface. The invention combines an IPTV service platform and an IPTV terminal capability to achieve an IPTV-supported VR panoramic multimedia service capability.

Description

A VR panoramic video playback method and system based on IPTV service

Cross-references to related applications

This application claims priority from the following patent applications:

A Chinese patent application was submitted to the China Patent Office on August 22, 2022, with the application number 202211008514.2 and the name "A VR panoramic video playback method and system based on IPTV services".

Technical field

This application relates to the field of IPTV technology, and in particular to a VR panoramic video playback method and system based on IPTV services.

Background technique

IPTV (Interactive Internet Television) panoramic multimedia service refers to providing users with 360-degree panoramic video, pictures, audio, text and other multimedia content services through the use of panoramic playback technology and IPTV terminals. Through human-computer interaction, users can obtain visual perception, auditory perception and immersive experience in the virtual environment.

With the development of IPTV business intelligent technology and video diversification, integrating VR panoramic video into IPTV business systems has become an inevitable trend. At present, the IPTV business platform has been deployed, including IPTV business navigation, IPTV media delivery and IPTV content operation. IPTV business navigation provides EPG (Electronic Program Guide, Electronic Program Guide) pages for IPTV terminals, and some provide broadcast control UI and unified button processing functions. However, the EPG page provided is a 2D page, not a 360-degree panoramic page. IPTV media delivery provides streaming video content, but the current content provision of panoramic video is also based on existing IPTV media delivery. When IPTV media delivery provides video content, it cannot distinguish whether the video content is panoramic video or non-panoramic video.

How to implement VR panoramic multimedia service capabilities based on IPTV services based on existing IPTV systems combined with IPTV terminal capabilities is an important core issue to be solved in this field. These problems include: first, IPTV service navigation needs to enable users to uniformly control the panoramic video content displayed on the IPTV terminal by switching up, down, left, and right 360-degree angles, and to uniformly control the playback status of fast forward, backward, and pause. Unified control of zooming in and out for viewing is a core issue that needs to be solved in this field. Second, it is necessary to realize the function that the 2D pages in IPTV service navigation, including EPG and broadcast control UI, can be displayed normally on a flat surface, and the panoramic video content obtained by IPTV media delivery can be modeled and rendered into a 3D sphere without conflicting with the 2D flat display of IPTV service navigation. Third, since IPTV panoramic video is currently in the early stages of development, IPTV business navigation does not have unified control capabilities, so it is still necessary to achieve compatibility with IPTV panoramic video. During the development process, IPTV business navigation does not have unified control capabilities, and can be played locally through IPTV terminals. The application obtains video from IPTV media delivery, and the local panoramic playback application implements video rendering, playback control and UI display; it can also be compatible with the later development of IPTV business navigation and support IPTV panoramic video unified control capabilities.

Existing technology: The existing IPTV service navigation includes EPG, broadcast control UI, and some remote control operation logic on the page, which can create and control ordinary flat-panel players. However, the flat player does not have the functions of 360-degree viewing angle switching and viewing angle zooming, and the IPTV service navigation does not have the ability to operate the VR panoramic player. IPTV media delivery has the function of transmitting video content data, but it cannot distinguish whether it is a flat video or a 360-degree panoramic video, so it cannot tell the IPTV terminal the video characteristics. Further, the IPTV terminal cannot know whether the video needs to be sphere rendered, nor can it perform the video rendering. Perspective switching and perspective zooming.

Problem 1 with the existing technology: It is impossible to realize the function of notifying the IPTV terminal to play the panoramic video based on the existing IPTV service navigation, and controlling the panoramic video (rotation, zooming, and playback control) without conflicting with the control logic of remote control of ordinary video. For example, if you press the left and right buttons, the function corresponding to ordinary video is fast forward and rewind, but the function corresponding to panoramic video is rotation angle.

Problem 2 in the existing technology: If the IPTV service navigation only transmits the URL (Uniform Resource Locator, Uniform Resource Locator) address of the panoramic video, all the broadcast control logic can only be implemented by the IPTV terminal, and unified navigation and unified button processing logic cannot be achieved. , the purpose of unifying the UI. This will cause users to have different operating logic when operating different videos. When users use terminals from different manufacturers, the operation logic is different and the display content is also different.

In view of this, how to overcome the shortcomings of existing technologies and how to implement IPTV's ability to support VR panoramic multimedia business services based on existing IPTV platforms combined with IPTV terminal capabilities are difficult problems to be solved in this technical field.

Application content

In view of the deficiencies or improvement needs in the existing technology: First, it is necessary to establish a system that supports IPTV service navigation. The content displayed on the IPTV terminal (set-top box) can be switched up, down, left, and right at 360-degree angles for unified control, and fast forwarding and fast forwarding can be performed. IPTV terminal panoramic media processing system with unified control of unpause and playback status, and unified control of zoom in and out for viewing. Second, it is necessary to implement the function of modeling and rendering panoramic video content into a 3D sphere without conflicting with the 2D flat display of IPTV service navigation. Third, it needs to be compatible with IPTV panoramic video. During the development process, IPTV business navigation does not have a unified control period. Videos can be obtained from IPTV media delivery through the local panoramic playback application of the IPTV terminal, and the local panoramic playback application can implement video rendering and playback control. and UI display functions; it can also be compatible with the ability to support IPTV panoramic video unified control after the IPTV service navigation matures. In response to the above needs, this application provides a VR panoramic video playback method and system based on IPTV services to achieve the above needs.

The embodiments of this application adopt the following technical solutions:

In the first aspect, this application provides a VR panoramic video playback method based on IPTV services, including two ways of starting the IPTV terminal panoramic media processing system:

Led by IPTV business navigation, API calls are used to decide whether to perform the action of changing the viewing angle or changing the playback state; when the IPTV terminal receives the key event, it sends the key event to the IPTV business navigation; and/or:

By calling the local panoramic video playback application of the IPTV terminal, it is dominated by the local panoramic video playback application of the IPTV terminal: when the IPTV terminal receives a key event, the local panoramic video playback application decides to perform the action of changing the viewing angle or changing the playback state. UI The display is also implemented by a local panoramic video playback application.

Further, the startup of the IPTV terminal panoramic media processing system is led by IPTV business navigation, and when called through API, specifically includes:

The browser subsystem parses and initializes the VR panoramic player object, including initializing the VR panoramic feature capabilities of the API of the VR panoramic player object; the VR panoramic feature capabilities of the API include video playback state control capabilities and perspective changing functions;

Obtain the 3D spherical model and 2D video data and map the output frames of the 2D video data to the 3D spherical model;

When a key event is triggered, the browser subsystem calls the corresponding VR panoramic feature capability according to the key event and displays the corresponding UI;

The 2D display layer of the UI display and the 3D display layer of the 3D spherical model are combined and displayed.

Further, obtaining the 3D spherical model and 2D video data and mapping the output frames of the 2D video data to the 3D spherical model specifically includes:

The browser subsystem calls the media control module to bind the 3D spherical model to the 3D display layer, combine it with the VR panoramic player object, and provide VR panoramic feature capabilities;

The VR panoramic player object connects with the streaming media subsystem through the VR panoramic video address URL parameter passed in by the IPTV service navigation to obtain 2D video data;

Map the output frames of the 2D video data decoded by the VR panorama player object to a 3D spherical model.

Further, when the key event is triggered, the browser subsystem calls the corresponding VR panoramic feature capability according to the key event, and displays the corresponding UI, which specifically includes:

When a key event is triggered, the browser subsystem determines whether the perspective control enable state of the VR panoramic player object is enabled;

If it is not enabled, the video playback status control capability interface is called through IPTV service navigation and the corresponding playback control UI is displayed; if it is enabled, it is further determined whether the perspective control attribute of the VR panoramic player object is IPTV service navigation control;

If it is an IPTV service navigation control, the corresponding perspective change function is called according to the key event through the IPTV service navigation, and the corresponding UI is displayed; if it is not an IPTV service navigation control, the browser subsystem avoids this by modifying the key processing logic in the kernel. The key event is sent to the IPTV business navigation, and the browser subsystem calls the perspective change function itself, and sends the current state after the perspective change to the IPTV business navigation through a message callback, so that the IPTV business navigation changes according to the perspective sent by the browser subsystem. The corresponding UI is displayed after the current status.

Further, when the IPTV terminal panoramic media processing system is started by calling the IPTV terminal's local panoramic video playback application and is dominated by the IPTV terminal's local panoramic video playback application, it specifically includes:

When the IPTV service navigation starts VR panoramic video playback through ACTION, the VR panoramic video address URL parameter is added to identify the video as a panoramic video; the browser subsystem parses ACTION and obtains the VR panoramic video address URL parameter; the browser subsystem starts the IPTV terminal Local panoramic video playback application, and pass in the VR panoramic video address URL parameters;

The local panoramic video playback application calls the UI display module and obtains the corresponding UI display based on the obtained VR panoramic video address URL parameters;

The local panoramic video playback application obtains the 3D spherical model and synthesizes its 3D display layer with the 2D display layer displayed by the UI;

The key event is passed to the local panoramic video playback application for processing. If the perspective changes, the 2D display layer of the UI display and the 3D display layer of the 3D spherical model are reacquired and combined for display.

Further, the local panoramic video playback application obtains the 3D spherical model and synthesizes its 3D display layer with the 2D display layer of the UI display, specifically including:

The local panoramic video playback application calls the media control module to bind the 3D spherical model to the 3D display layer and provide VR panoramic feature capabilities;

Call the 2D player and connect the VR panoramic video address URL parameter passed in through the IPTV service navigation to the streaming media subsystem;

After mapping the output frame of the 2D player to the 3D spherical model, the 2D display layer of the UI display and the 3D display layer of the 3D spherical model are combined and displayed.

Furthermore, when the IPTV business platform delivers the live broadcast list, a parameter is added to identify whether the live broadcast program is a panoramic video; when the live broadcast playback interface of the IPTV terminal is called, it is judged whether the live broadcast to be played is a panoramic video, and if so, the panoramic display and Panoramic playback, and capturing the direction keys of the remote control to change the perspective of the panoramic display;

During the panoramic playback process, when performing operations such as pausing playback, double-speed playback, positioning playback, or switching back to the live broadcast state, the IPTV terminal will send key events to the IPTV business navigation or local panoramic video playback application, and the IPTV business navigation or local panoramic video playback application will Call the corresponding broadcast control interface to implement the corresponding function; when the IPTV terminal receives the call request of the corresponding broadcast control interface, it must ensure that the video is displayed in panoramic view during the broadcast control operation; when the IPTV terminal's exit live broadcast interface is called, it exits the panoramic display .

Furthermore, when the IPTV business platform delivers the on-demand program address, a parameter is added to identify whether the on-demand program is a panoramic video; when the on-demand playback interface of the IPTV terminal is called, it is judged whether the on-demand program to be played is a panoramic video, and if so, the panoramic display is started. , and capture the direction keys of the remote control to change the perspective of the panoramic display;

During the panoramic playback process, when performing operations such as pausing playback, double-speed playback, or positioning playback, the IPTV terminal sends key events to the IPTV business navigation or local panoramic video playback application, and the IPTV business navigation or local panoramic video playback application calls the corresponding playback control interface to implement the corresponding function; when the IPTV terminal receives a call request from the corresponding broadcast control interface, it must ensure that the video is displayed in panoramic view during the broadcast control operation. When the IPTV terminal's exit on-demand interface is called, it exits the panoramic display.

Furthermore, it also includes a static image browsing function: when the panoramic video is paused or when static images are browsed, the IPTV terminal sends key events to the IPTV business navigation or local panoramic video playback application, and the IPTV business navigation or local panoramic video playback application calls the corresponding The broadcast control interface is used to implement the corresponding functions, so that users can rotate the paused video screen or static image at any angle, and watch the content of the video screen or static image at any angle in an all-round way.

In a second aspect, this application also provides a VR panoramic video playback system based on IPTV services to implement the method described in the first aspect. The system includes an IPTV service platform and an IPTV terminal; the IPTV service platform includes IPTV content. Operation, IPTV service navigation, and IPTV media delivery; the IPTV terminal includes an IPTV terminal panoramic media processing system, a display device, and an external device; the IPTV terminal panoramic media processing system includes a browser subsystem and a streaming media subsystem, wherein, The above-mentioned streaming media subsystem provides VR panoramic feature capabilities and interacts with IPTV media delivery through streaming media. The browser subsystem is responsible for parsing the control instructions of IPTV service navigation for VR panoramic content playback, receiving key events, and controlling the playback of VR panoramic content for the streaming media subsystem according to the requirements of IPTV service navigation. To implement response to media playback events; the startup method of the IPTV terminal panoramic media processing system includes:

Led by IPTV business navigation, through API calls: when the IPTV terminal receives a key event, it sends the key event to the IPTV business navigation, and the IPTV business navigation decides to perform the action of changing the viewing angle or changing the playback status; and/or:

Further, the streaming media subsystem includes a VR panoramic video control module. The VR panoramic video control module creates a VR panoramic player object and provides a set of APIs containing VR panoramic feature capabilities. The VR panoramic view of this set of APIs Characteristic capabilities include video playback state control capabilities and perspective changing functions. The video playback state control capabilities include pausing playback, double-speed playback, positioning playback, and resuming live broadcast status. The perspective changing functions include perspective direction changing capabilities and perspective distance changing capabilities. ability.

Compared with the existing technology, the beneficial effects of this application are:

1. The IPTV terminal panoramic media processing system decides whether to start the IPTV panoramic playback application based on the extended JS interface (i.e. API) called by the IPTV business navigation. When the IPTV business navigation chooses to call the IPTV panoramic playback API, the IPTV terminal panoramic media processing system will return the VR panoramic player object called by the IPTV terminal to the IPTV business navigation. The IPTV business navigation calls the IPTV panoramic playback external API according to the current status. Realize panoramic video playback, angle of view rotation, change of angle of view size, and corresponding UI display. JS extended VR panoramic player object is an object newly created by IPTV business navigation. IPTV's VR panoramic business is implemented by calling the method of VR panoramic player object. It can support operators' unified control logic and unified display UI solution for VR panoramic services. Facilitates unified and rapid deployment of VR panoramic services.

2. When the local panoramic video playback application is started, the local panoramic video playback application calls the playback interface, calls the 3D model, renders the decoded video image to the 3D model, and displays the corresponding UI interface when necessary. Compatible with ACTION mode: The ACTION startup of the local panoramic video playback application is delivered through the extended JS object, and the key processing is handed over to the local panoramic video playback application. The two solutions 1 and 2 are compatible and coexist to meet the needs of operators in the initial stage and mature stage of building and developing VR panoramic video.

3. The VR panoramic video control module, video protocol and data processing module are in a shared state for the two startup methods (i.e. IPTV business navigation calls the API for panoramic playback and the local panoramic video playback application).

4. The browser subsystem avoids sending key events to the IPTV business navigation by transforming the key processing logic in the kernel. By customizing the button processing logic of the browser subsystem, it simplifies the logical transformation of the IPTV business navigation direction key processing. .

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the drawings required to be used in the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is an architectural block diagram of a VR panoramic video playback system based on IPTV services provided in Embodiment 1 of the present application;

Figure 2 is a schematic diagram of the module flow of two different startup methods provided in Embodiment 1 of the present application;

Figure 3 is a business flow chart of two different startup methods provided in Embodiment 1 of the present application;

Figure 4 is a schematic diagram of two ways to start the IPTV terminal panoramic media processing system of a VR panoramic video playback method based on IPTV services provided in Embodiment 2 of the present application;

Figure 5 is a detailed flow chart of the first method of starting an IPTV terminal panoramic media processing system provided in Embodiment 2 of the present application;

Figure 6 is an expanded flow chart of step 120 provided in Embodiment 2 of the present application;

Figure 7 is an expanded flow chart of step 130 provided in Embodiment 2 of the present application;

Figure 8 is a detailed flow chart of the second method of starting an IPTV terminal panoramic media processing system provided in Embodiment 2 of the present application;

Figure 9 is an expanded flow chart of step 230 provided in Embodiment 2 of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

This application is an architecture of a specific functional system. Therefore, in the specific embodiments, the functional logical relationship of each structural module is mainly explained, and the specific software and hardware implementation methods are not limited.

In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other, and the order of each step can also be reversed if it is logical and does not conflict. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

In addition, it is necessary to first explain the definitions of some terms in this application, such as:

IPTV panoramic multimedia service refers to providing users with 360-degree panoramic video, pictures, audio, text and other multimedia content services through the use of panoramic playback technology and IPTV terminals. Through human-computer interaction, users can obtain visual perception, auditory perception and immersive experience in the virtual environment. IPTV panoramic content mainly includes panoramic videos and panoramic pictures. IPTV panoramic playback refers to playing panoramic videos and panoramic pictures.

Panoramic video uses professional panoramic cameras to collect video content. Later, it is spliced into a seamless "ball" through panoramic video stitching software, and finally a spherical panoramic video with a 360-degree panoramic perspective is output. A panoramic picture is a group or multiple groups of photos taken by the camera ring 360 degrees into a panoramic image.

Panoramic playback technology dynamically restores the real environment, making the audience feel that they are in the live environment. Through interactive operations, 360-degree panoramic observation can be performed left, right, up, or down, giving the audience a virtual experience that spans space.

Example 1:

As shown in Figure 1, Embodiment 1 of the present application provides a VR panoramic video playback system based on IPTV services. The architectural block diagram of the system is shown in Figure 1, including an IPTV service platform and an IPTV terminal; the IPTV service platform includes IPTV Content operation, IPTV service navigation, and IPTV media delivery; the IPTV terminal includes an IPTV terminal panoramic media processing system, display equipment, and external equipment. Among them, the IPTV terminal panoramic media processing system interacts with IPTV business navigation and IPTV media delivery. External devices can interact with the IPTV terminal panoramic media processing system via buttons. The IPTV terminal panoramic media processing system displays the processed panoramic video output on the display device. superior.

The above IPTV media delivery is an existing system and has not been changed in this embodiment. This embodiment obtains VR panoramic video from IPTV media delivery.

IPTV service navigation is an existing system that supports JS extension. This embodiment requires adding JS to the IPTV service navigation to extend the VR panoramic player object called by this application. To realize the operation of VR panoramic player object through IPTV service navigation. The entire VR panoramic player object provides a set of APIs containing VR panoramic feature capabilities. This set of APIs has one or more functions such as pausing playback, double-speed playback, positioning playback, switching back to live broadcast status, and changing the perspective. The viewing angle changing function also includes one or more of: the ability to change the viewing angle direction; the ability to change the viewing angle far and near, and other functions.

IPTV service navigation supports EPG UI display logic, which is an existing technology. However, in the prior art, there is a conflict between the UI display logic in panoramic mode and the UI display logic in non-panoramic mode. Existing IPTV service navigation cannot distinguish the UI display logic of an IPTV terminal in panoramic mode and the UI display logic of non-panoramic mode. Therefore, it is necessary to support the EPG UI display logic of IPTV service navigation in these two modes through the IPTV terminal panoramic media processing system of the embodiment of this application.

As shown in Figure 1, the IPTV terminal panoramic media processing system of this embodiment is the key to this application. Specifically, the IPTV terminal panoramic media processing system mainly includes a browser subsystem and a streaming media subsystem, wherein the streaming media subsystem The system provides VR panoramic feature capabilities and has the ability to interact with IPTV media delivery to obtain video content through streaming media protocols; the browser subsystem is responsible for parsing the control statements of IPTV business navigation for panoramic videos, and is responsible for receiving key events, and According to the requirements of IPTV business navigation, the streaming media subsystem performs VR playback control and responds to media playback events; among them, key events refer to events sent by the user using the remote control, including up, down, left, right and other key events; while media Playback events include playback state events: pause playback, double-speed playback, positioning playback, switch back to live broadcast state, perspective direction change, perspective distance change, etc. The startup method of the IPTV terminal panoramic media processing system includes:

By calling the local panoramic video playback application of the IPTV terminal, it is dominated by the local panoramic video playback application of the IPTV terminal: when the IPTV terminal receives a key event, the local panoramic video playback application decides to perform the action of changing the viewing angle or changing the playback state. UI The display is also broadcast by local panoramic video Put application implementation.

Referring to Figure 1, in addition to the browser subsystem and streaming media subsystem, the IPTV terminal panoramic media processing system also includes hardware drivers and operating system subsystems, graphics rendering subsystems, and presentation subsystems. The presentation subsystem includes WEB Page presentation window module, VR video presentation window module, synthesis module.

Specifically, the hardware driver and operating system subsystem are existing technologies.

Referring to Figure 2, the graphics rendering subsystem includes a 3D modeling module, a 3D model parsing module and a texture mapping module. The 3D modeling module is used for 3D modeling, and the 3D model parsing module is used for 3D modeling. Analysis, the texture mapping module is used to perform mapping processing on 3D models. In this application, the browser subsystem's ability to control the graphics rendering subsystem is an innovative point.

Referring to Figure 1, the WEB page presentation window module of the presentation subsystem is responsible for presenting the UI corresponding to the 2D EPG page and IPTV service navigation, and the VR panoramic video presentation window module is responsible for displaying the 3D spherical model of the VR panoramic video rendering. The synthesis module is responsible for synthesizing and displaying the presentation content of the WEB page presentation window module and the VR panoramic video presentation window module.

In the browser subsystem of this embodiment, mode judgment and key processing are the innovative points of this application. Specifically, the browser subsystem includes a judgment module, which is used to judge whether the viewing angle control enabling state of the VR panoramic video player object is enabled; if it is not enabled, the video playback state control capability is called through IPTV service navigation interface, and displays the corresponding broadcast control UI; if it is enabled, it will further determine whether the perspective control attribute of the VR panoramic video player object is IPTV service navigation control; if it is IPTV service navigation control, the IPTV service navigation will be based on the key event (key press The event is a button or direction key triggered by the user) to call the corresponding perspective change function and display the corresponding UI; if it is not for IPTV service navigation control, the browser subsystem avoids sending key events by transforming the key processing logic in the kernel. For IPTV business navigation, the browser subsystem calls the perspective change function itself, and sends the current state after the perspective change to the IPTV business navigation through a message callback, so that the IPTV business navigation can be based on the current state after the perspective change sent by the browser subsystem. Display the corresponding UI.

It should be noted that the entity of the VR panoramic player object in this embodiment is located in the streaming media subsystem. This object can be called by the local panoramic video playback application or the browser subsystem.

Referring to Figures 2 and 3, the streaming media subsystem includes the picture decoding module and video decoding module in Figure 2, as well as the VR panoramic video control module and video transmission protocol and data acquisition capabilities in Figure 3. Video transmission protocols and data acquisition capabilities are existing technologies, and their function is to obtain video streams from IPTV media delivery through streaming media protocols such as IGMP\RTSP\HLS, etc. The VR panoramic video control module is created by this application. The innovation lies in creating a VR panoramic player object and providing a set of APIs containing VR panoramic feature capabilities. The VR panoramic feature capabilities of this API include video playback status control capabilities and A viewing angle changing function, wherein the video playback state control capabilities include pausing playback, doubled speed playback, positioning playback, and switching back to live broadcast state; the viewing angle changing function includes a viewing angle direction changing capability and a viewing angle distance changing capability. Regarding the set of APIs in this embodiment, it should be noted that this set of APIs is provided by the browser subsystem for IPTV service navigation. The capabilities of the APIs include the streaming media subsystem (the entity of the VR panoramic player object belongs to the streaming media subsystem). media subsystem). The browser subsystem calls the VR panoramic player object. The entity of the VR panoramic player object is the streaming media subsystem and the graph. Combining some capabilities of the shape rendering subsystem. IPTV service navigation calls the API of the VR panoramic player object through the browser subsystem.

This application also created the function of controlling the viewing angle change through the browser subsystem, including providing JS objects (VR panoramic player objects) to realize interaction with IPTV business navigation. The buttons for user operation are processed by the browser subsystem and IPTV service navigation, and the model position and angle control instructions of the IPTV service navigation are applied to the graphics rendering module. Referring to Figure 1, this application also creates VR playback control and media playback event interaction functions between the browser subsystem and the streaming media subsystem. The VR playback control interactive function specifically includes: sending the control of user key operations and IPTV service navigation to the streaming media subsystem through the browser subsystem to realize pause playback, double-speed playback, positioning playback, and switch back to the VR panoramic player. Live status capability. The media playback event interaction function specifically includes: after the streaming media subsystem responds to the VR playback control command, it notifies the browser subsystem of the playback status events including paused playback, doubled speed playback, positioning playback, switching back to live broadcast status, etc. .

Referring to Figure 2, Figure 2 also includes a media control module, a UI display module, and a button processing module.

Media control module: The main function is to respond to video playback control key operations and process the video, such as starting playback, exiting playback, pausing playback, double-speed playback, positioning playback, switching back to live broadcast status, etc.

UI display module: used to display user interface related to panoramic playback. Including mode display, such as full screen mode, menu mode, pause mode.

Full screen mode one:

1) When the user clicks the OK button, enter the menu mode.

2) In full-screen mode, it supports prompting the user to click the OK button to perform fast forward/rewind operations. The prompt box can automatically disappear after a certain period of time.

Full screen mode two:

1) After entering the video, when you turn the viewing angle for the first time, a prompt box of "Double-click the OK button" will be displayed. The prompt box will automatically disappear after a certain period of time.

2) When the user double-clicks the OK button, the screen will switch back to the initial perspective.

Menu mode:

1) In menu mode, the playback control bar appears. Press and hold the left and right keys to perform fast forward and rewind operations.

2) Click the OK button to enter "Pause Mode".

Pause mode:

In pause mode, click OK to enter full-screen mode and start playing the video.

Key processing module: Key events from external devices (mainly remote controls) are processed. Key processing includes operations on the media control module, such as starting playback, exiting playback, pausing playback, double-speed playback, positioning playback, switching back to live broadcast status, etc. It also includes operations on UI display, such as displaying progress bars, displaying playback status, etc. It also includes operations on the browser subsystem, such as up, down, left, right, etc. It also includes operations on the 3D model analysis module, such as changes in the direction of the viewing angle and changes in the distance of the viewing angle.

As shown in Figure 2, the solid and dotted arrows represent the IPTV terminal panoramic media processing system created by this application. There are two startup methods. First, it is dominated by IPTV business navigation. JavaScriptAPI call, when the IPTV terminal receives user keys from the key processing module, the browser subsystem receives the keys and sends the keys (mainly direction keys) to the IPTV business navigation, which decides to change the viewing angle or change the playback state. Actions. After parsing the key processing logic of the IPTV service navigation page, the browser subsystem calls the media control module and 3D modeling module capabilities.

Second, by calling the local panoramic video playback application of the IPTV terminal, the local panoramic video playback application of the IPTV terminal is dominated. When the local panoramic video playback application of the IPTV terminal receives user keys, the key processing module includes the media control keys. Notifications such as starting playback, exiting playback, pausing playback, double-speed playback, positioning playback, switching back to live broadcast status, etc. are handled by the media control module. The key processing module will display progress bars, display playback status, mode display and other operations on the UI display and notify the UI display module for processing. The key processing module notifies the 3D model analysis module of changes in the direction of the viewing angle and changes in the distance of the viewing angle. The local panoramic video playback application decides to perform the action of changing the viewing angle or changing the playback state. The UI display is also implemented by the local panoramic video playback application.

As shown in Figure 3, it is a business flow chart of two different startup methods in this application. The solid line frame on the right is public capabilities, which are implemented by the streaming media subsystem, including providing VR panoramic video control module (jar), providing VR panoramic feature capabilities; providing video transmission protocol and data acquisition function (ctc_so), and IPTV streaming media Server docking, while also binding the rendering layer to the VR panoramic video control module; providing a synthesis module for synthesis display processing. The lower part of the business flow (the dotted box part) is dominated by IPTV business navigation and is implemented through JavaScript API calls. Specifically, the business navigation system JS calls the VR panoramic player object, and the browser subsystem parses and analyzes the VR panoramic player object. Initialization; the VR panoramic player object calls the video transmission protocol in the streaming media subsystem to obtain data from the IPTV media delivery, and binds it to the VR panoramic feature rendering layer of the 3D model parsing module to output the VR panoramic video. The business navigation system provides key processing logic and UI display logic, browser subsystem analysis, and key processing; the above processing is passed to the browser subsystem, and the browser subsystem interacts with the IPTV business navigation to obtain the UI, so the UI all comes from IPTV Business Navigation. Combine the UI derived from IPTV service navigation and VR panoramic video to display the final image. The upper part of the business flow (the part without dotted lines) is dominated by calling the local panoramic video playback application of the IPTV terminal. Specifically, the business navigation system JS calls ACTION or other application calls. ACTION, carries the panoramic video URL and passes it to the streaming media subsystem. The streaming media subsystem obtains the video from the IPTV media delivery through the video transmission protocol and binds it to the VR panoramic feature rendering layer of the 3D model parsing module, through the local panoramic video playback application. Comes with UI display combination to display the final image.

The above is an introduction to each subsystem and each module of the VR panoramic video playback system based on IPTV services in this embodiment. Below, the VR panoramic video based on IPTV services in this embodiment is explained through business scenarios, system functional requirements, and application layers. The playback system is further explained.

For business scenarios, IPTV panoramic services include the following typical business scenarios:

IPTV live program playback: When playing IPTV live programs, if the video content is panoramic video, it needs to support the panoramic playback function, and have functions such as pausing playback, double-speed playback, positioning playback, switching back to live broadcast status, and changing perspective.

IPTV on-demand program playback: When playing IPTV on-demand programs, if the video content is panoramic video, It needs to support the panoramic playback function, and have the functions of pausing playback, double-speed playback, positioning playback, switching back to live broadcast state, and changing perspective.

Static image browsing: When browsing static images or pausing panoramic videos, you can rotate the static images or paused videos 360 degrees to watch them from any angle.

Start the IPTV terminal panoramic media processing system: There are two ways. First, it is led by IPTV business navigation and called through JavaScript API. When receiving user keys, the IPTV terminal sends the keys (mainly direction keys) to the IPTV service navigation. The IPTV service navigation determines whether to perform the action of changing the viewing angle or changing the playback status. Second, by calling the local panoramic video playback application of the IPTV terminal. It is dominated by the local panoramic video playback application of the IPTV terminal. When the IPTV terminal receives user buttons, the local panoramic video playback application decides to perform the action of changing the viewing angle or changing the playback state. The UI display is also implemented by the local panoramic video playback application.

For system functional requirements:

IPTV live broadcast: When the IPTV business platform issues a live broadcast list, a parameter is added to identify whether the live program is a panoramic video. When the terminal's live broadcast interface (joinchannel) is called, it determines whether the live broadcast to be played is a panoramic video. If so, the panoramic display is started and the direction keys of the remote control are captured to switch between different viewing angles. During the panoramic playback process, when the user operates to pause playback, double-speed playback, position playback, or switch back to the live broadcast state, the IPTV terminal will send the key value to the IPTV business navigation or local panoramic video playback application, and the IPTV business navigation or local panoramic video playback application will Call the corresponding broadcast control interface to implement the function. The terminal receives the corresponding broadcast control interface and needs to ensure that the video is displayed in panorama during the broadcast control operation. When the IPTV terminal's live broadcast interface (leavechannel) is called, exit the panoramic display.

IPTV on-demand live broadcast: When the IPTV business platform issues the on-demand program address, a parameter is added to identify whether the on-demand program is a panoramic video.

When the terminal's on-demand playback interface (playfromstart) is called, it is determined whether the on-demand video to be played is a panoramic video. If so, the panoramic display is started and the direction keys of the remote control are captured to switch between different viewing angles. And during the panoramic playback process, when the user operates to pause playback, double-speed playback, or position playback, the IPTV terminal sends the key value to the IPTV business navigation or local panoramic video playback application, and the IPTV business navigation or local panoramic video playback application calls the corresponding playback function. The control interface implements the function. The terminal receives the corresponding broadcast control interface and needs to ensure that the video is displayed in panorama during the broadcast control operation. When the terminal's exit on-demand interface (stop) is called, exit the panoramic display.

IPTV terminal panoramic media processing system: It has the function of zooming in and out of the viewing angle, the function of converting the viewing angle (including the four directions of up, down, left and right), the reporting function of the viewing angle change status and the reporting function of the broadcast control status. It has two startup methods. First, it is started by JavaScript API call. The IPTV terminal panoramic media processing system provides JS extension objects for IPTV business navigation. IPTV service navigation can be started by calling JS extension object. Second, the IPTV terminal panoramic media processing system supports ACTION mode startup. IPTV service navigation can be started by extending the JS object and issuing ACTION. It can also be started by sending ACTION from other applications.

For the application layer, the IPTV panoramic playback application decides whether to start the IPTV panoramic playback application based on the extended JS interface called by the IPTV business navigation. When the local panoramic playback application is started, the local panoramic playback application calls the playback interface, calls the 3D model, and renders the decoded video image onto the 3D model. And display the corresponding UI interface when necessary. When the IPTV business navigation chooses to call the API of IPTV panoramic playback, the IPTV terminal panoramic media processing system will return the VR panoramic player object called by the IPTV terminal to the IPTV business navigation. The IPTV business navigation calls the external API of IPTV panoramic playback according to the current status. Realize panoramic video playback, angle of view rotation, change of angle of view size, and corresponding UI display.

In summary, the IPTV panoramic playback application in the embodiment of the present application determines whether to start the IPTV panoramic playback application based on the extended JS interface (that is, API) called by the IPTV service navigation. When the IPTV business navigation chooses to call the IPTV panoramic playback API, the IPTV terminal panoramic media processing system will return the VR panoramic player object called by the IPTV terminal to the IPTV business navigation. The IPTV business navigation calls the IPTV panoramic playback external API according to the current status. Realize panoramic video playback, angle of view rotation, change of angle of view size, and corresponding UI display. JS extended VR panoramic player object is an object newly created by IPTV business navigation. IPTV's VR panoramic business is implemented by calling the method of VR panoramic player object. It can support operators' unified control logic and unified display UI solution for VR panoramic services. Facilitates unified and rapid deployment of VR panoramic services.

In the embodiment of this application, when the local panoramic video playback application is started, the local panoramic video playback application calls the playback interface, calls the 3D model, renders the decoded video image to the 3D model, and displays the corresponding UI interface when necessary. Compatible with ACTION mode: The ACTION startup of the local panoramic video playback application is delivered through the extended JS object, and the key processing is handed over to the local panoramic video playback application.

The above two solutions are compatible and coexist, and meet the needs of operators in the initial stage and mature stage of building and developing VR panoramic video.

The VR panoramic video control module, video protocol and data processing module of the embodiment of this application are shared. The embodiment of the present application also customizes and transforms the key processing logic of the browser subsystem to simplify the logical transformation of IPTV service navigation processing direction keys.

Example 2:

As shown in Figure 4, based on the system provided in Embodiment 1, Embodiment 2 of the present application provides a VR panoramic video playback method based on IPTV services. The method mainly includes two ways of starting the IPTV terminal panoramic media processing system:

Type 1: Led by IPTV service navigation, through API calls: when the IPTV terminal receives a key event, it sends the key event to the IPTV service navigation, and the IPTV service navigation decides to perform the action of changing the viewing angle or changing the playback state; and/or :

Type 2: By calling the local panoramic video playback application of the IPTV terminal, the local panoramic video playback application of the IPTV terminal is dominant: when the IPTV terminal receives a key event, the local panoramic video playback application decides to change the viewing angle or change the playback state. The action and UI display are also implemented by the local panoramic video playback application.

For the first method of starting the IPTV terminal panoramic media processing system, it is led by the IPTV business navigation. When calling through the API, that is, when the IPTV business navigation starts the panoramic playback through the VR panoramic player object, refer to Figure 5, which specifically includes the following steps:

Step 110: The browser subsystem parses and initializes the VR panoramic player object. including initial The API of the VR panoramic player object has VR panoramic feature capabilities; the VR panoramic feature capabilities of the API include video playback state control capabilities and perspective changing functions, wherein the video playback state control capabilities include pause playback, double-speed playback , positioning playback, switching back to live broadcast state, and other functions; the viewing angle changing function includes one or more of the viewing angle direction changing capabilities, viewing angle distance changing capabilities, and other functions.

Step 120: Obtain the 3D spherical model and 2D video data and map the output frames of the 2D video data to the 3D spherical model.

Step 130: When a key event is triggered, the browser subsystem calls the corresponding VR panoramic feature capability according to the key event and displays the corresponding UI.

Step 140: Combine and display the 2D display layer of the UI display and the 3D display layer of the 3D spherical model.

Referring to Figure 6, the above step 120 can also be extended to the following steps:

Step 121: The browser subsystem calls the media control module to bind the 3D spherical model to the 3D display layer, combine it with the VR panoramic player object, and provide VR panoramic feature capabilities. The media control module also needs to receive the keys passed to the media control module by the key processing module. Button processing capabilities include: video playback status control capabilities include pausing playback, double-speed playback, positioning playback, and switching back to live broadcast status; viewing angle changing functions include viewing angle direction changing capabilities and viewing angle distance changing capabilities; the media control module receives button information and then controls the VR Module called by panorama player object capabilities.

Step 122: The VR panoramic player object connects with the streaming media subsystem through the VR panoramic video address URL parameter passed in by the IPTV service navigation to obtain 2D video data.

Step 123: Map the output frames of the 2D video data decoded by the VR panoramic player object to the 3D spherical model.

Referring to Figure 7, the above step 130 can also be extended to the following steps:

Step 131: When a key event is triggered, the browser subsystem determines whether the perspective control enable state of the VR panoramic player object is enabled.

Step 132: If it is not enabled, call the video playback status control capability interface through IPTV service navigation and display the corresponding playback control UI; if it is enabled, further determine whether the perspective control attribute of the VR panoramic player object is IPTV service navigation control.

Step 133: If it is an IPTV service navigation control, the corresponding perspective change function is called according to the key event through the IPTV service navigation, and the corresponding UI is displayed; if it is not an IPTV service navigation control, the browser subsystem processes the keys by modifying the kernel Logic to avoid sending key events to the IPTV business navigation. The browser subsystem calls the perspective change function by itself, and sends the current state after the perspective change to the IPTV business navigation through a message callback, so that the IPTV business navigation can be sent according to the browser subsystem. The corresponding UI is displayed in the current state after the perspective is changed.

For the second way to start the panoramic media processing system of the IPTV terminal, by calling the local panoramic video playback application of the IPTV terminal, when the local panoramic video playback application of the IPTV terminal is dominated, that is, when the IPTV service navigation starts the VR panoramic video playback through ACTION When referring to Figure 8, the specific steps include the following:

Step 210: Add the VR panoramic video address URL parameter to identify this video as a panoramic video; the browser subsystem parses ACTION and obtains the VR panoramic video address URL parameter; the browser subsystem starts Start the local panoramic video playback application of the IPTV terminal and pass in the VR panoramic video address URL parameters.

Step 220: The local panoramic video playback application calls the UI display module to obtain the corresponding UI display according to the obtained VR panoramic video address URL parameter.

Step 230: The local panoramic video playback application obtains the 3D spherical model and synthesizes its 3D display layer with the 2D display layer displayed by the UI for display.

Step 240: Pass the key event to the local panoramic video playback application for processing. If the perspective changes, reacquire the 2D display layer of the UI display and the 3D display layer of the 3D spherical model and synthesize them for display.

Referring to Figure 9, the above step 230 can also be extended to the following steps:

Step 231: The local panoramic video playback application calls the media control module to bind the 3D spherical model to the 3D display layer and provide VR panoramic feature capabilities.

Step 232: Call the 2D player and connect the VR panoramic video address URL parameter passed in through the IPTV service navigation to the streaming media subsystem.

Step 233: After mapping the output frame of the 2D player to the 3D spherical model, combine the 2D display layer of the UI display with the 3D display layer of the 3D spherical model for display.

Based on the above two method steps for starting the panoramic media processing system of IPTV terminals, when the IPTV business platform issues a live broadcast list, a parameter is added to identify whether the live broadcast program is a panoramic video; when the live broadcast playback interface of the IPTV terminal is called, it determines the live broadcast to be broadcast Whether it is a panoramic video, if so, start the panoramic display, and capture the remote control direction keys to switch between different viewing angles; during the panoramic playback process, when performing operations such as pausing playback, double-speed playback, positioning playback, or switching back to the live broadcast state, the IPTV terminal The key event is sent to the IPTV business navigation or local panoramic video playback application, and the IPTV business navigation or local panoramic video playback application calls the corresponding broadcast control interface to implement the corresponding function; when the IPTV terminal receives the call request of the corresponding broadcast control interface, it needs to Ensure that the video is in panoramic display during the broadcast control operation. When the IPTV terminal's live broadcast interface is called, the panoramic display will be exited.

When the IPTV business platform delivers the on-demand program address, a parameter is added to identify whether the on-demand program is a panoramic video; when the on-demand playback interface of the IPTV terminal is called, it is judged whether the on-demand program to be played is a panoramic video, and if so, the panoramic display is started, and Capture the direction keys of the remote control to switch between different viewing angles; during the panoramic playback process, when performing operations such as pausing playback, double-speed playback, positioning playback, and switching back to live broadcast status, the IPTV terminal will send key events to IPTV business navigation or local panoramic video playback Application, the IPTV business navigation or local panoramic video playback application calls the corresponding broadcast control interface to implement the corresponding function; when the IPTV terminal receives the call request of the corresponding broadcast control interface, it must ensure that the video is displayed in panorama during the broadcast control operation. When the IPTV When the terminal's exit on-demand interface is called, exit the panoramic display.

In addition, this embodiment also supports the static image browsing function: when the panoramic video is paused or the static image is browsed, the IPTV terminal sends the key event to the IPTV service navigation or local panoramic video playback application, which is called by the IPTV service navigation or local panoramic video playback application. The corresponding broadcast control interface is used to realize the corresponding functions, so that the user can rotate the paused video screen or static image at any angle, and watch the content of the video screen or static image at any angle in an all-round way.

To sum up, this embodiment has at least the following three innovations compared with the existing technology:

First, provide a set of APIs that include VR panoramic feature capabilities. This set of API has the functions of pausing playback, double-speed playback, positioning playback, switching back to live broadcast state, and changing perspective. Perspective changing features also include: The ability to change the direction of the viewing angle; the ability to change the distance of the viewing angle. This set of APIs is provided to IPTV business navigation calls in the form of JS extension objects (VR panoramic player objects). VR panoramic feature capabilities are provided to IPTV business navigation calls through the properties of VR panoramic player objects.

When playing an IPTV live program, if the video content is a panoramic video, this API is provided as a JS extension object for IPTV business navigation calls, which enables the IPTV terminal to support the panoramic playback function, and has the functions of pausing playback, double-speed playback, positioning playback, and switching back Live broadcast status, perspective change and other functions.

When playing an IPTV on-demand program, if the video content is a panoramic video, this API is provided as a JS extension object for IPTV business navigation calls, which enables the IPTV terminal to support the panoramic playback function, and has the functions of pause playback, double-speed playback, positioning playback, Viewing angle changing function.

Second, when the IPTV terminal receives a request to play a VR panoramic video, it calls the 3D spherical model through OPENGL technology, binds the 3D spherical model to the 3D display layer by calling GLSurfaceView, and calls the 2D player to convert the decoded panoramic video frames As a texture, map it to the 3D spherical model and finally display it to the 3D display layer.

Third, the IPTV business navigation can start the VR panoramic playback application by extending the JS object and issuing ACTION, and carry the VR panoramic video playback address URL parameter. The IPTV terminal will start the local VR panoramic playback application, which will be processed by the local panoramic playback application. The user presses a key to display the required display UI. When the IPTV navigation system starts VR panoramic playback through the VR panoramic player object, the IPTV terminal calls the panoramic playback system. Since the IPTV service navigation determines whether the user's direction key changes the perspective by setting the perspective control enable attribute of the VR panoramic player. Or change the playback state; when the viewing angle control is enabled, it is decided by the IPTV service navigation by setting the viewing angle control attribute of the VR panoramic player object, and it is processed by the IPTV terminal or the IPTV business navigation. The user adjusts the viewing angle direction through the arrow keys and Requests for far and near viewing angles; the above-mentioned UIs for state changes through the VR panoramic player are all displayed by the IPTV service navigation.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium can include: Read Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, etc.

The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the protection of the present application. within the range. Contents not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

A VR panoramic video playback system based on IPTV business, which is characterized by including an IPTV business platform and an IPTV terminal; the IPTV business platform includes IPTV content operation, IPTV business navigation, and IPTV media delivery; the IPTV terminal includes an IPTV terminal panoramic media processing system, display devices, external devices;

IPTV terminal panoramic media processing system includes browser subsystem, streaming media subsystem, graphics rendering subsystem, WEB page presentation window module, VR video presentation window module and synthesis module;

The browser subsystem is responsible for parsing the control instructions of IPTV business navigation for VR panoramic content playback, receiving key events, and controlling VR playback of the streaming media subsystem according to the requirements of IPTV business navigation, and responding to media playback events;

The streaming media subsystem provides VR panoramic feature capabilities, interactive capabilities with IPTV media delivery, and access to video content through streaming media protocols;

The graphics rendering subsystem includes a 3D modeling module and a 3D model analysis module, which are used to texture the 3D model;

The WEB page presentation window module is responsible for presenting the UI corresponding to the 2D EPG page and IPTV service navigation;

The VR panoramic video presentation window module is responsible for displaying the 3D spherical model of VR panoramic video rendering;

The synthesis module is responsible for synthesizing and displaying the presentation content of the WEB page presentation window module and the VR panoramic video presentation window module.
The VR panoramic video playback system based on IPTV services according to claim 1, characterized in that the browser subsystem parses and initializes the VR panoramic player object, including initializing the VR panoramic feature capabilities of the API of the VR panoramic player object. ;

The VR panoramic feature capabilities of the API include video playback state control capabilities and perspective changing functions. Among them, video playback state control capabilities include one or more of pause playback, doubled speed playback, positioning playback, and switching back to live broadcast state; perspective change functions include One or more of the ability to change the viewing angle direction and the ability to change the viewing angle distance.
The VR panoramic video playback system based on IPTV services according to claim 1, characterized in that when the IPTV service platform issues a live broadcast list, a parameter is added to identify whether the live program is a panoramic video; when the live broadcast playback interface of the IPTV terminal is called, it determines whether Whether the upcoming live broadcast is a panoramic video, if so, start the panoramic display, and capture the direction keys of the remote control to switch between different viewing angles;

During the panoramic playback process, when the user operates to pause playback, double-speed playback, position playback, or switch back to the live broadcast state, the IPTV terminal will send the key value to the IPTV business navigation or local panoramic video playback application, and the IPTV business navigation or local panoramic video playback application will Call the corresponding broadcast control interface to implement the function;

When the IPTV terminal receives the call request of the corresponding broadcast control interface, it must ensure that the video is displayed in panoramic view during the broadcast control operation; when the IPTV terminal's exit live broadcast interface is called, it exits the panoramic display.
The VR panoramic video playback system based on IPTV services according to claim 1, characterized in that when the IPTV service platform issues an on-demand program address, a parameter is added to identify whether the on-demand program is a panoramic video, and when the on-demand playback interface of the IPTV terminal is called, Determine whether the on-demand video to be played is a panoramic video. If so, start the panoramic display and capture the direction keys of the remote control to switch between different viewing angles;

During the panoramic playback process, when the user operates pause playback, double-speed playback, or positioning playback, the IPTV terminal sends the key value to the IPTV business navigation or local panoramic video playback application, and the IPTV business navigation or local panoramic video playback application calls the corresponding playback control Interface implementation function;

When the IPTV terminal receives the call request of the corresponding broadcast control interface, it must ensure that the video is displayed in panorama during the broadcast control operation;

When the exit on-demand interface of the IPTV terminal is called, the panoramic display is exited.
The VR panoramic video playback system based on IPTV services according to claim 1, characterized in that the VR panoramic video playback system based on IPTV services includes two ways of starting the IPTV terminal panoramic media processing system:

Led by IPTV business navigation, it decides to perform the action of changing the viewing angle or changing the playback state through API calls; when the IPTV terminal receives the key event, it sends the key event to the IPTV business navigation;

By calling the local panoramic video playback application of the IPTV terminal, it is dominated by the local panoramic video playback application of the IPTV terminal: when the IPTV terminal receives a key event, the local panoramic video playback application decides to perform the action of changing the viewing angle or changing the playback state. UI The display is also implemented by a local panoramic video playback application.
The VR panoramic video playback system based on IPTV services according to claim 5, characterized in that the IPTV panoramic playback application decides to start the IPTV panoramic playback application according to the extended JS interface called by the IPTV service navigation, and decides not to start the local panoramic view of the IPTV terminal. When playing the application, it includes:

The IPTV business navigation calls the IPTV panoramic playback API. The IPTV terminal panoramic media processing system will return the VR panoramic player object called by the IPTV terminal to the IPTV business navigation. The IPTV business navigation calls the IPTV panoramic playback external API according to the current status to realize the panoramic video Playback, rotation of perspective, change of perspective size, and corresponding UI display;

Led by IPTV business navigation, API calls are used to decide whether to perform the action of changing the viewing angle or changing the playback state. When the IPTV terminal receives the key event, it sends the key event to the IPTV business navigation.
The VR panoramic video playback system based on IPTV services according to claim 5, characterized in that the IPTV panoramic playback application decides not to start the IPTV panoramic playback application according to the extended JS interface called by the IPTV service navigation, and decides to start the local panoramic view of the IPTV terminal. Video playback applications include:

The local panoramic playback application calls the playback interface, creates a 3D model, renders the decoded video image onto the 3D model, and displays the corresponding UI interface when necessary;

It is dominated by the local panoramic video playback application of the IPTV terminal. When the IPTV terminal receives a key event, the local panoramic video playback application decides to perform the action of changing the viewing angle or changing the playback state.
A VR panoramic video playback method based on IPTV service, which is characterized by including:

When the IPTV business platform delivers the live broadcast list, a parameter is added to identify whether the live broadcast program is a panoramic video; when the live broadcast playback interface of the IPTV terminal is called, it is judged whether the live broadcast to be played is a panoramic video. If so, the panoramic display is started and the remote control is captured. Direction keys enable switching between different viewing angles; during the panoramic playback process, when the user operates to pause playback, double-speed playback, position playback, or switch back to the live broadcast state, the IPTV terminal sends the key value to the IPTV business navigation or local panoramic video playback application, which is controlled by IPTV The business navigation or local panoramic video playback application calls the corresponding broadcast control interface to implement the function; when the IPTV terminal receives the call request of the corresponding broadcast control interface, it must ensure that the video is displayed in panorama during the broadcast control operation; when the IPTV terminal leaves the live broadcast interface When called, exit the panoramic display;

When the IPTV business platform issues an on-demand program address, a parameter is added to identify whether the on-demand program is a panoramic video. When the on-demand playback interface of the IPTV terminal is called, it is judged whether the on-demand program to be played is a panoramic video. If so, the panoramic display is started and the remote control is captured. The direction keys of the device are used to switch between different viewing angles; during the panoramic playback process, when the user operates to pause playback, double-speed playback, or position playback, the IPTV terminal sends the key value to the IPTV business navigation or local panoramic video playback application, which is controlled by the IPTV business navigation or local panoramic video playback application. The local panoramic video playback application calls the corresponding broadcast control interface to implement the function; when the IPTV terminal receives the call request of the corresponding broadcast control interface, it needs to ensure that the video is displayed in panorama during the broadcast control operation; when the IPTV terminal's leaving on-demand interface is called, Exit panoramic display.
The VR panoramic video playback method based on IPTV service according to claim 8, characterized in that when dominated by IPTV service navigation, it specifically includes:

The IPTV panoramic playback application decides whether to start the IPTV panoramic playback application based on the extended JS interface called by the IPTV business navigation; when the IPTV business navigation chooses to call the IPTV panoramic playback API, the IPTV terminal panoramic media processing system will play the VR panoramic playback called by the IPTV terminal. The handler object is returned to the IPTV business navigation, which calls the external API of IPTV panoramic broadcast according to the current status to realize the playback of the panoramic video, the rotation of the viewing angle, the change of the viewing angle size, and the corresponding UI display;

Led by IPTV business navigation, API calls are used to decide whether to perform the action of changing the viewing angle or changing the playback state. When the IPTV terminal receives the key event, it sends the key event to the IPTV business navigation.
The VR panoramic video playback method based on IPTV services according to claim 8, characterized in that the local panoramic video playback application calls the corresponding broadcast control interface to implement the function, which specifically includes:

The IPTV panoramic playback application decides whether to start the IPTV panoramic playback application based on the extended JS interface called by the IPTV business navigation;

When the local panoramic playback application is started, the local panoramic playback application calls the playback interface and calls the 3D model, and render the decoded video image onto the 3D model, and display the corresponding UI interface when necessary;

It is dominated by the local panoramic video playback application of the IPTV terminal. When the IPTV terminal receives a key event, the local panoramic video playback application decides to perform the action of changing the viewing angle or changing the playback state.