WO2020098504A1 - Video switching control method and display device - Google Patents

Video switching control method and display device Download PDF

Info

Publication number
WO2020098504A1
WO2020098504A1 PCT/CN2019/114628 CN2019114628W WO2020098504A1 WO 2020098504 A1 WO2020098504 A1 WO 2020098504A1 CN 2019114628 W CN2019114628 W CN 2019114628W WO 2020098504 A1 WO2020098504 A1 WO 2020098504A1
Authority
WO
WIPO (PCT)
Prior art keywords
decoder
data
video
playback
type
Prior art date
Application number
PCT/CN2019/114628
Other languages
French (fr)
Chinese (zh)
Inventor
刘吉昊
Original Assignee
青岛海信传媒网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 青岛海信传媒网络技术有限公司 filed Critical 青岛海信传媒网络技术有限公司
Publication of WO2020098504A1 publication Critical patent/WO2020098504A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H04N21/2393Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808Management of client data
    • H04N21/25825Management of client data involving client display capabilities, e.g. screen resolution of a mobile phone
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H04N21/26216Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the channel capacity, e.g. network bandwidth
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H04N21/26233Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving content or additional data duration or size, e.g. length of a movie, size of an executable file
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331Caching operations, e.g. of an advertisement for later insertion during playback
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/437Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network

Definitions

  • the present application relates to streaming media technology, in particular to a video switching control method and display device.
  • the decoder there must be at least two decoders in the display device. While the first decoder decodes and plays the positive film, the second channel decoder decodes the cached advertising data. Then, at the switching moment, the decoder ’s Play, start advertising. In this way, it is equivalent to downloading and demultiplexing the relevant data of the advertisement in advance, initializing the second decoder, and decoding the relevant data of the advertisement, which greatly saves the video switching time.
  • the video switching operation can be controlled in a short time by using the dual-channel decoder decoding method, and the subjective experience is seamless switching.
  • the purpose of the present application is to provide a video switching control method and a display device, so as to improve the timeliness of video switching and ensure the video playback performance of the display device.
  • a display device including:
  • Display used to display video playback screen
  • the first decoder and the second decoder are used for processing the video including at least the decoding operation
  • the starting part of the data corresponding to the second type of video is received from the server and distributed to the second decoder for buffering.
  • the playback resolution of the starting part of the data is not high The highest playback resolution supported by the second decoder;
  • controlling the first decoder to switch to the second decoder to play the start-up part of the data buffered in the second decoder;
  • the playback resolution of the resume part data is higher than the highest playback resolution supported by the second decoder ;
  • the second decoder is controlled to switch to the first decoder to play the continued part of the data buffered in the first decoder.
  • a method for controlling video switching including:
  • the starting part of the data corresponding to the second type of video is received from the server and distributed to the second decoder for buffering.
  • the playback resolution of the starting part of the data is not high The highest playback resolution supported by the second decoder;
  • controlling the first decoder to switch to the second decoder to play the start-up part of the data buffered in the second decoder;
  • the playback resolution of the resume part data is higher than the highest playback resolution supported by the second decoder ;
  • the second decoder is controlled to switch to the first decoder to play the continued part of the data buffered in the first decoder.
  • a storage medium characterized in that a program for implementing a control method for video switching is stored, and when the program is run by a controller, the following steps are performed:
  • the starting part of the data corresponding to the second type of video is received from the server and distributed to the second decoder for buffering.
  • the playback resolution of the starting part of the data is not high The highest playback resolution supported by the second decoder;
  • controlling the first decoder to switch to the second decoder to play the start-up part of the data buffered in the second decoder;
  • the playback resolution of the resume part data is higher than the highest playback resolution supported by the second decoder ;
  • the second decoder is controlled to switch to the first decoder to play the continued part of the data buffered in the first decoder.
  • FIG. 1A exemplarily shows a schematic diagram of an operation scene between a display device and a control device
  • FIG. 1B exemplarily shows a configuration block diagram of the control device 100 in FIG. 1A;
  • FIG. 1C exemplarily shows a configuration block diagram of the display device 200 in FIG. 1A;
  • FIG. 1D exemplarily shows a block diagram of the architecture configuration of the operating system in the memory of the display device 200
  • FIG. 2 exemplarily shows a block diagram of another configuration of the display device 200
  • FIG. 3 exemplarily shows a schematic flowchart of video switching performed by the display device.
  • FIG. 1A exemplarily shows a schematic diagram of an operation scene between a display device and a control device.
  • the control apparatus 100 and the display device 200 may communicate in a wired or wireless manner.
  • the control device 100 is configured to control the display device 200, which can receive operation instructions input by the user and convert the operation instructions into instructions that the display device 200 can recognize and respond to, acting as an intermediary between the user and the display device 200 effect.
  • the user operates the channel addition and subtraction keys on the control device 100, and the display device 200 responds to the channel addition and subtraction operations.
  • the control device 100 may be a remote controller 100A, including infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, etc., to control the display device 200 through wireless or other wired methods.
  • the user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, and control panel input.
  • the user can control the display device 200 by inputting corresponding control commands through the volume up and down keys on the remote control, channel control keys, up / down / left / right movement keys, voice input keys, menu keys, power on / off keys, etc. Function.
  • the control device 100 may also be a smart device, such as a mobile terminal 100B, tablet computer, computer, notebook computer, or the like.
  • the display device 200 is controlled using an application program running on a smart device.
  • the application can be configured to provide users with various controls on the screen associated with the smart device through an intuitive user interface (UI).
  • UI intuitive user interface
  • the mobile terminal 100B may install a software application with the display device 200, implement connection communication through a network communication protocol, and realize one-to-one control operation and data communication purposes.
  • the mobile terminal 100B and the display device 200 can establish a control instruction protocol, and the functions of the physical keys such as the remote controller 100A can be realized by operating various function keys or virtual buttons of the user interface provided on the mobile terminal 100B.
  • the audio and video content displayed on the mobile terminal 100B may also be transmitted to the display device 200 to realize the synchronous display function.
  • the display device 200 may provide a network receiving function of a broadcast receiving function and a computer supporting function.
  • the display device may be implemented as a digital TV, an Internet TV, an Internet Protocol TV (IPTV), etc.
  • the display device 200 may be a liquid crystal display, an organic light emitting display, or a projection device.
  • the specific display device type, size and resolution are not limited.
  • the display device 200 also performs data communication with the server 300 through various communication methods.
  • the display device 200 may be allowed to perform communication connection through a local area network (LAN), a wireless local area network (WLAN), and other networks.
  • the server 300 can provide various contents and interactions to the display device 200.
  • the display device 200 can send and receive information, for example: receive electronic program guide (EPG) data, receive software program updates, or access a remotely stored digital media library.
  • EPG electronic program guide
  • the server 300 may be one group or multiple groups, and may be one or more types of servers.
  • the server 300 provides other network service content such as video on demand and advertising services.
  • FIG. 1B A configuration block diagram of the control device 100 is exemplarily shown in FIG. 1B.
  • the control device 100 includes a controller 110, a memory 120, a communicator 130, a user input interface 140, an output interface 150, and a power supply 160.
  • the controller 110 includes a random access memory (RAM) 111, a read only memory (ROM) 112, a processor 113, a communication interface, and a communication bus.
  • RAM random access memory
  • ROM read only memory
  • the controller 110 is used to control the operation and operation of the control device 100, as well as communication cooperation between internal components, external and internal data processing functions.
  • the controller 110 may control to generate a signal corresponding to the detected interaction, and This signal is sent to the display device 200.
  • the memory 120 is used to store various operating programs, data, and applications that drive and control the control device 100 under the control of the controller 110.
  • the memory 120 can store various control signal instructions input by the user.
  • the communicator 130 realizes communication of control signals and data signals with the display device 200 under the control of the controller 110.
  • the control device 100 transmits a control signal (such as a touch signal or a button signal) to the display device 200 via the communicator 130, and the control device 100 can receive the signal transmitted by the display device 200 via the communicator 130.
  • the communicator 130 may include an infrared signal interface 131 and a radio frequency signal interface 132.
  • the infrared signal interface when the infrared signal interface is used, the user input command needs to be converted into an infrared control signal according to the infrared control protocol, and then sent to the display device 200 via the infrared sending module.
  • the radio frequency signal interface the user input command needs to be converted into a digital signal, and then modulated according to the radio frequency control signal modulation protocol, and then sent to the display device 200 by the radio frequency transmission terminal.
  • the user input interface 140 may include at least one of a microphone 141, a touch pad 142, a sensor 143, a key 144, and the like, so that the user can input user instructions for controlling the display device 200 to the control device through voice, touch, gesture, pressing 100.
  • the output interface 150 outputs the user instruction received by the user input interface 140 to the display device 200, or outputs the image or voice signal received by the display device 200.
  • the output interface 150 may include an LED interface 151, a vibration interface 152 that generates vibration, a sound output interface 153 that outputs sound, a display 154 that outputs an image, and the like.
  • the remote controller 100A may receive output signals such as audio, video, or data from the output interface 150, and display the output signals as an image form on the display 154, output as an audio form on the sound output interface 153, or output as vibration on the vibration interface 152 form.
  • the power supply 160 is used to provide operating power support for each element of the control device 100 under the control of the controller 110.
  • the form can be battery and related control circuit.
  • the display device 200 may include a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a memory 260, a user interface 265, a video processor 270, a display 275, Audio processor 280, audio output interface 285, power supply 290.
  • Tuner demodulator 210 which receives broadcast TV signals by wire or wireless, can perform modulation, demodulation and other processing of amplification, mixing and resonance, and is used to demodulate the TV selected by the user from multiple wireless or cable broadcast TV signals
  • the audio and video signals carried in the frequency of the channel, as well as additional information (such as EPG data).
  • Tuner demodulator 210 can respond to the frequency of the TV channel selected by the user and the TV signal carried by the frequency according to the user's selection and controlled by controller 250.
  • Tuner demodulator 210 according to the different broadcasting systems of TV signals, there are many ways to receive the signal, such as: terrestrial broadcasting, cable broadcasting, satellite broadcasting or Internet broadcasting, etc .; and depending on the type of modulation, digital modulation or analog Modulation method; and according to the type of TV signal received, analog and digital signals can be demodulated.
  • the tuning demodulator 210 may also be in an external device, such as an external set-top box.
  • the set-top box outputs a TV signal after being modulated and demodulated, and is input into the display device 200 through the external device interface 240.
  • the communicator 220 is a component for communicating with an external device or an external server according to various types of communication protocols.
  • the display device 200 may transmit content data to an external device connected via the communicator 220, or browse and download content data from an external device connected via the communicator 220.
  • the communicator 220 may include a network communication protocol module such as a WIFI module 221, a Bluetooth communication protocol module 222, a wired Ethernet communication protocol module 223, or a near field communication protocol module, so that the communicator 220 can receive Control signals, and implement the control signals as WIFI signals, Bluetooth signals, radio frequency signals, etc.
  • the detector 230 is a component used by the display device 200 to collect signals from the external environment or interact with the external environment.
  • the detector 230 may include a sound collector 231, such as a microphone, which may be used to receive the user's voice, such as a voice signal of a user's control instruction to control the display device 200; or, it may collect environmental sounds used to identify the type of environmental scene to achieve display
  • the device 200 can adapt to environmental noise.
  • the detector 230 may further include an image collector 232, such as a camera, a camera, etc., which may be used to collect external environment scenes to adaptively change the display parameters of the display device 200; The user's attributes or interactive gestures with the user to achieve the interactive function between the display device and the user.
  • an image collector 232 such as a camera, a camera, etc., which may be used to collect external environment scenes to adaptively change the display parameters of the display device 200; The user's attributes or interactive gestures with the user to achieve the interactive function between the display device and the user.
  • the detector 230 may further include a light receiver for collecting ambient light intensity to adapt to changes in display parameters of the display device 200 and the like.
  • the detector 230 may further include a temperature sensor.
  • the display device 200 may adaptively adjust the display color temperature of the image. Exemplarily, when the temperature is relatively high, the display device 200 may be adjusted to display a cold color tone of the image temperature; when the temperature is relatively low, the display device 200 may be adjusted to display a warm color tone of the image color temperature.
  • the external device interface 240 is a component that provides the controller 250 to control the data transmission between the display device 200 and the external device.
  • the external device interface 240 can be connected to external devices such as set-top boxes, game devices, notebook computers, etc. in a wired / wireless manner, and can receive external devices such as video signals (such as moving images), audio signals (such as music), and additional information (such as EPG) ) And other data.
  • the external device interface 240 may include: a high-definition multimedia interface (HDMI) terminal 241, a composite video blanking synchronization (CVBS) terminal 242, an analog or digital component terminal 243, a universal serial bus (USB) terminal 244, and a component Any one or more of terminals (not shown in the figure), red, green and blue (RGB) terminals (not shown in the figure) and the like.
  • HDMI high-definition multimedia interface
  • CVBS composite video blanking synchronization
  • USB universal serial bus
  • the controller 250 controls various operations of the display device 200 and responds to user operations by running various software control programs (such as an operating system and various application programs) stored on the memory 260.
  • various software control programs such as an operating system and various application programs
  • the controller 250 includes a random access memory (RAM) 251, a read only memory (ROM) 252, a graphics processor 253, a CPU processor 254, a communication interface 255, and a communication bus 256.
  • RAM random access memory
  • ROM read only memory
  • CPU CPU processor
  • communication interface 255 a communication bus 256.
  • the RAM 251, ROM 252, graphics processor 253, and CPU processor 254 communication interface 255 are connected through a communication bus 256.
  • ROM252 is used to store various system startup instructions. For example, when the power-on signal is received, the power of the display device 200 starts to start, the CPU processor 254 runs the system start instruction in the ROM 252, and copies the operating system stored in the memory 260 to the RAM 251 to start the start operating system. After the startup of the operating system is completed, the CPU processor 254 copies the various application programs in the memory 260 to the RAM 251, and then starts to run and start the various application programs.
  • the graphics processor 253 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics.
  • the graphics processor 253 may include an operator for performing operations by receiving various interactive instructions input by the user, and then display various objects according to display attributes; and a renderer for generating various objects obtained based on the operator, which will be performed The rendered result is displayed on the display 275.
  • the CPU processor 254 is used to execute operating system and application program instructions stored in the memory 260. And according to the received user input instructions, to perform various applications, data and content processing, so as to finally display and play various audio and video content.
  • the CPU processor 254 may include multiple processors.
  • the multiple processors may include a main processor and multiple or one sub-processors.
  • the main processor is used to perform some initialization operations of the display device 200 in the pre-loading mode of the display device, and / or display screen operations in the normal mode.
  • Multiple or one sub-processors are used to perform an operation in the standby mode of the display device.
  • the communication interface 255 may include a first interface to an n-th interface. These interfaces may be network interfaces connected to external devices via a network.
  • the controller 250 may control the overall operation of the display device 200. For example, in response to receiving a user input command for selecting a GUI object displayed on the display 275, the controller 250 may perform an operation related to the object selected by the user input command.
  • the object may be any one of selectable objects, such as a hyperlink or an icon.
  • the operation related to the selected object for example, the operation of connecting to a hyperlinked page, document, image, etc., or the operation of executing a program corresponding to the object.
  • the user input command for selecting a GUI object may be a command input through various input devices (eg, mouse, keyboard, touch pad, etc.) connected to the display device 200 or a voice command corresponding to a voice spoken by the user.
  • the memory 260 is used to store various types of data, software programs, or application programs that drive and control the operation of the display device 200.
  • the memory 260 may include volatile and / or non-volatile memory.
  • the term "memory" includes the memory 260, the RAM 251 and the ROM 252 of the controller 250, or the memory card in the display device 200.
  • the memory 260 is specifically used to store operating programs that drive the controller 250 in the display device 200; store various application programs built in the display device 200 and downloaded by users from external devices; and store for configuration provided by the display 275 Data of various GUIs, various objects related to the GUI, and visual effects images of selectors for selecting GUI objects.
  • the memory 260 is specifically used to store drivers and related data of the tuner-demodulator 210, communicator 220, detector 230, external device interface 240, video processor 270, display 275, audio processor 280, etc.
  • external data received from an external device interface such as audio and video data
  • user data received from a user interface such as key information, voice information, touch information, etc.
  • the memory 260 specifically stores software and / or programs for representing an operating system (OS), and these software and / or programs may include, for example, a kernel, middleware, application programming interface (API), and / or application.
  • OS operating system
  • these software and / or programs may include, for example, a kernel, middleware, application programming interface (API), and / or application.
  • the kernel can control or manage system resources, and functions implemented by other programs (such as the middleware, API, or application program); meanwhile, the kernel can provide an interface to allow middleware, API, or application program access control To control or manage system resources.
  • FIG. 1D exemplarily shows a block diagram of the architecture configuration of the operating system in the memory of the display device 200.
  • the operating system architecture from top to bottom is the application layer, middleware layer and kernel layer.
  • the application layer application programs built in the system, and non-system-level application programs belong to the application layer. Responsible for direct interaction with users.
  • the application layer may include multiple applications, such as a setting application, an e-post application, and a media center application. These applications can be implemented as Web applications, which are executed based on the WebKit engine, and specifically can be developed and executed based on HTML5, Cascading Style Sheets (CSS), and JavaScript.
  • Web applications which are executed based on the WebKit engine, and specifically can be developed and executed based on HTML5, Cascading Style Sheets (CSS), and JavaScript.
  • CSS Cascading Style Sheets
  • HTML which is called HyperText Markup Language (HyperText Markup Language)
  • HTML tags are used to describe text, graphics, animation, sound, tables, Links, etc., the browser will read the HTML document, explain the content of the tags in the document, and display it in the form of a web page.
  • CSS which is called Cascading Style Sheets, is a computer language used to express the style of HTML files. It can be used to define the style structure, such as fonts, colors, and positions. CSS styles can be stored directly with HTML web pages or separate style files to control the styles in web pages.
  • JavaScript is a language used in the programming of Web pages. It can be inserted into HTML pages and interpreted by the browser. The interaction logic of the Web application is implemented through JavaScript. JavaScript can encapsulate the JavaScript extension interface through the browser to achieve communication with the kernel layer,
  • the middleware layer can provide some standardized interfaces to support the operation of various environments and systems.
  • the middleware layer can be implemented as a multimedia and hypermedia information coding expert group (MHEG) for middleware related to data broadcasting, as well as DLNA middleware for middleware related to external device communication, and can also be implemented as providing Middleware that displays the browser environment in which each application in the device runs.
  • MHEG multimedia and hypermedia information coding expert group
  • the kernel layer provides core system services, such as file management, memory management, process management, network management, and system security authority management.
  • the kernel layer may be implemented as a kernel based on various operating systems, for example, a kernel based on a Linux operating system.
  • the kernel layer also provides communication between system software and hardware, and provides device driver services for various hardware, such as: display driver for display, camera driver for camera, key driver for remote control, and WIFI module Provide WiFi drivers, audio drivers for audio output interfaces, and power management drivers for power management (PM) modules.
  • the user interface 265 receives various user interactions. Specifically, it is used to send the user's input signal to the controller 250, or to transmit the output signal from the controller 250 to the user.
  • the remote controller 100A may send input signals such as a power switch signal, a channel selection signal, a volume adjustment signal, etc. input by the user to the user interface 265, and then the user interface 265 forwards the signal to the controller 250;
  • the output signal such as audio, video or data output from the user interface 265 processed by the controller 250 is received, and the received output signal is displayed or output as an audio or vibration form.
  • a user may input a user command on a graphical user interface (GUI) displayed on the display 275, and the user interface 265 receives the user input command through the GUI.
  • GUI graphical user interface
  • the user interface 265 may receive user input commands for controlling the position of the selector in the GUI to select different objects or items.
  • the user can input a user command by inputting a specific sound or gesture, and the user interface 265 recognizes the sound or gesture through the sensor to receive the user input command.
  • the video processor 270 is used to receive external video signals and perform video data processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard codec protocol of the input signal.
  • video data processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard codec protocol of the input signal.
  • the video signal displayed or played directly on the display 275.
  • the video processor 270 includes a demultiplexing module, a video decoding module (decoder), an image synthesis module, a frame rate conversion module, a display formatting module, and so on.
  • the demultiplexing module is used to input audio and video data stream demultiplexing processing, such as MPEG-2 input stream (based on the digital storage medium the moving image and voice compression standard), the demultiplexing module which de Multiplexed into video signals and audio signals.
  • audio and video data stream demultiplexing processing such as MPEG-2 input stream (based on the digital storage medium the moving image and voice compression standard)
  • the demultiplexing module which de Multiplexed into video signals and audio signals.
  • the video decoding module is used to process the demultiplexed video signal, including decoding, scaling, and buffer processing.
  • the number of video decoding modules may be one or more.
  • An image synthesis module such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to user input or generated by itself with the zoomed video image to generate an image signal for display.
  • the frame rate conversion module is used to convert the frame rate of the input video, such as converting the frame rate of the input 60 Hz video to a frame rate of 120 Hz or 240 Hz.
  • the usual format is implemented by inserting frames, for example.
  • the display formatting module is used to change the signal output by the frame rate conversion module into a signal conforming to the display format of the display, for example, format-converting the signal output by the frame rate conversion module to output RGB data signals.
  • the display 275 is used to receive the image signal input from the video processor 270 and display the video content, image, and menu control interface.
  • the displayed video content may come from the video content in the broadcast signal received by the tuner demodulator 210, or from the video content input from the communicator 220 or the external device interface 240.
  • the display 275 also displays a user control interface UI generated in the display device 200 and used to control the display device 200.
  • the display 275 may include a display screen component for presenting a picture and a driving component that drives image display.
  • the display 275 may further include a projection device and a projection screen.
  • the audio processor 280 is used to receive external audio signals, perform decompression and decoding, and perform audio data processing such as noise reduction, digital-to-analog conversion, and amplification processing according to the standard codec protocol of the input signal, which can be obtained in the speaker 286 Play audio signal.
  • the audio processor 280 may support various audio formats. For example, MPEG-2, MPEG-4, Advanced Audio Coding (AAC), High Efficiency AAC (HE-AAC) and other formats.
  • AAC Advanced Audio Coding
  • HE-AAC High Efficiency AAC
  • the audio output interface 285 is used to receive the audio signal output by the audio processor 280 under the control of the controller 250.
  • the audio output interface 285 may include a speaker 286, or an external audio output terminal 287 output to a generator of an external device, such as a headset Output terminal.
  • the video processor 270 may include one or more chips.
  • the audio processor 280 may also include one or more chips.
  • the video processor 270 and the audio processor 280 may be separate chips, or may be integrated with the controller 250 in one or more chips.
  • the power supply 290 is used to provide power supply support for the display device 200 under the control of the controller 250 to input power from an external power supply.
  • the power supply 290 may be a built-in power supply circuit installed inside the display device 200 or a power supply installed outside the display device 200.
  • FIGS. 1A and 1C continue to refer to FIG. 2, wherein the video processor 270 in the display device includes at least a Main decoder 271 and a Sub decoder 272.
  • the controller 250 receives video data from the server 300, and creates corresponding types of resources for the Main decoder 271 and Sub decoder 272, respectively, for corresponding processing of the video data, These resources include at least a media playback background, a protocol analysis terminal, a player, etc., which will be described in detail in subsequent embodiments, and will not be repeated here.
  • the Main decoder 271 supports playback resolutions up to 2160P.
  • Sub-channel decoder 272 supports playback resolutions up to 1080P.
  • the performance of display devices is also required to increase. This high performance brings more resource occupancy, mainly reflected in the memory occupancy of the display device and the chip bandwidth. For high-resolution and high-bitrate video, the resource occupancy is even greater. Since the decoder is hardware, the memory occupied by the decoder during operation must be static memory allocated at the compilation stage, which is a fixed address segment, that is, the memory limit of the decoder is very strict.
  • the memory allocation principles adopted by the display device for the decoder are as follows:
  • the two decoders must support 2106p, and the video data of most advertisements is only up to 1080p, so the large Some hardware manufacturers choose to use the 2106P + 1080p decoder combination to support the advertisement insertion scene: that is, the first film is decoded by the first decoder (also known as the main (main) decoder), and the highest support is 2106p; the second decoder (also known as the decoder) is also used for advertising Sub (Sub) decoder] decoding, up to 1080p.
  • the first decoder also known as the main (main) decoder
  • the second decoder also known as the decoder
  • the decoder is also used for advertising Sub (Sub) decoder] decoding, up to 1080p.
  • the Main decoder and Sub decoder need to be used to play the main movie and each advertisement in turn.
  • the last advertisement will occupy the Main decoder to decode.
  • you need to switch back to the main film for playback you can only perform the preprocessing of the relevant data in the Sub decoder.
  • the playback resolution of the positive film will be limited to 1080p, thus affecting the playback of the positive film.
  • the display device specifically needs to perform the following operations:
  • Main decoder is used to decode and play the positive film.
  • the Sub-channel decoder is used to create and initialize the relevant resources of Advertisement 1, and decode the relevant data of Advertisement 1.
  • the Main decoder is used to create and initialize the relevant resources of Advertisement 2 and decode the relevant data of Cache Advertisement 2.
  • the advertisement of the main channel 2 is started to be played, and at the same time, the related resources of the sub-channel decoder are released.
  • the display device can only sacrifice switching Performance, that is, after the advertisement 2 is played, the relevant resources of the Main decoder are released, re-initialized, and then the Main decoder is used to decode and play the positive film. In this way, a black screen waiting process of 1 to 2 seconds will occur at the end of the advertisement playback.
  • the display device will have a 50% probability of the above black screen waiting process, that is, the rapid switching failure, which seriously affects the timeliness of video switching, and seriously reduces the user experience, unable to meet market needs.
  • streaming media has three major protocols, Dynamic Adaptive Streaming Protocol (Dynamic Adaptive Streaming over HTTP, DASH), Apple-based HTTP-based streaming media communication protocol (Apple HTTP Live Streaming, HLS), Microsoft Corporation
  • Dynamic Adaptive Streaming Protocol Dynamic Adaptive Streaming over HTTP, DASH
  • Apple-based HTTP-based streaming media communication protocol Apple HTTP Live Streaming, HLS
  • Microsoft Corporation The realized streaming media solution (Microsoft Smoothing, MSS)
  • MSS Microsoft Smoothing
  • the display device may divide the playback process of the main video (that is, the positive film) into two stages: The start-up phase with a lower playback resolution and the resume phase with a higher playback resolution.
  • the display device may first use the Sub decoder to play the low-resolution starting part of the main video data After the multimedia display information is played, switch to the main channel decoder to play the high playback resolution of the main video. In this way, the main video can be played in high playback resolution all the time.
  • a multimedia presentation information for example, an advertisement
  • Step 310 During the process of playing the multimedia presentation information through the Main decoder, the display device determines that the main video needs to be played.
  • the server may notify the display device in advance and start preparing to continue playing the main video.
  • Step 320 The display device receives the start-up part data corresponding to the main video from the server and distributes it to the Sub-channel decoder for buffering.
  • the playback resolution of the start-up part data is not higher than the highest playback supported by the Sub-channel decoder Resolution.
  • the video data will be divided into a number of small segments, which are recorded as segments.
  • Each segment generally contains 1 to 2 seconds of video data.
  • the playback resolution and bit rate of the same segment are completely Similarly, these parameters can be passed to the display device through the segment header.
  • the server or display device can decide which playback resolution video data to send or request to download based on the current network comprehensive status (eg, network speed, bandwidth, data cache amount, etc.), and the display device can be based on the received The data packet header of the video data, and the parameters such as the playback resolution of the currently received segment.
  • the current network comprehensive status e.g, network speed, bandwidth, data cache amount, etc.
  • the display device may request the server to deliver video data with a lower playback resolution according to the current network comprehensive status, that is, start-up part of the data.
  • the Sub-channel decoder can be initialized in advance, and the analysis and decoding operations of the start-up part data are completed in advance to prepare for subsequent playback
  • the playback resolution of the starting part of the data is 1080P.
  • Step 330 After the multimedia display information is played, the display device switches from the Main decoder to the Sub decoder to play back the start-up portion of data buffered in the Sub decoder.
  • the display device plays the multimedia display information cached in the Main decoder, it will quickly switch to the Sub decoder. At this time, the data of the starting part cached on the Sub decoder has been parsed and decoded. Play directly.
  • the display device will release various resources on the Main decoder and initialize the Main decoder. After receiving the resume data sent by the server, Then you can directly perform operations such as parsing and decoding to prepare for subsequent playback.
  • Step 340 The display device receives the resume data corresponding to the main video from the server and distributes it to the Main decoder for buffering.
  • the playback resolution of the resume data is higher than the highest supported by the Sub decoder. Play resolution.
  • the display device may request the server to deliver video data with a higher playback resolution according to the current network comprehensive status, that is, start-up part of the data.
  • the playback resolution of the resumed partial data is 2106P.
  • segment header of each segment can carry the playback resolution of the segment, when the display device determines that the video data corresponding to the received segment has a playback resolution higher than 1080p (ie, resumes part of the data), it will start from this segment Initially, all subsequent received segments are directly injected into the Main decoder.
  • the cut-off start data and continued playback data can be played separately.
  • the display device recognizes that the part of the resume data received from the server is recognized, that is, when the playback resolution of the subsequent arrival segment changes and exceeds 1080p, the first segment of the data of the resume broadcast will be saved Time stamp, marked as T1.
  • step 320 usually occurs during the display device playing the multimedia presentation information cached in the Main decoder
  • step 340 usually occurs when the display device plays the Sub channel In the process of starting part of the data buffered in the decoder.
  • Step 350 The display device switches from the Sub-channel decoder to the Main-channel decoder after the playback of the starting part of the data is completed, and plays the continued-play-part data buffered in the Main-channel decoder.
  • the display device plays the starting portion of the data buffered in the Sub channel decoder and reaches the recorded timestamp, that is, when it reaches the position of T1
  • the Sub channel decoder is suspended, the Main channel decoder is started, and playback starts from T1
  • the resumed part of the data buffered in the Main decoder so that the resumed part of the data with high playback resolution can be played in the Main decoder.
  • the display device will release various resources on the Sub decoder, and initialize the Sub decoder to prepare for subsequent playback of other video data.
  • the display device may receive the multimedia presentation information buffered in the Main decoder during the process. To continue to broadcast some data, at this time, because the various resources on the Main decoder have not been released, the display device needs to wait for the multimedia display information to be played, release the various resources on the Main decoder, and then The Main decoder is initialized, and further, the received resume data is distributed to the Main decoder for buffering. On the other hand, the display device may also receive the resume part of the data during the playback of the start part of the buffered data in the Sub decoder.
  • the display device can directly distribute the received part of the resume data to the Main decoder for buffering.
  • the playback screen When the display device is playing video data, regardless of whether two decoders are used, the playback screen will cause obvious visual changes when switching the playback resolution, and the use of two decoders will make the screen switch quickly without a black screen. Therefore, adopt When the above technical solution is used to play the main video (that is, the main film), compared with the technical solution that does not distinguish between the starting part data and the continuing part data, the subjective experience is basically the same, which can ensure the playback effect.
  • the content to be played by the display device is: positive film, advertisement 1, advertisement 2.
  • the various resources occupied include the main media playback background, which is denoted as media_backend_main, the main protocol resolution end, denoted as protocol_stack_main, the main player, denoted as player_main, the main decoder, and denoted as decoder_main.
  • the server instructs Advertisement 1 to be inserted at Tad1 at a later time, and the display device starts to create and initialize various related resources, including: Advertisement 1 media playback background, denoted as media_backend_ad1, Advertisement 1 protocol parsing end, denoted as protocol_stack_ad1 1. Advertisement 1 player, denoted as player_ad1, and auxiliary decoder, denoted as decoder_sub. Except for decoder_sub, all other resources are logic modules created by the controller. Each of the above logic modules can create multiple instances without affecting each other.
  • the display device passes the Uniform Resource Locator (URL) of Advertisement 1 to media_backend_ad1 to protocol_stack_ad1, and protocol_stack_ad1 downloads and parses the media presentation content description (MPD) file at the specified URL to obtain the initial information of Advertisement 1 , Including Codec, duration, code rate, frame rate, initial resolution, audio sampling rate, etc., then pass to player_ad1 and initialize decoder_sub, then download the video data of advertisement 1 to decoder_sub for decoding, and cache the decoded video data.
  • URL Uniform Resource Locator
  • MPD media presentation content description
  • the display device When the positive film is played to Tad1, the display device issues a pause command to suspend the operation of all logic modules on the Main decoder, including: stopping player_main playback, and switching the video output device (Vout) and sound output device (Aout) to Advertisement 1, and the start command is issued to start the playback of player_ad1. Since the video data of Advertisement 1 has been downloaded and decoded in advance, the phenomenon that the user can watch is as follows: the positive film seamlessly switches to Advertisement 1 at Tad1 moment.
  • the display device will actively release various resources related to the positive film, including media_backend_main, protocol_stack_main, player_main, and decoder_main.
  • the server After the display device plays advertisement 1 through player_ad1 for a period of time, the server indicates that advertisement 2 will be inserted after advertisement 1 is played. At this time, the display device creates and initializes various related resources, including: advertising 2 media playback background, denoted as media_backend_ad2, advertising 2 protocol parsing end, denoted as protocol_stack_ad2, advertising 2 player, denoted as player_ad2, and main decoder, Recording decoder_main, except for decoder_main, all other resources are logic modules created by the controller, which will not be repeated here.
  • advertising 2 media playback background denoted as media_backend_ad2
  • advertising 2 protocol parsing end denoted as protocol_stack_ad2
  • advertising 2 player denoted as player_ad2
  • main decoder Recording decoder_main, except for decoder_main, all other resources are logic modules created by the controller, which will not be repeated here.
  • the process is the same as that of Advertisement 1.
  • the display device pre-downloads, decodes, and caches the video data of Advertisement 2, and immediately switches to player_ad2 after Advertisement 1 finishes, starts to play Advertisement 2, and releases the advertisement.
  • All kinds of related resources including media_backend_ad1, protocol_stack_ad1, player_ad1 and decoder_sub.
  • the server After the display device plays Advertisement 2 through player_ad2 for a period of time, the server indicates that the main film will be inserted after the advertisement 2 is played. At this time, the display device creates and initializes various types of related resources, including the new media playback background, denoted as media_backend_main_resume, the new protocol parser, denoted as protocol_stack_main_resume, the new player, denoted as player_main_resume, auxiliary decoder, and denoted as decoder_sub.
  • the display device passes the URL of the positive film to protocol_stack_main_resume through media_backend_main_resume, protocol_stack_main_resume downloads the corresponding MPD file under the URL, and parses out the maximum playback resolution of the video data.
  • decoder_sub can support the playback of positive films, and the subsequent process does not need to switch between decoder_sub and decoder_main; conversely, if the maximum playback resolution of video data is greater than the maximum playback resolution (1080p) supported by decoder_sub, you need to use
  • the technical solutions proposed in the embodiments of the present application will be introduced in the following steps.
  • the display device negotiates with the server, by judging factors such as network speed, bandwidth, and amount of cached data at the current time, combined with the algorithm of the streaming media protocol, it is concluded that the playback resolution of the video data of the positive film to be downloaded is less than or equal to 1080p. Further, it can also Confirm the information such as the code rate and frame rate of the video data.
  • the display device downloads the video data after the time of Tad1 from the server. This part of the video data is the starting part of the data, and its playback resolution is less than or equal to 1080p.
  • the display device allocates the starting part of the data to decoder_sub for decoding and buffering, as long as it is not subsequently received When the video data with a resolution above 1080p is played, the video data received subsequently is always decoded using decoder_sub.
  • the display device switches the video output device and the sound output device to the positive film. Since the system state (network speed, etc.) does not support the playback of video data greater than 1080p at this time, use decoder_sub to decode first. Broadcast some data.
  • the display device negotiates with the server. By judging factors such as network speed, bandwidth, and the amount of cached data at the current time, combined with the algorithm of the streaming media protocol, it is concluded that the playback resolution of the video data of the positive film that needs to be downloaded is greater than 1080p. Further, it can also Confirm the information such as the code rate and frame rate of the video data.
  • the display device will record the start time of the positive film that continues to download, that is, the position of the display timestamp (Presentation Time Stamp), used to measure when the decoded video data is displayed, recorded as T2K4K, created and initialized All kinds of resources related to the new positive film, including the new new media playback background, referred to as media_backend_main_resume_2K4K, the new and new protocol parser, referred to as protocol_stack_main_resume_2K4K, the new new player, referred to as player_main_resume_2K4K, the main decoder, decoder_main.
  • media_backend_main_resume_2K4K the new and new protocol parser
  • protocol_stack_main_resume_2K4K the new new player
  • player_main_resume_2K4K the main decoder
  • the display device will continue to download the video data of the positive film after the T2K4K moment. This part of the video data is the resume data.
  • the display device will distribute the resume video data to the decoder_main for decoding and caching.
  • the display device When the display device plays the main film to the time of T2K4K through player_main_resume, it will pause the playback of player_main_resume and switch to player_main_resume_2K4K. At this time, the resumed part of the data has been downloaded, parsed, and decoded in decoder_main. The playback resolution of all the video data is above 1080p, then the display device will distribute all the downloaded video data to decoder_main to decode and cache, and no longer switch the decoder.
  • the display device downloads the starting part data and the resuming part data of the main video from the server respectively according to the low playback resolution and the high playback resolution.
  • the Main decoder When the Main decoder is occupied by the multimedia display information, it first passes the Sub The channel decoder plays the starting part of the data, and after the multimedia display information is played, it switches to the Main channel decoder to play the continued part of the data, so that the main video and multimedia display information can be seamlessly switched to save memory as much as possible.
  • the defect of the dual-channel decoder cannot be switched back to the main video with high playback resolution normally when inserting even-numbered multimedia display information in the main video, which effectively improves the timeliness of streaming video switching.
  • the system guarantees the streaming video playback performance of the system and improves the user experience.
  • a display device includes at least a controller 250, a Main decoder 271, and a Sub decoder 272, where,
  • the controller 250 is used to perform the following operations:
  • the starting part data corresponding to the main video is received from the server and distributed to the Sub decoder 272 for buffering.
  • the playback resolution of the starting part data is not higher than The highest playback resolution supported by the Sub decoder 272;
  • the Main decoder 271 is switched to the Sub decoder 272, and the start-up part of the data buffered in the Sub decoder 272 is played;
  • the playback resolution of the resume part data is higher than the highest playback resolution supported by the Sub decoder 272 ;
  • the Sub-channel decoder 272 is switched to the Main-channel decoder 271, and the continued-play-part data buffered in the Main-channel decoder 271 is played.
  • the controller 250 before receiving the start part data corresponding to the main video from the server, the controller 250 is further used to:
  • the controller 250 Before receiving the resume part data corresponding to the main video from the server, the controller 250 is further used to:
  • the controller 250 when allocating the start-up part of the data to the Sub decoder 272 for buffering, the controller 250 is used to:
  • the display device distributes part of the resumed data to the Main decoder 271 for buffering, including:
  • the Main decoder 271 analyzes and decodes the resumed part of the data.
  • the controller 250 when receiving the resume data corresponding to the main video from the server and distributing it to the Main decoder 271 for buffering, the controller 250 is used to:
  • the received resume part data is directly distributed to the Main decoder 271 for buffering.
  • the controller 250 when receiving the resume data corresponding to the main video from the server and distributing it to the Main decoder 271 for buffering, the controller 250 is used to:
  • the Sub-channel decoder 272 is switched to the Main-channel decoder 271, and when the continued-play part of the data buffered in the Main-channel decoder 271 is played, the The controller 250 is used to:
  • the device switches to the Main decoder, and continues to play the part of the resumed data buffered in the Main decoder.
  • embodiments of the present application provide a storage medium for video switching, which stores a program for implementing a control method for streaming media video switching.
  • the program is run by a controller, the following steps are performed:
  • the starting part data corresponding to the main video is received from the server and distributed to the Sub decoder for buffering.
  • the playback resolution of the starting part data is not higher than the The highest playback resolution supported by Sub decoder;
  • the Main decoder decodes to all
  • the Sub-channel decoder playing the starting part of the data buffered in the Sub-channel decoder
  • the display device receives the resume data corresponding to the main video from the server and distributes it to the Main decoder for buffering.
  • the playback resolution of the resume data is higher than the highest playback resolution supported by the Sub decoder ;
  • the Sub-channel decoder is switched to the Main-channel decoder, and the continued-play part of the data buffered in the Main-channel decoder is played.
  • the display device selects the service according to the low playback resolution and the high playback resolution.
  • the device downloads the starting part data and the resuming part data of the main video separately.
  • the starting part data is played through the Sub channel decoder first, and then switched after the multimedia display information is completed.
  • the maximum probability of eliminating even number of multimedia display information inserted in the main video can effectively improve the timeliness of the streaming video switching, ensure the streaming video playback performance of the system, and improve the user experience.
  • the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.
  • computer usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions can be provided to a general-purpose computer, special-purpose computer, embedded processing machine, or controller of other programmable data processing equipment to generate a machine that enables the generation of instructions executed by the computer or other programmable data processing equipment's controller
  • These computer program instructions may also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, the instructions The device implements the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processing, which is executed on the computer or other programmable device
  • the instructions provide steps for implementing the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.

Abstract

Disclosed is a video switching control method. The method comprises: respectively downloading, according to low play resolution and high play resolution, play start portion data and play resuming portion data of a second type of video from a server; and when a first decoder is occupied by a first type of video, first playing the play start portion data by means of a second decoder, and after playing the first type of video is completed, then switching to the first decoder to play the play resuming portion data.

Description

一种视频切换的控制方法及显示设备Video switching control method and display device
本申请要求在2018年11月12日提交中国专利局、申请号为201811338363.0、发明名称为“一种流媒体视频切换的方法及智能电视”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application requires the priority of the Chinese patent application submitted to the China Patent Office on November 12, 2018, with the application number 201811338363.0 and the invention titled "a method for streaming video switching and smart TV", the entire content of which is incorporated by reference In this application.
技术领域Technical field
本申请涉及流媒体技术,尤其涉及一种视频切换的控制方法及显示设备。The present application relates to streaming media technology, in particular to a video switching control method and display device.
背景技术Background technique
随着流媒体技术的发展,用户对于视频的切换速度要求越来越高。欧美市场的部分视频播放应用,都对视频切换、广告插播有着明确的性能要求。例如,部分播放器甚至要求广告插入的切换时间在250ms之内。With the development of streaming media technology, users have higher and higher requirements for video switching speed. Some video playback applications in the European and American markets have clear performance requirements for video switching and ad insertion. For example, some players even require that the switching time for ad insertion be within 250ms.
为满足这一要求,显示设备需存在至少两路解码器(decoder),在第一路decoder解码播放正片的同时,通过第二路decoder解码缓存广告的数据,然后,在切换时刻,暂停正片的播放,开始进行广告播放。这样,相当于提前进行了广告的相关数据的下载、解复用,第二路decoder的初始化,以及广告的相关数据的解码等动作,大大节省了视频的切换时间。To meet this requirement, there must be at least two decoders in the display device. While the first decoder decodes and plays the positive film, the second channel decoder decodes the cached advertising data. Then, at the switching moment, the decoder ’s Play, start advertising. In this way, it is equivalent to downloading and demultiplexing the relevant data of the advertisement in advance, initializing the second decoder, and decoding the relevant data of the advertisement, which greatly saves the video switching time.
由于视频数据的准备时间多数消耗在下载、解复用、解码过程,因此,采用双路解码器解码方式可以将视频切换操作控制在很短时间,主观感受上即为无缝切换。Since the preparation time of video data is mostly consumed in the downloading, demultiplexing, and decoding processes, the video switching operation can be controlled in a short time by using the dual-channel decoder decoding method, and the subjective experience is seamless switching.
但是,随着用户对视频清晰度的需求越来越高,例如正片的分辨率为2106p, 广告的分辨率为1080p,两者切换时仍然会产生1~2秒黑屏等待过程,影响用户感受。However, as the user's demand for video clarity becomes higher and higher, for example, the resolution of the positive film is 2106p and the resolution of the advertisement is 1080p, a black screen waiting process of 1 to 2 seconds will still occur when the two are switched, which affects the user experience.
发明内容Summary of the invention
本申请的目的是提供一种视频切换的控制方法及显示设备,以提高视频切换的时效性,保障显示设备的视频播放性能。The purpose of the present application is to provide a video switching control method and a display device, so as to improve the timeliness of video switching and ensure the video playback performance of the display device.
第一方面,提供一种显示设备,包括:In a first aspect, a display device is provided, including:
显示器,用于显示视频播放画面;Display, used to display video playback screen;
第一解码器和第二解码器,用于对视频进行至少包括解码操作的处理;The first decoder and the second decoder are used for processing the video including at least the decoding operation;
控制器,用于执行:Controller, used to perform:
通过第一解码器播放第一类视频的过程中,从服务器接收第二类视频对应的起播部分数据,并分配至第二解码器进行缓存,所述起播部分数据的播放分辨率不高于所述第二解码器支持的最高播放分辨率;In the process of playing the first type of video through the first decoder, the starting part of the data corresponding to the second type of video is received from the server and distributed to the second decoder for buffering. The playback resolution of the starting part of the data is not high The highest playback resolution supported by the second decoder;
在所述第一类视频播放完毕后,控制所述第一解码器切换至所述第二解码器,播放所述第二解码器中缓存的所述起播部分数据;After the first type of video is finished playing, controlling the first decoder to switch to the second decoder to play the start-up part of the data buffered in the second decoder;
从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存,所述续播部分数据的播放分辨率高于所述第二解码器支持的最高播放分辨率;Receiving the resume part data corresponding to the second type of video from the server and distributing it to the first decoder for buffering, the playback resolution of the resume part data is higher than the highest playback resolution supported by the second decoder ;
在所述起播部分数据播放完毕后,控制所述第二解码器切换至所述第一解码器,播放所述第一解码器中缓存的所述续播部分数据。After the playback of the starting part of the data is completed, the second decoder is controlled to switch to the first decoder to play the continued part of the data buffered in the first decoder.
第二方面,提供一种视频切换的控制方法,包括:In a second aspect, a method for controlling video switching is provided, including:
通过第一解码器播放第一类视频的过程中,从服务器接收第二类视频对应 的起播部分数据,并分配至第二解码器进行缓存,所述起播部分数据的播放分辨率不高于所述第二解码器支持的最高播放分辨率;In the process of playing the first type of video through the first decoder, the starting part of the data corresponding to the second type of video is received from the server and distributed to the second decoder for buffering. The playback resolution of the starting part of the data is not high The highest playback resolution supported by the second decoder;
在所述第一类视频播放完毕后,控制所述第一解码器切换至所述第二解码器,播放所述第二解码器中缓存的所述起播部分数据;After the first type of video is finished playing, controlling the first decoder to switch to the second decoder to play the start-up part of the data buffered in the second decoder;
从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存,所述续播部分数据的播放分辨率高于所述第二解码器支持的最高播放分辨率;Receiving the resume part data corresponding to the second type of video from the server and distributing it to the first decoder for buffering, the playback resolution of the resume part data is higher than the highest playback resolution supported by the second decoder ;
在所述起播部分数据播放完毕后,控制所述第二解码器切换至所述第一解码器,播放所述第一解码器中缓存的所述续播部分数据。After the playback of the starting part of the data is completed, the second decoder is controlled to switch to the first decoder to play the continued part of the data buffered in the first decoder.
第三方面,提供一种存储介质,其特征在于,存储有用于实现视频切换的控制方法的程序,所述程序被控制器运行时,执行以下步骤:In a third aspect, a storage medium is provided, characterized in that a program for implementing a control method for video switching is stored, and when the program is run by a controller, the following steps are performed:
通过第一解码器播放第一类视频的过程中,从服务器接收第二类视频对应的起播部分数据,并分配至第二解码器进行缓存,所述起播部分数据的播放分辨率不高于所述第二解码器支持的最高播放分辨率;In the process of playing the first type of video through the first decoder, the starting part of the data corresponding to the second type of video is received from the server and distributed to the second decoder for buffering. The playback resolution of the starting part of the data is not high The highest playback resolution supported by the second decoder;
在所述第一类视频播放完毕后,控制所述第一解码器切换至所述第二解码器,播放所述第二解码器中缓存的所述起播部分数据;After the first type of video is finished playing, controlling the first decoder to switch to the second decoder to play the start-up part of the data buffered in the second decoder;
从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存,所述续播部分数据的播放分辨率高于所述第二解码器支持的最高播放分辨率;Receiving the resume part data corresponding to the second type of video from the server and distributing it to the first decoder for buffering, the playback resolution of the resume part data is higher than the highest playback resolution supported by the second decoder ;
在所述起播部分数据播放完毕后,控制所述第二解码器切换至所述第一解码器,播放所述第一解码器中缓存的所述续播部分数据。After the playback of the starting part of the data is completed, the second decoder is controlled to switch to the first decoder to play the continued part of the data buffered in the first decoder.
附图说明BRIEF DESCRIPTION
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简要介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域的普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。In order to more clearly explain the technical solutions in the embodiments of the present invention, the drawings required in the description of the embodiments will be briefly described below. Obviously, the drawings in the following description are only some embodiments of the present invention. Those of ordinary skill in the art can obtain other drawings based on these drawings without paying any creative labor.
图1A中示例性示出了显示设备与控制装置之间操作场景的示意图;FIG. 1A exemplarily shows a schematic diagram of an operation scene between a display device and a control device;
图1B中示例性示出了图1A中控制装置100的配置框图;FIG. 1B exemplarily shows a configuration block diagram of the control device 100 in FIG. 1A;
图1C中示例性示出了图1A中显示设备200的配置框图;FIG. 1C exemplarily shows a configuration block diagram of the display device 200 in FIG. 1A;
图1D中示例性示出了显示设备200存储器中操作系统的架构配置框图;FIG. 1D exemplarily shows a block diagram of the architecture configuration of the operating system in the memory of the display device 200;
图2中示例性示出了显示设备200的另一种配置框图;FIG. 2 exemplarily shows a block diagram of another configuration of the display device 200;
图3中示例性示出了显示设备进行视频切换的流程示意图。FIG. 3 exemplarily shows a schematic flowchart of video switching performed by the display device.
具体实施方式detailed description
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,并不是全部的实施例。本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.
图1A中示例性示出了显示设备与控制装置之间操作场景的示意图。如图1A所示,控制装置100和显示设备200之间可以有线或无线方式进行通信。FIG. 1A exemplarily shows a schematic diagram of an operation scene between a display device and a control device. As shown in FIG. 1A, the control apparatus 100 and the display device 200 may communicate in a wired or wireless manner.
其中,控制装置100被配置为控制显示设备200,其可接收用户输入的操作指令,且将操作指令转换为显示设备200可识别和响应的指令,起着用户与显示设备200之间交互的中介作用。如:用户通过操作控制装置100上频道加 减键,显示设备200响应频道加减的操作。The control device 100 is configured to control the display device 200, which can receive operation instructions input by the user and convert the operation instructions into instructions that the display device 200 can recognize and respond to, acting as an intermediary between the user and the display device 200 effect. For example, the user operates the channel addition and subtraction keys on the control device 100, and the display device 200 responds to the channel addition and subtraction operations.
控制装置100可以是遥控器100A,包括红外协议通信或蓝牙协议通信,及其他短距离通信方式等,通过无线或其他有线方式来控制显示设备200。用户可以通过遥控器上按键、语音输入、控制面板输入等输入用户指令,来控制显示设备200。如:用户可以通过遥控器上音量加减键、频道控制键、上/下/左/右的移动按键、语音输入按键、菜单键、开关机按键等输入相应控制指令,来实现控制显示设备200的功能。The control device 100 may be a remote controller 100A, including infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, etc., to control the display device 200 through wireless or other wired methods. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, and control panel input. For example, the user can control the display device 200 by inputting corresponding control commands through the volume up and down keys on the remote control, channel control keys, up / down / left / right movement keys, voice input keys, menu keys, power on / off keys, etc. Function.
控制装置100也可以是智能设备,如移动终端100B、平板电脑、计算机、笔记本电脑等。例如,使用在智能设备上运行的应用程序控制显示设备200。该应用程序通过配置可以在与智能设备关联的屏幕上,通过直观的用户界面(UI)为用户提供各种控制。The control device 100 may also be a smart device, such as a mobile terminal 100B, tablet computer, computer, notebook computer, or the like. For example, the display device 200 is controlled using an application program running on a smart device. The application can be configured to provide users with various controls on the screen associated with the smart device through an intuitive user interface (UI).
示例性的,移动终端100B可与显示设备200安装软件应用,通过网络通信协议实现连接通信,实现一对一控制操作的和数据通信的目的。如:可以使移动终端100B与显示设备200建立控制指令协议,通过操作移动终端100B上提供的用户界面的各种功能键或虚拟按钮,来实现如遥控器100A布置的实体按键的功能。也可以将移动终端100B上显示的音视频内容传输到显示设备200上,实现同步显示功能。Exemplarily, the mobile terminal 100B may install a software application with the display device 200, implement connection communication through a network communication protocol, and realize one-to-one control operation and data communication purposes. For example, the mobile terminal 100B and the display device 200 can establish a control instruction protocol, and the functions of the physical keys such as the remote controller 100A can be realized by operating various function keys or virtual buttons of the user interface provided on the mobile terminal 100B. The audio and video content displayed on the mobile terminal 100B may also be transmitted to the display device 200 to realize the synchronous display function.
显示设备200可提供广播接收功能和计算机支持功能的网络电视功能。显示设备可以实施为,数字电视、网络电视、互联网协议电视(IPTV)等。The display device 200 may provide a network receiving function of a broadcast receiving function and a computer supporting function. The display device may be implemented as a digital TV, an Internet TV, an Internet Protocol TV (IPTV), etc.
显示设备200,可以是液晶显示器、有机发光显示器、投影设备。具体显示设备类型、尺寸大小和分辨率等不作限定。The display device 200 may be a liquid crystal display, an organic light emitting display, or a projection device. The specific display device type, size and resolution are not limited.
显示设备200还与服务器300通过多种通信方式进行数据通信。这里可允 许显示设备200通过局域网(LAN)、无线局域网(WLAN)和其他网络进行通信连接。服务器300可以向显示设备200提供各种内容和互动。示例的,显示设备200可以发送和接收信息,例如:接收电子节目指南(EPG)数据、接收软件程序更新、或访问远程储存的数字媒体库。服务器300可以一组,也可以多组,可以一类或多类服务器。通过服务器300提供视频点播和广告服务等其他网络服务内容。The display device 200 also performs data communication with the server 300 through various communication methods. Here, the display device 200 may be allowed to perform communication connection through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 300 can provide various contents and interactions to the display device 200. Illustratively, the display device 200 can send and receive information, for example: receive electronic program guide (EPG) data, receive software program updates, or access a remotely stored digital media library. The server 300 may be one group or multiple groups, and may be one or more types of servers. The server 300 provides other network service content such as video on demand and advertising services.
图1B中示例性示出了控制装置100的配置框图。如图1B所示,控制装置100包括控制器110、存储器120、通信器130、用户输入接口140、输出接口150、供电电源160。A configuration block diagram of the control device 100 is exemplarily shown in FIG. 1B. As shown in FIG. 1B, the control device 100 includes a controller 110, a memory 120, a communicator 130, a user input interface 140, an output interface 150, and a power supply 160.
控制器110包括随机存取存储器(RAM)111、只读存储器(ROM)112、处理器113、通信接口以及通信总线。控制器110用于控制控制装置100的运行和操作,以及内部各部件之间的通信协作、外部和内部的数据处理功能。The controller 110 includes a random access memory (RAM) 111, a read only memory (ROM) 112, a processor 113, a communication interface, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as communication cooperation between internal components, external and internal data processing functions.
示例性的,当检测到用户按压在遥控器100A上布置的按键的交互或触摸在遥控器100A上布置的触摸面板的交互时,控制器110可控制产生与检测到的交互相应的信号,并将该信号发送到显示设备200。Exemplarily, when it is detected that the user interacts with pressing a key arranged on the remote controller 100A or touches a touch panel arranged on the remote controller 100A, the controller 110 may control to generate a signal corresponding to the detected interaction, and This signal is sent to the display device 200.
存储器120,用于在控制器110的控制下存储驱动和控制控制装置100的各种运行程序、数据和应用。存储器120,可以存储用户输入的各类控制信号指令。The memory 120 is used to store various operating programs, data, and applications that drive and control the control device 100 under the control of the controller 110. The memory 120 can store various control signal instructions input by the user.
通信器130在控制器110的控制下,实现与显示设备200之间控制信号和数据信号的通信。如:控制装置100经由通信器130将控制信号(例如触摸信号或按钮信号)发送至显示设备200上,控制装置100可经由通信器130接收由显示设备200发送的信号。通信器130可以包括红外信号接口131和射频信 号接口132。例如:红外信号接口时,需要将用户输入指令按照红外控制协议转化为红外控制信号,经红外发送模块进行发送至显示设备200。再如:射频信号接口时,需将用户输入指令转化为数字信号,然后按照射频控制信号调制协议进行调制后,由射频发送端子发送至显示设备200。The communicator 130 realizes communication of control signals and data signals with the display device 200 under the control of the controller 110. For example, the control device 100 transmits a control signal (such as a touch signal or a button signal) to the display device 200 via the communicator 130, and the control device 100 can receive the signal transmitted by the display device 200 via the communicator 130. The communicator 130 may include an infrared signal interface 131 and a radio frequency signal interface 132. For example, when the infrared signal interface is used, the user input command needs to be converted into an infrared control signal according to the infrared control protocol, and then sent to the display device 200 via the infrared sending module. As another example: when the radio frequency signal interface, the user input command needs to be converted into a digital signal, and then modulated according to the radio frequency control signal modulation protocol, and then sent to the display device 200 by the radio frequency transmission terminal.
用户输入接口140,可包括麦克风141、触摸板142、传感器143、按键144等中至少一者,从而用户可以通过语音、触摸、手势、按压等将关于控制显示设备200的用户指令输入到控制装置100。The user input interface 140 may include at least one of a microphone 141, a touch pad 142, a sensor 143, a key 144, and the like, so that the user can input user instructions for controlling the display device 200 to the control device through voice, touch, gesture, pressing 100.
输出接口150,通过将用户输入接口140接收的用户指令输出至显示设备200,或者,输出由显示设备200接收的图像或语音信号。这里,输出接口150可以包括LED接口151、产生振动的振动接口152、输出声音的声音输出接口153和输出图像的显示器154等。例如,遥控器100A可从输出接口150接收音频、视频或数据等输出信号,并且将输出信号在显示器154上显示为图像形式、在声音输出接口153输出为音频形式或在振动接口152输出为振动形式。The output interface 150 outputs the user instruction received by the user input interface 140 to the display device 200, or outputs the image or voice signal received by the display device 200. Here, the output interface 150 may include an LED interface 151, a vibration interface 152 that generates vibration, a sound output interface 153 that outputs sound, a display 154 that outputs an image, and the like. For example, the remote controller 100A may receive output signals such as audio, video, or data from the output interface 150, and display the output signals as an image form on the display 154, output as an audio form on the sound output interface 153, or output as vibration on the vibration interface 152 form.
供电电源160,用于在控制器110的控制下为控制装置100各元件提供运行电力支持。形式可以为电池及相关控制电路。The power supply 160 is used to provide operating power support for each element of the control device 100 under the control of the controller 110. The form can be battery and related control circuit.
图1C中示例性示出了显示设备200的硬件配置框图。如图1C所示,显示设备200中可以包括调谐解调器210、通信器220、检测器230、外部装置接口240、控制器250、存储器260、用户接口265、视频处理器270、显示器275、音频处理器280、音频输出接口285、供电电源290。A block diagram of the hardware configuration of the display device 200 is exemplarily shown in FIG. 1C. As shown in FIG. 1C, the display device 200 may include a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a memory 260, a user interface 265, a video processor 270, a display 275, Audio processor 280, audio output interface 285, power supply 290.
调谐解调器210,通过有线或无线方式接收广播电视信号,可以进行放大、混频和谐振等调制解调处理,用于从多个无线或有线广播电视信号中解调出用户所选择的电视频道的频率中所携带的音视频信号,以及附加信息(例如EPG 数据)。 Tuner demodulator 210, which receives broadcast TV signals by wire or wireless, can perform modulation, demodulation and other processing of amplification, mixing and resonance, and is used to demodulate the TV selected by the user from multiple wireless or cable broadcast TV signals The audio and video signals carried in the frequency of the channel, as well as additional information (such as EPG data).
调谐解调器210,可根据用户选择,以及由控制器250控制,响应用户选择的电视频道的频率以及该频率所携带的电视信号。 Tuner demodulator 210 can respond to the frequency of the TV channel selected by the user and the TV signal carried by the frequency according to the user's selection and controlled by controller 250.
调谐解调器210,根据电视信号的广播制式不同,可以接收信号的途径有很多种,诸如:地面广播、有线广播、卫星广播或互联网广播等;以及根据调制类型不同,可以数字调制方式或模拟调制方式;以及根据接收电视信号的种类不同,可以解调模拟信号和数字信号。 Tuner demodulator 210, according to the different broadcasting systems of TV signals, there are many ways to receive the signal, such as: terrestrial broadcasting, cable broadcasting, satellite broadcasting or Internet broadcasting, etc .; and depending on the type of modulation, digital modulation or analog Modulation method; and according to the type of TV signal received, analog and digital signals can be demodulated.
在其他一些示例性实施例中,调谐解调器210也可在外部设备中,如外部机顶盒等。这样,机顶盒通过调制解调后输出电视信号,经过外部装置接口240输入至显示设备200中。In some other exemplary embodiments, the tuning demodulator 210 may also be in an external device, such as an external set-top box. In this way, the set-top box outputs a TV signal after being modulated and demodulated, and is input into the display device 200 through the external device interface 240.
通信器220,是用于根据各种通信协议类型与外部设备或外部服务器进行通信的组件。例如显示设备200可将内容数据发送至经由通信器220连接的外部设备,或者,从经由通信器220连接的外部设备浏览和下载内容数据。通信器220可以包括WIFI模块221、蓝牙通信协议模块222、有线以太网通信协议模块223等网络通信协议模块或近场通信协议模块,从而通信器220可根据控制器250的控制接收控制装置100的控制信号,并将控制信号实现为WIFI信号、蓝牙信号、射频信号等。The communicator 220 is a component for communicating with an external device or an external server according to various types of communication protocols. For example, the display device 200 may transmit content data to an external device connected via the communicator 220, or browse and download content data from an external device connected via the communicator 220. The communicator 220 may include a network communication protocol module such as a WIFI module 221, a Bluetooth communication protocol module 222, a wired Ethernet communication protocol module 223, or a near field communication protocol module, so that the communicator 220 can receive Control signals, and implement the control signals as WIFI signals, Bluetooth signals, radio frequency signals, etc.
检测器230,是显示设备200用于采集外部环境或与外部交互的信号的组件。检测器230可以包括声音采集器231,如麦克风,可以用于接收用户的声音,如用户控制显示设备200的控制指令的语音信号;或者,可以采集用于识别环境场景类型的环境声音,实现显示设备200可以自适应环境噪声。The detector 230 is a component used by the display device 200 to collect signals from the external environment or interact with the external environment. The detector 230 may include a sound collector 231, such as a microphone, which may be used to receive the user's voice, such as a voice signal of a user's control instruction to control the display device 200; or, it may collect environmental sounds used to identify the type of environmental scene to achieve display The device 200 can adapt to environmental noise.
在其他一些示例性实施例中,检测器230,还可以包括图像采集器232,如 相机、摄像头等,可以用于采集外部环境场景,以自适应变化显示设备200的显示参数;以及用于采集用户的属性或与用户交互手势,以实现显示设备与用户之间互动的功能。In some other exemplary embodiments, the detector 230 may further include an image collector 232, such as a camera, a camera, etc., which may be used to collect external environment scenes to adaptively change the display parameters of the display device 200; The user's attributes or interactive gestures with the user to achieve the interactive function between the display device and the user.
在其他一些示例性实施例中,检测器230,还可以包括光接收器,用于采集环境光线强度,以自适应显示设备200的显示参数变化等。In some other exemplary embodiments, the detector 230 may further include a light receiver for collecting ambient light intensity to adapt to changes in display parameters of the display device 200 and the like.
在其他一些示例性实施例中,检测器230,还可以包括温度传感器,如通过感测环境温度,显示设备200可自适应调整图像的显示色温。示例性的,当温度偏高的环境时,可调整显示设备200显示图像色温偏冷色调;当温度偏低的环境时,可以调整显示设备200显示图像色温偏暖色调。In some other exemplary embodiments, the detector 230 may further include a temperature sensor. For example, by sensing the ambient temperature, the display device 200 may adaptively adjust the display color temperature of the image. Exemplarily, when the temperature is relatively high, the display device 200 may be adjusted to display a cold color tone of the image temperature; when the temperature is relatively low, the display device 200 may be adjusted to display a warm color tone of the image color temperature.
外部装置接口240,是提供控制器250控制显示设备200与外部设备间数据传输的组件。外部装置接口240可按照有线/无线方式与诸如机顶盒、游戏装置、笔记本电脑等外部设备连接,可接收外部设备的诸如视频信号(例如运动图像)、音频信号(例如音乐)、附加信息(例如EPG)等数据。The external device interface 240 is a component that provides the controller 250 to control the data transmission between the display device 200 and the external device. The external device interface 240 can be connected to external devices such as set-top boxes, game devices, notebook computers, etc. in a wired / wireless manner, and can receive external devices such as video signals (such as moving images), audio signals (such as music), and additional information (such as EPG) ) And other data.
其中,外部装置接口240可以包括:高清多媒体接口(HDMI)端子241、复合视频消隐同步(CVBS)端子242、模拟或数字分量端子243、通用串行总线(USB)端子244、组件(Component)端子(图中未示出)、红绿蓝(RGB)端子(图中未示出)等任一个或多个。The external device interface 240 may include: a high-definition multimedia interface (HDMI) terminal 241, a composite video blanking synchronization (CVBS) terminal 242, an analog or digital component terminal 243, a universal serial bus (USB) terminal 244, and a component Any one or more of terminals (not shown in the figure), red, green and blue (RGB) terminals (not shown in the figure) and the like.
控制器250,通过运行存储在存储器260上的各种软件控制程序(如操作系统和各种应用程序),来控制显示设备200的工作和响应用户的操作。The controller 250 controls various operations of the display device 200 and responds to user operations by running various software control programs (such as an operating system and various application programs) stored on the memory 260.
如图1C所示,控制器250包括随机存取存储器(RAM)251、只读存储器(ROM)252、图形处理器253、CPU处理器254、通信接口255、以及通信总线256。其中,RAM251、ROM252以及图形处理器253、CPU处理器254 通信接口255通过通信总线256相连接。As shown in FIG. 1C, the controller 250 includes a random access memory (RAM) 251, a read only memory (ROM) 252, a graphics processor 253, a CPU processor 254, a communication interface 255, and a communication bus 256. Among them, the RAM 251, ROM 252, graphics processor 253, and CPU processor 254 communication interface 255 are connected through a communication bus 256.
ROM252,用于存储各种系统启动指令。如在接收到开机信号时,显示设备200电源开始启动,CPU处理器254运行ROM252中的系统启动指令,将存储在存储器260的操作系统拷贝至RAM251中,以开始运行启动操作系统。当操作系统启动完成后,CPU处理器254再将存储器260中各种应用程序拷贝至RAM251中,然后,开始运行启动各种应用程序。ROM252 is used to store various system startup instructions. For example, when the power-on signal is received, the power of the display device 200 starts to start, the CPU processor 254 runs the system start instruction in the ROM 252, and copies the operating system stored in the memory 260 to the RAM 251 to start the start operating system. After the startup of the operating system is completed, the CPU processor 254 copies the various application programs in the memory 260 to the RAM 251, and then starts to run and start the various application programs.
图形处理器253,用于产生各种图形对象,如图标、操作菜单、以及用户输入指令显示图形等。图形处理器253可以包括运算器,用于通过接收用户输入各种交互指令进行运算,进而根据显示属性显示各种对象;以及包括渲染器,用于产生基于运算器得到的各种对象,将进行渲染的结果显示在显示器275上。The graphics processor 253 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics. The graphics processor 253 may include an operator for performing operations by receiving various interactive instructions input by the user, and then display various objects according to display attributes; and a renderer for generating various objects obtained based on the operator, which will be performed The rendered result is displayed on the display 275.
CPU处理器254,用于执行存储在存储器260中的操作系统和应用程序指令。以及根据接收的用户输入指令,来执行各种应用程序、数据和内容的处理,以便最终显示和播放各种音视频内容。The CPU processor 254 is used to execute operating system and application program instructions stored in the memory 260. And according to the received user input instructions, to perform various applications, data and content processing, so as to finally display and play various audio and video content.
在一些示例性实施例中,CPU处理器254,可以包括多个处理器。多个处理器可包括一个主处理器以及多个或一个子处理器。主处理器,用于在显示设备预加载模式中执行显示设备200的一些初始化操作,和/或,在正常模式下显示画面的操作。多个或一个子处理器,用于执行在显示设备待机模式等状态下的一种操作。In some exemplary embodiments, the CPU processor 254 may include multiple processors. The multiple processors may include a main processor and multiple or one sub-processors. The main processor is used to perform some initialization operations of the display device 200 in the pre-loading mode of the display device, and / or display screen operations in the normal mode. Multiple or one sub-processors are used to perform an operation in the standby mode of the display device.
通信接口255,可包括第一接口到第n接口。这些接口可以是经由网络被连接到外部设备的网络接口。The communication interface 255 may include a first interface to an n-th interface. These interfaces may be network interfaces connected to external devices via a network.
控制器250可以控制显示设备200的整体操作。例如:响应于接收到用于选择在显示器275上显示的GUI对象的用户输入命令,控制器250便可以执行 与由用户输入命令选择的对象有关的操作。The controller 250 may control the overall operation of the display device 200. For example, in response to receiving a user input command for selecting a GUI object displayed on the display 275, the controller 250 may perform an operation related to the object selected by the user input command.
其中,该对象可以是可选对象中的任何一个,例如超链接或图标。该与所选择的对象有关的操作,例如显示连接到超链接页面、文档、图像等操作,或者执行与对象相对应的程序的操作。该用于选择GUI对象的用户输入命令,可以是通过连接到显示设备200的各种输入装置(例如,鼠标、键盘、触摸板等)输入命令或者与由用户说出语音相对应的语音命令。Among them, the object may be any one of selectable objects, such as a hyperlink or an icon. The operation related to the selected object, for example, the operation of connecting to a hyperlinked page, document, image, etc., or the operation of executing a program corresponding to the object. The user input command for selecting a GUI object may be a command input through various input devices (eg, mouse, keyboard, touch pad, etc.) connected to the display device 200 or a voice command corresponding to a voice spoken by the user.
存储器260,用于存储驱动和控制显示设备200运行的各种类型的数据、软件程序或应用程序。存储器260可以包括易失性和/或非易失性存储器。而术语“存储器”包括存储器260、控制器250的RAM251和ROM252、或显示设备200中的存储卡。The memory 260 is used to store various types of data, software programs, or application programs that drive and control the operation of the display device 200. The memory 260 may include volatile and / or non-volatile memory. The term "memory" includes the memory 260, the RAM 251 and the ROM 252 of the controller 250, or the memory card in the display device 200.
在一些实施例中,存储器260具体用于存储驱动显示设备200中控制器250的运行程序;存储显示设备200内置的和用户从外部设备下载的各种应用程序;存储用于配置由显示器275提供的各种GUI、与GUI相关的各种对象及用于选择GUI对象的选择器的视觉效果图像等数据。In some embodiments, the memory 260 is specifically used to store operating programs that drive the controller 250 in the display device 200; store various application programs built in the display device 200 and downloaded by users from external devices; and store for configuration provided by the display 275 Data of various GUIs, various objects related to the GUI, and visual effects images of selectors for selecting GUI objects.
在一些实施例中,存储器260具体用于存储调谐解调器210、通信器220、检测器230、外部装置接口240、视频处理器270、显示器275、音频处理器280等的驱动程序和相关数据,例如从外部装置接口接收的外部数据(例如音视频数据)或用户接口接收的用户数据(例如按键信息、语音信息、触摸信息等)。In some embodiments, the memory 260 is specifically used to store drivers and related data of the tuner-demodulator 210, communicator 220, detector 230, external device interface 240, video processor 270, display 275, audio processor 280, etc. For example, external data received from an external device interface (such as audio and video data) or user data received from a user interface (such as key information, voice information, touch information, etc.).
在一些实施例中,存储器260具体存储用于表示操作系统(OS)的软件和/或程序,这些软件和/或程序可包括,例如:内核、中间件、应用编程接口(API)和/或应用程序。示例性的,内核可控制或管理系统资源,以及其它程序所实施的功能(如所述中间件、API或应用程序);同时,内核可以提供接口,以允许中 间件、API或应用程序访问控制器,以实现控制或管理系统资源。In some embodiments, the memory 260 specifically stores software and / or programs for representing an operating system (OS), and these software and / or programs may include, for example, a kernel, middleware, application programming interface (API), and / or application. Exemplarily, the kernel can control or manage system resources, and functions implemented by other programs (such as the middleware, API, or application program); meanwhile, the kernel can provide an interface to allow middleware, API, or application program access control To control or manage system resources.
图1D中示例性示出了显示设备200存储器中操作系统的架构配置框图。该操作系统架构从上到下依次是应用层、中间件层和内核层。FIG. 1D exemplarily shows a block diagram of the architecture configuration of the operating system in the memory of the display device 200. The operating system architecture from top to bottom is the application layer, middleware layer and kernel layer.
应用层,系统内置的应用程序以及非系统级的应用程序都是属于应用层。负责与用户进行直接交互。应用层可包括多个应用程序,如设置应用程序、电子帖应用程序、媒体中心应用程序等。这些应用程序可被实现为Web应用,其基于WebKit引擎来执行,具体可基于HTML5、层叠样式表(CSS)和JavaScript来开发并执行。The application layer, application programs built in the system, and non-system-level application programs belong to the application layer. Responsible for direct interaction with users. The application layer may include multiple applications, such as a setting application, an e-post application, and a media center application. These applications can be implemented as Web applications, which are executed based on the WebKit engine, and specifically can be developed and executed based on HTML5, Cascading Style Sheets (CSS), and JavaScript.
这里,HTML,全称为超文本标记语言(HyperText Markup Language),是一种用于创建网页的标准标记语言,通过标记标签来描述网页,HTML标签用以说明文字、图形、动画、声音、表格、链接等,浏览器会读取HTML文档,解释文档内标签的内容,并以网页的形式显示出来。Here, HTML, which is called HyperText Markup Language (HyperText Markup Language), is a standard markup language for creating web pages. The web pages are described by markup tags. HTML tags are used to describe text, graphics, animation, sound, tables, Links, etc., the browser will read the HTML document, explain the content of the tags in the document, and display it in the form of a web page.
CSS,全称为层叠样式表(Cascading Style Sheets),是一种用来表现HTML文件样式的计算机语言,可以用来定义样式结构,如字体、颜色、位置等的语言。CSS样式可以直接存储与HTML网页或者单独的样式文件中,实现对网页中样式的控制。CSS, which is called Cascading Style Sheets, is a computer language used to express the style of HTML files. It can be used to define the style structure, such as fonts, colors, and positions. CSS styles can be stored directly with HTML web pages or separate style files to control the styles in web pages.
JavaScript,是一种应用于Web网页编程的语言,可以插入HTML页面并由浏览器解释执行。其中Web应用的交互逻辑都是通过JavaScript实现。JavaScript可以通过浏览器,封装JavaScript扩展接口,实现与内核层的通信,JavaScript is a language used in the programming of Web pages. It can be inserted into HTML pages and interpreted by the browser. The interaction logic of the Web application is implemented through JavaScript. JavaScript can encapsulate the JavaScript extension interface through the browser to achieve communication with the kernel layer,
中间件层,可以提供一些标准化的接口,以支持各种环境和系统的操作。例如,中间件层可以实现为与数据广播相关的中间件的多媒体和超媒体信息编码专家组(MHEG),还可以实现为与外部设备通信相关的中间件的DLNA中间 件,还可以实现为提供显示设备内各应用程序所运行的浏览器环境的中间件等。The middleware layer can provide some standardized interfaces to support the operation of various environments and systems. For example, the middleware layer can be implemented as a multimedia and hypermedia information coding expert group (MHEG) for middleware related to data broadcasting, as well as DLNA middleware for middleware related to external device communication, and can also be implemented as providing Middleware that displays the browser environment in which each application in the device runs.
内核层,提供核心系统服务,例如:文件管理、内存管理、进程管理、网络管理、系统安全权限管理等服务。内核层可以被实现为基于各种操作系统的内核,例如,基于Linux操作系统的内核。The kernel layer provides core system services, such as file management, memory management, process management, network management, and system security authority management. The kernel layer may be implemented as a kernel based on various operating systems, for example, a kernel based on a Linux operating system.
内核层也同时提供系统软件和硬件之间的通信,为各种硬件提供设备驱动服务,例如:为显示器提供显示驱动程序、为摄像头提供摄像头驱动程序、为遥控器提供按键驱动程序、为WIFI模块提供WiFi驱动程序、为音频输出接口提供音频驱动程序、为电源管理(PM)模块提供电源管理驱动等。The kernel layer also provides communication between system software and hardware, and provides device driver services for various hardware, such as: display driver for display, camera driver for camera, key driver for remote control, and WIFI module Provide WiFi drivers, audio drivers for audio output interfaces, and power management drivers for power management (PM) modules.
用户接口265,接收各种用户交互。具体的,用于将用户的输入信号发送给控制器250,或者,将从控制器250的输出信号传送给用户。示例性的,遥控器100A可将用户输入的诸如电源开关信号、频道选择信号、音量调节信号等输入信号发送至用户接口265,再由用户接口265转送至控制器250;或者,遥控器100A可接收经控制器250处理从用户接口265输出的音频、视频或数据等输出信号,并且显示接收的输出信号或将接收的输出信号输出为音频或振动形式。The user interface 265 receives various user interactions. Specifically, it is used to send the user's input signal to the controller 250, or to transmit the output signal from the controller 250 to the user. Exemplarily, the remote controller 100A may send input signals such as a power switch signal, a channel selection signal, a volume adjustment signal, etc. input by the user to the user interface 265, and then the user interface 265 forwards the signal to the controller 250; The output signal such as audio, video or data output from the user interface 265 processed by the controller 250 is received, and the received output signal is displayed or output as an audio or vibration form.
在一些实施例中,用户可在显示器275上显示的图形用户界面(GUI)输入用户命令,则用户接口265通过GUI接收用户输入命令。确切的说,用户接口265可接收用于控制选择器在GUI中的位置以选择不同的对象或项目的用户输入命令。In some embodiments, a user may input a user command on a graphical user interface (GUI) displayed on the display 275, and the user interface 265 receives the user input command through the GUI. Specifically, the user interface 265 may receive user input commands for controlling the position of the selector in the GUI to select different objects or items.
或者,用户可通过输入特定的声音或手势进行输入用户命令,则用户接口265通过传感器识别出声音或手势,来接收用户输入命令。Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user interface 265 recognizes the sound or gesture through the sensor to receive the user input command.
视频处理器270,用于接收外部的视频信号,根据输入信号的标准编解码协 议,进行解压缩、解码、缩放、降噪、帧率转换、分辨率转换、图像合成等视频数据处理,可得到直接在显示器275上显示或播放的视频信号。The video processor 270 is used to receive external video signals and perform video data processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard codec protocol of the input signal. The video signal displayed or played directly on the display 275.
示例的,视频处理器270,包括解复用模块、视频解码模块(解码器)、图像合成模块、帧率转换模块、显示格式化模块等。For example, the video processor 270 includes a demultiplexing module, a video decoding module (decoder), an image synthesis module, a frame rate conversion module, a display formatting module, and so on.
其中,解复用模块,用于对输入音视频数据流进行解复用处理,如输入MPEG-2流(基于数字存储媒体运动 图像和语音的压缩标准),则解复用模块将其进行解复用成视频信号和音频信号等。 Wherein the demultiplexing module is used to input audio and video data stream demultiplexing processing, such as MPEG-2 input stream (based on the digital storage medium the moving image and voice compression standard), the demultiplexing module which de Multiplexed into video signals and audio signals.
视频解码模块,用于对解复用后的视频信号进行处理,包括解码、缩放和缓存处理等。这里,视频解码模块的数量可以一个或多个。The video decoding module is used to process the demultiplexed video signal, including decoding, scaling, and buffer processing. Here, the number of video decoding modules may be one or more.
图像合成模块,如图像合成器,其用于将图形生成器根据用户输入或自身生成的GUI信号,与缩放处理后视频图像进行叠加混合处理,以生成可供显示的图像信号。An image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to user input or generated by itself with the zoomed video image to generate an image signal for display.
帧率转换模块,用于对输入视频的帧率进行转换,如将输入的60Hz视频的帧率转换为120Hz或240Hz的帧率,通常的格式采用如插帧方式实现。The frame rate conversion module is used to convert the frame rate of the input video, such as converting the frame rate of the input 60 Hz video to a frame rate of 120 Hz or 240 Hz. The usual format is implemented by inserting frames, for example.
显示格式化模块,用于将帧率转换模块输出的信号,改变为符合诸如显示器显示格式的信号,如将帧率转换模块输出的信号进行格式转换以输出RGB数据信号。The display formatting module is used to change the signal output by the frame rate conversion module into a signal conforming to the display format of the display, for example, format-converting the signal output by the frame rate conversion module to output RGB data signals.
显示器275,用于接收源自视频处理器270输入的图像信号,进行显示视频内容、图像以及菜单操控界面。显示视频内容,可以来自调谐解调器210接收的广播信号中的视频内容,也可以来自通信器220或外部装置接口240输入的视频内容。显示器275,同时显示显示设备200中产生且用于控制显示设备200的用户操控界面UI。The display 275 is used to receive the image signal input from the video processor 270 and display the video content, image, and menu control interface. The displayed video content may come from the video content in the broadcast signal received by the tuner demodulator 210, or from the video content input from the communicator 220 or the external device interface 240. The display 275 also displays a user control interface UI generated in the display device 200 and used to control the display device 200.
以及,显示器275可以包括用于呈现画面的显示屏组件以及驱动图像显示的驱动组件。或者,倘若显示器275为一种投影显示器,还可以包括一种投影装置和投影屏幕。And, the display 275 may include a display screen component for presenting a picture and a driving component that drives image display. Alternatively, if the display 275 is a projection display, it may further include a projection device and a projection screen.
音频处理器280,用于接收外部的音频信号,根据输入信号的标准编解码协议,进行解压缩和解码,以及降噪、数模转换、和放大处理等音频数据处理,得到可以在扬声器286中播放的音频信号。The audio processor 280 is used to receive external audio signals, perform decompression and decoding, and perform audio data processing such as noise reduction, digital-to-analog conversion, and amplification processing according to the standard codec protocol of the input signal, which can be obtained in the speaker 286 Play audio signal.
示例性的,音频处理器280可以支持各种音频格式。例如MPEG-2、MPEG-4、高级音频编码(AAC)、高效AAC(HE-AAC)等格式。Exemplarily, the audio processor 280 may support various audio formats. For example, MPEG-2, MPEG-4, Advanced Audio Coding (AAC), High Efficiency AAC (HE-AAC) and other formats.
音频输出接口285,用于在控制器250的控制下接收音频处理器280输出的音频信号,音频输出接口285可包括扬声器286,或输出至外接设备的发生装置的外接音响输出端子287,如耳机输出端子。The audio output interface 285 is used to receive the audio signal output by the audio processor 280 under the control of the controller 250. The audio output interface 285 may include a speaker 286, or an external audio output terminal 287 output to a generator of an external device, such as a headset Output terminal.
在其他一些示例性实施例中,视频处理器270可以包括一个或多个芯片组成。音频处理器280,也可以包括一个或多个芯片组成。In some other exemplary embodiments, the video processor 270 may include one or more chips. The audio processor 280 may also include one or more chips.
以及,在其他一些示例性实施例中,视频处理器270和音频处理器280,可以为单独的芯片,也可以与控制器250一起集成在一个或多个芯片中。And, in some other exemplary embodiments, the video processor 270 and the audio processor 280 may be separate chips, or may be integrated with the controller 250 in one or more chips.
供电电源290,用于在控制器250的控制下,将外部电源输入的电力为显示设备200提供电源供电支持。供电电源290可以是安装在显示设备200内部的内置电源电路,也可以是安装在显示设备200外部的电源。The power supply 290 is used to provide power supply support for the display device 200 under the control of the controller 250 to input power from an external power supply. The power supply 290 may be a built-in power supply circuit installed inside the display device 200 or a power supply installed outside the display device 200.
在上述图1A和图1C的基础上,继续参见图2,其中,显示设备内的视频处理器270至少包括Main路解码器解码器271和Sub路解码器解码器272。On the basis of the foregoing FIGS. 1A and 1C, continue to refer to FIG. 2, wherein the video processor 270 in the display device includes at least a Main decoder 271 and a Sub decoder 272.
在本申请一些实施例中,控制器250,从服务器300接收视频数据,并针对Main路解码器271和Sub路解码器272,分别创建相应的各类资源,用于 对视频数据进行相应处理,这些资源至少包括媒体播放后台,协议解析端,播放器等等,后续实施例中将会详细介绍,在此不再赘述。In some embodiments of the present application, the controller 250 receives video data from the server 300, and creates corresponding types of resources for the Main decoder 271 and Sub decoder 272, respectively, for corresponding processing of the video data, These resources include at least a media playback background, a protocol analysis terminal, a player, etc., which will be described in detail in subsequent embodiments, and will not be repeated here.
Main路解码器271,例如支持最高2160P的播放分辨率。The Main decoder 271, for example, supports playback resolutions up to 2160P.
Sub路解码器272,例如支持最高1080P的播放分辨率。 Sub-channel decoder 272, for example, supports playback resolutions up to 1080P.
随着用户对视频清晰度的需求越来越高,显示设备的性能也随之要求提高。这种高性能带来的是更多的资源占用,主要体现在显示设备的内存占用和芯片带宽占用,对于高分辨率高码率的视频而言,资源占用程度更甚。由于decoder是硬件,因此decoder运行时占用的内存必须是编译阶段就分配好的静态内存,其为固定地址段,即decoder的内存限制非常严格。As users' demands for video definition become higher and higher, the performance of display devices is also required to increase. This high performance brings more resource occupancy, mainly reflected in the memory occupancy of the display device and the chip bandwidth. For high-resolution and high-bitrate video, the resource occupancy is even greater. Since the decoder is hardware, the memory occupied by the decoder during operation must be static memory allocated at the compilation stage, which is a fixed address segment, that is, the memory limit of the decoder is very strict.
例如,在一些场景下,显示设备采用的针对decoder的内存分配原则如下:For example, in some scenarios, the memory allocation principles adopted by the display device for the decoder are as follows:
针对支持1080p视频的decoder,至少需分配50M内存;For decoders that support 1080p video, at least 50M of memory must be allocated;
针对支持2160p视频的decoder,至少需分配100M内存寻For decoders that support 2160p video, at least 100M of memory must be allocated
基于上述原则,如果要在两个2160p视频之间实现快速视频切换,则需要在编译时就为两路decoder分配200M内存,这在实际操作中是不能被接受的。Based on the above principles, if you want to achieve fast video switching between two 2160p videos, you need to allocate 200M of memory for the two decoders during compilation, which is not acceptable in actual operation.
对于智能电视等高端显示设备而言,支持4K分辨率(即2106p)几乎为刚性需求,因此,两路decoder中,必须有一路支持2106p,而绝大多数广告的视频数据最高只有1080p,所以大部分硬件厂商选择采用2106P+1080p的decoder组合来支持广告插播场景:即正片采用第一路decoder〔又称主(Main)路解码器〕解码,最高支持2106p;广告采用第二路decoder〔又称辅(Sub)路解码器〕解码,最高支持1080p。For high-end display devices such as smart TVs, supporting 4K resolution (that is, 2106p) is almost a rigid requirement. Therefore, one of the two decoders must support 2106p, and the video data of most advertisements is only up to 1080p, so the large Some hardware manufacturers choose to use the 2106P + 1080p decoder combination to support the advertisement insertion scene: that is, the first film is decoded by the first decoder (also known as the main (main) decoder), and the highest support is 2106p; the second decoder (also known as the decoder) is also used for advertising Sub (Sub) decoder] decoding, up to 1080p.
另外,绝大多数硬件厂商的中低端芯片带宽只能支持到一路2106p加一路1080p,所以2106p+1080p的decoder组合方案是满足要求的最佳方案,总共 占用150M内存,可以支持绝大多数场景。In addition, most hardware manufacturers' mid-to-low-end chip bandwidth can only support one channel 2106p plus one channel 1080p, so the decoder combination of 2106p + 1080p is the best solution to meet the requirements. It takes up a total of 150M of memory and can support most scenarios .
然而,上述decoder组合方案却存在以下缺陷:However, the above decoder combination scheme has the following defects:
由于只有两个decoder,那么,在需要在正片播放过程中插播多个广告时,需要采用Main路解码器和Sub路解码器轮流播放正片和各个广告。而当需要插播广告的数量为偶数时,最后一个广告会占用Main路解码器解码,此时,如果需要切换回正片进行播放,只能在Sub路解码器进行相关数据的预处理操作,这样,正片的播放分辨率会限制在1080p,从而影响正片的播放效果。Since there are only two decoders, when multiple advertisements need to be inserted during the playback of the main movie, the Main decoder and Sub decoder need to be used to play the main movie and each advertisement in turn. When the number of advertisements to be inserted is even, the last advertisement will occupy the Main decoder to decode. At this time, if you need to switch back to the main film for playback, you can only perform the preprocessing of the relevant data in the Sub decoder. The playback resolution of the positive film will be limited to 1080p, thus affecting the playback of the positive film.
例如,假设正片的最高播放分辨率为2106p,需要插播的两个广告的最高播放分辨率均为1080p,并且在第二个广告播放结束后切换回正片。那么,显示设备具体需要执行以下操作:For example, suppose that the highest playback resolution of the positive film is 2106p, the highest playback resolution of the two advertisements that need to be inserted are both 1080p, and switch back to the positive film after the second advertisement ends. Then, the display device specifically needs to perform the following operations:
1、采用Main路解码器,解码播放正片。1. Main decoder is used to decode and play the positive film.
2、在正片播放过程中,接收到广告1的播放指令,则采用Sub路解码器创建并初始化广告1的相关资源,解码缓存广告1的相关数据。2. In the process of playing the positive film, when receiving the play instruction of Advertisement 1, the Sub-channel decoder is used to create and initialize the relevant resources of Advertisement 1, and decode the relevant data of Advertisement 1.
3、在广告1的指定播放时间点,暂停播放正片,开始通过Sub路解码器播放广告1,同时,释放Main路解码器的相关资源。3. At the designated playback time of advertisement 1, pause the playback of the positive film, start playing advertisement 1 through the Sub decoder, and at the same time, release the relevant resources of the Main decoder.
4、在广告1的播放过程中,采用Main路解码器创建并初始化广告2的相关资源,解码缓存广告2的相关数据。4. During the playback of Advertisement 1, the Main decoder is used to create and initialize the relevant resources of Advertisement 2 and decode the relevant data of Cache Advertisement 2.
5、在广告1播放结束后,开始通过Main路解码器播放广告2,同时,释放Sub路解码器的相关资源。5. After the advertisement 1 is played, the advertisement of the main channel 2 is started to be played, and at the same time, the related resources of the sub-channel decoder are released.
6、在广告2的播放过程中,接收到切换回正片的播放指令,而此时Main路解码器正在被广告2占用,为了正片的播放过程中可以达到2106p分辨率,显示设备只能牺牲切换性能,即在广告2播放结束后释放Main路解码器的相关 资源,重新初始化,再使用Main路解码器对正片进行解码及播放。这样,会在广告播放结束时产生1~2秒黑屏等待过程。6. During the playback of Advertisement 2, I received the instruction to switch back to the main movie, and the Main decoder is being occupied by Advertisement 2. In order to achieve a resolution of 2106p during the playback of the main movie, the display device can only sacrifice switching Performance, that is, after the advertisement 2 is played, the relevant resources of the Main decoder are released, re-initialized, and then the Main decoder is used to decode and play the positive film. In this way, a black screen waiting process of 1 to 2 seconds will occur at the end of the advertisement playback.
实际使用过程中,广告数量是完全随机的,因此,只要正片的最高播放分In actual use, the number of advertisements is completely random, so as long as the highest playback score of the main film
辨率大于1080p,显示设备便有50%的概率出现上述黑屏等待过程,即出现快速切换失效的情况,这严重影响了视频切换的时效性,以及严重降低了用户体验,无法达到市场需要。If the resolution is greater than 1080p, the display device will have a 50% probability of the above black screen waiting process, that is, the rapid switching failure, which seriously affects the timeliness of video switching, and seriously reduces the user experience, unable to meet market needs.
需要说明的是,流媒体具有三大协议,动态自适应流媒体协议(Dynamic Adaptive Streaming over HTTP,DASH)、苹果公司实现的基于HTTP的流媒体通信协议(Apple HTTP Live Streaming,HLS)、微软公司实现的流媒体解决方案(Microsoft Smooth Streaming,MSS),其主要特性就是根据网络状况自适应切换视频的播放分辨率。It should be noted that streaming media has three major protocols, Dynamic Adaptive Streaming Protocol (Dynamic Adaptive Streaming over HTTP, DASH), Apple-based HTTP-based streaming media communication protocol (Apple HTTP Live Streaming, HLS), Microsoft Corporation The realized streaming media solution (Microsoft Smoothing, MSS), its main feature is to adaptively switch the video playback resolution according to the network conditions.
基于此,本申请中考虑到:在视频播放的前几秒内,由于此时显示设备缓存的视频数据较少以及网络波动问题,绝大多数情况下显示设备不会直接使用支持的最大播放分辨率开始播放,而是先使用较低的播放分辨率进行一段时间的视频播放,在中途再切换至更高的播放分辨率。在实验中,这种情况发生概率大于90%,因此,本申请实施例中,根据视频播放过程体现出的上述特性,显示设备可以将主视频(即正片)的播放过程分为两个阶段:使用较低的播放分辨率的起播阶段,和切换到较高的播放分辨率的续播阶段。Based on this, it is considered in this application that within the first few seconds of video playback, due to the low video data cached by the display device and network fluctuations at this time, the display device will not directly use the maximum playback resolution supported in most cases The rate starts to play, but first use a lower playback resolution for a period of video playback, and then switch to a higher playback resolution in the middle. In the experiment, the occurrence probability of this situation is greater than 90%. Therefore, in the embodiments of the present application, according to the above characteristics embodied in the video playback process, the display device may divide the playback process of the main video (that is, the positive film) into two stages: The start-up phase with a lower playback resolution and the resume phase with a higher playback resolution.
因此,本申请实施例中,当一个多媒体展示信息(如,一个广告)占用Main路解码器进行播放时,显示设备可以先使用Sub路解码器播放主视频的低播放分辨率的起播部分数据,等多媒体展示信息播放完毕后,再切换至Main路解码器播放主视频的高播放分辨率的续播部分数据。这样,后续就可以一直使用高 播放分辨率播放主视频了。Therefore, in the embodiment of the present application, when a multimedia presentation information (for example, an advertisement) occupies the Main decoder to play, the display device may first use the Sub decoder to play the low-resolution starting part of the main video data After the multimedia display information is played, switch to the main channel decoder to play the high playback resolution of the main video. In this way, the main video can be played in high playback resolution all the time.
参阅图3所示,本申请实施例中,显示设备进行视频切换的详细流程如下:Referring to FIG. 3, in the embodiment of the present application, the detailed process of video switching by the display device is as follows:
步骤310:显示设备通过Main路解码器播放多媒体展示信息的过程中,确定需要播放主视频。Step 310: During the process of playing the multimedia presentation information through the Main decoder, the display device determines that the main video needs to be played.
例如,在主视频中插播多媒体展示信息时,由于多媒体展示信息的播放时长是已知的,因此,服务器可以预先通知显示设备,开始准备继续播放主视频。For example, when inserting multimedia presentation information in the main video, since the playing time of the multimedia presentation information is known, the server may notify the display device in advance and start preparing to continue playing the main video.
步骤320:显示设备从服务器接收主视频对应的起播部分数据,并分配至Sub路解码器进行缓存,所述起播部分数据的播放分辨率不高于所述Sub路解码器支持的最高播放分辨率。Step 320: The display device receives the start-up part data corresponding to the main video from the server and distributes it to the Sub-channel decoder for buffering. The playback resolution of the start-up part data is not higher than the highest playback supported by the Sub-channel decoder Resolution.
具体的,在流媒体协议中,视频数据会被分割成若干小片段,记为segment,每个segment一般包含有1~2秒的视频数据,同一个segment的播放分辨率、码率等参数完全相同,这些参数可以通过segment的数据包头传递给显示设备。Specifically, in the streaming media protocol, the video data will be divided into a number of small segments, which are recorded as segments. Each segment generally contains 1 to 2 seconds of video data. The playback resolution and bit rate of the same segment are completely Similarly, these parameters can be passed to the display device through the segment header.
服务器或显示设备均可以根据当前的网络综合状态(如,网速,带宽,数据缓存量等),决定下发或请求下载哪种播放分辨率的视频数据,而显示设备则可以根据接收到的视频数据的数据包头,获知当前接收的segment的播放分辨率等参数。The server or display device can decide which playback resolution video data to send or request to download based on the current network comprehensive status (eg, network speed, bandwidth, data cache amount, etc.), and the display device can be based on the received The data packet header of the video data, and the parameters such as the playback resolution of the currently received segment.
因此,本申请实施例中,显示设备可以根据当前的网络综合状态,请求服务器下发具有较低播放分辨率的视频数据,即起播部分数据。Therefore, in the embodiment of the present application, the display device may request the server to deliver video data with a lower playback resolution according to the current network comprehensive status, that is, start-up part of the data.
相应的,显示设备将接收的起播部分数据发送至Sub路解码器进行缓存时,可以预先对Sub路解码器进行初始化,并预先完成起播部分数据的解析、解码操作,为后续播放做准备,在本申请某些实施例中,起播部分数据的播放分辨率为1080P。Correspondingly, when the display device sends the received start-up part data to the Sub-channel decoder for buffering, the Sub-channel decoder can be initialized in advance, and the analysis and decoding operations of the start-up part data are completed in advance to prepare for subsequent playback In some embodiments of the present application, the playback resolution of the starting part of the data is 1080P.
步骤330:显示设备在所述多媒体展示信息播放完毕后,由所述Main路解码器切换至所述Sub路解码器,播放所述Sub解码器中缓存的所述起播部分数据。Step 330: After the multimedia display information is played, the display device switches from the Main decoder to the Sub decoder to play back the start-up portion of data buffered in the Sub decoder.
具体的,显示设备将Main路解码器中缓存的多媒体展示信息播放完毕后,会快速切换至Sub路解码器,此时,Sub路解码器上缓存的起播部分数据已经解析、解码完毕,可以直接播放。Specifically, after the display device plays the multimedia display information cached in the Main decoder, it will quickly switch to the Sub decoder. At this time, the data of the starting part cached on the Sub decoder has been parsed and decoded. Play directly.
进一步地,在多媒体展示信息播放完毕后,显示设备会将Main路解码器上的各种资源进行释放,并对Main路解码器进行初始化处理,之后,若接收到服务器发送的续播部分数据,则可以直接进行解析、解码等操作,为后续播放做准备。Further, after the multimedia presentation information is played, the display device will release various resources on the Main decoder and initialize the Main decoder. After receiving the resume data sent by the server, Then you can directly perform operations such as parsing and decoding to prepare for subsequent playback.
步骤340:显示设备从服务器接收主视频对应的续播部分数据,并分配至所述Main路解码器进行缓存,所述续播部分数据的播放分辨率高于所述Sub路解码器支持的最高播放分辨率。Step 340: The display device receives the resume data corresponding to the main video from the server and distributes it to the Main decoder for buffering. The playback resolution of the resume data is higher than the highest supported by the Sub decoder. Play resolution.
同理,本申请实施例中,显示设备可以根据当前的网络综合状态,请求服务器下发具有较高播放分辨率的视频数据,即起播部分数据。Similarly, in the embodiment of the present application, the display device may request the server to deliver video data with a higher playback resolution according to the current network comprehensive status, that is, start-up part of the data.
在本申请某些实施例中,续播部分数据的播放分辨率为2106P。In some embodiments of the present application, the playback resolution of the resumed partial data is 2106P.
由于各个segment的数据包头部分可以携带segment的播放分辨率,因此,当显示设备确定接收到的segment对应的播放分辨率高于1080p的视频数据(即续播部分数据)时,会从这部分segment开始,将后续接收到的segment均直接注入Main路解码器。Since the segment header of each segment can carry the playback resolution of the segment, when the display device determines that the video data corresponding to the received segment has a playback resolution higher than 1080p (ie, resumes part of the data), it will start from this segment Initially, all subsequent received segments are directly injected into the Main decoder.
由于每个segment包含的第一帧数据一定是视频关键帧(即I帧),所以只要按照segment进行视频切割,切割后的起播部分数据和续播部分数据都是可 以被单独播放的。Since the first frame of data contained in each segment must be the key frame of the video (that is, I frame), as long as the video is cut according to the segment, the cut-off start data and continued playback data can be played separately.
进一步地,当显示设备识别出从服务器接收的是续播部分数据时,即识别出后续到达的segment的播放分辨率有变化且超过1080p时,会保存下续播部分数据中第一个segment的时间戳,记为T1。Further, when the display device recognizes that the part of the resume data received from the server is recognized, that is, when the playback resolution of the subsequent arrival segment changes and exceeds 1080p, the first segment of the data of the resume broadcast will be saved Time stamp, marked as T1.
实际应用中,服务器会将视频数据持续发送给显示设备,因此,步骤320通常发生在显示设备播放Main路解码器中缓存的多媒体展示信息的过程中,而步骤340通常发生在显示设备播放Sub路解码器中缓存的起播部分数据的过程中。In actual applications, the server will continuously send video data to the display device. Therefore, step 320 usually occurs during the display device playing the multimedia presentation information cached in the Main decoder, while step 340 usually occurs when the display device plays the Sub channel In the process of starting part of the data buffered in the decoder.
步骤350:显示设备在所述起播部分数据播放完毕后,从所述Sub路解码器切换至所述Main路解码器,播放所述Main路解码器中缓存的所述续播部分数据。Step 350: The display device switches from the Sub-channel decoder to the Main-channel decoder after the playback of the starting part of the data is completed, and plays the continued-play-part data buffered in the Main-channel decoder.
具体的,当显示设备播放Sub路解码器中缓存的起播部分数据到达已记载的时间戳时,即到达上述T1的位置时,暂停Sub路解码器,启动Main路解码器,从T1开始播放Main路解码器中缓存的续播部分数据,这样,具有高播放分辨率的续播部分数据就可以在Main路解码器一直播放了。Specifically, when the display device plays the starting portion of the data buffered in the Sub channel decoder and reaches the recorded timestamp, that is, when it reaches the position of T1, the Sub channel decoder is suspended, the Main channel decoder is started, and playback starts from T1 The resumed part of the data buffered in the Main decoder, so that the resumed part of the data with high playback resolution can be played in the Main decoder.
进一步地,显示设备会将Sub路解码器上的各种资源进行释放,并对Sub路解码器进行初始化处理,为后续播放其他视频数据做准备。Further, the display device will release various resources on the Sub decoder, and initialize the Sub decoder to prepare for subsequent playback of other video data.
本申请实施例中,由于多媒体展示信息的播放时长不固定,起播部分数据的播放时长也不固定,因此,显示设备有可能在播放Main路解码器中缓存的多媒体展示信息的过程中,接收到续播部分数据,此时,由于Main路解码器上的各类资源还没有释放,因此,显示设备需要等待多媒体展示信息播放完毕后,将Main路解码器上的各种资源释放,然后对Main路解码器进行初始化,进一 步地,再将接收到的续播部分数据分配至Main路解码器进行缓存。另一方面,显示设备也有可能在播放Sub路解码器中缓存的起播部分数据的过程中,接收到续播部分数据,此时,由于Main路解码器中缓存的多媒体展示信息已播放完毕,则Main路解码器上的各种资源已经得到释放并进行了初始化,因此,显示设备可以直接将接收到的续播部分数据分配至Main路解码器进行缓存。In the embodiment of the present application, since the playback duration of the multimedia presentation information is not fixed, and the playback duration of the start-up part of the data is also not fixed, the display device may receive the multimedia presentation information buffered in the Main decoder during the process. To continue to broadcast some data, at this time, because the various resources on the Main decoder have not been released, the display device needs to wait for the multimedia display information to be played, release the various resources on the Main decoder, and then The Main decoder is initialized, and further, the received resume data is distributed to the Main decoder for buffering. On the other hand, the display device may also receive the resume part of the data during the playback of the start part of the buffered data in the Sub decoder. At this time, because the multimedia presentation information cached in the Main decoder has been played, Then, various resources on the Main decoder have been released and initialized. Therefore, the display device can directly distribute the received part of the resume data to the Main decoder for buffering.
显示设备在播放视频数据时,无论是否使用两个decoder,播放画面在切换播放分辨率时都会造成视觉上的明显变化,并且使用两个decoder会令画面切换迅速,不会有黑屏,因此,采用上述技术方案进行主视频(即正片)播放时,相较于不区分起播部分数据与续播部分数据的技术方案,主观感受基本相同,能够保证播放效果。When the display device is playing video data, regardless of whether two decoders are used, the playback screen will cause obvious visual changes when switching the playback resolution, and the use of two decoders will make the screen switch quickly without a black screen. Therefore, adopt When the above technical solution is used to play the main video (that is, the main film), compared with the technical solution that does not distinguish between the starting part data and the continuing part data, the subjective experience is basically the same, which can ensure the playback effect.
下面结合一个具体的应用场景,对上述实施例作出进一步详细说明。The above embodiment will be further described in detail in conjunction with a specific application scenario.
以下实施例中,假设显示设备需要播放的内容为:正片、广告1、广告2。In the following embodiments, it is assumed that the content to be played by the display device is: positive film, advertisement 1, advertisement 2.
显示设备播放正片时,占用的各类资源包括主媒体播放后台,记为media_backend_main、主协议解析端,记为protocol_stack_main、主播放器,记为player_main、主解码器,记为decoder_main。When the display device plays the positive film, the various resources occupied include the main media playback background, which is denoted as media_backend_main, the main protocol resolution end, denoted as protocol_stack_main, the main player, denoted as player_main, the main decoder, and denoted as decoder_main.
此时,服务器指示广告1在之后一段时间的Tad1时刻插播,则显示设备开始创建并初始化相关的各类资源,包括:广告1媒体播放后台,记为media_backend_ad1、广告1协议解析端,记为protocol_stack_ad1、广告1播放器,记为player_ad1,以及辅解码器,记为decoder_sub,除decoder_sub之外,其他资源均为控制器创建的逻辑模块,上述各个逻辑模块均可创建多个实例,互不影响。At this time, the server instructs Advertisement 1 to be inserted at Tad1 at a later time, and the display device starts to create and initialize various related resources, including: Advertisement 1 media playback background, denoted as media_backend_ad1, Advertisement 1 protocol parsing end, denoted as protocol_stack_ad1 1. Advertisement 1 player, denoted as player_ad1, and auxiliary decoder, denoted as decoder_sub. Except for decoder_sub, all other resources are logic modules created by the controller. Each of the above logic modules can create multiple instances without affecting each other.
显示设备将广告1的统一资源定位符(Uniform Resource Locator,URL) 通过media_backend_ad1传递给protocol_stack_ad1,protocol_stack_ad1下载指定URL下的媒体呈现内容描述(Media Presentation Description,MPD)文件并解析,得到广告1的初始信息,包括Codec、时长、码率、帧率、初始分辨率、音频采样率等,然后传递给player_ad1并初始化decoder_sub,随后下载广告1的视频数据注入给decoder_sub进行解码,并缓存解码后的视频数据。The display device passes the Uniform Resource Locator (URL) of Advertisement 1 to media_backend_ad1 to protocol_stack_ad1, and protocol_stack_ad1 downloads and parses the media presentation content description (MPD) file at the specified URL to obtain the initial information of Advertisement 1 , Including Codec, duration, code rate, frame rate, initial resolution, audio sampling rate, etc., then pass to player_ad1 and initialize decoder_sub, then download the video data of advertisement 1 to decoder_sub for decoding, and cache the decoded video data.
当正片播放至Tad1时刻,显示设备下发pause命令,暂停Main路解码器上所有逻辑模块的运行,包含:停止player_main的播放,同时将视频输出设备(Vout)和声音输出设备(Aout)切换至广告1,以及下发start命令,开始player_ad1的播放,由于广告1的视频数据已经提前下载解码,所以用户可以观看到的现象为:正片在Tad1时刻无缝切换到广告1。When the positive film is played to Tad1, the display device issues a pause command to suspend the operation of all logic modules on the Main decoder, including: stopping player_main playback, and switching the video output device (Vout) and sound output device (Aout) to Advertisement 1, and the start command is issued to start the playback of player_ad1. Since the video data of Advertisement 1 has been downloaded and decoded in advance, the phenomenon that the user can watch is as follows: the positive film seamlessly switches to Advertisement 1 at Tad1 moment.
而此时,显示设备会主动释放正片相关的各类资源,包括media_backend_main、protocol_stack_main、player_main和decoder_main。At this time, the display device will actively release various resources related to the positive film, including media_backend_main, protocol_stack_main, player_main, and decoder_main.
显示设备通过player_ad1播放广告1一段时间后,服务器指示广告2将在广告1播放结束后插播。此时,显示设备创建并初始化相关的各类资源,包括:广告2媒体播放后台,记为media_backend_ad2、广告2协议解析端,记为protocol_stack_ad2、广告2播放器,记为player_ad2、以及主解码器,记灵decoder_main,除decoder_main之外,其他资源均为控制器创建的逻辑模块,在此不再赘述。After the display device plays advertisement 1 through player_ad1 for a period of time, the server indicates that advertisement 2 will be inserted after advertisement 1 is played. At this time, the display device creates and initializes various related resources, including: advertising 2 media playback background, denoted as media_backend_ad2, advertising 2 protocol parsing end, denoted as protocol_stack_ad2, advertising 2 player, denoted as player_ad2, and main decoder, Recording decoder_main, except for decoder_main, all other resources are logic modules created by the controller, which will not be repeated here.
与广告1的处理过程相同,在广告1播放过程中,显示设备预下载、解码、缓存广告2的视频数据,并在广告1播放结束后,立即切换至player_ad2,开始播放广告2,以及释放广告1相关的各类资源,包括media_backend_ad1、 protocol_stack_ad1、player_ad1和decoder_sub。The process is the same as that of Advertisement 1. In the process of Advertisement 1, the display device pre-downloads, decodes, and caches the video data of Advertisement 2, and immediately switches to player_ad2 after Advertisement 1 finishes, starts to play Advertisement 2, and releases the advertisement. 1 All kinds of related resources, including media_backend_ad1, protocol_stack_ad1, player_ad1 and decoder_sub.
显示设备通过player_ad2播放广告2一段时间后,服务器指示正片将在广告2播放结束后插播。此时,显示设备创建并初始化相关的各类资源,包括新媒体播放后台,记为media_backend_main_resume、新协议解析端,记为protocol_stack_main_resume、新播放器,记为player_main_resume、辅解码器,记为decoder_sub。After the display device plays Advertisement 2 through player_ad2 for a period of time, the server indicates that the main film will be inserted after the advertisement 2 is played. At this time, the display device creates and initializes various types of related resources, including the new media playback background, denoted as media_backend_main_resume, the new protocol parser, denoted as protocol_stack_main_resume, the new player, denoted as player_main_resume, auxiliary decoder, and denoted as decoder_sub.
显示设备通过media_backend_main_resume将正片的URL传递给protocol_stack_main_resume,protocol_stack_main_resume下载URL下对应的MPD文件,并解析出视频数据的最大播放分辨率,如果视频数据的最大播放分辨率小于等于decoder_sub支持的最大播放分辨率(1080p),则decoder_sub就能支持正片的播放,后续过程不需要进行decoder_sub和decoder_main之间的切换;反之,如果视频数据的最大播放分辨率大于decoder_sub支持的最大播放分辨率(1080p),则需要使用本申请实施例提出的技术方案,将在以下步骤中介绍。The display device passes the URL of the positive film to protocol_stack_main_resume through media_backend_main_resume, protocol_stack_main_resume downloads the corresponding MPD file under the URL, and parses out the maximum playback resolution of the video data. If the maximum playback resolution of the video data is less than or equal to the maximum playback resolution supported by the decoder_sub ( 1080p), decoder_sub can support the playback of positive films, and the subsequent process does not need to switch between decoder_sub and decoder_main; conversely, if the maximum playback resolution of video data is greater than the maximum playback resolution (1080p) supported by decoder_sub, you need to use The technical solutions proposed in the embodiments of the present application will be introduced in the following steps.
显示设备与服务器协商,通过判断当前时刻网速、带宽、缓存数据量等因素,结合流媒体协议的算法,得出需要下载的正片的视频数据的播放分辨率小于等于1080p,进一步地,还可以确认视频数据的码率、帧率等信息。The display device negotiates with the server, by judging factors such as network speed, bandwidth, and amount of cached data at the current time, combined with the algorithm of the streaming media protocol, it is concluded that the playback resolution of the video data of the positive film to be downloaded is less than or equal to 1080p. Further, it can also Confirm the information such as the code rate and frame rate of the video data.
显示设备从服务器下载Tad1时刻之后的视频数据,这部分视频数据便是起播部分数据,其播放分辨率小于等于1080p,显示设备将起播部分数据分配至decoder_sub进行解码并缓存,只要后续不接收到播放分辨率在1080p以上的视频数据,则后续接收到的视频数据一直使用decoder_sub进行解码。The display device downloads the video data after the time of Tad1 from the server. This part of the video data is the starting part of the data, and its playback resolution is less than or equal to 1080p. The display device allocates the starting part of the data to decoder_sub for decoding and buffering, as long as it is not subsequently received When the video data with a resolution above 1080p is played, the video data received subsequently is always decoded using decoder_sub.
广告2播放结束后,显示设备将视频输出设备和声音输出设备切换至正片, 由于,此时系统状态(网速等)不支持播放大于1080p的视频数据,因此,先使用decoder_sub解码,即播放起播部分数据。After the advertisement 2 is played, the display device switches the video output device and the sound output device to the positive film. Since the system state (network speed, etc.) does not support the playback of video data greater than 1080p at this time, use decoder_sub to decode first. Broadcast some data.
显示设备与服务器协商,通过判断当前时刻网速、带宽、缓存数据量等因素,结合流媒体协议的算法,得出需要继续下载的正片的视频数据的播放分辨率大于1080p,进一步地,还可以确认视频数据的码率、帧率等信息。The display device negotiates with the server. By judging factors such as network speed, bandwidth, and the amount of cached data at the current time, combined with the algorithm of the streaming media protocol, it is concluded that the playback resolution of the video data of the positive film that needs to be downloaded is greater than 1080p. Further, it can also Confirm the information such as the code rate and frame rate of the video data.
那么,显示设备会记录下继续下载的正片的起始时刻,即显示时间戳(Presentation Time Stamp,PTS)位置,用于度量解码后的视频数据什么时候被显示出来,记为T2K4K,创建并初始化新的正片相关的各类资源,包括,新新媒体播放后台,记为media_backend_main_resume_2K4K、新新协议解析端,记为protocol_stack_main_resume_2K4K、新新播放器,记为player_main_resume_2K4K、主解码器,decoder_main。Then, the display device will record the start time of the positive film that continues to download, that is, the position of the display timestamp (Presentation Time Stamp), used to measure when the decoded video data is displayed, recorded as T2K4K, created and initialized All kinds of resources related to the new positive film, including the new new media playback background, referred to as media_backend_main_resume_2K4K, the new and new protocol parser, referred to as protocol_stack_main_resume_2K4K, the new new player, referred to as player_main_resume_2K4K, the main decoder, decoder_main.
显示设备会从T2K4K时刻之后,继续下载正片的视频数据,这部分视频数据便是续播部分数据,显示设备会将续播视频数据分配至decoder_main进行解码并缓存。The display device will continue to download the video data of the positive film after the T2K4K moment. This part of the video data is the resume data. The display device will distribute the resume video data to the decoder_main for decoding and caching.
当显示设备通过player_main_resume将正片播放至T2K4K时刻时,会暂停player_main_resume的播放,切换至player_main_resume_2K4K,此时,续播部分数据已经在decoder_main下载、解析、解码完毕,则可以直接通过player_main_resume_2K4K开始播放,如果之后的视频数据的播放分辨率都在1080p以上,则显示设备会将之后下载的视频数据全部分配至decoder_main进行解码并缓存,并且不再切换decoder。When the display device plays the main film to the time of T2K4K through player_main_resume, it will pause the playback of player_main_resume and switch to player_main_resume_2K4K. At this time, the resumed part of the data has been downloaded, parsed, and decoded in decoder_main. The playback resolution of all the video data is above 1080p, then the display device will distribute all the downloaded video data to decoder_main to decode and cache, and no longer switch the decoder.
本申请实施例中,显示设备按照低播放分辨率和高播放分辨率从服务器分别下载主视频的起播部分数据和续播部分数据,在Main路解码器被多媒体展示 信息占用时,先通过Sub路解码器播放起播部分数据,等多媒体展示信息播放完毕后,再切换至Main路解码器播放续播部分数据,这样,可以实现主视频和多媒体展示信息的无缝切换,在尽可以节省内存的前提下,最大概率地消除了在主视频中插播偶数条多媒体展示信息时,双路解码器无法正常切换回高播放分辨率的主视频的缺陷,有效提高了流媒体视频切换的时效性,保障了系统的流媒体视频播放性能,提高了用户体验。In the embodiment of the present application, the display device downloads the starting part data and the resuming part data of the main video from the server respectively according to the low playback resolution and the high playback resolution. When the Main decoder is occupied by the multimedia display information, it first passes the Sub The channel decoder plays the starting part of the data, and after the multimedia display information is played, it switches to the Main channel decoder to play the continued part of the data, so that the main video and multimedia display information can be seamlessly switched to save memory as much as possible. Under the premise of, the defect of the dual-channel decoder cannot be switched back to the main video with high playback resolution normally when inserting even-numbered multimedia display information in the main video, which effectively improves the timeliness of streaming video switching. The system guarantees the streaming video playback performance of the system and improves the user experience.
基于上述实施例,参阅图2所示,一种显示设备,至少包括控制器250、Main路解码器271和Sub路解码器272,其中,Based on the above embodiment, referring to FIG. 2, a display device includes at least a controller 250, a Main decoder 271, and a Sub decoder 272, where,
控制器250,用于执行以下操作:The controller 250 is used to perform the following operations:
通过Main路解码器271播放多媒体展示信息的过程中,从服务器接收主视频对应的起播部分数据,并分配至Sub路解码器272进行缓存,所述起播部分数据的播放分辨率不高于所述Sub路解码器272支持的最高播放分辨率;In the process of playing the multimedia presentation information through the Main decoder 271, the starting part data corresponding to the main video is received from the server and distributed to the Sub decoder 272 for buffering. The playback resolution of the starting part data is not higher than The highest playback resolution supported by the Sub decoder 272;
在所述多媒体展示信息播放完毕后,由所述Main路解码器271切换至所述Sub路解码器272,播放所述Sub路解码器272中缓存的所述起播部分数据;After the multimedia presentation information is played, the Main decoder 271 is switched to the Sub decoder 272, and the start-up part of the data buffered in the Sub decoder 272 is played;
从服务器接收主视频对应的续播部分数据,并分配至所述Main路解码器271进行缓存,所述续播部分数据的播放分辨率高于所述Sub路解码器272支持的最高播放分辨率;Receive the resume part data corresponding to the main video from the server and distribute it to the Main decoder 271 for buffering. The playback resolution of the resume part data is higher than the highest playback resolution supported by the Sub decoder 272 ;
在所述起播部分数据播放完毕后,从所述Sub路解码器272切换至所述Main路解码器271,播放所述Main路解码器271中缓存的所述续播部分数据。After the playback of the start-up part data is completed, the Sub-channel decoder 272 is switched to the Main-channel decoder 271, and the continued-play-part data buffered in the Main-channel decoder 271 is played.
在本申请某些实施例中,从服务器接收主视频对应的起播部分数据之前, 控制器250进一步用于:In some embodiments of the present application, before receiving the start part data corresponding to the main video from the server, the controller 250 is further used to:
确定当前的第一网络综合状态,并请求服务器下发播放分辨率符合所述第一网络综合状态的主视频;Determine the current comprehensive state of the first network, and request the server to deliver a master video with a resolution that meets the comprehensive state of the first network;
从服务器接收主视频对应的续播部分数据之前,控制器250进一步用于:Before receiving the resume part data corresponding to the main video from the server, the controller 250 is further used to:
确定当前的第二网络综合状态,并请求服务器下发播放分辨率符合所述第二网络综合状态的主视频。Determine the current integrated state of the second network, and request the server to deliver a master video with a resolution that meets the integrated state of the second network.
在本申请某些实施例中,将所述起播部分数据分配至所述Sub路解码器272进行缓存时,控制器250用于:In some embodiments of the present application, when allocating the start-up part of the data to the Sub decoder 272 for buffering, the controller 250 is used to:
将所述起播部分数据发送至所述Sub路解码器272;Send the starting part of the data to the Sub decoder 272;
通过所述Sub路解码器272对所述起播部分数据进行解析及解码操作;Analyze and decode the starting part of the data through the Sub decoder 272;
显示设备将所述续播部分数据分配至所述Main路解码器271进行缓存,包括:The display device distributes part of the resumed data to the Main decoder 271 for buffering, including:
将所述续播部分数据发送至所述Main路解码器271;Sending the resumed part of the data to the Main decoder 271;
通过所述Main路解码器271对所述续播部分数据进行解析及解码操作。The Main decoder 271 analyzes and decodes the resumed part of the data.
在本申请某些实施例中,从服务器接收主视频对应的续播部分数据,并分配至所述Main路解码器271进行缓存时,所述控制器250用于:In some embodiments of the present application, when receiving the resume data corresponding to the main video from the server and distributing it to the Main decoder 271 for buffering, the controller 250 is used to:
若在通过Main路解码器271播放多媒体展示信息的过程中,接收到所述续播部分数据,则等待多媒体展示信息播放完毕后,释放Main路解码器271上的各类资源,对Main路解码器271进行初始化,再将接收到的续播部分数据分配至Main路解码器271进行缓存;If during the playback of the multimedia presentation information through the Main decoder 271, part of the resumed data is received, after waiting for the multimedia presentation information to finish playing, various resources on the Main decoder 271 are released to decode the Main presentation The device 271 initializes, and then distributes part of the received resume data to the Main decoder 271 for buffering;
若在通过Sub路解码器272播放起播部分数据的过程中,接收到所述续播部分数据,则直接将接收到的续播部分数据分配至Main路解码器271进行缓存。If the resume part data is received during the playback of the starting part data through the Sub decoder 272, the received resume part data is directly distributed to the Main decoder 271 for buffering.
在本申请某些实施例中,从服务器接收主视频对应的续播部分数据,并分配至所述Main路解码器271进行缓存时,所述控制器250用于:In some embodiments of the present application, when receiving the resume data corresponding to the main video from the server and distributing it to the Main decoder 271 for buffering, the controller 250 is used to:
记录续播部分数据中第一个片段的起始时刻;Record the start time of the first segment in the resumed part of the data;
在所述起播部分数据播放完毕后,从所述Sub路解码器272切换至所述Main路解码器271,播放所述Main路解码器271中缓存的所述续播部分数据时,所述控制器250用于:After the playback of the starting part of the data is completed, the Sub-channel decoder 272 is switched to the Main-channel decoder 271, and when the continued-play part of the data buffered in the Main-channel decoder 271 is played, the The controller 250 is used to:
通过Sub路解码器播放所述Sub路解码器中缓存的所述起播部分数据的过程中,在到达所述起始时刻时,确定所述起播部分数据播放完毕,从所述Sub路解码器切换至所述Main路解码器,继续播放所述Main路解码器中缓存的所述续播部分数据。During the playback of the start-up part data buffered in the Sub-channel decoder through the Sub-channel decoder, when the starting time is reached, it is determined that the playback of the start-up part of the data is completed, and decoding is performed from the Sub-channel The device switches to the Main decoder, and continues to play the part of the resumed data buffered in the Main decoder.
基于同一发明构思,本申请实施例提供一种视频切换的存储介质,存储有用于实现流媒体视频切换的控制方法的程序,所述程序被控制器运行时,执行以下步骤:Based on the same inventive concept, embodiments of the present application provide a storage medium for video switching, which stores a program for implementing a control method for streaming media video switching. When the program is run by a controller, the following steps are performed:
通过Main路解码器播放多媒体展示信息的过程中,从服务器接收主视频对应的起播部分数据,并分配至Sub路解码器进行缓存,所述起播部分数据的播放分辨率不高于所述Sub路解码器支持的最高播放分辨率;During the process of playing the multimedia presentation information through the Main decoder, the starting part data corresponding to the main video is received from the server and distributed to the Sub decoder for buffering. The playback resolution of the starting part data is not higher than the The highest playback resolution supported by Sub decoder;
在所述多媒体展示信息播放完毕后,由所述Main路解码器切换至所After the multimedia display information is played, the Main decoder decodes to all
述Sub路解码器,播放所述Sub路解码器中缓存的所述起播部分数据;The Sub-channel decoder, playing the starting part of the data buffered in the Sub-channel decoder;
显示设备从服务器接收主视频对应的续播部分数据,并分配至所述Main路解码器进行缓存,所述续播部分数据的播放分辨率高于所述Sub路解码器支持的最高播放分辨率;The display device receives the resume data corresponding to the main video from the server and distributes it to the Main decoder for buffering. The playback resolution of the resume data is higher than the highest playback resolution supported by the Sub decoder ;
在所述起播部分数据播放完毕后,从所述Sub路解码器切换至所述Main 路解码器,播放所述Main路解码器中缓存的所述续播部分数据。After the playback of the start-up part data is completed, the Sub-channel decoder is switched to the Main-channel decoder, and the continued-play part of the data buffered in the Main-channel decoder is played.
本申请实施例中,显示设备按照低播放分辨率和高播放分辨率从服务In the embodiment of the present application, the display device selects the service according to the low playback resolution and the high playback resolution.
器分别下载主视频的起播部分数据和续播部分数据,在Main路解码器被多媒体展示信息占用时,先通过Sub路解码器播放起播部分数据,等多媒体展示信息播放完毕后,再切换至Main路解码器播放续播部分数据,这样,可以实现主视频和多媒体展示信息的无缝切换,在尽可以节省内存的前提下,最大概率地消除了在主视频中插播偶数条多媒体展示信息时,双路main+sub解码器无法正常切换回高播放分辨率的主视频的缺陷,有效提高了流媒体视频切换的时效性,保障了系统的流媒体视频播放性能,提高了用户体验。The device downloads the starting part data and the resuming part data of the main video separately. When the Main channel decoder is occupied by the multimedia display information, the starting part data is played through the Sub channel decoder first, and then switched after the multimedia display information is completed. To the Main decoder to play the continued broadcast part of the data, so that the main video and multimedia display information can be seamlessly switched, under the premise of saving memory, the maximum probability of eliminating even number of multimedia display information inserted in the main video At this time, the defect that the dual main + sub decoder cannot switch back to the main video with high playback resolution can effectively improve the timeliness of the streaming video switching, ensure the streaming video playback performance of the system, and improve the user experience.
本领域内的技术人员应明白,本申请的实施例可提供为方法、系统、或计算机程序产品。因此,本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的控制器以产生一个机器,使得通过计算机或其他可编程数据处理设备的控制器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。This application is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowchart and / or block diagram and a combination of the flow and / or block in the flowchart and / or block diagram may be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, special-purpose computer, embedded processing machine, or controller of other programmable data processing equipment to generate a machine that enables the generation of instructions executed by the computer or other programmable data processing equipment's controller An apparatus for realizing the functions specified in one block or multiple blocks of one flow or multiple flows of a flowchart and / or one block or multiple blocks of a block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备 以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, the instructions The device implements the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processing, which is executed on the computer or other programmable device The instructions provide steps for implementing the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.
尽管已描述了本申请的优选实施例,但本领域内的技术人员一旦得知了基本创造性概念,则可对这些实施例作出另外的变更和修改。所以,所附权利要求意欲解释为包括优选实施例以及落入本申请范围的所有变更和修改。Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
显然,本领域的技术人员可以对本申请实施例进行各种改动和变型而不脱离本申请实施例的精神和范围。这样,倘若本申请实施例的这些修改和变型属于本申请权利要求及其等同技术的范围之内,则本申请也意图包含这些改动和变型在内。Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. In this way, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims (11)

  1. 一种显示设备,其特征在于,包括:A display device is characterized by comprising:
    显示器,用于显示视频播放画面;Display, used to display video playback screen;
    第一解码器和第二解码器,用于对视频进行至少包括解码操作的处理;The first decoder and the second decoder are used for processing the video including at least the decoding operation;
    以及与所述显示器、所述第一解码器和所述第二解码器通信的控制器,用于执行:And a controller in communication with the display, the first decoder, and the second decoder, for executing:
    通过第一解码器播放第一类视频的过程中,从服务器接收第二类视频对应的起播部分数据,并分配至第二解码器进行缓存,所述起播部分数据的播放分辨率不高于所述第二解码器支持的最高播放分辨率;In the process of playing the first type of video through the first decoder, the starting part of the data corresponding to the second type of video is received from the server and distributed to the second decoder for buffering. The playback resolution of the starting part of the data is not high The highest playback resolution supported by the second decoder;
    在所述第一类视频播放完毕后,控制所述第一解码器切换至所述第二解码器,播放所述第二解码器中缓存的所述起播部分数据;After the first type of video is finished playing, controlling the first decoder to switch to the second decoder to play the start-up part of the data buffered in the second decoder;
    从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存,所述续播部分数据的播放分辨率高于所述第二解码器支持的最高播放分辨率;Receiving the resume part data corresponding to the second type of video from the server and distributing it to the first decoder for buffering, the playback resolution of the resume part data is higher than the highest playback resolution supported by the second decoder ;
    在所述起播部分数据播放完毕后,控制所述第二解码器切换至所述第一解码器,播放所述第一解码器中缓存的所述续播部分数据。After the playback of the starting part of the data is completed, the second decoder is controlled to switch to the first decoder to play the continued part of the data buffered in the first decoder.
  2. 如权利要求1所述的显示设备,其特征在于,从服务器接收第二类视频对应的起播部分数据之前,所述控制器进一步用于:The display device according to claim 1, wherein before receiving the start part data corresponding to the second type of video from the server, the controller is further configured to:
    确定当前的第一网络综合状态,并请求服务器下发播放分辨率符合所述第一网络综合状态的第二类视频;Determine the current comprehensive status of the first network, and request the server to deliver a second type of video whose resolution meets the comprehensive status of the first network;
    从服务器接收第二类视频对应的续播部分数据之前,所述控制器进一步用于:Before receiving the resume data corresponding to the second type of video from the server, the controller is further used to:
    确定当前的第二网络综合状态,并请求服务器下发播放分辨率符合所述第二网络综合状态的第二类视频。Determine the current comprehensive state of the second network, and request the server to deliver and play the second type of video whose resolution meets the comprehensive state of the second network.
  3. 如权利要求1所述的显示设备,其特征在于,将所述起播部分数据分配至所述第二解码器进行缓存时,所述控制器用于:The display device according to claim 1, wherein the controller is configured to: when allocating the start-up portion of data to the second decoder for buffering:
    将所述起播部分数据发送至所述第二解码器;Sending the starting part of the data to the second decoder;
    通过所述第二解码器对所述起播部分数据进行解析及解码操作;Analyzing and decoding the start-up part data through the second decoder;
    将所述续播部分数据分配至所述第一解码器进行缓存时,所述控制器用于:When allocating part of the resumed data to the first decoder for buffering, the controller is used to:
    将所述续播部分数据发送至所述第一解码器;Sending the resumed part of the data to the first decoder;
    通过所述第一解码器对所述续播部分数据进行解析及解码操作。The first decoder performs parsing and decoding operations on the resumed part of the data.
  4. 如权利要求1、2或3所述的显示设备,其特征在于,从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存时,所述控制器用于:The display device according to claim 1, 2 or 3, wherein the controller is used to receive the resume data corresponding to the second type of video from the server and distribute it to the first decoder for buffering :
    若在通过第一解码器播放第一类视频的过程中,接收到所述续播部分数据,则等待第一类视频播放完毕后,释放第一解码器上的各类资源,对第一解码器进行初始化,再将接收到的续播部分数据分配至第一解码器进行缓存;If, during the playback of the first type of video through the first decoder, the partial resume data is received, after waiting for the first type of video to finish playing, the various resources on the first decoder are released to decode the first video The device initializes, and then distributes the part of the received resume data to the first decoder for buffering;
    若在通过第二解码器播放起播部分数据的过程中,接收到所述续播部分数据,则直接将接收到的续播部分数据分配至第一解码器进行缓存。If the resume part data is received during the playback of the start-up part data through the second decoder, the received resume part data is directly allocated to the first decoder for buffering.
  5. 如权利要求4所述的显示设备,其特征在于,从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存时,所述控制器用于:The display device according to claim 4, wherein when receiving the resume part data corresponding to the second type of video from the server and distributing it to the first decoder for buffering, the controller is used to:
    记录续播部分数据中第一个片段的起始时刻;Record the start time of the first segment in the resumed part of the data;
    在所述起播部分数据播放完毕后,控制所述第二解码器切换至所述第一解码器,播放所述第一解码器中缓存的所述续播部分数据时,所述控制器用于:After the playback of the start-up part data is completed, the second decoder is controlled to switch to the first decoder, and when playing the resumed part data buffered in the first decoder, the controller is used :
    通过第二解码器播放所述第二解码器中缓存的所述起播部分数据的过程中,在到达所述起始时刻时,确定所述起播部分数据播放完毕,从所述第二解码器切换至所述第一解码器,继续播放所述第一解码器中缓存的所述续播部分数据。In the process of playing the start-up part data buffered in the second decoder through the second decoder, when the starting time is reached, it is determined that the start-up part data is finished playing, and the second decoding The decoder switches to the first decoder, and continues to play the resumed part of the data buffered in the first decoder.
  6. 一种显示设备中视频切换的控制方法,其特征在于,包括:A control method for video switching in a display device is characterized in that it includes:
    通过第一解码器播放第一类视频的过程中,从服务器接收第二类视频对应的起播部分数据,并分配至第二解码器进行缓存,所述起播部分数据的播放分辨率不高于所述第二解码器支持的最高播放分辨率;In the process of playing the first type of video through the first decoder, the starting part of the data corresponding to the second type of video is received from the server and distributed to the second decoder for buffering. The playback resolution of the starting part of the data is not high The highest playback resolution supported by the second decoder;
    在所述第一类视频播放完毕后,控制所述第一解码器切换至所述第二解码器,播放所述第二解码器中缓存的所述起播部分数据;After the first type of video is finished playing, controlling the first decoder to switch to the second decoder to play the start-up part of the data buffered in the second decoder;
    从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存,所述续播部分数据的播放分辨率高于所述第二解码器支持的最高播放分辨率;Receiving the resume part data corresponding to the second type of video from the server and distributing it to the first decoder for buffering, the playback resolution of the resume part data is higher than the highest playback resolution supported by the second decoder ;
    在所述起播部分数据播放完毕后,控制所述第二解码器切换至所述第一解码器,播放所述第一解码器中缓存的所述续播部分数据。After the playback of the starting part of the data is completed, the second decoder is controlled to switch to the first decoder to play the continued part of the data buffered in the first decoder.
  7. 如权利要求6所述的方法,其特征在于,从服务器接收第二类视频对应的起播部分数据之前,进一步包括:The method according to claim 6, wherein before receiving the start part data corresponding to the second type of video from the server, the method further comprises:
    确定当前的第一网络综合状态,请求服务器下发播放分辨率符合所述第一网络综合状态的第二类视频;Determining the current comprehensive state of the first network, and requesting the server to deliver a second type of video with a resolution consistent with the comprehensive state of the first network;
    从服务器接收第二类视频对应的续播部分数据之前,进一步包括:Before receiving the resume data corresponding to the second type of video from the server, the method further includes:
    确定当前的第二网络综合状态,并请求服务器下发播放分辨率符合所述第二网络综合状态的第二类视频。Determine the current comprehensive state of the second network, and request the server to deliver and play the second type of video whose resolution meets the comprehensive state of the second network.
  8. 如权利要求6所述的方法,其特征在于,将所述起播部分数据分配至所 述第二解码器进行缓存,包括:The method according to claim 6, wherein allocating the start-up part data to the second decoder for buffering includes:
    将所述起播部分数据发送至所述第二解码器;Sending the starting part of the data to the second decoder;
    通过所述第二解码器对所述起播部分数据进行解析及解码操作;Analyzing and decoding the start-up part data through the second decoder;
    将所述续播部分数据分配至所述第一解码器进行缓存,包括:Distributing part of the resumed data to the first decoder for buffering includes:
    将所述续播部分数据发送至所述第一解码器;Sending the resumed part of the data to the first decoder;
    通过所述第一解码器对所述续播部分数据进行解析及解码操作。The first decoder performs parsing and decoding operations on the resumed part of the data.
  9. 如权利要求6、7或8所述的方法,其特征在于,从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存,包括:The method according to claim 6, 7 or 8, wherein receiving data of the resume part corresponding to the second type of video from the server and distributing it to the first decoder for buffering includes:
    若在通过第一解码器播放第一类视频的过程中,接收到所述续播部分数据,则等待第一类视频播放完毕后,释放第一解码器上的各类资源,对第一进行初始化,再将接收到的续播部分数据分配至第一解码器进行缓存;If in the process of playing the first type of video through the first decoder, part of the resume data is received, after waiting for the first type of video to finish playing, the various resources on the first decoder are released and the first Initialization, and then distribute the received part of the resume data to the first decoder for buffering;
    若在通过第二解码器播放起播部分数据的过程中,接收到所述续播部分数据,则直接将接收到的续播部分数据分配至第一解码器进行缓存。If the resume part data is received during the playback of the start-up part data through the second decoder, the received resume part data is directly allocated to the first decoder for buffering.
  10. 如权利要求9所述的方法,其特征在于,从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存,进一步包括:The method according to claim 9, wherein receiving the resume data corresponding to the second type of video from the server and distributing it to the first decoder for buffering further comprises:
    记录续播部分数据中第一个片段的起始时刻;Record the start time of the first segment in the resumed part of the data;
    在所述起播部分数据播放完毕后,控制所述第二解码器切换至所述第一解码器,播放所述第一解码器中缓存的所述续播部分数据,包括:After the playback of the starting part of the data is completed, controlling the second decoder to switch to the first decoder to play the continued part of the data buffered in the first decoder includes:
    通过第二解码器播放所述第二解码器中缓存的所述起播部分数据的过程中,在到达所述起始时刻时,确定所述起播部分数据播放完毕,并从所述第二解码器切换至所述第一解码器,继续播放所述第一解码器中缓存的所述续播部分数据。In the process of playing the start-up part data buffered in the second decoder through the second decoder, when the starting time is reached, it is determined that the start-up part data is finished playing, and the second The decoder switches to the first decoder, and continues to play the resumed part of the data buffered in the first decoder.
  11. 一种存储介质,其特征在于,存储有用于实现视频切换的控制方法的程序,所述程序被控制器运行时,执行以下步骤:A storage medium is characterized by storing a program for implementing a control method for video switching. When the program is run by a controller, the following steps are performed:
    通过第一解码器播放第一类视频的过程中,从服务器接收第二类视频对应的起播部分数据,并分配至第二解码器进行缓存,所述起播部分数据的播放分辨率不高于所述第二解码器支持的最高播放分辨率;In the process of playing the first type of video through the first decoder, the starting part of the data corresponding to the second type of video is received from the server and distributed to the second decoder for buffering. The playback resolution of the starting part of the data is not high The highest playback resolution supported by the second decoder;
    在所述第一类视频播放完毕后,控制所述第一解码器切换至所述第二解码器,播放所述第二解码器中缓存的所述起播部分数据;After the first type of video is finished playing, controlling the first decoder to switch to the second decoder to play the start-up part of the data buffered in the second decoder;
    从服务器接收第二类视频对应的续播部分数据,并分配至所述第一解码器进行缓存,所述续播部分数据的播放分辨率高于所述第二解码器支持的最高播放分辨率;Receiving the resume part data corresponding to the second type of video from the server and distributing it to the first decoder for buffering, the playback resolution of the resume part data is higher than the highest playback resolution supported by the second decoder ;
    在所述起播部分数据播放完毕后,控制所述第二解码器切换至所述第一解码器,播放所述第一解码器中缓存的所述续播部分数据。After the playback of the starting part of the data is completed, the second decoder is controlled to switch to the first decoder to play the continued part of the data buffered in the first decoder.
PCT/CN2019/114628 2018-11-12 2019-10-31 Video switching control method and display device WO2020098504A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811338363.0 2018-11-12
CN201811338363.0A CN109218763A (en) 2018-11-12 2018-11-12 A kind of method and smart television of streaming media video switching

Publications (1)

Publication Number Publication Date
WO2020098504A1 true WO2020098504A1 (en) 2020-05-22

Family

ID=64995448

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/114628 WO2020098504A1 (en) 2018-11-12 2019-10-31 Video switching control method and display device

Country Status (2)

Country Link
CN (1) CN109218763A (en)
WO (1) WO2020098504A1 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109218763A (en) * 2018-11-12 2019-01-15 青岛海信传媒网络技术有限公司 A kind of method and smart television of streaming media video switching
CN111479164A (en) * 2019-01-23 2020-07-31 上海哔哩哔哩科技有限公司 Hardware decoding dynamic resolution seamless switching method and device and storage medium
CN109754771B (en) * 2019-03-08 2020-09-01 南昌黑鲨科技有限公司 Image display system and display method
CN110677689A (en) * 2019-09-29 2020-01-10 杭州当虹科技股份有限公司 VR video advertisement seamless insertion method based on user view angle
CN111372127B (en) * 2020-03-10 2022-08-30 深圳市腾讯信息技术有限公司 Data transmission method and related device
CN113453046B (en) * 2020-03-24 2022-07-12 腾讯科技(深圳)有限公司 Immersive media providing method, immersive media obtaining device, immersive media equipment and storage medium
CN114071187B (en) * 2020-07-31 2024-02-27 海信视像科技股份有限公司 Display device, server and resolution fast switching method

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1518832A (en) * 2001-04-24 2004-08-04 ��ķɭ��ɹ�˾ System and data format for providing seamless stream switching in digital video decoder
CN103222276A (en) * 2010-09-20 2013-07-24 数码士有限公司 Processing method to be implemented upon the occurrence of an expression switch in HTTP streaming
CN104010232A (en) * 2014-05-23 2014-08-27 惠州Tcl移动通信有限公司 Method and system for intelligently playing online videos, player and mobile terminal
CN104602032A (en) * 2015-01-13 2015-05-06 乐视致新电子科技(天津)有限公司 Video playing processing method, playing application proxy device and playing server
CN105100644A (en) * 2015-07-15 2015-11-25 西安诺瓦电子科技有限公司 Seamless switching method for video source
CN105120184A (en) * 2015-10-09 2015-12-02 深圳市捷视飞通科技有限公司 High-definition video seamless matrix based on FPGA (field programmable gate array)
WO2017140685A1 (en) * 2016-02-16 2017-08-24 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Efficient adaptive streaming
CN107707976A (en) * 2017-09-29 2018-02-16 维沃移动通信有限公司 A kind of video encoding/decoding method and mobile terminal
CN108063904A (en) * 2018-01-18 2018-05-22 康佳集团股份有限公司 A kind of television set double-channel signal switching system, method and television set
CN108156487A (en) * 2016-12-02 2018-06-12 北京邦天信息技术有限公司 A kind of video broadcasting method and device
CN109218763A (en) * 2018-11-12 2019-01-15 青岛海信传媒网络技术有限公司 A kind of method and smart television of streaming media video switching

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020129374A1 (en) * 1991-11-25 2002-09-12 Michael J. Freeman Compressed digital-data seamless video switching system
CN102572421B (en) * 2010-12-31 2014-03-26 福建星网视易信息系统有限公司 Rapid double-line decoding switching method based on song preview application
CN103607634A (en) * 2013-11-19 2014-02-26 四川长虹电器股份有限公司 Method for reducing network video ordering terminal user waiting time
CN104768051B (en) * 2015-03-06 2017-12-15 深圳市九洲电器有限公司 The adaptive method for switching and system of odd encoder formatted program stream
CN105263034A (en) * 2015-10-13 2016-01-20 深圳Tcl数字技术有限公司 Spot video advertisement playing method and device
CN105721921B (en) * 2016-01-29 2019-07-12 四川长虹电器股份有限公司 A kind of adaptive selection method of multiwindow Video Decoder
CN106375837B (en) * 2016-11-14 2017-10-10 武汉斗鱼网络科技有限公司 A kind of advertisement cut-in method and device
CN107249144B (en) * 2017-05-12 2020-05-15 北京小鸟看看科技有限公司 Method for decoding multimedia file, multimedia player and playing equipment
CN107295403A (en) * 2017-08-24 2017-10-24 四川长虹电器股份有限公司 The method that seamless connection is played between video
CN107786890A (en) * 2017-10-30 2018-03-09 深圳Tcl数字技术有限公司 Video switching method, device and storage medium
CN108600800B (en) * 2018-04-28 2021-05-28 海信视像科技股份有限公司 Smart television and starting method thereof

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1518832A (en) * 2001-04-24 2004-08-04 ��ķɭ��ɹ�˾ System and data format for providing seamless stream switching in digital video decoder
CN103222276A (en) * 2010-09-20 2013-07-24 数码士有限公司 Processing method to be implemented upon the occurrence of an expression switch in HTTP streaming
CN104010232A (en) * 2014-05-23 2014-08-27 惠州Tcl移动通信有限公司 Method and system for intelligently playing online videos, player and mobile terminal
CN104602032A (en) * 2015-01-13 2015-05-06 乐视致新电子科技(天津)有限公司 Video playing processing method, playing application proxy device and playing server
CN105100644A (en) * 2015-07-15 2015-11-25 西安诺瓦电子科技有限公司 Seamless switching method for video source
CN105120184A (en) * 2015-10-09 2015-12-02 深圳市捷视飞通科技有限公司 High-definition video seamless matrix based on FPGA (field programmable gate array)
WO2017140685A1 (en) * 2016-02-16 2017-08-24 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Efficient adaptive streaming
CN108156487A (en) * 2016-12-02 2018-06-12 北京邦天信息技术有限公司 A kind of video broadcasting method and device
CN107707976A (en) * 2017-09-29 2018-02-16 维沃移动通信有限公司 A kind of video encoding/decoding method and mobile terminal
CN108063904A (en) * 2018-01-18 2018-05-22 康佳集团股份有限公司 A kind of television set double-channel signal switching system, method and television set
CN109218763A (en) * 2018-11-12 2019-01-15 青岛海信传媒网络技术有限公司 A kind of method and smart television of streaming media video switching

Also Published As

Publication number Publication date
CN109218763A (en) 2019-01-15

Similar Documents

Publication Publication Date Title
US11895360B2 (en) Method for waking up audio device, and display apparatus
WO2020098504A1 (en) Video switching control method and display device
CN110022495B (en) Method for pushing media file to display device by mobile terminal and display device
WO2021109491A1 (en) Method for displaying epg user interface, and display device
CN111601135B (en) Method for synchronously injecting audio and video elementary streams and display equipment
CN114073098B (en) Streaming media synchronization method and display device
CN104202677A (en) Multi-screen display and control method and multi-screen display and control device supportive to multi-window applications
WO2021109354A1 (en) Media stream data playback method and device
CN111601144B (en) Streaming media file playing method and display equipment
CN111601142B (en) Subtitle display method and display equipment
CN111836104B (en) Display apparatus and display method
CN111654743B (en) Audio playing method and display device
CN113473194B (en) Intelligent device and response method
CN111935510B (en) Double-browser application loading method and display equipment
CN111757181A (en) Method for reducing network media definition jitter and display equipment
CN113115088A (en) Control method of user interface of display equipment and display equipment
CN111885415B (en) Audio data rapid output method and display device
CN111343498B (en) Mute control method and device and smart television
CN111629250A (en) Display device and video playing method
CN113382291A (en) Display device and streaming media playing method
CN111246282B (en) Program information acquisition method in display equipment and display equipment
CN113407346B (en) Browser memory adjusting method and display device
CN111601158B (en) Method for optimizing audio track cutting of streaming media pipeline and display equipment
CN111683272A (en) Streaming media playing method and display equipment
CN117641024A (en) Display equipment and media data display method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19884347

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25.08.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19884347

Country of ref document: EP

Kind code of ref document: A1