CN117812387A

CN117812387A - Display equipment and streaming media playing method

Info

Publication number: CN117812387A
Application number: CN202310513022.7A
Authority: CN
Inventors: 聂继生; 庄尚志; 游灿虹; 蒋建群
Original assignee: Vidaa Netherlands International Holdings BV
Current assignee: Vidaa Netherlands International Holdings BV
Priority date: 2023-05-08
Filing date: 2023-05-08
Publication date: 2024-04-02

Abstract

The embodiment of the application discloses a display device and a streaming media playing method, wherein the method comprises the following steps: receiving stream media data sent by a stream media server; starting a streaming media player, and controlling the streaming media player to decode and play streaming media data; when receiving a decryption request sent by a streaming media server, acquiring a secret key; restarting the streaming media player if the currently received streaming media data is the dense stream data; controlling the restarted streaming media player to decrypt the streaming media data by using the key; and controlling the restarted streaming media player to decode the decrypted streaming media data and play the decoded streaming media data. The method aims at media resource playing scenes of firstly clearing and then sealing the stream, does not restart the media player when the clearing is not finished, so as to reduce system load, improve fluency of stream media playing, and restart the stream media player when the clearing is switched to sealing the stream, so that the influence of the flash black screen problem generated by restarting the stream media player on the viewing of the media resource by a user is eliminated.

Description

Display equipment and streaming media playing method

Technical Field

The present disclosure relates to the field of display devices, and in particular, to a display device and a streaming media playing method.

Background

The user can watch the media provided by application programs such as a browser, a video application and the like on the display device, and can watch the video signals received by devices such as a set top box, a television box and the like, and the audio and video data of the media can be transmitted in a streaming media mode.

In some application scenarios, the streaming server may send an clear stream to the display device before sending a dense stream, where the clear stream refers to unencrypted streaming data and the dense stream refers to encrypted streaming data. When the clear stream is not played, if the display device receives a decryption request sent by the streaming media server, the streaming media player needs to be restarted so as to switch the streaming media player from the clear stream player to the dense stream player. When the streaming media player is restarted by the display device, the media playing interface presents a flash screen, and as the streaming data is not completely played, the streaming data is played by the dense-stream player, so that the decryption resource waste is caused, the workload of a system chip is increased, even the media playing is blocked, and the watching experience of a user is influenced.

Disclosure of Invention

The embodiment of the application provides a display device and a streaming media playing method, aiming at a playing scene of firstly clearing and then sealing, the influence of a flash screen generated when a streaming media player is restarted on a media playing interface can be eliminated, an invalid decryption flow is avoided, and the system workload is reduced.

In a first aspect, embodiments of the present application provide a display device, including:

the communicator is used for being in communication connection with the streaming media server;

a controller for performing:

receiving stream media data sent by a stream media server through a communicator;

starting a streaming media player, and controlling the streaming media player to decode and play streaming media data;

when receiving a decryption request sent by a streaming media server, acquiring a secret key;

restarting the streaming media player if the currently received streaming media data is the dense stream data;

the restarted streaming media player is controlled to decrypt the streaming media data by using the secret key;

and controlling the restarted streaming media player to decode the decrypted streaming media data and play the decoded streaming media data.

In some embodiments, after obtaining the key, the controller is further to perform: and controlling the streaming media middleware to create a content decryption module matched with the decryption request, wherein the content decryption module is used for analyzing the dense stream data so as to separate playing configuration information from encrypted audio and video data.

In some embodiments, after restarting the streaming media player, the controller is further configured to perform: the secret key is issued to the restarted streaming media player; establishing a binding relation between the content decryption module and the restarted streaming media player; the binding relationship is used for indicating the content decryption module to transmit the play configuration information and the encrypted audio and video data to the restarted streaming media player, so that the restarted streaming media player decrypts the encrypted audio and video data by using a secret key, and the restarted streaming media player decodes the decrypted audio and video data by using the play configuration information.

In some embodiments, after creating the content decryption module, the controller is further to perform: if the current received streaming media data is clear streaming data, temporarily not restarting the streaming media player, and not creating a binding relation between the content decryption module and the streaming media player; and controlling the streaming media player to play the clear stream data.

In some embodiments, the communicator is further configured to communicatively connect with an authentication server, and the controller obtains a key, including: sending an authentication request to an authentication server through a communicator, wherein the authentication request comprises equipment information of a display device, and the authentication request is used for indicating the authentication server to authenticate the display device according to the equipment information and sending the secret key to the display device after the authentication is successful; the key sent by the authentication server is received by the communicator.

In some embodiments, the controller establishes a binding relationship between the content decryption module and the restarted streaming media player, including: generating a dense stream handle, wherein the dense stream handle is used for indicating a target storage address in a memory of the encrypted audio and video data; and sending the dense stream handle to the restarted streaming media player so that the restarted streaming media player can acquire the encrypted audio and video data from the target storage address.

In some embodiments, the streaming media player comprises a decoder and a display and audio playback device linked to the decoder, the controller restarting the streaming media player comprising: resetting the decoder, wherein the reset decoder comprises a decryption module, and the decryption module is used for storing the secret key and decrypting the dense stream data; a display and an audio decoder are linked to the reset decoder.

In some embodiments, after obtaining the key, the controller is further to perform: reading a preset flag bit in a data packet of the currently received streaming media data; acquiring a streaming media type identifier of the preset flag bit record; and determining the streaming media type of the data packet according to the streaming media type identifier, wherein the streaming media type comprises clear streaming data and dense streaming data.

In a second aspect, an embodiment of the present application further provides a streaming media playing method, including:

receiving stream media data sent by a stream media server;

In some embodiments, the method further comprises:

after the secret key is acquired, controlling the streaming media middleware to create a content decryption module matched with the decryption request, wherein the content decryption module is used for analyzing the secret stream data so as to separate playing configuration information from encrypted audio and video data;

after restarting the streaming media player, issuing the secret key to the restarted streaming media player;

establishing a binding relation between the content decryption module and the restarted streaming media player; the binding relationship is used for indicating the content decryption module to transmit the play configuration information and the encrypted audio and video data to the restarted streaming media player, so that the restarted streaming media player decrypts the encrypted audio and video data by using a secret key, and the restarted streaming media player decodes the decrypted audio and video data by using the play configuration information.

In the foregoing embodiments of the present application, for a media play scenario in which streaming is performed first and then in which streaming is performed second, before receiving a decryption request, the controller performs a streaming play procedure by default, that is, the streaming media player does not decrypt streaming media data, but directly decodes and plays the streaming media data. After receiving the decryption request, the controller obtains the key for decryption after knowing that the streaming media server is about to transmit the encrypted streaming data, and performs category detection on the streaming media data so as to determine the time for restarting the streaming media player. If the controller detects that the current received streaming media data is the dense streaming data, restarting the streaming media player, and decrypting, decoding and playing the streaming media data by utilizing the restarted streaming media player.

If the clear stream data is not played, the controller does not restart the stream media player temporarily, so that the clear stream data can be played continuously by the clear stream player, unnecessary waste of decryption resources is avoided, the workload of a system chip is reduced, and the fluency of the clear stream data playing is improved. When stream media data is switched from clear stream to dense stream, the prior clear stream data is indicated to be played, at the moment, the controller restarts the stream media player, so that a black screen picture which is displayed when the media asset itself transits during the conversion of clear stream to dense stream covers the short-time black screen phenomenon generated by restarting the stream media player, thereby eliminating the influence of restarting the stream media player on the viewing of the media asset by a user and improving the experience of the user on viewing the media asset.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is an operation scenario between a display device and a control apparatus 100 provided in some embodiments of the present application;

fig. 2 is a block diagram of a hardware configuration of a control device 100 according to some embodiments of the present application;

fig. 3 is a block diagram of a hardware configuration of a display device 200 provided in some embodiments of the present application;

fig. 4 is a software configuration diagram of a display device 200 according to some embodiments of the present application;

fig. 5 is a schematic diagram of a streaming media playing architecture of clear streaming data according to some embodiments of the present application;

fig. 6 is a schematic diagram of a streaming media playing architecture of the dense stream data according to some embodiments of the present application;

fig. 7 is a schematic diagram of a first streaming media playing architecture with clear-before-dense streaming according to some embodiments of the present application;

FIG. 8 is a schematic diagram of a media playback frame flash screen caused by restarting a streaming media player according to some embodiments of the present application;

Fig. 9 is a schematic diagram of a second streaming media playing architecture with clear-before-dense streaming according to some embodiments of the present application;

fig. 10 is a flowchart of a first streaming media playing method according to some embodiments of the present application;

fig. 11 is a flowchart of a second streaming media playing method according to some embodiments of the present application.

Detailed Description

For purposes of clarity and implementation of the present application, the following description will make clear and complete descriptions of exemplary implementations of the present application with reference to the accompanying drawings in which exemplary implementations of the present application are illustrated, it being apparent that the exemplary implementations described are only some, but not all, of the examples of the present application.

It should be noted that the brief description of the terms in the present application is only for convenience in understanding the embodiments described below, and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.

The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between similar or similar objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The display device provided in the embodiment of the application may have various implementation forms, for example, may be a television, an intelligent television, a laser projection device, a display 260 (monitor), an electronic whiteboard (electronic bulletin board), an electronic desktop (electronic table), and the like. Fig. 1 and 2 are specific embodiments of a display device of the present application.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus 100 according to an embodiment. As shown in fig. 1, a user may operate the display device 200 through the smart device 300 or the control apparatus 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or bluetooth protocol communication, and other short-range communication modes, and the display device 200 is controlled by a wireless or wired mode. The user may control the display device 200 by inputting user instructions through keys on a remote control, voice input, control panel input, etc. Alternatively, the control device 100 may be a mouse, and the mouse and the display device may be connected by a wired or wireless manner.

In some embodiments, a smart device 300 (e.g., mobile terminal, tablet, computer, notebook, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on a smart device.

In some embodiments, the display device may receive instructions not using the smart device or control device described above, but rather receive control of the user by touch or gesture, or the like.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be permitted to make communication connections via a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200. The server 400 may be a cluster, or may be multiple clusters, and may include one or more types of servers.

In some embodiments, server 400 may comprise a streaming server. The display device 200 may acquire streaming media data from the streaming media server after establishing a communication connection with the streaming media server, and play the received streaming media data.

Fig. 2 shows a block diagram of a configuration of the control apparatus 100 according to an exemplary embodiment in some embodiments. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction of a user and convert the operation instruction into an instruction recognizable and responsive to the display device 200, and function as an interaction between the user and the display device 200.

Fig. 3 is a block diagram of a hardware configuration of a display device 200 provided in some embodiments of the present application. As shown in fig. 3, the display apparatus 200 includes at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, a first interface for input/output to an nth interface.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, or other network communication protocol chip or a near field communication protocol chip, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

In some embodiments, the communicator 220 may be used to establish a communication connection between the display device 200 and a streaming server to enable interaction between the display device 200 and the streaming server. The controller 250 may transmit a download request to the streaming server through the communicator 220 to cause the streaming server to transmit streaming data to the display device 200 in response to the download request. After receiving the streaming media data through the communicator 220, the controller 250 may start the streaming media player and control the streaming media player to play the streaming media data, which includes audio and video data.

In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, for receiving image signals from the controller output, for displaying video content, image content, and a menu manipulation interface, and for manipulating a UI interface by a user. The display 260 may also be used to display video data of the streaming media.

In some embodiments, audio output interface 270 may be coupled to an audio playback device that plays audio data of the streaming media. Audio output interface 270 may be coupled to speakers internal to display device 200. The audio output interface may further include an external audio output terminal, where the external audio output terminal is used to connect to an external audio playing device (e.g., an audio device) in a wired manner. The audio playing device may also be an external power amplifier (e.g., bluetooth sound, bluetooth headset, etc.) that is wirelessly connected via the communicator 220.

The user interface may be configured to receive a control signal of the control device 100 (e.g., an infrared remote control, a mouse, etc.), where the control signal includes a streaming media on-demand operation input by a user through the control device 100.

The external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. The input/output interface may be a composite input/output interface formed by a plurality of interfaces.

The controller 250 controls the operation of the display device and responds to the user's operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command to select a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments the controller includes at least one of a central processing unit (Central Processing Unit, CPU), video processor, audio processor, graphics processor (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read-Only Memory, ROM), first to nth interfaces for input/output, a communication Bus (Bus), and the like.

The user may input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.

A "user interface" is a media interface for interaction and exchange of information between an application or operating system and a user, which enables conversion between an internal form of information and a user-acceptable form. A commonly used presentation form of the user interface is a graphical user interface (Graphic User Interface, GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in a display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In some embodiments, a graphics processor is used to generate various graphical objects, such as: at least one of icons, operation menus, page contents displayed based on an input instruction of a user, and the like. The graphic processor comprises an arithmetic unit, which is used for receiving various interactive instructions input by a user to operate and displaying various objects according to display attributes; the device also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, as shown in fig. 4, the system of the display device is divided into three layers, an application layer, a middleware layer, and a hardware layer, respectively, from top to bottom.

In some embodiments, the application layer mainly comprises common applications on the television, and an application framework (Application Framework), wherein the common applications are mainly applications developed based on Browser, such as: HTML5 APPs; native applications (Native APPs);

in some embodiments, the application framework (Application Framework) is a complete program model with all the basic functions required by standard application software, such as: file access, data exchange, and the interface for the use of these functions (toolbar, status column, menu, dialog box).

In some embodiments, native APPs (Native APPs) may support online or offline, message push, or local resource access.

In some embodiments, the middleware layer includes middleware such as various television protocols, multimedia protocols, and system components. The middleware can use basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, so that the purposes of resource sharing and function sharing can be achieved.

In some embodiments, the hardware layer mainly includes a HAL interface, hardware and a driver, where the HAL interface is a unified interface for all television chips to interface, and specific logic is implemented by each chip. The driving mainly comprises: audio drive, display drive, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (e.g., fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply drive, etc.

For devices with the same or similar hardware and software configuration as the aforementioned display device 200, a user may view a web feed through an application such as a browser, video APP, or the like, or may receive and play television signals through a device such as a set-top box, television box, or the like. In some embodiments, the media data can be transmitted by adopting a streaming media technology, that is, the media data is decomposed into a plurality of data packets and sent through network segmentation, so that the data packets can be sent in real time like 'streaming', and the real-time performance and the interactivity are high. The streaming mode realizes that a user does not need to wait for the complete downloading of audio and video data when watching the media asset, thereby reducing the playing delay of the media asset and also reducing the requirement on the system buffer capacity.

In some embodiments, most of feature videos need copyright maintenance, a part of feature videos need payment or member rights are opened to be completely watched, so as to prevent piracy risks caused by re-etching of audio and video streams in the transmission process, therefore, a streaming media server needs to implement encryption protection on such media, namely, the media is presented as a dense stream in the streaming media, so that the display device 200 receives the dense stream data, decrypts the dense stream data by using a secret key, and decodes and plays the dense stream after the dense stream is restored to a clear stream.

In some embodiments, for non-feature video such as advertisements, trailers, clips, etc., users can watch the video through free channels, and such video is often expected to be widely spread, so the streaming server may not encrypt the non-feature video, that is, present as clear streaming in the streaming media, so the display device 200 receives the clear streaming data, and directly controls the streaming media player to play the clear streaming data without decryption. Thus, streaming media data can be divided into: clear stream data and dense stream data.

In some embodiments, streaming media data is provided and sent by a streaming media server to the display device 200. For example, after the user clicks on the link of the clip source a, the display apparatus 200 may send a download request to the streaming server, where the download request is used to pull the audio/video stream of the clip source a to the streaming server. In response to the download request, the streaming server determines whether to take encrypted transmission for the clip source A based on the related information of the clip source A (e.g., ID, URL, video details, etc.). If encrypted transmission is not adopted, the streaming media server sends the audio/video stream (clear stream) of the clip source a to the display device 200, that is, each data packet of the clip source a is not encrypted.

In some embodiments, if encrypted transmission is adopted, the streaming media server may utilize an encryption scheme to encrypt the audio/video data of the source a, for example, an encryption algorithm such as HLS (Http Live Streaming, http real-time streaming media) may be adopted, where HLS is a streaming media transmission protocol that supports streaming media encryption and can be applied to a browser, and by cutting the entire media stream into a plurality of data packets that can be downloaded through Http, and then providing a matched media list file to the display device 200, the display device 200 receives the data packets and plays the data packets in sequence, so that a "one-stream" play effect is presented, and dense streaming is realized.

In some embodiments, the encryption protocols and algorithms used by the different encryption schemes are also different, and the encryption scheme used by the streaming server is not limited and may be specifically configured by the operator. Encryption schemes may include, but are not limited to:or other customized encryption schemes, etc.

Fig. 5 is a schematic diagram of a streaming media playing architecture of clear streaming data according to some embodiments of the present application. As shown in fig. 5, when a user requests a first media asset, the media asset application sends a download request to a streaming media server; the streaming server may determine whether the first asset is an encrypted asset in response to the download request. If the first asset is an unencrypted asset (such as an advertisement, a trailer, etc.), the streaming media server sends an audio-video stream of the first asset to the asset application, where the audio-video stream is clear stream data, so that the asset application forwards the received clear stream data to the streaming media middleware via the browser. The streaming middleware knows that the current media stream is clear stream, does not create CDM (Content Decryption Module ), and starts the streaming player by the foreground and sends the clear stream data to the streaming player so that the streaming player plays the clear stream data. The CDM module is configured to parse the dense stream to separate the playing configuration information from the encrypted audio/video data, where the playing configuration information may include configuration parameters related to decoding and playing of the first media asset.

In some embodiments, after determining to transmit the audio and video data of the first asset in the clear stream mode, the streaming media server does not send a decryption request to the asset application, so that the asset application cannot receive the decryption request, and starts the clear stream playing program by default, where the class of the streaming media player is the clear stream player. The clear stream player may include a video playback link, an audio playback link, and a sound and picture synchronization module. The video playing link comprises a video decoder and a display which are sequentially linked, and the audio playing link comprises an audio decoder and an audio playing device which are sequentially linked. And the sound and picture synchronization module is used for realizing sound and picture playing synchronization of the display and the audio playing device according to the information such as the display time stamp and the like.

Fig. 6 is a schematic diagram of a streaming media playing architecture of the dense stream data according to some embodiments of the present application. As shown in fig. 6, when a user requests a first media asset, the media asset application sends a download request to a streaming media server; the streaming media server responds to the downloading request, and if the first media asset is determined to be an encrypted media asset (for example, pure feature video), the streaming media server sends a decryption request to the media asset application, wherein the decryption request is used for informing that the first media asset is encrypted and indicating that the media asset application starts a dense stream playing program.

In some embodiments, referring to fig. 6, the close-stream playback program is configured to: the media resource application transmits the decryption request to the decoder after passing through the browser and the streaming media middleware in sequence; the decoder generates an authentication request according to the decryption request, and sends the authentication request to the browser through the streaming middleware, and the browser sends the authentication request to the authentication server; the authentication server responds to the authentication request to carry out authentication verification on the display equipment 200, if the authentication is successful, the key is sent to the browser, and the browser forwards the key to the streaming media middleware; after receiving the key, the streaming media middleware issues the key to the decoder to create a CDM module, and the foreground starts the streaming media player to bind the CDM module with the streaming media player. Thus, the execution of the front preparation link of the dense stream playing is finished.

In some embodiments, if the streaming server fails to authenticate the display device 200, i.e., the display device 200 does not have decryption rights, authentication failure information is sent to the browser without sending a key. And the browser receives the authentication failure information and forwards the authentication failure information to the streaming media middleware.

In some embodiments, the streaming middleware may destroy the created CDM module if it receives authentication failure information, and does not perform the procedure of detecting the streaming class.

In some embodiments, the streaming middleware may send authentication failure information to the decoder. The decoder receives the authentication failure information, does not reset the decoder, and does not establish a binding relationship with the CDM module.

In some embodiments, the browser may send authentication failure information to the media asset application. The media asset application receives the authentication failure information, and presents first prompt information in a media asset playing interface, wherein the first prompt information is used for prompting that the first media asset does not have the authority of decryption and viewing.

In some embodiments, the authentication request may contain device information for display device 200, including, but not limited to: verification information such as factory company, product model, software version, expiration date and the like.

In some embodiments, the authentication server may maintain a rights list, where the rights list stores device information with decryption rights, a validity period of the decryption rights, a key of the trusted device, and the like. The authentication server verifies whether the display device 200 has decryption rights and whether the current decryption rights are within a valid period in response to an authentication request transmitted from the browser. If the display device has valid decryption rights, the authentication is successful and the authentication server can send the key to the browser. The browser sends the key to the decryption module of the decoder via the streaming middleware so that the decryption module decrypts the data using the key.

In some embodiments, the key includes, but is not limited to, information such as a decryption type, a decryption algorithm, a decryption expiration date, and a decryption identification. Wherein the key is related to an encryption scheme used by the streaming server. The decryption module can store the key and decrypt the encrypted audio/video stream after CDM analysis by using the key, thereby restoring the dense stream into the clear stream.

In some embodiments, when the streaming media server sends a decryption request to the media asset application, encryption identification information corresponding to an encryption scheme used by the first media asset may be added to the decryption request, where the encryption scheme may include, but is not limited to:or other customized encryption schemes, etc. Thus, the streaming middleware receives the solutionAnd when the encryption request is received, acquiring encryption identification information from the decryption request, wherein the encryption identification information is used for enabling the streaming media middleware to create a CDM module which is adapted to the encryption scheme of the first media asset.

If the stream media middleware does not control the CDM module to analyze the dense stream data, the dense stream data is directly transmitted to the stream media player, and the stream media player does not have the dense stream analysis capability, so that the stream media player cannot normally decrypt, decode and play the dense stream data.

In some embodiments, the purpose of binding the CDM module to the streaming media player in this application is to: the front-end parsing link can be added for the streaming media player, so that the CDM module can transmit the encrypted audio and video data obtained after parsing to the decryption module, and transmit the play configuration information obtained after parsing to the decoder, so that the streaming media player becomes a dense streaming player adapted to the encryption scheme of the first media asset.

In some embodiments, the step of binding the CDM module with the streaming media player includes: the CDM module may create a dense stream handle associated with a storage address of a target memory pre-allocated by the system, where the target memory may be a memory space pre-allocated by the system for the encrypted audio-video data separated after parsing; the CDM module sends the dense stream handle to the decryption module of the streaming media player, and the decryption module stores the dense stream handle, so that the decryption module can read the data to be decrypted from the target memory pointed by the dense stream handle.

In some embodiments, the system may allocate a certain area in the memory for each process to store a handle, where the handle is an unsigned integer with a specific number of bits, and the handle corresponds to an object pointer, and is used to point to another area in the memory (assumed to be referred to as an area a), and it is the address of the object stored in the area a in the memory. When the position of the object in the memory changes, the value of the area A is updated and changed to the address of the object in the memory at the current moment, and in the process, the position of the area A and the value of the corresponding handle remain unchanged. A fixed handle points to a fixed position (i.e. area a), the value in area a can change dynamically, and the value in area a records the address of the object in the memory at the current moment, so that no matter how the address of the object in the memory changes, the area a can be found and the object can be located as long as the value of the handle is grasped, and the value of the handle is unchanged absolutely during the running of the program.

In some embodiments, after the pre-preparation link of the dense stream playing is executed, referring to fig. 6, the media asset application receives the dense stream data sent by the streaming media server, and forwards the dense stream data to the streaming media middleware through the browser. After receiving the dense stream data, the streaming media middleware controls the CDM module to analyze the dense stream data so as to separate the playing configuration information from the encrypted audio and video stream and send the playing configuration information and the encrypted audio and video stream to the streaming media player, wherein the streaming media player operated at the moment is the dense stream player, the dense stream player comprises a decoder, and a decryption module is configured in the decoder. After the decoder acquires and stores the secret key, when the encrypted audio and video stream is received, the decryption module is controlled to decrypt the audio and video stream by using the secret key, the decrypted audio and video stream is decoded by using the playing configuration information, the decoded video data is transmitted to the display, and the decoded audio data is transmitted to the audio playing device, so that the decryption, decoding and audio and video synchronous playing of the encrypted stream data are realized.

In the pure-dense stream playback architecture illustrated in fig. 6, the streaming server sends a decryption request to the asset application before the streaming player foreground starts, and the encryption scheme used by the current asset is informed by the decryption request, so that the streaming middleware creates an adapted CDM module.

In some embodiments, mixed stream playback may occur, where mixed streams include clear stream followed by dense stream, for example: the media asset header has tens of seconds of advertisements (clear stream) and links up with the feature content (dense stream) after the advertisement ends. For a media playback scenario with clear-before-dense streaming, the streaming server may not issue a decryption request before the streaming player is started, so the display device 200 does not receive the decryption request, and starts the clear streaming player by default. If the media asset application receives a decryption request issued by the streaming media server before the end of the streaming media play, the media asset application needs to be switched from the streaming media player to the dense streaming media player.

Fig. 7 is a schematic diagram of a first streaming media playing architecture with clear-before-dense streaming according to some embodiments of the present application. As shown in fig. 7, in response to an operation of requesting a first media asset by a user, a media asset application transmits a download request to a streaming media server; the streaming media server responds to the downloading request, and if the first media asset is queried to be firstly clear streaming media asset and then the streaming media asset is dense streaming media asset, the streaming media server firstly sends clear streaming data to the media asset application; the media resource application receives the clear stream data, does not receive the decryption request, defaults to start the clear stream player, transmits the clear stream data to the clear stream player after passing through the browser and the streaming media middleware, and controls the decoder to decode and play the clear stream data.

In some embodiments, before sending the encrypted stream data, the streaming server needs to send a decryption request to the media asset application in advance, so that the display device 200 completes the pre-preparation link of the encrypted stream playing. Referring to fig. 7, the decryption request is sent at a timing before the streaming server sends the streamlet data, and the streamlet data connected to the front part of the streamlet data has not yet been played, so that the media application needs to switch from the streamlet playing program to the streamlet playing program when receiving the decryption request.

In some embodiments, the media asset application sends the decryption request to the streamcast player via the browser and the streaming middleware, the streamcast player generates an authentication request, and sends the authentication request to the authentication server via the streaming middleware and the browser; the authentication server performs authentication verification on the equipment information, and feeds back a secret key to the browser after the authentication is passed; the browser forwards the secret key to the streaming media middleware; the streaming media middleware receives the secret key, creates a CDM module and controls the streaming media player to restart so as to switch from the clear stream player to the dense stream player; the streaming media middleware sends the secret key to the dense stream player, binds the CDM module with the dense stream player, and finishes the front preparation link of the dense stream play.

In some embodiments, restarting the streaming media player includes resetting the decoder and linking the display and the audio playback device to the decoder after the resetting. Restarting the streaming media player does not affect the self-configuration of the display and the audio playback device. Referring to fig. 7, the decoder in the clear stream player has no decryption module and stores no key, only for decoding the clear stream data; the decoder in the dense stream player is configured with a decryption module and stores a secret key issued by the streaming media middleware, so that the decoder is used for controlling the decryption module to decrypt the audio and video data in advance and then decode the decrypted audio and video data. It should be noted that, the clear stream player can only play clear stream data, and the dense stream player can play clear stream data and dense stream data compatibly.

In some embodiments, after switching to the dense stream player, the streaming server continues to send the streamlet data to the media asset application because the streamlet data has not yet been played. The media resource application transmits the clear stream data to the dense stream player after passing through the browser and the streaming media middleware, and the dense stream player plays the clear stream data.

In some embodiments, after the clear stream data is played, the streaming media server starts sending the encrypted stream data to the media asset application, and the media asset application forwards the encrypted stream data to the streaming media middleware through the browser. The streaming media middleware controls the CDM module to analyze the dense stream data to obtain separated play configuration information and encrypted audio/video data, and sends the play configuration information and the encrypted audio/video data to the dense stream player; the decryption module in the decoder is controlled by the dense stream player to decrypt the audio and video data, the decoder is controlled to decode the decrypted audio and video data, and the display and the audio playing device are controlled to synchronously play the decoded audio and video data, so that the playing of the dense stream data is completed.

The streaming media playing architecture of the clear-before-dense stream illustrated in fig. 7 has at least the following problems:

problem one: if the streaming media server does not issue the decryption request before the streaming media player is started, but issues the decryption request at such a time after the start of the streaming media player and before the transmission of the dense stream data, the streaming media player needs to restart to switch from the streaming media player to the dense stream player, and the streaming media player restarting can cause unsmooth media playing pictures, and a significant problem of flashing black screen occurs, which may last from several frames to several tens of frames, affecting the media playing effect and the user viewing experience.

Fig. 8 is a schematic diagram of a media playback frame flash screen caused by restarting a streaming media player according to some embodiments of the present application. Referring to fig. 8, when the playing progress of the current media asset is 00:54, the streaming media player is restarted to switch to the dense-stream player, the display 260 presents a problem of flashing black screen, and when the playing progress is changed to 00:55, the media asset playing picture is restored to be normal, i.e. the display presents a state of flashing black screen which can be obviously perceived by the user in the time period of 00:54-00:55.

And a second problem: when the clear stream data is not yet played, the stream media player is switched to the dense stream player, so that the stream media player can execute the dense stream playing program on the clear stream data, thereby causing the waste of decryption resources and the consumption of system calculation power, reducing the decoding efficiency of the audio and video data and even causing media resource playing to be blocked.

In order to solve the problem existing in the scene of playing the clear stream before the dense stream, the embodiment of the application is based on the characteristic that the coupling between the CDM module and the streaming media player is not strong, and the provided solving strategy is as follows: after the clear stream player is started, if the media asset application receives a decryption request issued by the stream media server, the stream media middleware temporarily does not restart the stream media player, and delays binding the CDM module and the stream media player. And restarting the streaming media player until the streaming media middleware receives the real dense stream data, and binding the CDM module with the dense stream player, thereby eliminating the defects of the first problem and the second problem.

Fig. 9 is a schematic diagram of a second streaming media playing architecture with clear-before-dense streaming according to some embodiments of the present application.

As shown in fig. 9, compared to the first streaming media playing architecture of clear-before-dense streaming illustrated in fig. 7, the difference is that:

when receiving the secret key sent by the browser, the streaming media middleware firstly creates a CDM module, but does not immediately restart the streaming media player, does not bind the CDM module and the current running clear stream player, and detects the streaming media category, wherein the streaming media category comprises clear stream data and dense stream data.

In some embodiments, the streaming media server may set the streaming media category identification within a data packet sent to the media asset application. For example, the streaming media server may set a preset flag bit in the data packet, and write a streaming media type identifier in the preset flag bit, where the streaming media type identifier is used to indicate that the data packet is dense stream data or clear stream data. After the stream media middleware creates the CDM module, the stream media type identification in the received data packet is read, so that the stream media type corresponding to the data packet is detected.

In some embodiments, the streaming media middleware may create a detection module, where the detection module is configured to read the streaming media class identifier from a preset flag bit of the data packet, so as to obtain a streaming media class corresponding to the data packet.

In some embodiments, if the streaming middleware detects that the streaming media class is clear streaming data, the streaming media player is not restarted temporarily, the clear streaming data is transmitted to the clear streaming player, the clear streaming player controls the decoder to decode the clear streaming data, and controls the display and the audio and video playing device to synchronously play the decoded audio and video data.

In some embodiments, if the streaming middleware detects that the streaming class is the dense stream data, indicating that the previous clean stream data has been played, immediately restarting the streaming player to switch to the dense stream player, issuing a key to the dense stream player, and creating a binding relationship between the CDM module and the dense stream player. The streaming media middleware controls the CDM module to analyze the dense stream data and sends the playing configuration information and the encrypted audio and video data obtained after analysis to the dense stream player; the decryption module is controlled by the dense-stream player to decrypt the audio and video data, the decoder is controlled to decode the decrypted audio and video data according to the playing configuration information, and the display and the audio playing device are controlled to synchronously play the decoded audio and video data.

In the embodiment of the present application, the streaming middleware does not restart the streaming media player immediately after creating the CDM module, but performs detection of the streaming media category. On the one hand, if the clear stream data is not played, the stream media player is not restarted temporarily, and the CDM and the stream media player are bound in a delayed mode, so that the clear stream data can be played continuously by the clear stream player, unnecessary waste of decryption resources is avoided, the workload of a system chip is reduced, and the fluency of playing the clear stream data is improved.

On the other hand, when the streaming middleware detects that the streaming media class is switched from clear stream to dense stream, the streaming media middleware indicates that the playing of the previous clear stream data is finished, and then the streaming media player is restarted, and the CDM module and the streaming media player are bound. For media assets, there is a video transition when switching from clear stream to dense stream, the video transition is generally represented by a short-time black screen phenomenon, and the duration of the black screen is assumed to be T when the video transition is represented ₁ The time consumed for restarting the streaming media player is T ₂ . Wherein T is ₂ The time is short (e.g., hundreds of milliseconds), while T ₁ Typically up to a few seconds due to T ₂ Less than T ₁ The method and the device have the advantages that the flash black screen generated when the streaming media player is restarted is covered by the black screen picture when the video is transited, so that a user cannot perceive the operation of restarting the streaming media player, and the influence of restarting the streaming media player on the watching of media by the user is eliminated.

In some embodiments, the mixed stream may also include a clear-first stream, a medium-dense stream, and a clear-after-stream, such as a pre-advertisement-joining feature, where the feature plays for a period of time before inserting an advertisement in a segment. For such a play scenario, the first clear in stream dense stream stage may execute the streaming media play scheme illustrated in fig. 9; when the medium-density stream is switched to the post-density stream, the playing time of the density stream data is shorter, and the density stream player can be compatible with the playing of the density stream data, so that the playing of the density stream data by the density stream player can be kept temporarily without restarting the stream media player, when the advertisement in the film is switched back to the positive film after the playing is finished, the stream media playing process of the density stream after the clearing is not needed to be repeatedly executed, and the playing efficiency of the stream media is improved.

In some embodiments, the controller 250 may control the streaming media playback architecture referred to in fig. 9 and communicatively interact with the streaming media server via the communicator 220. The streaming media playing architecture may include, but is not limited to, media applications, browsers, streaming media middleware, and streaming media players.

It should be noted that, the streaming media playing architecture is not limited to the examples of the embodiments of the present application, for example, related applications and modules in the streaming media playing architecture may be added or deleted according to requirements of a system structure, a streaming media protocol, a playing scene, and the like, and the configuration of software and hardware of the streaming media player is not limited.

Fig. 10 is a flowchart of a first streaming media playing method according to some embodiments of the present application. The streaming media playing method is directed to the execution angle of the controller 250, as shown in fig. 10, and the method includes:

step S101, receiving streaming media data sent by a streaming media server.

Step S102, the streaming media player is started, and the streaming media player is controlled to decode and play the streaming media data. At this time, the streaming media player is a clear streaming player, that is, the clear streaming player program is executed by default when the decryption request is not received.

Step S103, when receiving the decryption request sent by the streaming media server, acquiring a key. The key may be obtained by an authentication server or other viable means, and the present embodiment is not limited.

Step S104, if the current received stream media data is the dense stream data, restarting the stream media player. The restarted streaming media player is a dense stream player, thereby converting to executing a dense stream playing program.

In step S105, the restarted streaming media player is controlled to decrypt the streaming media data by using the key.

Step S106, the restarted stream media player is controlled to decode the decrypted stream media data and play the decoded stream media data. Thus, decryption, decoding and playing of streaming media data can be completed through the dense stream player.

In the embodiment corresponding to fig. 10, if the playing of the clear stream data is not finished, the controller 250 does not restart the streaming media player temporarily, so that the clear stream data can be played by the clear stream player continuously, thereby avoiding unnecessary waste of decryption resources, reducing the workload of the system chip, and improving the fluency of playing the clear stream data. When the streaming media data is switched from clear stream to dense stream, the previous clear stream data is indicated to be played, and then the controller 250 restarts the streaming media player, so that a black screen picture which is displayed by the media asset itself in transition when the clear stream is switched to the dense stream covers a short-time black screen phenomenon generated by restarting the streaming media player, thereby eliminating the influence of restarting the streaming media player on the viewing of the media asset by the user and improving the experience of the user on viewing the media asset.

Fig. 11 is a flowchart of a second streaming media playing method according to some embodiments of the present application. The streaming media playing method is directed to the execution angle of the controller 250, as shown in fig. 11, and the method includes:

step S111, receiving the streaming media data sent by the streaming media server.

Step S112, the streaming media player is started to control the streaming media player to decode and play the streaming media data.

Step S113, when receiving the decryption request sent by the streaming media server, obtaining the secret key and controlling the streaming media middleware to create a content decryption module.

Step S114, judging whether the current received stream media data is the dense stream data.

If the received streaming media data is clear streaming data, executing step S115; if the received streaming media data is the dense stream data, step S116 is performed.

In step S115, the streaming media player is controlled to play the streaming media data.

Step S116, restarting the streaming media player, and establishing the binding relation between the streaming media player and the content decryption module.

Step S117, the control content decryption module analyzes the streaming media data and sends the secret key, the playing configuration information obtained after analysis and the encrypted audio/video data to the streaming media player.

In step S118, the streaming media player is controlled to decrypt the encrypted audio/video data using the key.

In step S119, the streaming media player is controlled to decode the decrypted audio/video data by using the play configuration information.

Step S1110, controlling the streaming media player to play the decoded audio/video data.

In the streaming media playing method, the controller 250 does not restart the streaming media player immediately after the streaming media middleware creates the CDM module, but performs detection of the streaming media category. If the clear stream data is not played, the stream media player is not restarted temporarily, and the CDM and the stream media player are bound in a delayed mode, so that the clear stream data can be played continuously by the clear stream player, unnecessary waste of decryption resources is avoided, the workload of a system chip is reduced, and the fluency of the clear stream data playing is improved. When the streaming media data is switched from clear stream to dense stream, the controller 250 restarts the streaming media player and binds the CDM module with the streaming media player, which can eliminate the influence of restarting the streaming media player on the viewing of media by the user.

The streaming media playing method illustrated in fig. 10 and fig. 11 is a concept and description of the foregoing streaming media playing architecture and flow of clear-before-dense streaming from the execution point of the controller 250, and the technical details involved in each step may be adaptively referred to the description of the foregoing related embodiments, which is not repeated herein.

Some embodiments of the present application also provide a computer storage medium, which may store a program. When the computer storage medium is configured in the display apparatus 200, the program may include program steps involved in the streaming media playing method configured by the display apparatus 200 in the above embodiment when the program is executed. The computer storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the disclosure and to enable others skilled in the art to best utilize the embodiments.

Claims

1. A display device, characterized by comprising:

a controller for performing:

2. The display device of claim 1, wherein after obtaining the key, the controller is further to perform:

and controlling the streaming media middleware to create a content decryption module matched with the decryption request, wherein the content decryption module is used for analyzing the dense stream data so as to separate playing configuration information from encrypted audio and video data.

3. The display device of claim 2, wherein after restarting the streaming media player, the controller is further configured to perform:

the secret key is issued to the restarted streaming media player;

4. The display device of claim 2, wherein after creating the content decryption module, the controller is further configured to perform:

If the current received streaming media data is clear streaming data, temporarily not restarting the streaming media player, and not creating a binding relation between the content decryption module and the streaming media player;

and controlling the streaming media player to play the clear stream data.

5. The display device of claim 1, wherein the communicator is further configured to communicatively connect with an authentication server, and wherein the controller obtains a key, comprising:

sending an authentication request to an authentication server through a communicator, wherein the authentication request comprises equipment information of a display device, and the authentication request is used for indicating the authentication server to authenticate the display device according to the equipment information and sending the secret key to the display device after the authentication is successful;

the key sent by the authentication server is received by the communicator.

6. The display device of claim 3, wherein the controller establishes a binding relationship between the content decryption module and the restarted streaming media player, comprising:

generating a dense stream handle, wherein the dense stream handle is used for indicating a target storage address in a memory of the encrypted audio and video data;

And sending the dense stream handle to the restarted streaming media player so that the restarted streaming media player can acquire the encrypted audio and video data from the target storage address.

7. The display device of claim 1, wherein the streaming media player comprises a decoder and a display and audio playback means linked to the decoder, and wherein the controller restarts the streaming media player comprising:

resetting the decoder, wherein the reset decoder comprises a decryption module, and the decryption module is used for storing the secret key and decrypting the dense stream data;

a display and an audio decoder are linked to the reset decoder.

8. The display device of claim 1, wherein after obtaining the key, the controller is further to perform:

reading a preset flag bit in a data packet of the currently received streaming media data;

acquiring a streaming media type identifier of the preset flag bit record;

and determining the streaming media type of the data packet according to the streaming media type identifier, wherein the streaming media type comprises clear streaming data and dense streaming data.

9. A streaming media playing method, comprising:

Receiving stream media data sent by a stream media server;

10. The method according to claim 9, wherein the method further comprises: