WO2021184181A1

WO2021184181A1 - Secure output method and electronic device

Info

Publication number: WO2021184181A1
Application number: PCT/CN2020/079589
Authority: WO
Inventors: 方中华; 王玺林
Original assignee: 华为技术有限公司
Priority date: 2020-03-17
Filing date: 2020-03-17
Publication date: 2021-09-23
Also published as: CN115211129A

Abstract

The present application relates to the field of multimedia technologies, and discloses a secure output method and an electronic device. The secure output method comprises: in a stage in which an electronic device decodes a video code stream to obtain a first decoded video frame, the electronic device generating frame-level protection information of the first decoded video frame according to an output control policy and the resolution of the first decoded video frame; and then, the electronic device determining, according to the frame-level protection information, whether to output a first display video frame, the first display video frame being obtained according to the first decoded video frame. Hence, every time the electronic device outputs a video frame, the electronic device can perform output protection on the video frame according to frame-level protection information corresponding to the video frame, thereby providing more precise output protection and improving the protection performance.

Description

Safe output method and electronic equipment

Technical field

This application relates to the field of multimedia technology, and in particular to a safe output method and electronic equipment.

Background technique

In order to protect the media content with protection requirements, the electronic device can process the transmission process according to the content usage rules (CUR) from decoding the code stream to transmitting the decoded video stream to the output port. Media content in. CUR can include protection strategies such as high-bandwidth digital content protection (HDCP) strategies, video watermark protection strategies, and so on.

However, the existing safe output mechanism protects the media content from the code stream level, which may cause problems such as video error display, and the protection performance of the media content is poor.

Summary of the invention

This application provides a safe output method and electronic equipment to solve the problem of poor protection performance of existing media content protection mechanisms.

In the first aspect, the present application provides a safe output method. The method includes: in the stage of decoding a video code stream to obtain a first decoded video frame, generating according to the output control strategy and the resolution of the first decoded video frame Frame-level protection information of the first decoded video frame; determining whether to output the first display video frame according to the frame-level protection information, and the first display video frame is obtained according to the first decoded video frame.

Among them, the electronic equipment involved in the field can support a trusted execution environment (TEE) and a common execution environment (rich execution environment, REE). TEE is used to provide a protected execution environment for protected application software, and REE is used to provide an execution environment for unprotected application software. The media content transmitted under the TEE may be processed in accordance with media content usage rules (content usage rules, CUR), for example, to realize the protection of the media content. CUR can include protection strategies such as HDCP strategy, video watermark protection strategy, prohibition of transcoding and prohibition of recording. In this application, the HDCP strategy and the video watermark protection strategy are collectively referred to as "output protection strategy" or "output control strategy".

In the technical solution of the present application, the electronic device decodes the video code stream to obtain the first decoded video frame, and generates the frame-level protection information of the first decoded video frame. Further, the electronic device may determine whether to output the first display video frame according to the frame-level protection information. It can be seen that the protection information in the embodiments of this application is frame-level, and each video frame has its own corresponding frame-level protection information, and each video frame output can be based on the frame-level protection information corresponding to the video frame. For output protection, the safety is higher and the flexibility is better.

In a possible implementation manner, after the first decoded video frame is obtained by decoding the video bitstream, the method further includes: processing the first decoded video frame to obtain the first display video frame; The resolution of the displayed video frame updates the frame-level protection information.

In some embodiments, the electronic device may process the first decoded video frame to obtain the output first display video frame. Correspondingly, after the electronic device processes the first decoded video frame, the resolution of the first decoded video frame may change accordingly, that is, the resolution of the first displayed video frame may be different from the resolution of the first decoded video frame . Based on this, the electronic device can update the frame-level protection information according to the resolution of the first display video frame. It can be seen that with this implementation, the electronic device can update the frame-level protection information of the video frame according to the change in the resolution of the video frame, so that each video frame corresponds to the protection information of the video frame.

In a possible implementation manner, the processing the first decoded video frame to obtain the first display video frame includes: enlarging the first decoded video frame to obtain the first display video frame; or reducing the first decoded video frame The video frame obtains the first display video frame.

In a possible implementation manner, the frame-level protection information includes the resolution and minimum resolution of the first decoded video frame, and the minimum resolution is the minimum resolution corresponding to the video stream in the entire media path. Updating the frame-level protection information with the resolution of the first display video frame includes: when the resolution of the first display video frame is less than the minimum resolution, modifying the minimum resolution in the frame-level protection information to the second One shows the resolution of the video frame. Among them, the frame-level protection information includes the minimum resolution, and the minimum resolution is the minimum resolution corresponding to the video stream in the entire media path.

It should be understood that the minimum resolution is usually set in the initial information of the frame-level protection information. In the frame-level protection information of the first decoded video frame, if the resolution of the first decoded video frame is less than the initially set minimum resolution, modify the minimum resolution in the frame-level protection information to the resolution of the first decoded video frame . Similarly, when the resolution of the first display video frame is less than the minimum resolution in the frame-level protection information (that is, the resolution of the first decoded video frame), the minimum resolution in the frame-level protection information is modified to the first display The resolution of the video frame.

In a possible implementation manner, the determining whether to output the first display video frame according to the frame-level protection information includes: obtaining an output protection state allowed by an output port; determining a target output control strategy that matches the output protection state; determining Whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy; when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, the first display is output Video frame.

Digital high definition multimedia interface (HDMI) is a hardware interface that supports HDCP strategy. The HDMI of the electronic device can determine the output protection status of the output video frame (also referred to as the HDCP protection status) according to the HDCP support status of the connected display device. For example, the HDMI of the electronic device supports HDCP2.2 and HDCP1.4, and the interface connected to the HDMI only supports HDCP1.4, then the HDCP protection state of the output video frame determined by HDMI is HDCP1.4.

Before outputting the first display video, the electronic device may obtain the HDCP protection status of HDMI, and further, determine a target output control strategy matching the HDCP protection status from the frame-level protection information of the first display video frame. Then, when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, the electronic device outputs the first display video frame. It can be seen that every time the electronic device outputs a video frame, the video frame can be output protected according to the frame-level protection information corresponding to the video frame, so that the security is higher. When the resolution of the video frame exceeds the resolution allowed by the output control strategy, the output of the video frame will be prohibited. It can be seen that this implementation can effectively protect the output of the high-definition video frame. The protection information in the embodiments of this application is at the frame level. Each video frame has its own corresponding frame-level protection information. Every time a video frame is output, the video frame can be output according to the frame-level protection information corresponding to the video frame. Protection, security, and flexibility are also better.

In a possible implementation manner, before decoding the video code stream to obtain the first decoded video frame, the method further includes: the trusted application TA configures the output control strategy for the video code stream. The software system that performs output protection for video content includes a condition access system (CAS) or a digital rights management (digitial rights management, DRM) system. In this application, CAS or DRM is expressed as "CAS/DRM". The TEE software application layer can include CAS/DRM trusted applications (CAS/DRM trusted application, CAS/DRM TA). After receiving the encrypted video code stream, the TA decrypts the encrypted video code stream to obtain the video code stream and the protection requirements corresponding to the video code stream. After that, the TA can configure the output control strategy according to the protection requirements. With this implementation, it is possible to provide information for generating frame-level protection information.

In a possible implementation manner, the method further includes: determining whether the video bitstream and the first decoded video frame correspond to the same media channel; and determining whether the first decoded video frame and the first display video frame correspond to the same media channel . In this technical field, the path for transmitting video content is the media path. The media path includes a series of hardware modules and memory used for video processing. In this application, it is the hardware decoding module (video decoder, VDEC) that decodes the video code stream to obtain the first decoded video frame. It is a video processor (VPSS) that processes the first decoded video frame to obtain the first display video frame. Before outputting the first decoded video frame, VDEC should determine whether the video bitstream and the first decoded video frame correspond to the same media channel. This can avoid the problem of accessing data of other media channels when VDEC outputs the first decoded video frame, thereby improving the security of video frame transmission. Similarly, before outputting the first display video frame, the VPSS should determine whether the first decoded video frame and the first display video frame correspond to the same media channel. This can avoid the problem of accessing data of other media channels when the VPSS outputs the first display video frame, thereby improving the security of video frame transmission.

In a possible implementation manner, the determining whether the video code stream and the first decoded video frame correspond to the same media path includes: obtaining the first path identifier corresponding to the video code stream and the frame-level protection of the first decoded video frame The second path identifier in the information; when the first path identifier and the second path identifier are the same, it is determined that the video bitstream and the frame-level protection information of the first decoded video frame correspond to the same media path; it is determined that the first Whether the decoded video frame and the first display video frame correspond to the same media path includes: obtaining the third path identifier in the frame-level protection information of the first decoded video frame, and the fourth path identifier in the updated frame-level protection information ; When the third channel identifier and the fourth channel identifier are the same, it is determined that the first decoded video frame and the first display video frame correspond to the same media channel. Wherein, when the electronic device creates a media channel, the corresponding media channel is provided with a channel identifier, and the channel identifier is used to identify the media channel. After that, the electronic device can maintain the corresponding relationship between the path identifier and each data memory on the media path, and write the path identifier as initial information into the frame-level protection information. Furthermore, in this application, the electronic device can determine whether two video frames correspond to the same media channel by comparing whether the channel identifiers are the same, so as to avoid the problem of hardware modules accessing data of other media channels, thereby improving the security of video frame transmission .

In a possible implementation manner, before determining whether to output the first display video frame according to the frame-level protection information, and before updating the frame-level protection information according to the resolution of the first display video frame, the method further includes: detecting the Whether the frame-level protection information is valid. Among them, any hardware module on the media channel needs to check the validity of the corresponding frame-level protection information before reading the frame-level protection information and before writing the relevant information of the frame-level protection information. For example, before writing the output control strategy and the resolution of the first decoded video frame, VDEC should check whether the corresponding initial information of the frame-level protection information is valid. Before the VPSS reads the frame-level protection information of the first decoded video frame, it needs to check whether the frame-level protection information of the first decoded video frame is valid. Before the VPSS updates the minimum resolution in the frame-level protection information, it needs to check whether the frame-level protection information to be updated is valid. Before reading the frame-level protection information of the first display video frame, the VDP needs to check whether the frame-level protection information of the first display video frame is valid. This can ensure that the corresponding video frame is output protected according to the correct protection information.

In a possible implementation manner, the detecting whether the frame-level protection information is valid includes: detecting at least one of the following: detecting whether the type of the frame-level protection information matches a pre-configured type, and the type is used to indicate the frame-level protection Whether the information is bound to the video frame; determine whether the check value of the frame-level protection information is the same as the initial check value; or, detect the correspondence between the storage address of the first display video frame in the frame-level protection information and the length of the storage address is it right or not. Among them, detecting whether the frame-level protection information is valid is essentially detecting at least one piece of initial protection information in the frame-level protection information, and determining the corresponding frame under the condition that the detected at least one piece of initial protection information is valid Level protection information is valid.

Taking VDP as an example, the VDP can detect whether the type (tag) of the frame-level protection information of the first displayed video frame matches the pre-configured tag. If the tag of the frame-level protection information of the first display video frame matches the pre-configured tag, the tag information is valid. Among them, the tag is used to indicate whether the frame-level protection information is bound to the video frame. If the frame-level protection information is bound to a video frame, then the VDP should perform an operation on the first display video frame according to the resolution of the video frame in the frame-level protection information. If the frame-level protection information is not bound to the video frame, then the frame-level protection information is global protection information, and the VDP should perform operations on the first display video frame according to the resolution in the extended output strategy. With this implementation, after determining the correctness of the frame-level protection information type, the VDP can process the first display video frame according to the frame-level protection information according to the matching processing logic.

The VDP can also detect whether the corresponding relationship between the storage address of the first display video frame and the length of the storage address in the frame-level protection information is correct. If the corresponding relationship between the storage address of the first display video frame and the length of the storage address is correct, it indicates that the frame-level protection information is the frame-level protection information corresponding to the first display video frame. With this implementation method, VDP can ensure whether the frame-level protection information is associated with the video frame, thereby ensuring that the protection information of the corresponding video frame is read from the frame-level protection information, and further, realizing accurate output protection for the video frame.

VDP can also determine whether the check value of the frame-level protection information is the same as the initial check value. If the check value of the frame-level protection information is the same as the initial check value, it indicates that the frame-level protection information has not been tampered with. If the check value of the frame-level protection information is different from the initial check value, it indicates that the frame-level protection information has been tampered with. With this implementation method, VDP can perform output protection on the first display video frame according to the frame-level protection information while ensuring that the frame-level protection information is not damaged, forged or tampered, so as to implement the correct output protection for the first display video frame. Output protection, optimize the performance of output protection.

It should be understood that the detection sequence of the above three items of information can be arbitrary, which is not limited in this application. In addition, the detection of the validity of the frame-level protection information by VDEC and the detection of the validity of the frame-level protection information by the VPSS are similar to the detection process of the above-mentioned VDP, and will not be described in detail here.

In a possible implementation manner, the judging whether the check value of the frame-level protection information is the same as the initial check value includes: performing an exclusive OR operation on all the information in the frame-level protection information to obtain the first result; The first result is XORed with the security random number to obtain the check value of the frame-level protection information; whether the check value of the frame-level protection information is the same as the initial check value is compared.

The electronic device is provided with a mask register, and the mask register is, for example, a 32-bit register. When the electronic device is powered on, the TEE side can read a 32-bit random number from the hardware random number module, write it into the mask register and latch it. The REE side has no access rights to the mask register. Based on this, the 32-bit random number in the mask register is relatively safe and can be called a "secure random number". Further, the electronic device may determine the initial checksum in the frame-level protection information according to the algorithm checksum=(Word0^Word1^...Wordn)^Mask, where n is an integer greater than or equal to 1, and Word0 to Wordn refer to frames For each information in the level protection information, Mask refers to a secure random number, and ^ refers to an exclusive OR operation. When checking checksum, the corresponding hardware module can calculate the check value of the current frame-level protection information according to the same algorithm. Since the secure random number cannot be tampered with, if the calculated check value is the same as the checksum, it indicates that the frame-level protection information has not been tampered with. If the calculated check value is different from the checksum, it indicates that the frame-level protection information has been tampered with by the REE side.

In a possible implementation manner, the method further includes: adding a watermark to the first display video frame according to the frame-level protection information and the position and size of the first display video frame.

Among them, the frame-level protection information includes the storage address of the watermark information. The watermark information may include the content of the watermark information, the position added to the video frame, the relative size of the video frame, and so on. The display position and size of the first display video frame are all controlled by the user through the REE side input control instructions. Therefore, if the first display video frame needs video watermark protection, the electronic device can also obtain the position and size of the first display video frame from the REE driver. Then, the watermark information is obtained from the storage address of the watermark information. After that, the electronic device can use the position and size of the first display video frame, the relative position of the watermark information and the first display video frame, and the relationship between the watermark information and the first display video frame. Relative size, determine the location and size of the watermark information to be added. With this implementation, the electronic device can determine the position and size of the watermark to be added according to the position and size of the video frame when it is displayed, so as to ensure that the watermark is accurately added and displayed with the change of the video frame. In addition, the electronic device directly corresponds the video frame to the watermark information of the video frame through the frame-level protection information, which can avoid the problem of failure to add watermark due to the display level of the video frame in a scene where multiple videos are displayed at the same time.

In a possible implementation manner, it further includes: if the frame-level protection information of the first display video frame is lost, determining whether to output the first display video frame according to the resolution in the global protection information, and the resolution in the global protection information The rate is the minimum resolution in the output strategy corresponding to at least one bitstream.

In a possible implementation manner, before decoding the video code stream to obtain the first decoded video frame, the method further includes: obtaining a request to create a media path, which is used to transmit the video code stream; and generating a path of the media path Identification; configure the type of protection information in the frame-level protection information, the corresponding relationship between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value. With this implementation, the electronic device can pre-configure and maintain the software, hardware, related information, and the corresponding relationship between the above factors involved in the video transmission process, so as to realize the function of safely outputting the video.

In a second aspect, the present application provides a safety output device, which includes: a decoding module for generating frame-level protection information of the first decoded video frame according to the output control strategy and the resolution of the first decoded video frame, The first decoded video frame is obtained by decoding the video code stream by the decoding module; the display control module is configured to determine whether to output the first display video frame according to the frame-level protection information, and the first display video frame is based on the first display video frame. Obtained from a decoded video frame.

In a possible implementation manner, the device further includes: a processing module, configured to process the first decoded video frame to obtain the first display video frame; and an update module, configured to obtain the first display video frame according to the resolution of the first display video frame Update the frame-level protection information.

In a possible implementation manner, the processing module is specifically configured to enlarge the first decoded video frame to obtain the first display video frame; or, the processing module is specifically configured to reduce the first decoded video frame to obtain the first display video frame. Display video frames.

In a possible implementation manner, the frame-level protection information includes the resolution and minimum resolution of the first decoded video frame, and the minimum resolution is the minimum resolution corresponding to the video stream in the entire media path. The update The module is specifically configured to modify the minimum resolution in the frame-level protection information to the resolution of the first display video frame when the resolution of the first display video frame is less than the minimum resolution.

In a possible implementation, the display control module is specifically used to: obtain the output protection status allowed by the output port, determine the target output control strategy that matches the output protection status; determine the minimum resolution in the frame-level protection information Whether it is less than the resolution allowed by the target output control strategy; when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, output the first display video frame.

In a possible implementation manner, the device further includes a trusted application TA, which is used to configure the output control strategy for the video stream.

In a possible implementation manner, the decoding module is also used to determine whether the video bitstream and the first decoded video frame correspond to the same media path; and the processing module is also used to determine whether the first decoded video frame and the first decoded video frame correspond to the same media path. Whether the first display video frame corresponds to the same media channel.

In a possible implementation, the decoding module is specifically configured to: obtain the first path identifier corresponding to the video code stream and the second path identifier in the frame-level protection information of the first decoded video frame; When the channel identifier and the second channel identifier are the same, it is determined that the video bitstream and the frame-level protection information of the first decoded video frame correspond to the same media channel; and the processing module is specifically used to: obtain the information of the first decoded video frame The third path identifier in the frame-level protection information and the fourth path identifier in the updated frame-level protection information; when the third path identifier and the fourth path identifier are the same, it is determined that the first decoded video frame is the same as the fourth path identifier. The first display video frame corresponds to the same media channel.

In a possible implementation manner, the device further includes a detection module configured to detect whether the frame-level protection information is valid.

In a possible implementation manner, the detection module is specifically configured to perform at least one of the following detections: detecting whether the type of the frame-level protection information matches a pre-configured type, and the type is used to indicate whether the frame-level protection information is bound Determine the video frame; determine whether the check value of the frame-level protection information is the same as the initial check value; or check whether the corresponding relationship between the storage address of the first display video frame in the frame-level protection information and the length of the storage address is correct.

In a possible implementation, the detection module is specifically used to: perform an exclusive OR operation on all information in the frame-level protection information to obtain a first result; perform an exclusive OR operation on the first result and a secure random number, Obtain the check value of the frame-level protection information; compare whether the check value of the frame-level protection information is the same as the initial check value.

In a possible implementation manner, the device further includes a watermark adding module, configured to add a watermark to the first display video frame according to the frame-level protection information and the position and size of the first display video frame.

In a possible implementation manner, the display control module is further configured to determine whether to output the first display video frame according to the resolution in the global protection information when the frame-level protection information of the first display video frame is lost, the The resolution in the global protection information is the minimum resolution in the output strategy corresponding to at least one code stream.

In a possible implementation manner, the device further includes an acquisition module for acquiring a request to create a media channel, the media channel is used to transmit the video stream; a session management module, which is used to generate a channel identifier of the media channel; configuration The type of the protection information in the frame-level protection information, the correspondence between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value.

Among them, the technical effects produced by the second aspect and the implementation manners of the second aspect are the same as the technical effects produced by the implementation manners of the first aspect and the first aspect, and will not be repeated here.

In a third aspect, the present application provides an electronic device including a processor and a transmission interface, wherein the processor is configured to call software instructions stored in a memory to realize: decoding a video stream; In the stage of decoding the video stream to obtain the first decoded video frame, according to the output control strategy and the resolution of the first decoded video frame, the frame-level protection information of the first decoded video frame is generated; The level protection information determines whether to output the first display video frame, and the first display video frame is obtained according to the first decoded video frame.

In a possible implementation manner, the processor is further configured to process the first decoded video frame to obtain the first display video frame; and update the frame-level protection information according to the resolution of the first display video frame.

In a possible implementation manner, the processor is specifically configured to enlarge the first decoded video frame to obtain the first display video frame; or reduce the first decoded video frame to obtain the first display video frame.

In a possible implementation manner, the frame-level protection information includes the resolution and minimum resolution of the first decoded video frame, and the minimum resolution is the minimum resolution corresponding to the video stream in the entire media path. The device is also used to modify the minimum resolution in the frame-level protection information to the resolution of the first display video frame when the resolution of the first display video frame is less than the minimum resolution. Among them, the frame-level protection information includes the minimum resolution, and the minimum resolution is the minimum resolution corresponding to the video stream in the entire media path.

In a possible implementation, the processor specifically obtains the output protection status allowed by the output port, determines a target output control strategy that matches the output protection status; determines whether the minimum resolution in the frame-level protection information is less than the target The resolution allowed by the output control strategy; when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, the first display video frame is output.

In a possible implementation manner, the processor is further configured to configure the output control strategy for the video code stream.

In a possible implementation manner, the processor is further configured to determine whether the video bitstream and the first decoded video frame correspond to the same media path; the processor is also configured to determine whether the first decoded video frame and the first decoded video frame correspond to the same media path. One shows whether the video frame corresponds to the same media channel.

In a possible implementation manner, the processor is specifically configured to obtain the first path identifier corresponding to the video bitstream and the second path identifier in the frame-level protection information of the first decoded video frame; in the first path When the identifier and the second path identifier are the same, it is determined that the video bitstream and the frame-level protection information of the first decoded video frame correspond to the same media path; the processor is specifically configured to obtain the frame-level protection of the first decoded video frame The third path identifier in the information and the fourth path identifier in the updated frame-level protection information; when the third path identifier and the fourth path identifier are the same, it is determined that the first decoded video frame is the same as the first display The video frames correspond to the same media channel.

In a possible implementation manner, the processor is also used to detect whether the frame-level protection information is valid.

In a possible implementation manner, the processor is specifically configured to perform at least one of the following detections: detecting whether the type of the frame-level protection information matches a pre-configured type, and the type is used to indicate whether the frame-level protection information is bound Determine the video frame; determine whether the check value of the frame-level protection information is the same as the initial check value; or check whether the corresponding relationship between the storage address of the first display video frame in the frame-level protection information and the length of the storage address is correct.

In a possible implementation manner, the processor is specifically configured to perform an exclusive OR operation on all information in the frame-level protection information to obtain a first result; perform an exclusive OR operation on the first result and a secure random number to obtain The check value of the frame-level protection information; compare whether the check value of the frame-level protection information is the same as the initial check value.

In a possible implementation manner, the processor is further configured to add a watermark to the first display video frame according to the frame-level protection information and the position and size of the first display video frame.

In a possible implementation manner, the processor is further configured to determine whether to output the first display video frame according to the resolution in the global protection information when the frame-level protection information of the first display video frame is lost. The resolution in the protection information is the minimum resolution in the output strategy corresponding to at least one code stream.

In a possible implementation, the processor is also used to obtain a request to create a media path, which is used to transmit the video code stream; generate the path identifier of the media path; configure the protection in the frame-level protection information The type of information, the corresponding relationship between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value.

Among them, the technical effects produced by the third aspect and the implementation manners of the third aspect are the same as the technical effects produced by the implementation manners of the first aspect and the first aspect, and will not be repeated here.

In a fourth aspect, the present application provides an electronic device including a processor and a memory, where the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory, Complete the first aspect, or any one of the possible design methods of the first aspect. It should be understood that the electronic device may be a processor chip. In this case, the processor in the electronic device is a processor core or a central processing unit in the processor chip.

In the fifth aspect, the present application provides a computer-readable storage medium that stores instructions in the computer-readable storage medium, and when it runs on a computer or a processor, the computer or the processor executes the first aspect or the first aspect. In terms of any possible design method.

In the sixth aspect, this application provides a computer program product containing instructions that, when the instructions run on a computer or processor, cause the computer or processor to execute any possible design as in the first aspect or any possible design in the first aspect. In the method.

Using the technical solution of the present application, after the electronic device decodes the video code stream to obtain the video frame, it generates frame-level protection information corresponding to the video frame. After that, the electronic device updates the frame-level protection information of the video frame according to the change in the resolution of the video frame. Therefore, before outputting the to-be-displayed video frame, the electronic device can perform output protection on the to-be-displayed video frame according to the frame-level protection information corresponding to the to-be-displayed video frame. In this way, protection is performed at the level of the video frame. When the video frame changes, the electronic device can update the output protection information corresponding to the video frame in time, so that the electronic device can provide more accurate output protection, and avoid the video frame and the output protection strategy from being out of synchronization. The protection failure problem, improve the performance of the protection.

Description of the drawings

In order to explain the technical solutions of the present application more clearly, the following will briefly introduce the drawings needed in the embodiments. Obviously, for those of ordinary skill in the art, without paying creative labor, Other drawings can also be obtained from these drawings.

FIG. 1A is a system architecture diagram of a typical electronic device provided by this application;

FIG. 1B is a schematic diagram of the first exemplary application scenario of output protection provided by this application;

FIG. 1C is a schematic diagram of a second exemplary application scenario of output protection provided by this application;

FIG. 2 is a schematic diagram of the system architecture of the electronic device 10 provided by the present application;

Fig. 3 is a schematic diagram of an exemplary structure of frame-level protection information provided by the present application;

FIG. 4 is an exemplary method flowchart of the safe output method 100 provided by the present application;

FIG. 5 is a schematic diagram of an exemplary application scenario of the safe output method 100 provided by the present application;

FIG. 6A is a schematic diagram of an exemplary system architecture of an electronic device 20 provided by the present application;

FIG. 6B is a schematic diagram of the transmission flow of frame-level protection information provided by this application;

FIG. 7A is a schematic diagram of an exemplary structure of a safety output device 70 provided by the present application;

FIG. 7B is a schematic diagram of an exemplary structure of the safety output device 71 provided by the present application.

Detailed ways

The technical solutions in this application will be clearly described below in conjunction with the drawings in this application.

The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also Including plural expressions, unless the context clearly indicates to the contrary. It should also be understood that although the terms first, second, etc. may be used in the following embodiments to describe a certain type of object, the object should not be limited to these terms. These terms are only used to distinguish specific objects of this class of objects. For example, in the following embodiments, the terms first, second, etc. may be used to describe the path identification, but the path identification should not be limited to these terms. These terms are only used to distinguish the identification of different media channels. In the following embodiments, the terms first, second, etc. may be used to describe other types of objects in the same way, which will not be repeated here.

The architecture and business scenarios described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in this application. Those of ordinary skill in the art will know that as the architecture evolves and new With the emergence of business scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

The following describes the implementation scenarios of this application.

This application can be applied to electronic devices that support audio and video input/output, such as smart phones, smart set-top boxes, smart TVs, surveillance, computers, tablet computers, etc.

Figure 1A shows the system architecture of a typical electronic device. The system architecture includes: application layer, driver layer, operating system layer, hardware module and storage module. The application layer is used to run application software, such as Tencent, Youku, etc. The driver layer includes a driver program written for each hardware module, and the driver program is used to drive the corresponding hardware module to access the memory block. The operating system layer is responsible for memory management, stack management, task scheduling management, etc. When the application software of the application layer is running, it can apply to the operating system layer to allocate a memory block and obtain the virtual address of the allocated memory block. The allocated memory block is used to store data during the running of the application software, such as the following data such as video frames. The driver layer can drive the hardware module to access the corresponding memory block according to the virtual address of the memory block. The "access" involved in this application includes read operations and write operations. For example, the hardware module can process the data read from the input memory block in response to the instruction of the driver, and then write the processed data into the output memory block.

Before the application software transmits the media content, it can create a media path based on the application software's intention and the processing process of the media content. The media path is the path through which the application software processes the media content on the electronic device. The processing process includes but is not limited to Play, record, transcode, and forward, etc. Correspondingly, the media path may include, for example, a recording path, a playback path, and a transcoding path. The media path includes the resources used for audio and video processing, such as a series of hardware modules and memory.

With reference to FIG. 1A, by way of example, the application software may apply to the operating system layer to occupy the hardware module 1, the hardware module 2 and the hardware module 3, and apply to the operating system layer to allocate the memory block 1 and the memory block 2. Hardware module 1, hardware module 2, and hardware module 3, as well as memory block 1 and memory block 2, for example, can form a media channel, and the transmission process of the media stream on the media channel can be, for example: hardware module 1 to memory block 1, memory Block 1 to hardware module 2, hardware module 2 to memory block 2, and memory block 2 to hardware module 3. In actual operation, the driver layer receives instructions from the application software, and then drives the corresponding hardware modules in the media path to perform access operations on the corresponding memory blocks and process the corresponding media data.

The electronic device shown in FIG. 1A may, for example, support a trusted execution environment (TEE) and a rich execution environment (REE). TEE corresponds to REE. TEE is used to provide a protected execution environment for protected application software, and REE is used to provide an execution environment for unprotected application software. The media content transmitted under the TEE may be processed in accordance with media content usage rules (content usage rules, CUR), for example, to realize the protection of the media content. Correspondingly, the hardware modules and memory blocks of the media channel under TEE should process and transmit relevant data in accordance with CUR. Exemplarily, CUR may include protection policies such as HDCP policy, video watermark protection policy, prohibition of transcoding, and prohibition of recording.

The following takes HDCP strategy and video watermark protection strategy as examples. Among them, the HDCP strategy and the video watermark protection strategy act on the video output stage. Therefore, this application collectively refers to the HDCP strategy and the video watermark protection strategy as an "output protection strategy" or an "output control strategy". Exemplarily, taking video output as an example, the electronic device first decrypts the video source, that is, the encrypted video code stream, to obtain the video code stream to be decoded. Then, the electronic device decodes the video code stream to be decoded to obtain a video frame. After that, the electronic device can output the video frame through the hardware interface. Among them, the electronic device can adopt the HDCP strategy to provide protection to the decoded video frame, and the electronic device can add a watermark to the video frame to realize the watermark protection of the video frame.

In actual implementation, a software system that performs output protection on video content includes a condition access system (CAS) or a digital rights management (digitial rights management, DRM) system. In this application, CAS or DRM is expressed as "CAS/DRM", where "/" means "or" in this application. The "CAS/DRM" mentioned in this manual all mean CAS or DRM.

Among them, the HDCP strategy is a technology to prevent the illegal recording of high-definition digital media content. The HDCP policy can include HDCP1.4 protection level and HDCP2.2 protection level. Among them, HDCP1.4 protection level and HDCP2.2 protection level allow different resolutions. For example, the resolution supported by HDCP1.4 protection level is usually 1080 Progressive scan (P), that is, 1080P, for example, indicates that the number of pixels per frame is 1920*1080, and the resolution supported by HDCP2.2 protection level is usually 4K, and 4K, for example, indicates that the number of pixels per frame is 4096*2160. , "K", for example, is used to represent 2 to the 10*4 power. When the HDCP strategy is actually used for protection, it is required that the software and hardware for processing and displaying video frames support the HDCP strategy. Otherwise, the electronic device is prohibited from outputting the corresponding video frame or outputting the corresponding video frame at a reduced resolution.

The digital high definition multimedia interface (HDMI) is a commonly used hardware interface that supports the HDCP strategy. At present, HDMI commonly used by equipment includes HDMI1.0, HDMI1.3, HDMI1.4 and HDMI2.0, among which different versions of HDMI support different levels of HDCP. For example, HDMI1.3 supports HDCP1.4 Protection level, HDMI2.0 version supports HDCP2.2 protection level. Before outputting the video frame, the HDMI of the electronic device can determine the HDCP protection status of the output video frame according to the HDCP support status of the connected interface. For example, the HDMI of the electronic device supports HDCP2.2 and HDCP1.4, and the interface to which the HDMI is connected only supports HDCP1.4, then the electronic device can output video frames at a resolution corresponding to the HDCP1.4 protection level. Correspondingly, the HDCP protection state of the output video frame determined by the HDMI of the electronic device is HDCP1.4.

The video watermark protection strategy refers to adding the required watermark information to the output video frame to identify the source of the video frame, etc. The watermark information may include information such as the address, content identifier, and time stamp of the video source to which the video frame belongs. Among them, the electronic device usually adds watermark information to the video frame according to the display position and size of the video frame.

As shown in Figure 1B, the TEE application layer decrypts the encrypted video code stream, and then writes the decrypted video code stream into the memory block accessed by the decoding module in the corresponding media channel. Based on this, in an existing implementation, before decrypting the TEE application layer, HDMI determines the supported HDCP protection status from the connected device and sends it to the TEE application layer. Then, the TEE application layer determines the HDCP protection status and Whether the HDCP protection level corresponding to the video stream matches. If the HDCP protection status does not match the HDCP protection level of the video stream, for example, the HDCP protection level of the video stream is HDCP2.2, and the HDCP protection status supports HDCP1.4 protection. Then, the TEE application layer will not decrypt the encrypted video stream, and then the electronic device outputs a mute signal. Among them, output mute refers to outputting pure color video frames such as all black video frames or all blue video frames. If the HDCP protection status matches the HDCP protection level of the code stream, the TEE application layer will decrypt the encrypted code stream. Furthermore, the electronic device outputs a video frame corresponding to the video code stream. It can be seen that, in this solution, the electronic device has only two states of decryption and non-decryption for the encrypted video stream, and it will not adapt the output protection strategy of the video stream, and the applicability is poor.

As shown in FIG. 1C, before the electronic device outputs the video frame, the video display module in the corresponding media channel reads the to-be-displayed video frame from the memory block m, for example, and then configures the to-be-displayed video frame according to the output protection policy. The output protection strategy includes, for example, the output resolution corresponding to the video frame to be displayed, the location of the watermark, and the size of the watermark. In some embodiments, the output protection policy in the video display module is configured by the TEE application layer, and after the output protection policy is configured in the video display module, the corresponding output protection policy is immediately configured for the read video frame to be displayed. Based on this, on the one hand, the memory block m can buffer part of the video frames to be displayed. If the output protection strategy configured in the video display module is changed, the video frames that have been cached in memory block m correspond to the output protection strategy before the change, but the video display module will configure the output protection strategy that has been cached in memory block m according to the changed output protection strategy. The video frame to be displayed, which causes the output protection to fail. On the other hand, the to-be-displayed video frame stored in the memory block m may be a processed video frame, and the processing mechanism of the video frame is usually controlled by the REE application layer. Therefore, if the position and size of the video frame change, the TEE application layer cannot perceive it. Furthermore, the TEE application layer cannot update the output protection strategy in the video display module in time, which will also cause the output protection to fail. For example, it is not allowed in accordance with HDCP. The resolution output video frame, watermark adding error, etc.

Based on the above, the present application provides a safe output method and electronic device, which generates frame-level protection information corresponding to video frames, so that before outputting the video frame to be displayed, the video frame to be displayed can be displayed according to the frame-level protection information corresponding to the video frame to be displayed. Frame output protection. In this way, protection at the level of the video frame can provide more accurate output protection, and when the video frame changes, the corresponding output protection information can be updated in time, thereby improving the performance of the protection.

The software and hardware structure of the electronic equipment involved in this application is introduced below.

FIG. 2 illustrates a schematic diagram of a system architecture of an exemplary application environment of the electronic device 10. The electronic device 10 supports TEE. The following describes the REE system architecture and TEE system architecture respectively in conjunction with Figure 2.

The REE system architecture includes REE software application layer, REE software interface adaptation layer, REE software driver layer, REE software operating system layer, REE hardware module and REE storage module. The software application in the REE software application layer can control the REE hardware module through the REE software interface adaptation layer, the REE software driver layer and the REE software operating system layer, and access the memory blocks in the REE storage module. Among them, it should be understood that the REE software operating system layer, the REE software driver layer, the REE software interface adaptation layer, and the REE software application layer are implemented by software codes. Illustratively, these software codes can be stored in the memory and run. On the processor.

Among them, the REE software application layer is used to provide the operating environment of the REE software application, and is also used to apply to the REE software operating system layer to allocate the REE hardware module and REE memory block when the REE software application is running, and to store the virtual address of the REE memory block.

The REE software interface adaptation layer is used to match the driver in the REE software driver layer according to the virtual address accessed by the REE software application layer.

The REE software driver layer includes a driver program written for each hardware module. The driver program drives the corresponding hardware module to access the corresponding REE memory block according to the virtual address.

The REE software operating system layer is used to perform resource management of REE hardware modules, as well as stack management and task scheduling. The REE software operating system layer can, for example, respond to instructions from the REE software application layer to configure the REE hardware module to create a media path, which is used to transmit media data without security requirements. In addition, the REE software operating system layer is also used to maintain one or more computer programs and data. When the one or more computer programs are running, they can realize the functions of each software layer on the REE side. The data is used to provide support for the operation of the one or more computer programs.

The REE storage module may include, but is not limited to, double data rate (DDR) memory, flash memory (Flash), static random access memory (static random access memory, SRAM), etc., which are not limited in this application. The REE storage module includes multiple memory blocks, and each memory block has a different physical address.

The TEE system architecture includes TEE software application layer, TEE software interface adaptation layer, TEE software driver layer, TEE software operating system layer, TEE hardware module and TEE storage module. Among them, the TEE software operating system layer, the TEE software driver layer, the TEE software interface adaptation layer, and the TEE software application layer are implemented by software codes. Illustratively, these software codes can be stored in the memory and run on the processor. superior.

Among them, the software application in the TEE software application layer can control the TEE hardware module through the TEE software interface adaptation layer, the TEE software driver layer and the TEE software operating system layer, and access the memory block in the TEE storage module. It should be understood that the basic interaction process between the software layers is similar to the REE side, and will not be detailed here.

In this application, the TEE software application layer may include the CAS/DRM trusted application (CAS/DRM trusted application, CAS/DRM TA), and the CAS/DRM TA may also be referred to as TA for short. TA can be used to configure the output protection strategy of the video stream.

In addition, in this application, the TEE system architecture also includes a session management module, which is implemented by software code and runs on the processor. In some embodiments, the session management module runs on the TEE software driver layer, for example. In other embodiments, the session management module runs on the TEE software operating system layer, for example. There is no restriction here.

Wherein, the session management module may be used to configure a protection information memory block in the process of creating a media channel in the TEE system architecture, and the protection information memory block is used to store the frame-level protection information corresponding to the video frame transmitted in the media channel. The frame-level protection information includes output protection information corresponding to the video frame. The session management module can also be used to configure the initial information of the frame-level protection information in each protection information memory block. The initial information can include the type of the frame-level protection information, the virtual address of the memory block of the video frame corresponding to the frame-level protection information, and all the information. The corresponding relationship between the length of the virtual address and the initial check value of the frame-level protection information are described. For the embodiments of the frame-level protection information, please refer to the following description, which will not be described in detail here.

In addition, the session management module can also be used to configure the channel identifier for the media channel created by the TEE system architecture, and store the correspondence between the virtual address of the memory block belonging to the media channel and the channel identifier.

The TEE storage module may include multiple secure memory blocks, where each secure memory block can be identified by a physical address. Exemplarily, in the process of creating a media path for video output, part of the secure memory block may be configured as a memory block for storing video frames, and another part of the secure memory block may be configured as a memory block for storing frame-level protection information. And is written into the initial information of the frame-level protection information. In this application, the memory block used to store video frames may be referred to as a data memory block, and the memory block used to store frame-level protection information is referred to as a protected information memory block.

In this application, the hardware module can respond to instructions from the TEE software driver layer or the TEE software operating system layer to detect the validity of the frame-level protection information in the protection information memory block before accessing the protection information memory block. The hardware module may also detect whether the protected information memory block and the memory block corresponding to the read operation correspond to the same media path before performing the write operation on the protected information memory block.

It is understandable that, in some embodiments, the processor running each software layer of REE and the processor running each software layer of TEE may be physically the same processor. When the processor is running in REE mode, The processor implements the functions of each software layer of the REE. When the processor runs in the TEE mode, the processor implements the functions of each software layer of the TEE. The processor can be, for example, a system-level chip control logic unit, a microprocessor, a microcontroller (micro-controller unit, MCU), a central processing unit (CPU), a digital signal processing (digital signal processing, DSP) ), graphics processing unit (GPU), field programmable gate array (FPGA), application specific integrated circuit (ASIC), etc., which are not limited in this application.

The REE hardware module and the TEE hardware module shown in FIG. 2 may include, for example, a demux module (demux), a hardware decryption module, a hardware encryption module, a hardware decoding module (decoder), a video decoding module (video decoder, VDEC), and hardware Communication module, hardware video processing module (video processor, VPSS), hardware display module (video display, VDP), analog to digital converter (analog to digital converter, ADC), digital to analog converter (digital to analog converter, DAC), The communication interface, radio frequency unit, and microelectronic mechanical module, etc., are not limited in this application. Among them, the REE hardware module and the TEE hardware module can be implemented by two processing channels in the same hardware module.

Both the REE storage module and the TEE storage module may include, but are not limited to, DDR memory, flash memory (Flash), SRAM, etc., which are not limited in this application. The REE storage module and the TEE storage module are isolated from each other, so that the ordinary memory block and the secure memory block are isolated from each other.

The physical address (physical address) described in the embodiment shown in FIG. 2 may be: storing information in a byte as a unit in the storage module. In order to store or obtain information correctly, each byte unit has a unique memory block address. The physical address can also be called the actual address or the absolute address. The physical address can be addressed in the storage module through the address bus, and is the address where the data is actually stored.

The virtual address described in the embodiment illustrated in FIG. 2 may be a logical address used by application software to access the memory block. The virtual address does not actually store the data, but needs to be mapped to the actual physical address to obtain the data.

In some embodiments, the software and hardware of the REE system architecture and the software and hardware of the TEE system architecture shown in FIG. 2 may be located in the same system on chip (SOC). In other embodiments, the REE software application layer, REE software interface adaptation layer, REE software driver layer, REE software operating system layer, REE hardware module, and TEE software application layer, TEE software interface adaptation layer, The TEE software driver layer, the TEE software operating system layer, and the TEE hardware module are located in the same SOC, and the REE storage module and TEE storage module can be independent of the SOC.

It can be understood that FIG. 2 is only a schematic description, and does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

The following describes the safe output method of the present application in conjunction with the electronic device 10.

It is understandable that the safe output method described in this application is implemented in TEE. Correspondingly, the safe output method described in this application is executed by at least one of software, hardware, and a combination of software and hardware on the TEE side of the electronic device 10.

First, the frame-level protection information involved in the embodiments of the present application is introduced. Fig. 3 illustrates an exemplary structure of frame-level protection information. As shown in Fig. 3, frame-level protection information may include tag information, frame buffer information of video frames, resolution information of video frames, HDCP protection policy (HDCP policy) information, watermark control information (watermark control), extended output policy (output control extend) information, and initial checksum (checksum). Wherein, the memory block information of the video frame may specifically include, for example, the storage address of the video frame, the length information of the storage address, and the path identifier. The resolution information of the video frame may specifically include the current resolution and the minimum resolution, for example. The HDCP protection strategy may specifically include, for example, a protection strategy of No HDCP (NoHDCP), a protection strategy of HDCP1.4, and a protection strategy of HDCP2.2. The watermark control information may specifically include watermark parameter information and memory block information of the watermark parameter, for example.

The frame-level protection information shown in FIG. 3 includes various types of information, specific information contained in various types of information, and information descriptions of each specific piece of information. See Table 1 for details.

Table 1

Among them, the type of frame-level protection information is used to indicate whether the frame-level protection information is bound to a video frame. Exemplarily, if the type of frame-level protection information in Table 1 is frame, it means that the frame-level protection information is bound to the video frame, and the frame-level protection information contains the memory block information and resolution information of the bound video frame . If the type of frame-level protection information in Table 1 is global, it means that the frame-level protection information is not bound to the video frame. Correspondingly, the memory block information and resolution information of the video frame in the frame-level protection information can be empty. In addition, when the type of frame-level protection information is global, the frame-level protection information can be applied to all video frames that have lost frame-level protection information. In order to facilitate the distinction, this application refers to this type of protection information as "global protection information". ". The output resolution in the global protection information can be included in the extended output strategy information.

According to the above description of the secure memory block, the TEE system architecture is configured with a data memory block and a protection information memory block when the media channel is created. Correspondingly, the memory block information of the video frame in Table 1 is used to establish the correspondence between the frame-level protection information and the video frame bound to the frame-level protection information.

The resolution information of the video frame in Table 1 can be updated with the update of the video frame. For details, refer to the description of the following embodiments, which will not be described in detail here.

The watermark control information in Table 1 is used to maintain information such as the watermark type and the storage address of the watermark when a watermark needs to be added to the bound video frame. Among them, the storage address of the watermark is used to indicate the memory block where the watermark information is stored. The watermark information includes, for example, the content of the watermark information, the position added to the video frame, the relative size of the video frame, and so on. For example, the content of the watermark information is, for example, the address of the video source, and the position added to the video frame is, for example, from the lower left direction to the upper right direction of the video frame. The relative size of the video frame includes, for example, the watermark length is 80% of the longitudinal length of the video frame. , The width is 10% of the vertical length of the video frame.

The checksum in Table 1 is used to verify whether the frame-level protection information has been tampered with. For details, refer to the description of the following embodiments, which will not be described in detail here.

According to the description of the above embodiment, the corresponding relationship between the tag, the storage address of the video frame and the length of the storage address, the path identifier and the checksum in Table 1 can be the session management module added to the protection information memory when the protection information memory block is configured. In the block. The tag, the storage address of the video frame, the length of the storage address, and the corresponding relationship and checksum are the initial information of the frame-level protection information.

It is understandable that Table 1 is only a schematic description, and does not limit the frame-level protection information involved in this application. In some other embodiments, the frame-level protection information may also contain more or less information.

Refer to Fig. 4, which illustrates a safe output method 100 (hereinafter referred to as the method 100). The method 100 includes the following steps:

Step S101: In a stage where the first decoded video frame is obtained by decoding the video bitstream, the frame-level protection information of the first decoded video frame is generated according to the output control strategy and the resolution of the first decoded video frame.

Among them, the output control strategy is a protection strategy that matches the protection requirements of the video stream. The electronic device can learn the resolution of the first decoded video frame before decoding the video code stream. For example, the electronic device may learn the resolution of the first decoded video frame according to the video code stream.

Exemplarily, after the TEE software application layer of the electronic device receives the encrypted video code stream, it decrypts the encrypted video code stream to obtain the video code stream and the protection requirements corresponding to the video code stream. The protection requirements are, for example, 1080 progressive scan ( 1080progressive scan, 1080p), output the video corresponding to the video stream. After that, the TEE software application layer can configure the output control strategy according to the protection requirements. The output control strategy includes the video width and height allowed when the NoHDCP protection level is adopted, the video width and height allowed when the HDCP1.4 protection level is adopted, and the video width and height allowed when the HDCP2.2 protection level is adopted. After that, the electronic device decodes the video code stream to obtain the first decoded video frame, and further, the electronic device writes the resolution of the first decoded video frame into the protection information memory block corresponding to the first decoded video frame to obtain the first decoded video Frame-level protection information of the frame.

Wherein, the frame-level protection information of the first decoded video frame includes the resolution and minimum resolution of the first decoded video frame, and the minimum resolution is the minimum resolution corresponding to the video stream in the entire media path. It should be understood that the minimum resolution is usually set in the initial information of the frame-level protection information. The resolution of the first decoded video frame is the resolution of the currently decoded video frame. If the resolution of the current decoded video frame is less than the initially set minimum resolution , The minimum resolution in the frame-level protection information is modified to the resolution of the currently decoded video frame (that is, the resolution of the first decoded video frame).

Step S102: Determine whether to output the first display video frame according to the frame-level protection information.

Wherein, the first display video frame is obtained according to the first decoded video frame. In some embodiments, the first decoded video frame obtained by decoding may be directly used as an output. In this case, the first displayed video frame is the first decoded video frame. In other embodiments, the electronic device may process the first decoded video frame to obtain the output first display video frame.

Exemplarily, in some embodiments, the electronic device may amplify the first decoded video frame to obtain the first display video frame. In other embodiments, the electronic device may reduce the first decoded video frame to obtain the first display video frame.

It should be pointed out that if the electronic device performs an operation of zooming in or out on the first decoded video frame, the resolution of the first decoded video frame will also change accordingly. Based on this, the resolution of the first display video frame is different from the resolution of the first decoded video frame. Furthermore, after obtaining the first display video frame, the electronic device may update the frame-level protection information according to the resolution of the first display video frame. Exemplarily, when the resolution of the first display video frame is less than the minimum resolution in the frame-level protection information, the minimum resolution in the frame-level protection information is modified to the resolution of the first display video frame to obtain the first display Frame-level protection information for video frames.

Further, the electronic device may obtain the output protection state allowed by the output port, and then determine the target output control strategy that matches the output protection state from the frame-level protection information of the first displayed video frame. After that, the electronic device can determine whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy. When the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, the electronic device outputs The first video frame is displayed. When the minimum resolution in the frame-level protection information is greater than the resolution allowed by the target output control strategy, the electronic device outputs mute. The output protection state of the output port is, for example, the HDCP protection state. For the embodiments related to the output protection state of the output port of the output port, and the embodiment of mute, refer to the description of the foregoing embodiment for details, and will not be described in detail here.

With this implementation, when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, the electronic device can output the first display video frame according to the resolution allowed by the target output control strategy. It can be seen that every time the electronic device outputs a video frame, the video frame can be output protected according to the frame-level protection information corresponding to the video frame, so that the security is higher. When the minimum resolution in the frame-level protection information is greater than the resolution allowed by the target output control strategy, it means that the resolution of the first display video frame does not meet the output control strategy, and the electronic device outputs mute. That is, when the resolution of the video frame exceeds the resolution allowed by the output control strategy, the output of the video frame will be prohibited. It can be seen that this implementation can effectively protect the output of the high-definition video frame. The protection information in the embodiments of this application is at the frame level, and each video frame has its own corresponding frame-level protection information. Each output frame of video frame can be performed on the video frame according to the frame-level protection information corresponding to the video frame. Output protection, higher safety and flexibility.

The embodiments of the video frame with frame-level protection information have been described above. In other embodiments, if the frame-level protection information of the first display video frame is lost, the electronic device may determine whether to output the first display video frame according to the resolution in the global protection information. Wherein, the resolution in the global protection information is the minimum resolution in the output strategy corresponding to at least one code stream. The global protection information may be generated according to at least one code stream while the electronic device generates frame-level protection information, and the at least one code stream may include the video code stream. The global protection information is described in the embodiment shown in FIG. 3, and will not be described in detail here.

In addition, the display position and size of the first display video frame are controlled by the user through the REE side. Based on this, in some other embodiments, if the first display video frame needs video watermark protection, before outputting the first display video frame, the electronic device may also obtain the position and size of the first display video frame from the REE driver. Then, the electronic device can determine the memory block storing the watermark information according to the storage address information of the watermark in the frame-level protection information, and then read the watermark information from the corresponding memory block. After that, the electronic device can determine the position and size of the watermark information to be added according to the position and size of the first display video frame, the relative position of the watermark information and the first display video frame, and the relative size of the watermark information and the first display video frame. .

It can be seen that, with this implementation, the electronic device can determine the position and size of the watermark to be added according to the position and size of the video frame during display, so as to ensure that the watermark is accurately added and displayed with the change of the video frame. In addition, the electronic device directly corresponds the video frame to the watermark information of the video frame through the frame-level protection information, which can avoid the problem of failure to add watermark due to the display level of the video frame in a scene where multiple videos are displayed at the same time.

In summary, in the safe output method provided by this application, after the electronic device decodes the video code stream to obtain the video frame, it generates frame-level protection information corresponding to the video frame. After that, the electronic device updates the frame-level protection information of the video frame according to the change in the resolution of the video frame. Therefore, before outputting the to-be-displayed video frame, the electronic device can perform output protection on the to-be-displayed video frame according to the frame-level protection information corresponding to the to-be-displayed video frame. In this way, protection is performed at the level of the video frame. When the video frame changes, the electronic device can update the output protection information corresponding to the video frame in time, so that the electronic device can provide more accurate output protection, and avoid the video frame and the output protection strategy from being out of synchronization. The protection failure problem, improve the performance of the protection.

The method 100 is exemplarily introduced below.

Wherein, before performing the above step S101, the TEE software application layer in FIG. 2 may request the TEE software operating system layer and the session management module to create a media path through the TEE software interface adaptation layer. The created media path can be used to output protected video content. Correspondingly, the media path includes, for example, VDEC, VDP, and output port, etc., and also includes at least one data memory block and at least one protection information memory block. At least one data memory block is used to store video-related data transmitted by the media path, at least A protection information memory block is used to store the frame-level protection information corresponding to the video frame transmitted by the media channel.

In this embodiment, the session management module sets a path identifier A for the media path, and the path identifier A is used to indicate the media path. The session management module may also maintain the correspondence between at least one data memory block and the path identifier A. In addition, the session management module may configure the initial information of the frame-level protection information in the at least one protection information memory block. In this embodiment, the initial information includes, for example, the type of the frame-level protection information is frame, the path identifier A, and the correspondence between the address and length of the data memory block of the video frame bound to the frame-level protection information.

Further, as shown in Figure 5, the TA running on the application layer of the TEE software with CAS/DRM can receive the encrypted video stream, and then decrypt the encrypted video stream to obtain the video stream. Then, the TA can configure the output control strategy according to the protection requirements corresponding to the video code stream. After that, the TA can write the video code stream into the data memory block 01. In addition, in some embodiments, the TA can configure the output control strategy to VDEC. In other embodiments, the TA may configure the output control policy in a stream output control policy buffer. Among them, VDEC has read operation authority for the code stream output control strategy memory block.

Furthermore, VDEC can respond to the instruction of the TEE software driver layer to read the video code stream from the data memory block 01, and then decode the video code stream to obtain the first decoded video frame. After that, VDEC can write the first decoded video frame into the data memory block 02. VDEC can also read the output control strategy from the code stream output control strategy memory block, and then write the output control strategy and the resolution of the first decoded video frame into the protection information memory block 01 to obtain the frame level of the first decoded video frame Protect information. The frame-level protection information in the protection information memory block 01 includes the resolution and the minimum resolution of the first decoded video frame. VDEC knows the resolution of the first decoded video frame before performing the decoding operation.

In this application, before the VDEC writes the first decoded video frame into the data memory block 02, it can detect whether the video code stream and the first decoded video frame correspond to the same media channel. After determining that the video code stream and the first decoded video frame correspond to the same media path, the first decoded video frame is written into the data memory block 02. This can avoid the problem of accessing data of other media channels when VDEC outputs the first decoded video frame, thereby improving the security of video frame transmission.

Exemplarily, the VDEC may obtain the first path identifier corresponding to the data memory block 01 and the second path identifier in the frame-level protection information of the first decoded video frame. When the first path identifier and the second path identifier are the same, it can be determined that the video bitstream and the frame-level protection information of the first decoded video frame correspond to the same media path.

For example, the data memory block 01 corresponds to the path identifier A, and the path identifier in the frame-level protection information of the first decoded video frame is pre-configured by the session management module, that is, the second path identifier is the path identifier A. Based on this, the first path identifier should be the same as the second path identifier.

In addition, VDEC can detect whether the initial information in the protection information memory block 01 is valid before writing the output control strategy and the resolution of the first decoded video frame into the protection information memory block 01. After determining that the initial information in the protection information memory block 01 is valid, VDEC writes the output control strategy and the resolution of the first decoded video frame into the protection information memory block 01. For an embodiment of VDEC detecting whether the initial information is valid, see the following description, which will not be described in detail here.

In some embodiments, the first decoded video frame is the first display video frame. Further, the VDP can respond to instructions from the TEE software driver layer to read frame-level protection information from the protection information memory block 01, and obtain the HDCP protection status from the output port. Then, VDP can determine the target output control strategy that matches the HDCP protection status from the frame-level protection information. In this embodiment, the target output control strategy is, for example, an HDCP1.4 output control strategy. Then, VDP judges whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the HDCP1.4 output control strategy. When the minimum resolution in the frame-level protection information is less than the resolution allowed by the HDCP1.4 output control strategy, the first display video frame is output through the output port.

In this embodiment, before reading the frame-level protection information from the protection information memory block 01, the VDP needs to check whether the frame-level protection information in the protection information memory block 01 is valid, and then determine the frame-level protection information in the protection information memory block 01. After the protection information is valid, read the frame-level protection information from the protection information memory block 01. The process by which the VDP detects whether the frame-level protection information in the protection information memory block 01 is valid is described in the following, and will not be described in detail here.

In other embodiments, the media path may also include VPSS (not shown in FIG. 5). After VDEC writes the first decoded video frame into the data memory block 02, and writes the output control strategy and the resolution of the first decoded video frame into the protection information memory block 01, the VPSS can respond to the instruction of the TEE software driver layer, from the data The memory block 02 reads the first decoded video frame and processes the first decoded video frame. For example, the processing may be reduction processing or enlargement processing, for example, shrinking the first decoded video frame to obtain the first display video frame and the first display video frame Resolution. After that, the VPSS can write the first display video frame into the data memory block 03 (not shown in FIG. 5), and update the protection information memory block 02 (not shown in FIG. 5) according to the resolution of the first display video frame. The frame-level protection information in the first display video frame is obtained. Exemplarily, the resolution of the first display video frame is smaller than the resolution of the first decoded video frame, for example, the VPSS may write the resolution of the first display video frame into the protection information memory block 02, and store the protection information memory block 02 The minimum resolution of the frame-level protection information is updated to the resolution of the first display video frame, and the frame-level protection information of the first display video frame is obtained. Correspondingly, in this embodiment, the VDP reads the frame-level protection information of the first display video frame from the protection information memory block 02 to determine whether to output the first display video frame. No more details here.

In this embodiment, before the VPSS writes the first display video frame into the data memory block 03, it can detect whether the first decoded video frame and the first display video frame correspond to the same media channel. After determining that the first decoded video frame and the first display video frame correspond to the same media channel, the first display video frame is written into the data memory block 03. This can avoid the problem of accessing data of other media channels when the VPSS outputs the first display video frame, thereby improving the security of video frame transmission.

Exemplarily, the VPSS may obtain the third path identifier in the level protection information in the protection information memory block 01 and the fourth path identifier in the level protection information in the protection information memory block 02. When the third channel identifier and the fourth channel identifier are the same, it is determined that the first decoded video frame and the first displayed video frame correspond to the same media channel.

For example, the third path identifier is pre-configured by the session management module, that is, the third path identifier should be the path identifier A. The fourth path identifier is pre-configured by the session management module, that is, the fourth path identifier should also be the path identifier A, so the third path identifier should be the same as the fourth path identifier.

In addition, before the VPSS reads the frame-level protection information from the protection information memory block 01, it needs to check whether the frame-level protection information in the protection information memory block 01 is valid. After determining that the frame-level protection information in the protection information memory block 01 is valid, the VPSS reads the frame-level protection information from the protection information memory block 01. In addition, before updating the frame-level protection information in the protection information memory block 02, the VPSS needs to check whether the frame-level protection information in the protection information memory block 02 is valid. After determining that the frame-level protection information in the protection information memory block 02 is valid, the VPSS updates the frame-level protection information in the protection information memory block 02.

It should be pointed out that in this application, VDEC, VPSS, and VDP detect whether the frame-level protection information is valid. In essence, they all detect at least one piece of initial protection information in the frame-level protection information, and determine whether at least one of the detected frame-level protection information is valid. Under the condition that the initial protection information is valid, it is determined that the corresponding frame-level protection information is valid. In actual operation, for example, at least one item of information in the tag, the storage address of the video frame, and the length of the storage address in the frame-level protection information is detected, and at least one item of information in the checksum. In the following, combining the above three exemplary information, taking VDP as an example, the operation of detecting validity will be exemplarily introduced.

Detection tag: VDP can detect whether the tag of the frame-level protection information of the first displayed video frame matches the pre-configured tag. If the tag of the frame-level protection information of the first display video frame matches the pre-configured tag, the VDP can read the frame-level protection information according to the processing logic indicated by the instruction, and perform operations such as display processing.

According to the embodiment illustrated in FIG. 3, the tag is used to indicate whether the frame-level protection information is bound to the video frame. If the frame-level protection information is bound to a video frame, then the VDP should perform an operation on the first display video frame according to the resolution of the video frame in the frame-level protection information. If the frame-level protection information is not bound to the video frame, then the frame-level protection information is global protection information, and the VDP should perform operations on the first display video frame according to the resolution in the extended output strategy. It can be seen that when corresponding to different types of frame-level protection information, the information read by the VDP and the operation logic are different.

Based on this, using this implementation method, after determining the correctness of the frame-level protection information type, VDP can process the first display video frame according to the frame-level protection information according to the matching processing logic.

Detecting the correspondence between the storage address of the video frame and the length of the storage address: VDP can detect whether the correspondence between the storage address of the first display video frame and the length of the storage address in the frame-level protection information is correct. If the corresponding relationship between the storage address of the first display video frame and the length of the storage address is correct, it indicates that the frame-level protection information is the frame-level protection information corresponding to the first display video frame. If the corresponding relationship between the storage address of the first display video frame and the length of the storage address is incorrect, it indicates that the frame-level protection information is not the frame-level protection information corresponding to the first display video frame.

With this implementation method, VDP can ensure whether the frame-level protection information is associated with the video frame, thereby ensuring that the protection information of the corresponding video frame is read from the frame-level protection information, and further, realizing accurate output protection for the video frame.

Check checksum: VDP can determine whether the check value of the frame-level protection information is the same as the initial check value. If the check value of the frame-level protection information is the same as the initial check value, it indicates that the frame-level protection information has not been tampered with. If the check value of the frame-level protection information is different from the initial check value, it indicates that the frame-level protection information has been tampered with.

Exemplarily, the VDP may perform an exclusive OR operation on all information in the frame-level protection information to obtain the first result. Then, the VDP can XOR the first result with the secure random number to obtain the check value. After that, VDP compares whether the check value is the same as the initial check value.

Combined with the electronic device architecture shown in Figure 2, both the REE driver and the TEE driver have a driving effect on the hardware module. Although the frame-level protection information is stored in the secure memory block, the REE driver can control the tampering and forgery of the frame-level protection information by hardware modules that have access rights to the secure memory block. Based on this, this application sets up a checksum detection mechanism.

This embodiment relates to the feature of "secure random number", and the secure random number is a random number of the mask register of the electronic device. The mask register is, for example, a 32-bit register. When the electronic device is powered on, the TEE side can read a 32-bit random number from the hardware random number module, write it into the mask register and latch it. The REE side has no access rights to the mask register. Based on this, the 32-bit random number in the mask register is relatively safe and can be called a "secure random number".

Further, the session management module can determine the initial checksum value in the frame-level protection information according to the algorithm checksum=(Word0^Word1^...Wordn)^Mask, where n is an integer greater than or equal to 1, and Word0 to Wordn refer to frames Each information in the level protection information, Mask refers to a secure random number. When VDP detects checksum, it can calculate the check value of the current frame-level protection information according to the same algorithm. Since the secure random number cannot be tampered with, if the calculated check value is the same as the checksum, it indicates that the frame-level protection information has not been tampered with. If the calculated checksum is different from the checksum, it means that the frame-level protection information has been tampered by the REE side, and the VDP can output mute.

It can be seen that with this implementation method, VDP can perform output protection on the first display video frame according to the frame-level protection information while ensuring that the frame-level protection information is not damaged, forged or tampered, thereby realizing the realization of the first display video frame. Correct output protection, optimize the performance of output protection.

It is understandable that the above algorithm for calculating the check value is only a schematic description, and does not constitute a specific limitation on the determination of the check value. In some other embodiments of the present application, the electronic device session management module and each hardware module may use other mechanisms and other algorithms to determine the check value of the frame-level protection information.

In some embodiments, the VDP can only determine whether the frame-level protection information is valid through the above checksum. For example, when the checksum is the same as the check value, VDP determines that the frame-level protection information is valid. In other embodiments, the VDP can determine whether the frame-level protection information is valid through any two of the above three pieces of information. When the detected two pieces of information meet the conditions, VDP determines that the frame-level protection information is valid. In some other embodiments, for example, when the detection results of the above three items of information all meet the conditions, the VDP determines that the frame-level protection information is valid. In addition, the detection result of any of the above three pieces of information has no influence on the detection of other information. Correspondingly, the detection sequence of the above three items of information can be arbitrary, which is not limited in this application.

In addition, in this application, the detection of the validity of the frame-level protection information by VDEC and the detection of the validity of the frame-level protection information by VPSS are similar to the detection process of the above-mentioned VDP, except that the storage address of the video frame and the storage address are detected. In the scene of the length correspondence, the involved video frame and the storage address of the video frame may be different from the VDP. For example, VDEC should detect whether the corresponding relationship between the storage address of the first decoded video frame and the length of the storage address is correct. No more details here.

It is understandable that the foregoing operation of detecting the validity of frame-level protection information is only a schematic description, and does not constitute a restriction on the validity of the detection of frame-level protection information in this application. In other embodiments of the present application, there may be more or less information related to the validity of frame-level protection information. There is no restriction here.

The technical solution of the present application will be introduced below in conjunction with examples.

FIG. 6A provides a structural diagram of an electronic device 20, and the electronic device 20 supports TEE. The TEE side of the electronic device 20 includes a software part and a hardware part. The software part includes TEE application, session management module (session manager) and TEE driver module. TEE applications include TA. It should be understood that the software part is a functional module implemented by software instructions or software codes, and these software instructions or software codes run on the processor to implement corresponding functions. The hardware part includes VDEC, VPSS, VDP and HDMI. The hardware part also includes TEE storage module.

In this embodiment, the TEE application runs on the TEE software application layer in the electronic device 10. The session management module and the TEE driver module run on the TEE software driver layer of the electronic device 10, for example.

It can be understood that FIG. 6A is only an exemplary description of the electronic device of the present application, and does not constitute any limitation to the electronic device involved in the present application. In some other embodiments, the electronic device involved in this application may include more or fewer hardware modules. Accordingly, the electronic device involved in this application may include hardware modules with other functions. In addition, the functional software of the electronic device can also adopt other forms of expression. No more details here.

It can be understood that FIG. 6A is only a schematic description, and does not constitute a specific limitation on the electronic device 20. In other embodiments of the present application, the electronic device 20 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

Further, with reference to FIG. 6A, the operation process of the present application is exemplarily introduced.

Create target media channels

The TEE application can send a request to create a target media path to the session management module. Wherein, the target media channel is used to transmit a video code stream, for example. Furthermore, the session management module can create the target media channel according to the purpose of the target media channel, and configure the channel identifier Z for the target media channel.

Exemplarily, the target media path includes VDEC, VPSS, VDP, and HDMI, as well as secure memory block A, secure memory block B1, secure memory block B2, secure memory block C1, and secure memory block C2. Among them, the secure memory block A, the secure memory block B1, and the secure memory block C1 are used to store video-related data. The secure memory block B2 and the secure memory block C2 are used to store frame-level protection information. The frame-level protection information in the secure memory block B2 corresponds to the video frame data in the secure memory block B1. The frame-level protection information in the secure memory block C2 corresponds to the video frame data in the secure memory block C1.

In this embodiment, the session management module can maintain the correspondence between the secure memory block A, the secure memory block B1, and the secure memory block C1 and the path identifier Z. In addition, the session management module can also configure the initial information of the frame-level protection information in the secure memory block B2 and the initial information of the frame-level protection information in the secure memory block C2. Exemplarily, the initial information of the frame-level protection information in the secure memory block B2 includes the tag is frame, the virtual address of the secure memory block B1 and the corresponding relationship between the length of the virtual address, and the initial verification of the frame-level protection information in the secure memory block B2 value. The initial information of the frame-level protection information in the secure memory block C2 includes the tag is frame, the virtual address of the secure memory block C1 and the corresponding relationship of the virtual address length, and the initial check value of the frame-level protection information in the secure memory block C2. Exemplarily, the secure memory block B2 is used to store the frame-level protection information 01, and the secure memory block C2 is used to store the frame-level protection information 02.

With reference to the media path illustrated in FIG. 6A, FIG. 6B illustrates an exemplary transmission process of frame-level protection information. Among them, the control flow on the TEE side is shown by the dotted arrow on the TEE side in Figure 6B, the data transmission flow on the TEE side is shown by the solid arrow on the TEE side in Figure 6B, and the control flow on the REE side is shown by the solid line on the REE side in Figure 6B. The arrow shows.

Get video stream

After the target media channel is created, the REE application obtains the video code stream source, and after determining that the video code stream source is a protected video code stream source, the REE application transmits the protected video code stream source to the TA. The video stream source is the encrypted video stream. TA decrypts the source of the video code stream, and obtains the video code stream and the corresponding protection requirements of the video code stream. After that, the TA configures the output control strategy corresponding to the video code stream according to the protection requirements. Among them, the output control strategy is as described in the foregoing embodiment, and will not be described in detail here. In this embodiment, the display control driver on the TEE side configures the output control strategy configured by the TA to the VDEC, and the TA writes the video code stream into the secure memory block A. Among them, the display control drive is a kind of TEE drive module.

Decode the video stream to get the decoded video frame

VDEC responds to the instruction of the TEE driver module and reads the video stream from the secure memory block A. Then, VDEC decodes the video code stream to obtain decoded video frames. In addition, VDEC can obtain the resolution of the decoded video frame before decoding, and the resolution of the decoded video frame is, for example, resolution 01.

After that, VDEC obtains the path identifier corresponding to the secure memory block A, and obtains the path identifier Z. VDEC also obtains the path identifier in the initial information in the secure memory block B2, and also obtains the path identifier Z. According to the path identifier corresponding to the secure memory block A and the path identifier in the initial information in the secure memory block B2, it is determined that both the video code stream and the decoded video frame correspond to the target media path, and the VDEC writes the decoded video frame into the secure memory block B1. The process of VDEC from reading the video code stream to writing the decoded video frame into the secure memory block B1 is not shown in FIG. 6B.

In addition, in conjunction with Figure 6B, VDEC detects whether the initial information in the secure memory block B2 is valid. Exemplarily, VDEC detects whether the tag of the initial information in the secure memory block B2 is a frame, whether the correspondence between the virtual address of the secure memory block B1 and the length of the virtual address is correct, and the check value of the initial information in the secure memory block B2 Is it the same as the initial check value? In an optional case, when the detection results of the above three items of information are all "Yes", it indicates that the initial information in the secure memory block B2 is valid, and VDEC writes the resolution 01 to the "current" in the secure memory block B2. The information bits corresponding to the “resolution” and the “minimum resolution” obtain the frame-level protection information 01 of the decoded video frame.

Process decoded video frame to get display video frame

As shown in Figure 6B, the user can input instructions for processing and decoding video frames through the REE application layer. In this embodiment, the REE application layer can call the VPSS driver to send specific processing instructions to the VPSS. Correspondingly, the VPSS reads the decoded video frame from the secure memory block B1, and then performs a shrinking operation on the decoded video frame to obtain a display video frame (not shown in FIG. 6B).

In this embodiment, in conjunction with FIG. 6B, the VPSS also needs to detect whether the frame-level protection information 01 in the secure memory block B2 is valid. If the frame-level protection information 01 in the secure memory block B2 is valid, VPSS can obtain the resolution 01 from the frame-level protection information 01 in the secure memory block B2, and then, according to the resolution 01 and the display video frame is reduced relative to the decoded video frame The multiple of to determine the resolution of the displayed video frame 02. In this embodiment, the resolution 02 is smaller than the resolution 01, for example. The operation process of VPSS detecting whether the frame-level protection information 01 in the secure memory block B2 is valid is similar to the operation process of VDEC detecting whether the initial information in the secure memory block B2 is valid, and will not be repeated here.

After that, the VPSS can obtain the path identifier of the frame-level protection information 01 in the secure memory block B2 to obtain the path identifier Z. In addition, the VPSS can also obtain the path identifier of the initial information in the secure memory block C2 to obtain the path identifier Z. According to the channel identification of the frame-level protection information 01 in the secure memory block B2 and the initial information in the secure memory block C2, it is determined that both the decoded video frame and the displayed video frame correspond to the target media channel, and VPSS writes the displayed video frame into the secure memory Block C1.

In addition, VPSS detects whether the initial information in the secure memory block C2 is valid. Exemplarily, VPSS can detect whether the tag of the initial information in the secure memory block C2 is frame, whether the correspondence between the virtual address of the secure memory block C1 and the length of the virtual address is correct, and the frame-level protection information in the secure memory block C2 02 Whether the check value of is the same as the initial check value. After determining that the initial information in the secure memory block C2 is valid, VPSS writes the resolution 02 into the information bit corresponding to the "current resolution" in the secure memory block C2. Since the resolution 02 is smaller than the resolution 01, the VPSS also needs to update the information bit corresponding to the "minimum resolution" in the secure memory block C2 to the resolution 02 to obtain the frame-level protection information 02 of the displayed video frame.

Output display video frame

VDP reads the display video frame from the secure memory block C1.

The VDP also needs to check whether the frame-level protection information 02 in the secure memory block C2 is valid. If the frame-level protection information 02 in the secure memory block C2 is valid, VDP reads the frame-level protection information 02 from the secure memory block C2. Exemplarily, in this embodiment, the HDCP monitor module (HDCP monitor) in the VDP may read the frame-level protection information 02. In this embodiment, the VDP also obtains the HDCP protection status supported by HDMI from HDMI. The HDCP protection status includes the HDCP protection level allowed by HDMI. In this embodiment, the operation process of VDP detecting whether the frame-level protection information 02 in the secure memory block C2 is valid is similar to the operation process of VPSS detecting whether the initial information in the secure memory block C2 is valid, and will not be repeated here.

It should be pointed out that the operation of VDP to read the frame-level protection information 02 and the operation of VDP to obtain the HDCP protection status supported by HDMI are two separate operation processes. Therefore, the execution sequence of the two operation processes can be arbitrary, which is not limited in this application.

Furthermore, VDP determines a target output protection strategy that matches the HDCP protection status from the frame-level protection information 02. The target output protection strategy is, for example, HDCP1.4. Then, VDP detects whether the resolution 02 in the frame-level protection information 02 is less than the resolution allowed by HDCP1.4. If the resolution 02 is less than the resolution allowed by HDCP1.4, VDP can output and display video frames through HDMI according to the resolution allowed by HDCP1.4. If the resolution 02 is greater than the resolution allowed by HDCP1.4, VDP outputs mute through HDMI.

With reference to Figure 6B, in other implementation scenarios, if the frame-level protection information 02 also includes watermark control information, after VDP determines that the display video frame can be output, it can also obtain the position and position of the display video frame from the video output driver on the REE side. Then, read the watermark information from the watermark storage address in the frame-level protection information 02, and then add the watermark information to the display video frame according to the position and size of the display video frame. Among them, the video output driver on the REE side is one of the software driver modules on the REE side.

It can be understood that the embodiments involved in FIG. 6A and FIG. 6B are only schematic descriptions, and do not constitute a limitation to the technical solution of the present application. In some other embodiments, the target media path may also include more or fewer hardware modules and memory blocks. The operation process of each hardware module can also be different from the above description. The protection information in each memory block can also be different from the above description. No more details here.

In addition, this specification does not show all implementation scenarios applicable to this application. In other implementation scenarios, adopting other implementation methods based on the technical ideas of this application also belongs to the protection category of this application.

In summary, in the technical solution provided by this application, after the electronic device decodes the video code stream to obtain the video frame, it generates frame-level protection information corresponding to the video frame. After that, the electronic device updates the frame-level protection information of the video frame according to the change in the resolution of the video frame. Therefore, before outputting the to-be-displayed video frame, the electronic device can perform output protection on the to-be-displayed video frame according to the frame-level protection information corresponding to the to-be-displayed video frame. In this way, protection is performed at the level of the video frame. When the video frame changes, the electronic device can update the output protection information corresponding to the video frame in time, so that the electronic device can provide more accurate output protection, and avoid the video frame and the output protection strategy from being out of synchronization. The protection failure problem, improve the performance of the protection.

In the foregoing embodiment, the solutions of the safe output method provided in the present application are introduced from the perspective of the physical structure of the electronic device hardware, the software architecture, and the actions performed by each software and hardware. Those skilled in the art should easily realize that in combination with the establishment of the correspondence relationship described in the embodiments disclosed herein and the execution of the output processing steps according to the correspondence relationship, this application can not only be implemented in the form of hardware or a combination of hardware and computer software . Whether certain functions are executed by hardware or computer software-driven hardware depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each of the above specific applications, but such implementation should not be considered as going beyond the scope of the embodiments of the present application.

For example, the above-mentioned electronic device 10 and the electronic device 20 may implement the above-mentioned part of the functions in the form of functional modules. As shown in FIG. 7A, the safety output device 70 may include a decoding module 701 and a display control module 702. The safety output device 70 can be used to implement some or all of the embodiments of the safety output method in any of the embodiments shown in FIG. 4.

For example, the decoding module 701 is configured to generate frame-level protection information of the first decoded video frame according to the output control strategy and the resolution of the first decoded video frame, and the first decoded video frame is the video code of the decoding module. The stream is decoded. The display control module 702 is configured to determine whether to output a first display video frame according to the frame-level protection information, and the first display video frame is obtained according to the first decoded video frame.

It can be seen that, every time a video frame is output by the safety output device 70 provided by the present application, the video frame can be output protected according to the frame-level protection information corresponding to the video frame, thereby achieving higher security.

Optionally, the safety output device 70 may also include a processing module, an update module, a TA, a detection module, a watermark adding module, an acquisition module, and a session management module. In different embodiments, the above-mentioned modules are used to implement different functions.

For example, in some embodiments, the processing module may be used to process the first decoded video frame to obtain the first display video frame. The update module is configured to update the frame-level protection information according to the resolution of the first display video frame. Wherein, the processing module is specifically configured to enlarge the first decoded video frame to obtain the first display video frame; or, specifically, to reduce the first decoded video frame to obtain the first display video frame. Wherein, the frame-level protection information includes the resolution and minimum resolution of the first decoded video frame, and the minimum resolution is the minimum resolution corresponding to the video code stream in the entire media path. In this embodiment, the update module is specifically configured to modify the minimum resolution in the frame-level protection information to the first display when the resolution of the first display video frame is less than the minimum resolution. The resolution of the video frame.

Optionally, the display control module 702 is specifically configured to obtain the output protection status allowed by the output port, determine a target output control strategy that matches the output protection status; determine whether the minimum resolution in the frame-level protection information is less than The resolution allowed by the target output control strategy; and when the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, output the first display video frame.

Optionally, TA may be used to configure the output control strategy for the video code stream.

Optionally, the decoding module 701 is further configured to determine whether the video bitstream and the first decoded video frame correspond to the same media path. Wherein, the decoding module 701 is specifically configured to obtain the first path identifier corresponding to the video code stream and the second path identifier in the frame-level protection information of the first decoded video frame. When the second channel identifiers are the same, it is determined that the video code stream and the frame-level protection information of the first decoded video frame correspond to the same media channel. The processing module is also used to determine whether the first decoded video frame and the first display video frame correspond to the same media channel. Wherein, the processing module is used to obtain the third path identifier in the frame-level protection information of the first decoded video frame, and the fourth path identifier in the updated frame-level protection information, in the third path identifier and When the fourth channel identifiers are the same, it is determined that the first decoded video frame and the first display video frame correspond to the same media channel.

Optionally, the detection module is used to detect whether the frame-level protection information is valid. The detection module is specifically configured to perform at least one of the following detections: detecting whether the type of the frame-level protection information matches a pre-configured type, and the type is used to indicate whether the frame-level protection information is bound to a video frame, and determining Whether the check value of the frame-level protection information is the same as the initial check value, and whether the correspondence between the storage address of the first display video frame and the length of the storage address in the frame-level protection information is correct.

Optionally, the detection module is specifically configured to perform an exclusive OR operation on all information in the frame-level protection information to obtain a first result, and perform an exclusive OR operation on the first result and a secure random number to obtain the frame The check value of the level protection information, and compare whether the check value of the frame level protection information is the same as the initial check value.

Optionally, the watermark adding module is configured to add a watermark to the first display video frame according to the frame-level protection information and the position and size of the first display video frame.

Optionally, the display control module 702 is further configured to determine whether to output the first display video frame according to the resolution in the global protection information when the frame-level protection information of the first display video frame is lost. The resolution in the protection information is the minimum resolution in the output strategy corresponding to at least one code stream.

Optionally, the acquisition module is configured to acquire a request for creating a media path, and the media path is used to transmit the video code stream. In this embodiment, the session management module may be used to generate the path identifier of the media path. The configuration module is also used to configure the type of protection information in the frame-level protection information, the correspondence between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value .

For specific content, reference may be made to the method 100 and related descriptions in the embodiment corresponding to FIG. 6B, which will not be repeated here.

It is understandable that, in some embodiments, the decoding module 701 illustrated in FIG. 7A can implement the functions of the VDEC in FIG. 2, FIG. 5, and FIG. 6A. The display control module 702 illustrated in FIG. 7A can implement the functions of the VDP in FIG. 2, FIG. 5, and FIG. 6A. The display processing module illustrated in FIG. 7A can implement the functions of the VPSS in FIG. 2, FIG. 5, and FIG. 6A. The other functional modules shown in FIG. 7A can realize the functions of the software layer in FIG. 2 and FIG. 6A. For example, the acquisition module can implement the functions of the operating system layer of the TEE software. For details, refer to the description of the electronic device 10 or the electronic device 20 in the foregoing embodiment, which is not described in detail here.

It can be understood that the division of the above modules is only a division of logic functions, and in actual implementation, the functions of the above modules can be integrated into the processor for implementation. As shown in FIG. 7B, the safety output device 71 includes a processor 711, a transmission interface 712, and a memory 713. It should be understood that the transmission interface 712 may include an input interface and an output interface, or that the transmission interface 712 has functions of an input interface and an output interface at the same time, which is not limited in the embodiment of the present application. Among them, the memory 713 may be used to store programs/codes pre-installed in the safety output device 71, and may also store codes used for execution by the processor 711, and the like. The transmission interface 712 may perform operations of determining the HDCP state in the method 100 and outputting the first display video frame. The processor 711 may perform operations in the method 100 except for determining the HDCP state and outputting the first display video frame.

In specific implementation, the application also provides a computer storage medium corresponding to the electronic device. The computer storage medium set in any device can store a program. When the program is executed, it can implement each of the safe output methods provided by the method 100. Part or all of the steps in the embodiment. The storage medium in any device can be a magnetic disk, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

Anyone skilled in the art can also understand that the various illustrative logical blocks and steps listed in the embodiments of this application can be implemented by electronic hardware, computer software, or a combination of the two. Whether such a function is implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art can use various methods to implement the function for each specific application, but such implementation should not be understood as going beyond the protection scope of the embodiments of the present application.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the procedures or functions according to the application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instruction may be passed from a website, a computer, a server, or a message center. Wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server or message center for transmission. The computer-readable storage medium may be any available medium that can be accessed by a computer or a message storage device such as a server or a message center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

It should be understood that in the various embodiments of the present application, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, rather than the implementation process of the embodiment. Constitute any limitation.

Each part of this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments.

Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, then this application is also intended to include these modifications and variations.

Claims

A safe output method, characterized in that the method includes:

In the stage of decoding the video code stream to obtain the first decoded video frame, generating the frame-level protection information of the first decoded video frame according to the output control strategy and the resolution of the first decoded video frame;

Determine whether to output the first display video frame according to the frame-level protection information, the first display video frame being obtained according to the first decoded video frame.
The method according to claim 1, wherein, after the first decoded video frame is obtained by decoding the video bitstream, the method further comprises:

Processing the first decoded video frame to obtain the first display video frame;

The frame-level protection information is updated according to the resolution of the first display video frame.
3. The method of claim 2, wherein said processing said first decoded video frame to obtain said first display video frame comprises:

Amplify the first decoded video frame to obtain the first display video frame; or,

The first decoded video frame is reduced to obtain the first display video frame.
The method according to claim 2 or 3, wherein the frame-level protection information includes the resolution and the minimum resolution of the first decoded video frame, and the minimum resolution is the entire video code stream The corresponding minimum resolution in the media path, the updating the frame-level protection information according to the resolution of the first display video frame includes:

When the resolution of the first display video frame is less than the minimum resolution, the minimum resolution in the frame-level protection information is modified to the resolution of the first display video frame.
The method according to any one of claims 1 to 4, wherein the determining whether to output the first display video frame according to the frame-level protection information comprises:

Obtain the output protection status allowed by the output port;

Determining a target output control strategy that matches the output protection state;

Judging whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy;

When the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, output the first display video frame.
The method according to any one of claims 1 to 5, wherein before decoding the video bitstream to obtain the first decoded video frame, the method further comprises:

The trusted application TA configures the output control strategy for the video code stream.
The method according to any one of claims 2 to 6, wherein the method further comprises:

Determining whether the video code stream and the first decoded video frame correspond to the same media channel;

It is determined whether the first decoded video frame and the first display video frame correspond to the same media channel.
The method of claim 7, wherein:

The determining whether the video code stream and the first decoded video frame correspond to the same media path includes:

Acquiring the first path identifier corresponding to the video code stream and the second path identifier in the frame-level protection information of the first decoded video frame;

When the first path identifier and the second path identifier are the same, determining that the video bitstream and the frame-level protection information of the first decoded video frame correspond to the same media path;

The determining whether the first decoded video frame and the first display video frame correspond to the same media path includes:

Acquiring the third path identifier in the frame-level protection information of the first decoded video frame and the fourth path identifier in the updated frame-level protection information;

When the third channel identifier and the fourth channel identifier are the same, it is determined that the first decoded video frame and the first display video frame correspond to the same media channel.
The method according to any one of claims 1 to 8, wherein before determining whether to output the first display video frame according to the frame-level protection information, and before determining whether to output the first display video frame according to the resolution of the first display video frame Before updating the frame-level protection information, it also includes:

Detect whether the frame-level protection information is valid.
The method of claim 9, wherein the detecting whether the frame-level protection information is valid comprises:

Check at least one of the following:

Detecting whether the type of the frame-level protection information matches a pre-configured type, where the type is used to indicate whether the frame-level protection information is bound to a video frame;

Determine whether the check value of the frame-level protection information is the same as the initial check value; or

It is detected whether the corresponding relationship between the storage address of the first display video frame and the length of the storage address in the frame-level protection information is correct.
The method of claim 10, wherein the judging whether the check value of the frame-level protection information is the same as the initial check value comprises:

Performing an exclusive OR operation on all the information in the frame-level protection information to obtain the first result;

Performing an exclusive OR operation between the first result and a secure random number to obtain the check value of the frame-level protection information;

Compare whether the check value of the frame-level protection information is the same as the initial check value.
The method according to any one of claims 1 to 11, wherein the method further comprises:

Add a watermark to the first display video frame according to the frame-level protection information and the position and size of the first display video frame.
The method of claim 1, further comprising:

If the frame-level protection information of the first display video frame is lost, determine whether to output the first display video frame according to the resolution in the global protection information, where the resolution in the global protection information corresponds to at least one bit stream The minimum resolution in the output strategy.
The method according to any one of claims 1 to 13, characterized in that, before decoding the video bitstream to obtain the first decoded video frame, the method further comprises:

Acquiring a request for creating a media path, where the media path is used to transmit the video code stream;

Generating a path identifier of the media path;

Configure the type of protection information in the frame-level protection information, the correspondence between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address, and the initial check value.
A safety output device, characterized in that the device comprises:

The decoding module is used to generate the frame-level protection information of the first decoded video frame according to the output control strategy and the resolution of the first decoded video frame, and the first decoded video frame is used by the decoding module to perform Decoded

The determining module and the display control module are configured to determine whether to output a first display video frame according to the frame-level protection information, and the first display video frame is obtained according to the first decoded video frame.
The device according to claim 15, wherein the device further comprises:

A processing module, configured to process the first decoded video frame to obtain the first display video frame;

The update module is configured to update the frame-level protection information according to the resolution of the first display video frame.
The device of claim 16, wherein:

The processing module is specifically configured to amplify the first decoded video frame to obtain the first display video frame; or,

The processing module is specifically configured to shrink the first decoded video frame to obtain the first display video frame.
The device according to claim 16 or 17, wherein the frame-level protection information includes the resolution and the minimum resolution of the first decoded video frame, and the minimum resolution is the entire The corresponding minimum resolution in the media channel,

The update module is specifically configured to modify the minimum resolution in the frame-level protection information to the resolution of the first display video frame when the resolution of the first display video frame is less than the minimum resolution Rate.
The device according to any one of claims 15 to 18, wherein the display control module is specifically configured to:

Obtain the output protection state allowed by the output port, and determine a target output control strategy that matches the output protection state;

Judging whether the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy;

When the minimum resolution in the frame-level protection information is less than the resolution allowed by the target output control strategy, output the first display video frame.
The device according to any one of claims 15 to 19, wherein the device further comprises a trusted application TA, which is used to configure the output control strategy for the video stream.
The device according to any one of claims 16 to 20, wherein:

The decoding module is further configured to determine whether the video code stream and the first decoded video frame correspond to the same media channel; and,

The processing module is further configured to determine whether the first decoded video frame and the first display video frame correspond to the same media channel.
The method of claim 21, wherein:

The decoding module is specifically used for:

Acquiring a first path identifier corresponding to the video code stream and a second path identifier in the frame-level protection information of the first decoded video frame;

When the first path identifier and the second path identifier are the same, determining that the video bitstream and the frame-level protection information of the first decoded video frame correspond to the same media path; and

The processing module is specifically also used for:

Acquiring the third path identifier in the frame-level protection information of the first decoded video frame and the fourth path identifier in the updated frame-level protection information;

When the third channel identifier and the fourth channel identifier are the same, it is determined that the first decoded video frame and the first display video frame correspond to the same media channel.
The device according to any one of claims 15 to 22, wherein the device further comprises a detection module,

The detection module is used to detect whether the frame-level protection information is valid.
The device of claim 23, wherein:

The detection module is specifically configured to perform at least one of the following detections:

Detecting whether the type of the frame-level protection information matches a pre-configured type, where the type is used to indicate whether the frame-level protection information is bound to a video frame;

Determine whether the check value of the frame-level protection information is the same as the initial check value; or

It is detected whether the corresponding relationship between the storage address of the first display video frame and the length of the storage address in the frame-level protection information is correct.
The device of claim 24, wherein:

The detection module is specifically used for:

Performing an exclusive OR operation on all the information in the frame-level protection information to obtain the first result;

Performing an exclusive OR operation between the first result and a secure random number to obtain the check value of the frame-level protection information;

Compare whether the check value of the frame-level protection information is the same as the initial check value.
The device according to any one of claims 15 to 25, wherein the device further comprises a watermark adding module, configured to: A watermark is added to the first display video frame.
The device of claim 15, wherein:

The display control module is further configured to determine whether to output the first display video frame according to the resolution in the global protection information when the frame-level protection information of the first display video frame is lost. The resolution of is the minimum resolution in the output strategy corresponding to at least one bitstream.
The device according to any one of claims 15 to 27, wherein the device further comprises:

An obtaining module, configured to obtain a request for creating a media path, where the media path is used to transmit the video code stream;

The session management module is used to generate the path identifier of the media path; configure the type of protection information in the frame-level protection information, the correspondence between the storage address of the video frame corresponding to the frame-level protection information and the length of the storage address Relationship, and the initial check value.
A safe output device, characterized in that the device includes a processor and a transmission interface, wherein:

The processor is configured to call program instructions stored in the memory to execute the safe output method according to any one of claims 1 to 14.