WO2021077882A1 - 一种图像显示方法与电子设备 - Google Patents

一种图像显示方法与电子设备 Download PDF

Info

Publication number
WO2021077882A1
WO2021077882A1 PCT/CN2020/110337 CN2020110337W WO2021077882A1 WO 2021077882 A1 WO2021077882 A1 WO 2021077882A1 CN 2020110337 W CN2020110337 W CN 2020110337W WO 2021077882 A1 WO2021077882 A1 WO 2021077882A1
Authority
WO
WIPO (PCT)
Prior art keywords
tee
ree
display
layer
graphic
Prior art date
Application number
PCT/CN2020/110337
Other languages
English (en)
French (fr)
Inventor
张静超
闫彦
王莲
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2021077882A1 publication Critical patent/WO2021077882A1/zh
Priority to US17/727,148 priority Critical patent/US11948233B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1637—Details related to the display arrangement, including those related to the mounting of the display in the housing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/03—Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
    • G06F2221/032—Protect output to user by software means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/03—Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
    • G06F2221/034—Test or assess a computer or a system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]

Definitions

  • This application relates to the technical field of security verification, and in particular to an image display method and electronic equipment.
  • TEE trusted execution environment
  • the existing REE+TEE architecture has two safe operation mechanisms:
  • Mechanism 1 When the application is running on the TEE, the REE controls the display interface. The user enters the password and other information in the display interface, and the REE sends the detected password to the TEE for security authentication.
  • Mechanism 2 When the application is running in the TEE, the TEE controls the display interface. The password entered by the user in the display interface is detected by the TEE and then security authentication is performed. The password input does not go through the REE. Among them, the display interface controlled by the TEE is called a trusted user interface (TUI).
  • TTI trusted user interface
  • the above mechanism one is based on the powerful drawing ability of REE, so when the application is running on the TEE, the display interface presented by the electronic device is relatively rich, but because the security of the REE is low, the password is easily stolen after the REE detects the password, and the security is low .
  • the above mechanism two solves the problem of low security in mechanism one, it has the following shortcomings:
  • the TUI controlled by the TEE is usually relatively dull; on the other hand, there are relatively few applications or services running in the TEE.
  • the applications or services running in the REE The related information cannot be displayed in the TUI. For example, when the electronic device controls the TUI, if the application in the REE receives a new message or notification, the TEE cannot display the new message and notification in the TUI, so the TUI visual interaction experience is poor.
  • TUI The user interaction operation on TUI is less sensitive. For example, only limited operation types (such as button clicks) can be recognized, and input operations such as full-screen gestures, up/down menus, etc. cannot be recognized and processed. Therefore, the current TUI gives users the feeling of inflexible touch response, and the user experience is poor.
  • limited operation types such as button clicks
  • input operations such as full-screen gestures, up/down menus, etc. cannot be recognized and processed. Therefore, the current TUI gives users the feeling of inflexible touch response, and the user experience is poor.
  • the purpose of this application is to provide an image display method and electronic equipment for improving the visual interaction experience of the TUI.
  • an image display method is provided, which is applied to electronic devices including a trusted execution environment TEE and a rich execution environment REE, such as mobile phones and tablet computers.
  • the method includes: TEE in the electronic device draws a first graphic, and displays the first graphic on the first layer in the display screen of the electronic device; REE draws the second graphic, and displays the second graphic on the second layer in the display screen. Graphics, where the first layer is located above the second layer; the first graphics includes transparent areas and non-transparent areas, the non-transparent areas include content drawn by TEE, and the display area corresponding to the transparent areas on the second graphic includes REE The drawn content, when the first layer and the second layer are overlapped and displayed, the transparent area does not block the content drawn by the REE.
  • the final display effect is that the display interface includes both the content provided by the TEE and the content provided by the REE.
  • the second graphic can include richer information, such as status bar, background, etc., so the final display interface, namely TUI, is richer, which solves the existing problems.
  • the TUI is dull.
  • the first layer and the second layer can be determined in a variety of ways, for example, method 1 and method 2 as follows.
  • Method 1 The touch screen in the electronic device is located on the upper layer of the display screen.
  • the display screen includes multiple layers.
  • the first layer is one or more layers close to the touch screen, and the second layer is one or more images far away from the touch screen.
  • Floor is one or more images far away from the touch screen.
  • each layer on the display screen is overlapped and displayed, and the layer located on the upper layer will obscure the layer located on the lower layer.
  • the layer on the upper layer is the layer close to the touch screen
  • the layer on the lower layer is the layer far away from the touch screen. Therefore, in the embodiment of the present application, the TEE displays the first graphic in one or more layers close to the touch screen in the display screen, and the REE displays the second graphic in one or more layers away from the touch screen in the display screen.
  • the transparent area in the first graphic does not block the content drawn by the REE in the second graphic.
  • the richness of the TUI is enhanced.
  • Method 2 The first layer is the layer closest to the touch screen in the display screen, and the second layer is the other layers in the display screen except the first layer.
  • TEE displays the first graphic on layer 1 on the display screen
  • REE is in layer 2-layer N
  • One or more of the layers display the second graphic. It is understandable that limited by the global standard platform (global platform, GP) standard specifications, TEE has limited processing capabilities, so if TEE only controls layer 1, it can alleviate the pressure of TEE drawing, and it can achieve rich TUI. effect.
  • GP global platform
  • the TEE can determine the display template corresponding to the current scene from one or more display templates according to the current scene; then the TEE draws the first graph according to the display template, and the REE draws the second graph according to the display template.
  • the current scene may be an application currently running in the TEE, or a business within an application currently running in the TEE.
  • REE uses the same display module to draw graphics, which can be implemented in the following way 1 to way 3.
  • Manner 1 After determining a suitable display template according to the current scene, the TEE sends the identification of the display template to the REE, and then the REE draws graphics according to the display template corresponding to the identification.
  • Method 2 After the TEE determines a suitable display template according to the current scene, it sends the display template to the REE, and the REE uses the display template to draw graphics.
  • TEE and REE can also jointly determine the display template. Specifically, the two can be negotiated and determined through information transmission.
  • the above three methods are only examples and are not limiting, and other methods that enable TEE and REE to use the same display template are also possible.
  • the transparent area in the first graphic drawn by the TEE corresponds to the content drawn by the REE on the second graphic, which enhances the richness of the display interface.
  • the display template can be implemented in multiple ways, such as way A and way B as follows.
  • the display template is a graphic, the graphic includes at least one display area, and each display area is configured as TEE drawing content or REE drawing content.
  • TEE and the REE can agree which area on the display module is drawn by the TEE and which area is drawn by the REE.
  • the TEE draws the first graphic according to the display template
  • the content is drawn in the area drawn by the TEE on the display module, and other areas are set as transparent areas.
  • the REE draws the second graphic according to the display template, the content is drawn in the area drawn by the REE on the display template. In this case, when the first graphic and the second graphic are displayed overlapped, the transparent area in the first graphic does not block the content drawn by the REE in the second graphic.
  • the display template includes at least one graph, each graph corresponds to a layer on the display screen, and each graph is configured as TEE drawing or REE drawing. It should be understood that assuming that TEE controls layer 1, REE draws layer 2-layer N, TEE draws the first figure according to the figure corresponding to layer 1, and REE draws the second figure according to the figure corresponding to layer 2-layer N. There is no need to agree which areas are drawn by TEE and which areas are drawn by REE.
  • the content drawn by the TEE may include; information related to password input. It should be understood that the less content the TEE draws, the more it can relieve the graphic drawing pressure of the TEE. Therefore, in this embodiment of the application, the TEE can only draw the information related to the password input, such as the numeric key value, etc.; other content can be drawn with the help of REE .
  • TEE can also detect user input and determine whether the user input requires a TEE response; when the user input does not require a TEE response, the user input is sent to the REE, and the REE executes the corresponding processing flow in response to the user input .
  • the display interface of the electronic device includes both the content provided by the TEE and the content provided by the REE.
  • the user may perform user input on the display interface, and the user input may be for the content provided by the TEE or for the content provided by the REE. Therefore, in the embodiment of the present application, when the TEE detects user input, it can determine whether the user input requires a TEE response. If not, the user input is sent to the REE, and the REE responds to the user input to improve the user's display interface ( That is, the diversity and sensitivity of interactive operations on the TUI.
  • the way for the electronic device to determine whether the user input requires a TEE response may be: when the TEE determines that the user input is an input in an application currently running in the TEE, it determines that the user input requires a TEE response; when the TEE determines that the user input is not an input in an application When determining that user input does not require a TEE response.
  • the display interface when the application is running in the TEE, the display interface includes the content provided by the TEE and REE; after the TEE detects the user input, if it is determined that the user input belongs to the input in the application (such as password input) When the TEE responds to the user input to ensure safety; when the TEE determines that the user input does not belong to the input in the application, the TEE sends the user input to the REE, and the REE responds to the user input to improve the user’s display interface ( That is, the diversity and sensitivity of interactive operations on the TUI.
  • the manner in which the TEE judges whether the user input is an in-app input or a non-in-app input may be at least one of the following methods:
  • the first graphic drawn by the TEE includes a transparent area and a non-transparent area, and the non-transparent area includes information drawn by the TEE. Therefore, the TEE determines whether the touch position of the user input is located in the non-transparent area, and if it is, it determines that the user input is an in-app input; if not, it determines that the user input is a non-in-app input. In other words, TEE only responds to user input whose touch position is located in a non-transparent area.
  • Method 2 TEE judges whether the operation type input by the user is a preset operation type.
  • the preset operation types include click operation, long-press operation, etc.; if yes, confirm that the user input is an in-app input; if not (for example, slide up, slide down, etc.) And so on), the user input is determined to be non-in-app input. That is, the TEE only responds to user input whose operation type is the preset operation type. It is understandable that gesture operations such as sliding up/down are usually not operations for inputting a password, so TEE will send REE to respond to gesture operations such as sliding up/down.
  • the above method 1 and method 2 can be used alone, or, in order to improve accuracy, method 1 and method 2 can be used in combination.
  • the input in the application includes: input in a non-transparent area, and/ Or, the operation type meets the input of the preset operation type; the non-in-app input includes: input located outside the non-transparent area, and/or, the operation type is not the input of the preset operation type.
  • REE when an application in REE receives a notification message, REE updates the second graphic, and the updated second graphic displays the notification message in the display area corresponding to the transparent area in the first graphic; REE is in the first graphic.
  • the updated second graph is displayed in the second layer.
  • the application displays the TUI when running in the TEE.
  • the TEE cannot obtain the notification message, and the REE does not participate in the drawing of the TUI, so The notification message cannot be displayed in the TUI.
  • the REE can update the second image, and the updated second image in the display area corresponding to the transparent area on the first image Including the notification message, so the notification message can be displayed in the TUI to enhance the richness of the TUI.
  • an electronic device which includes a trusted execution environment TEE and a rich execution environment REE; wherein the TEE includes a first processing module and a first display module, and the REE includes a second processing module and a second processing module.
  • Second display module the first processing module is used to draw the first figure; the first display module is used to display the first figure on the first layer in the display screen of the electronic device; the second processing module is used to draw the second Graphics; a second display module for displaying the second graphics on the second layer of the display screen, wherein the first layer is located above the second layer; wherein, the first graphics include transparent areas and non- A transparent area, the non-transparent area includes the content drawn by the TEE, the display area on the second graphic corresponding to the transparent area includes the content drawn by the REE, when the first layer and the first layer When the two layers are overlapped and displayed, the transparent area does not block the content drawn by the REE.
  • the touch screen in the electronic device is located on the upper layer of the display screen, the display screen includes multiple layers, and the first layer is one or more images close to the touch screen.
  • the second layer is one or more layers away from the touch screen.
  • the first layer is the layer closest to the touch screen in the display screen
  • the second layer is the one in the display screen excluding the first layer.
  • Other layers outside.
  • the first processing module is further configured to: determine a display template corresponding to the current scene from one or more display templates according to the current scene; the first processing module specifically uses In: drawing the first graphic according to the display template, and the second processing module is specifically configured to: drawing the second graphic according to the display template.
  • the display template is a graphic, and the graphic includes at least one display area, wherein each display area is configured as the TEE drawing content or the REE drawing content; or,
  • the display template includes at least one graphic, each graphic corresponds to a layer, and each graphic is configured for the TEE drawing or the REE drawing.
  • the content drawn by the TEE includes: information related to password input.
  • the first processing module is further configured to: detect user input, and determine whether the user input requires the TEE response; when the user input does not require the TEE response, pass the The first communication module in the TEE sends the user input to the REE, and the REE executes a corresponding processing flow in response to the user input.
  • the first processing module when used to determine whether the user input requires the TEE response, it is specifically used to: when it is determined that the user input is in an application currently running in the TEE When it is determined that the user input requires the TEE response; when it is determined that the user input is not an input in the application, it is determined that the user input does not require the TEE response.
  • the input in the application includes: the input located in the non-transparent area, and/or the input whose operation type meets the preset operation type; the input not in the application includes: The input located outside the non-transparent area, and/or the operation type is not the input of the preset operation type.
  • the first processing module is further configured to: when the application in the REE receives a notification message, update the second graphic, and the updated second graphic is compared with the first graphic.
  • the notification message is displayed in the display area corresponding to the transparent area in the graph; the second display module is further configured to display the updated second graph in the second layer.
  • an electronic device that is deployed with a trusted execution environment TEE and a rich execution environment REE, and further includes a display screen, a processor, and a memory; wherein the memory stores a computer program, and the computer The program includes instructions, and when the instructions are executed by the processor, the electronic device realizes the image display method provided in the first aspect.
  • a computer storage medium stores a computer program.
  • the computer program runs on an electronic device deployed with a trusted execution environment TEE and a rich execution environment REE, the electronic The device implements the image display method provided in the above first aspect.
  • a computer program including instructions, when the instructions are run on a computer deployed with a trusted execution environment TEE and a rich execution environment REE, the computer realizes the image display provided in the first aspect. method.
  • a chip that can run in a trusted execution environment TEE and a rich execution environment REE, and the chip is used to read computer programs stored in a memory to realize the image display provided in the first aspect. method.
  • a graphical user interface on an electronic device is also provided.
  • the electronic device is deployed with a trusted execution environment TEE and a rich execution environment REE, and further includes a display screen, one or more memories, and one or more processing
  • the one or more processors are configured to execute one or more computer programs stored in the one or more memories
  • the graphical user interface includes the image display method provided by the first aspect of the electronic device Graphical user interface displayed.
  • Figure 1 is a schematic diagram of the existing TEE+REE architecture
  • Figure 2 is a schematic diagram of the logical structure of the display screen
  • FIG. 3 is a schematic flowchart of an image display method provided by an embodiment of the application.
  • FIG. 4 is a schematic diagram of two display templates provided by an embodiment of the application.
  • FIG. 5 is a schematic diagram of a first graphic, a second graphic, and a merged graphic provided by an embodiment of the application;
  • FIG. 6 is a schematic diagram of a logical structure of a display screen provided by an embodiment of the application.
  • FIG. 7 is a schematic diagram of a display interface when an application in REE receives a notification message according to an embodiment of the application
  • FIG. 8 is a schematic diagram of a TEE+REE architecture provided by an embodiment of this application.
  • FIG. 9 is a schematic flowchart of an image display method provided by an embodiment of the application.
  • FIG. 10 is a schematic diagram of a logical structure of an electronic device provided by an embodiment of the application.
  • FIG. 11 is a schematic structural diagram of an electronic device provided by an embodiment of this application.
  • references described in this specification to "one embodiment” or “some embodiments”, etc. mean that one or more embodiments of the present application include a specific feature, structure, or characteristic described in conjunction with the embodiment. Therefore, the sentences “in one embodiment”, “in some embodiments”, “in some other embodiments”, “in some other embodiments”, etc. appearing in different places in this specification are not necessarily All refer to the same embodiment, but mean “one or more but not all embodiments” unless it is specifically emphasized otherwise.
  • the terms “including”, “including”, “having” and their variations all mean “including but not limited to”, unless otherwise specifically emphasized.
  • the electronic device involved in the embodiments of this application can be any electronic device that supports the REE+TEE architecture, such as mobile phones, tablets, laptops, palmtops, and mobile internet devices (MID) , Wearable devices (for example, watches, bracelets, smart helmets, etc.), virtual reality (VR) devices, augmented reality (AR) devices, ultra-mobile personal computers (UMPC) , Netbooks, personal digital assistants (PDA), wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, and smart Wireless terminals in a smart grid, wireless terminals in transportation safety, wireless terminals in a smart city, wireless terminals in a smart home, etc.
  • MID mobile internet devices
  • Wearable devices for example, watches, bracelets, smart helmets, etc.
  • VR virtual reality
  • AR augmented reality
  • UMPC ultra-mobile personal computers
  • PDA personal digital assistants
  • wireless terminals in industrial control wireless terminals in self-driving
  • wireless terminals in remote medical surgery and smart Wireless terminals
  • the electronic devices involved in the embodiments of this application can also be any electronic devices that support two or more operating environments.
  • electronic devices that support REE and TEE are taken as examples, but this is not To constitute a limitation, electronic equipment supporting the other two operating environments is also possible.
  • Rich execution environment refers to the operating environment in electronic equipment that does not have specific safety functions.
  • the Android operating system is a rich execution environment.
  • REE can also be called “untrusted execution environment”, “ordinary execution environment”, “insecure execution environment” and so on. This is not limited.
  • TEE Trusted execution environment
  • REE+TEE architecture refers to the architecture that provides services for applications through the combination of TEE and REE.
  • TEE and REE co-exist in electronic equipment.
  • TEE can realize an operating mechanism isolated from REE through the support of hardware.
  • TEE has its own operating space, which has a higher security level than REE, and can protect assets (such as data, software, etc.) in TEE from software attacks. Only authorized security software can be executed in the TEE, and it also protects the confidentiality of the security software's resources and data. Compared with REE, TEE can better protect the security of data and resources due to its isolation and access control protection mechanisms.
  • Client application usually refers to an application running in REE.
  • CA can call TA under certain conditions; for example, if a service involving user information in CA is triggered, CA calls TA.
  • the CA may call the TA through a client application programming interface (application programming interface, API).
  • API application programming interface
  • Trusted application refers to an application running in a TEE, which can provide security-related services for a client application (CA) running outside the TEE. It is understandable that one CA corresponds to one TA (for example, one TA provides different services for the corresponding CA), or one CA can correspond to multiple TAs (for example, different services in a CA correspond to different TAs), etc., The embodiments of this application are not limited.
  • Trusted user interface is a security function provided by the TEE operating system. It can be understood as a secure display interface controlled by the TEE, which can provide a trusted user interface for each TA running in the TEE.
  • the user interaction interface can ensure that when TA is running in TEE, the displayed content cannot be obtained by REE, which can improve the security of the business.
  • the global platform organization has standardized the commonly used APIs of TUI, so that TA developers can use these APIs to develop TUI. Through TUI, the three most basic safe user interaction functions of safe display, safe input, and safe indication can be realized. For example, in the financial sector, the central bank’s mobile phone shield specification clearly requires the use of TUI to ensure the security of the display interface for password input and signature content.
  • the TUI controlled by TEE is relatively dull and unable to present rich content.
  • Mobile phone shield is a product or application that uses a mobile phone to realize the bank's USB key (usbkey) function.
  • the mobile phone shield can implement related services in mobile phones that support the TEE+REE architecture.
  • the TEE in the mobile phone can provide the mobile phone shield with a high-security environment required for all functions such as cryptographic operations and CA digital certificates.
  • Mobile Shield provides cryptographic computing support capabilities for mobile Internet applications, which are used for identity authentication, electronic signatures, and data protection.
  • the mobile phone shield signature key is stored in the TEE, and all signature operations are performed in the TEE, thereby ensuring that the signature process reaches financial level security.
  • the user can perform some large-value transactions or other highly sensitive operations on the mobile phone.
  • the signature information is sent to the TEE, displayed through the TUI and confirmed by the user, and entered after the user confirms that it is correct
  • Mobile phone shield personal identification number (PIN) the PIN is sent to the signature applet in the TEE for verification. After the verification is passed, the transaction information confirmed by the user is sent to the applet for signature, and the digital signature is passed Technology ensures that transactions cannot be denied.
  • TUI has two important functions: 1. To ensure the security of the user's input PIN, only the mobile phone shield can be accessed, and the application on the REE side cannot obtain user input through screen reading, keyboard reading, etc. Ensure that the PIN will not be leaked, while also ensuring that the PIN is entered in real time. 2. To ensure that "what you see is what you sign" (or conversely, "what you sign is what you see"), that is, the transaction information sent to the mobile phone shield for signature will not be tampered with or replaced through TUI. In this way, the user After confirming that the transaction information is correct, the information sent to the TEE for signature is the information that the user has confirmed.
  • TEE Transaction information and user PINs may be obtained and tampered by malicious programs, and malicious programs/hackers can skip them.
  • the user conducts a transaction, or modifies the content of the transaction to commit fraud.
  • TUI plays an obvious defensive role against possible malicious attacks on the REE side.
  • TUI due to the weak drawing ability of TEE, TUI is relatively dull and unable to present rich content.
  • the display method of the embodiment of the present application aims to enhance the richness of the TUI.
  • the application (application, app) involved in the embodiments of the present application may be referred to as an application, which is a software program that can implement one or more specific functions.
  • an application which is a software program that can implement one or more specific functions.
  • multiple applications can be installed in an electronic device, for example, instant messaging applications, video applications, audio applications, image capturing applications, payment applications, and so on.
  • instant messaging applications for example, may include short message applications, WeChat (WeChat), WhatsApp Messenger, Line, photo sharing (instagram), Kakao Talk, DingTalk, etc.
  • Image shooting applications for example, may include camera applications (system cameras or third-party camera applications).
  • Video applications such as Youtube, Twitter, Douyin, iQiyi, Tencent Video, etc.
  • Audio applications may include Kugou Music, Xiami, QQ Music, and so on.
  • Payment applications such as Alipay, WeChat, various banking apps, etc.
  • the applications mentioned in the embodiments of the present application may be applications that have been installed when the electronic device leaves the factory, or may be applications that a user downloads from the network or obtains from other electronic devices during the process of using the electronic device.
  • At least one of the embodiments of the present application includes one or more; wherein, multiple refers to greater than or equal to two.
  • words such as “first” and “second” are only used for the purpose of distinguishing description, and cannot be understood as indicating or implying relative importance, nor can it be understood as indicating Or imply the order.
  • “And/or” describes the association relationship of the associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, and B exists alone.
  • the character “/” generally indicates that the associated objects before and after are in an "or” relationship.
  • FIG. 1 is a schematic diagram of the existing TEE+REE architecture.
  • the architecture includes TEE and REE; among them, CA can run in REE, and TA can run in TEE.
  • This architecture can implement the second mechanism mentioned above.
  • REE controls the display and touch screen
  • TEE controls the display and touch screen.
  • the CA can control the display screen through the display screen driver (DSD) in the rich execution environment operating system kernel layer (rich execution environment operation system kernel, REE OS kernel), and through the touch screen driver in the REE OS kernel ( touch screen driver, TSD) controls the touch screen.
  • the TA can control the display screen through the DSD in the trusted execution environment operating system kernel (TEE OS kernel), and control the touch screen through the TSD in the TEE OS kernel. That is, when the TA is running in the TEE, the display screen displays the display content provided/drawn by the TEE, and the TEE detects user operations on the touch screen (for example, input of a password, etc.).
  • the way TEE controls the display is: only control the first layer in the display.
  • Figure 2 is a schematic diagram of the logical structure of the display screen.
  • the display screen includes multiple layers (display screen), for example, layer 1-layer N, where N is an integer greater than or equal to 2; the touch screen is on the upper layer of layer 1.
  • layer 1-layer N where N is an integer greater than or equal to 2
  • the touch screen is on the upper layer of layer 1.
  • REE exclusively occupies the display screen, that is, REE controls all layers in the display screen.
  • TEE controls layer 1 in the display. That is to say, TEE only draws the first graphic, and then displays the first graphic on layer 1.
  • the final TUI presented is only Including the content on layer 1, that is, the display content drawn by TEE.
  • TEE has limited processing capabilities, so TUI is usually relatively dull, and can only display some text, buttons, text boxes, etc., and other content such as the status bar (power Information, operator information, time information, etc.) cannot be displayed.
  • the TEE controls the touch screen.
  • the user may trigger various user operations, such as full-screen gestures, slide-up/drop-down menus, and other operations.
  • These operations are related to the business in the REE, not related to the business in the TEE.
  • the TEE cannot send these operations to the REE, and the TEE itself cannot respond to these operations, which causes the user to trigger various operations on the TUI, but the TUI can only identify
  • the limited operation types (such as button clicks) cannot recognize and process other operations, which causes the existing TUI to fail to provide a rich human-computer interaction experience.
  • the related services of the CA in the REE cannot be transferred to the TEE.
  • the electronic device controls the display screen to display the TUI
  • the application running in the REE receives a new message or notification
  • the new The message or notification cannot be delivered to the TEE, so the TEE cannot display the new message and notification in the TUI, so the current interactive experience of the TUI is poor.
  • an embodiment of the present application provides an image display method, which can be applied to electronic equipment deployed with TEE and REE.
  • the display method is a fusion display mechanism of TEE and REE.
  • TEE controls layer 1 in the display screen, and REE controls layer 2 to layer N in the display screen.
  • TEE draws the first graphic, and displays the first graphic on layer 1
  • REE draws the second graphic, and displays the second graphic on one or more of Layer 2-Layer N; first
  • the graphics and the second graphics are merged (or called superimposition, overlap, etc.) to obtain the TUI.
  • the first graphic includes a non-transparent area and a transparent area.
  • the non-transparent area includes content drawn by the TEE, such as password input information.
  • the transparent area on the first graphic does not block the display content of the area corresponding to the transparent area on the second graphic. Therefore, the display effect after the fusion display of the first graphic and the second graphic is: the display interface, that is, the TUI, includes both the content drawn by the TEE and the content drawn by the REE. That is to say, with the help of REE's powerful drawing capabilities and rich display resources, the second graphic can include richer information, such as status bar, background, etc., so the final display interface is richer, which solves the above 1) TUI dull problem.
  • the REE can update the second graphic. For example, when an application in the REE receives a notification message, the REE can add the notification to the area corresponding to the transparent area in the first graphic on the second graphic. Message, so the first graphic on layer 1 will not obscure the notification message, so the final display effect is that the notification message is displayed in the TUI. Therefore, the problem of poor interactive experience in the TUI in 3) above is solved.
  • the embodiment of the present application also provides a REE and TEE fusion input mechanism: a channel for transmitting user input is established between the REE and the TEE.
  • the REE includes a communication module 1, which is used to receive information sent by the TEE ;
  • TEE includes a communication module 2, which is used to receive and send information to the REE.
  • the TEE controls the touch screen. After the TEE detects the user's input on the TUI, it determines whether the user input is processed by the TEE or the REE. If it is processed by the REE, the user input is sent to the channel through the channel REE; If processed by TEE, the application in the TEE processes the user input.
  • the above-mentioned communication module 1 and communication module 2 may be interprocess communication (IPC) modules, may also be referred to as interprocess communication modules, or other underlying hardware communication modules.
  • IPC interprocess communication
  • FIG. 3 a schematic flow chart of a display method of an electronic device supporting TEE+REE architecture provided by an embodiment of this application. As shown in FIG. 3, the flow of the method includes:
  • the display template is the reference template when drawing graphics.
  • the display template includes the layout (position, size, etc.) of the display area, the display content in each area, and so on.
  • Method A a display template is a graphic template, the graphic template is divided into different areas, and each area is used to display corresponding content.
  • TEE and REE can agree to show which areas of the template are provided by TEE and which areas are provided by REE.
  • TEE and REE can agree that the content related to password input in the display template is provided by TEE, and other content is provided by REE.
  • the display template is relatively simple.
  • one display template includes multiple graphic templates, and one graphic template corresponds to one layer in the display screen.
  • the display template includes two graphic templates, graphic template 1 corresponds to layer 1 on the display screen, and graphic template 2 corresponds to layer 2 on the display screen.
  • TEE controls the layer 1 of the display screen
  • the REE controls the layer 2 on the display screen
  • TEE and REE do not need to agree to draw content, which improves efficiency.
  • the display template may also be unnecessary, that is, it is drawn according to the drawing logic provided by the application or the operating system every time it is drawn.
  • the part of REE and TEE that needs to be agreed can be controlled by the preset drawing logic.
  • 301 can be executed by REE or TEE. Assuming that the REE constructs the display template, after the construction is completed, the display template can be sent to the TEE. Assuming that the TEE constructs the display template, after the construction is completed, the display template can be sent to the REE. In short, it is enough to realize the sharing of the display template between the TEE and REE.
  • the display template has nothing to do with the application.
  • a shared pool is formed.
  • the shared pool includes the one or more display modules.
  • the TEE You can select a display module from the shared pool to use. For example, you can select a suitable display template from the shared pool according to the current scene (for example, the currently running application, or the segmented scene in the application), which will be described later. . It should be understood that REE and TEE need to use the same display template.
  • the display template is related to the application. For example, when an electronic device downloads an application, it also downloads a display template corresponding to the application.
  • the display module may be provided by a provider corresponding to the application.
  • REE and TEE can share the display template corresponding to each application.
  • the display template corresponding to the application can be used.
  • the scenario may include:
  • USB-Shield scenario that is, a scenario in which an electronic device runs USB-shield to implement related services.
  • This scenario can correspond to a display template.
  • the lock screen scene that is, the electronic device is in a locked and dark screen state.
  • the electronic device detects an input operation, it lights up the electronic device and displays the lock screen interface, which can correspond to a display template.
  • Scenario 3 an account login scenario, for example, the electronic device detects an input operation, and in response to the input operation starts a first application such as a bank APP, and displays an account login interface, which may correspond to a display template.
  • a first application such as a bank APP
  • Scenario 4 the transfer scenario, the electronic device runs a first application such as a certain bank APP, and when an input operation is detected, and a security-related service (for example, a transfer service) in the first application is started in response to the input operation, a password is displayed Input interface, the password input interface can correspond to a display template.
  • a security-related service for example, a transfer service
  • multiple scenes can also correspond to the same display template.
  • at least two of the aforementioned scenes 1 to 4 correspond to the same display template.
  • the above scene 3 and scene 4 correspond to the same display template.
  • FIG. 4 is a schematic diagram of two display templates provided in an embodiment of this application.
  • (A) in FIG. 4 is a schematic diagram of display template 1 corresponding to scene 2 in a lock screen scene
  • (b) in FIG. 4 is a schematic diagram of display template 2 corresponding to scene 3 in a login bank app scene.
  • a part of the area in the display template 1 displays the content provided by the TEE
  • another part of the area displays the content provided by the REE.
  • the application runs in the TEE.
  • the application in 302 may be software authorized by TEE, such as Alipay, bank client, screen lock App, and so on.
  • the electronic device displays a home screen, and the home screen includes one or more application icons.
  • the application runs in the TEE and displays the application lock interface of the application, which includes a password input area.
  • Case 2 The application is running in the REE.
  • the specific input operation is used to start a specific service in the application, for example, a service involving the user's private information, such as transfer, payment, etc.; in response to the specific input Operation, the application is switched to run in TEE.
  • the application switches from REE to running in TEE, that is, CA runs in REE, then CA calls TA in TEE corresponding to the CA, and then TA runs in TEE.
  • the TEE determines a suitable display template according to the current scene.
  • Method 1 one application is one scene, and one scene corresponds to one display template. Therefore, the TEE in 303 determines the display template corresponding to the application according to the running application. For example, when an electronic device runs the China Merchants Bank APP in the TEE, it can use the display template provided by the China Merchants Bank APP to draw graphics.
  • TEE 2 Different services in one application correspond to different scenarios.
  • one application includes transfer services, payment services, etc., and different services correspond to different scenarios.
  • the TEE can determine the corresponding display template according to the current scene. Exemplarily, assuming that the TEE determines that it is currently in the lock screen scene, select the display template 1 shown in Figure 4 (a); if the TEE determines that it is currently in the transfer scene, select the display shown in Figure 4 (b) Template two.
  • One CA corresponds to one or more TAs, and each TA is used to provide different security-related services for the CA. Therefore, when the CA in the REE calls the TA corresponding to the CA in the TEE, the TEE can determine the currently running application or the services in the currently running application according to the called TA, and then determine a suitable display template.
  • the electronic device when the electronic device detects an operation on the icon of application 1, it determines that application 1 is set with an application lock, CA calls the TA in the TEE that is used to process applications related to the application lock in application 1, and then , The TA runs in TEE. In this case, the TEE determines the currently running application as Application 1 according to the called TA, and then determines the appropriate display template. For another example, for the above case 2, the application is running in the REE. When a specific input operation is detected, it is determined that the specific input operation is used to start a specific service in the application, and the TA corresponding to the specific service in the TEE is invoked. TA runs in TEE. Therefore, the TEE can determine the currently running application and the services running in the application according to the TA, and then determine a suitable display template.
  • the TEE and the REE can also determine the display template together. Specifically, the two can be negotiated and determined through information transmission. In other embodiments, the TEE can also accept the display template determined by the REE.
  • the TEE draws a first graphic according to the determined display template.
  • the first graphic includes a transparent area and a non-transparent area, where the non-transparent area is content drawn by the TEE.
  • TEE and REE agree to show which areas in the template are provided by TEE and which areas are provided by REE.
  • the display template determined by TEE is display template one shown in (a) in Figure 4, and it is agreed that the content of the password input area in display template one is provided by TTE, so TEE can draw the first graphic, the first The area on a graph corresponding to the password input area includes password input information, such as numeric key values, etc., and other areas on the first graph are transparent areas.
  • the first graph drawn by the TEE is as shown in (a) of FIG. 5, and the first graph is a transparent area except for the numeric key value.
  • the display template includes the graphic template corresponding to each layer. Assuming that the graphics drawn by TEE are displayed on the first layer, and the graphics drawn by REE are displayed on the second layer to the Nth layer, the TEE only needs to draw the first graphics according to the graphics template corresponding to the first layer. can.
  • the TEE displays the first graphic on the first layer of the display screen.
  • the touch screen in the electronic device is located on the upper layer of the display screen, and the display screen includes multiple layers, so the first layer in the display screen can be one or more layers close to the touch screen, for example, the first image Layers, or, the first layer and the second layer.
  • the display screen includes multiple layers, and the final TUI on the display screen is obtained by fusion (or superimposition) of graphics on each layer.
  • the TEE can display the first graphic on the first layer, such as layer 1.
  • the TEE sends an identification of the display template to the REE.
  • the TEE and the REE share the display module. After the TEE sends the identification of the display module to the REE, the REE can determine the corresponding display template based on the identification.
  • the REE draws a second graphic according to the display template.
  • TEE and REE agree to show which areas in the template need to be provided by TEE and which areas need to be provided by REE.
  • the display template determined by TEE is the display template 1 shown in (a) in Figure 4, and it is agreed that only the content of the "password input area" in the display template 1 is provided by TTE, and the content of other areas is provided by REE , So REE draws the second graph according to the display template.
  • the REE can only draw the area on the second graphic corresponding to the transparent area on the first graphic, and not the area corresponding to the non-transparent area; of course, the REE can also draw all the second graphics.
  • the display template includes the graphic template corresponding to each layer, it is assumed that the graphics drawn by TEE are displayed on the first layer, and the graphics drawn by REE are displayed on the second layer to the Nth layer. Then REE draws the second graph according to the graph template corresponding to the second layer to the Nth layer.
  • the REE displays the second graph on the second layer of the display screen.
  • 307-308 can be executed before 303, 304 or 305, that is, REE first draws the second shape, then TEE draws the first shape; or, TEE first draws the first image, and then REE draws the second Graphics can be used, and the embodiment of the present application does not limit it.
  • the REE can determine a suitable display template according to the current scene, and then send the identification of the display template to the TEE, as long as the display template used by the TEE and the REE are the same.
  • the touch screen in the electronic device is located on the upper layer of the display screen, and the display screen includes multiple layers, so the second layer in the display screen may be one or more layers away from the touch screen.
  • the "first layer” in 305 may be the first layer
  • the "second layer” in 308 may be the second layer to the Nth layer; or
  • the "first layer” It can be the first layer to the third layer
  • the "second layer” can be the fourth layer to the Nth layer, etc., which are not limited in the embodiment of the present application.
  • the first graphic is displayed on layer 1, and the second graphic is displayed on layer 2-layer N.
  • the second graph generated by REE is shown in Figure 5(b)
  • the first graph drawn by TEE is shown in Figure 5(a).
  • the first graph is displayed on layer 1.
  • the transparent area of is unable to block the content of the corresponding area on the second graphic, and the first graphic and the second graphic are merged to obtain the final display interface.
  • the final display interface obtained by the fusion is the interface shown in (c) in FIG. 5. Therefore, when the application is running in the TEE, the display interface includes not only the content provided by the TEE, but also the content provided by the REE, such as the status bar, the background, etc., and the display content is relatively rich.
  • the lock screen interface is an interface generated by REE. Because REE has powerful drawing capabilities, the lock screen interface includes rich information, such as status However, due to the low security of REE, it is easy to reveal the password when the user enters the unlock password in the lock screen interface.
  • the lock screen interface in (c) in Figure 5 is a TUI obtained after fusing the first graphic drawn by TEE and the second graphic drawn by REE.
  • the key information in the display interface is, for example, a password.
  • Input information is drawn by TEE, and other information is drawn by REE. Therefore, REE cannot know the password input information, which improves security.
  • the richness of TUI is increased.
  • the REE draws the notification message in the area (for example, the status bar area) corresponding to the transparent area in the first graphic on the second graphic, so the first graphic and the updated After the second graphics are merged, a notification message is displayed, as shown in Figure 7.
  • the area for example, the status bar area
  • the application displays the TUI when running in the TEE.
  • the TEE cannot obtain the notification message, and the REE does not participate in the drawing of the TUI, so The notification message cannot be displayed in the TUI.
  • the notification message can be displayed in the TUI, which enhances the richness of the TUI.
  • the TUI on the display screen of the electronic device is the interface after fusing the first figure drawn by the TEE and the second figure drawn by the REE.
  • the user can perform user input on the TUI.
  • the embodiment of the application adopts the TEE
  • the fusion input mechanism with REE is specifically implemented through the following steps 309-311:
  • the TEE detects user input.
  • the TEE controls the touch screen, that is, the TEE detects user input on the touch screen.
  • the electronic device displays a TUI as shown in (c) in Figure 5
  • part of the content in the TUI is provided by the TEE
  • part of the content is provided by the REE
  • the user may perform any operation anywhere on the TUI For example, operations such as sliding up/down. Therefore, after the TEE detects the user input, it can be determined whether the user operation is responded by the TEE or the REE, that is, 310.
  • the TEE determines whether the user input is responded by the TEE or by the REE.
  • the TEE draws a first graphic according to the display template, and the first graphic includes information related to the application, such as password input information. Therefore, the TEE can determine whether the user input is the in-app input (hereinafter referred to as in-app input for short) or non-in-app input (hereinafter referred to as non-app input for short).
  • the input in the APP includes the control information input in the APP and the text information input in the APP; the text information input in the APP, such as the password entered in the TUI, other non-APP text information input, such as clicking the return button on the TUI or clicking for adjustment
  • the key input of the keyboard input method belongs to the control information input in the APP.
  • Non-APP input includes, for example, full-screen gesture operations, pull-down menu operations, virtual button operations at the bottom of the screen, and other inputs that are not in the application.
  • TEE sends non-APP input to REE for processing, and APP input to TEE internal application processing. It is understandable that before TEE sends non-APP input to REE, TUI Service exits, releases the touch screen and display layer 1, and clears the content in layer 1. After REE receives non-APP input, it passes through layer 1. -Layer N output response. In this method, when the application is running in the TEE, all input in the APP is processed by the TEE without going through the REE, and the security is high.
  • TEE sends the non-APP input and the control information input in the APP to the REE processing, and sends the text information input in the APP to the TEE internal application processing.
  • the application is running in the TEE
  • only the text information input in the APP for example, password input
  • other input is processed by the REE, which relieves the processing pressure of the TEE.
  • the screen lock App runs in the TEE, and the first graph drawn by the TEE only includes numeric key values.
  • TEE detects that the user enters the password, it determines that the user input is text information input in the APP, and then sends the user input to the lock screen application processing in the TEE. If it is determined that the user input is the control information input in the APP, for example, click the return button to send the user Enter REE, you can also send user input to the lock screen application in TEE.
  • TEE judges whether the user input is in-app or non-in-app input can be at least one of the following methods:
  • the first graphic drawn by the TEE includes a transparent area and a non-transparent area
  • the non-transparent area includes information drawn by the TEE. Therefore, the way for TEE to determine whether the user input is in-app input or non-in-app input can be to determine whether the touch position of the user input is located in a non-transparent area, if it is, it is determined that the user input is an in-app input; if not, it is determined User input is non-in-app input.
  • Method 2 TEE judges whether the user input is in-app input or non-in-app input. It can also be judged whether the operation type input by the user is a preset operation type.
  • the preset operation type includes click operation, long-press operation, etc.; if so, It is determined that the user input is an in-app input; if not (for example, it is an operation such as sliding up or down), it is determined that the user input is not an in-app input.
  • gesture operations such as sliding up/down are usually not operations for inputting a password, so the TEE can determine whether the user operation is an input operation in the APP according to the operation type of the user operation.
  • method 1 and method 2 can be used alone, or, in order to improve accuracy, method 1 and method 2 can be used in combination, for example, TEE judges whether the user input is in a non-transparent area, if so, continue to judge the user input Whether the operation type of is the preset operation type, if so, the application in the TEE responds to the user input; if the user input is not in the non-transparent area or the operation type is not the preset operation type, the user input is sent to REE for processing.
  • TEE sends user input to REE.
  • a channel for transmitting user input can be established between the TEE and the REE. If the TEE in 309 determines that the user input is processed by the REE, the user input is sent to the REE through the channel. After receiving user input, REE responds to user input. For example, taking (c) in Figure 5 as an example, the TEE detects a user operation, the TEE judges that the user operation is located in the status bar, and the operation type of the user operation is a downward sliding operation, the TEE sends the user operation to REE, and the REE responds User operation, the drop-down menu is displayed.
  • the application in the TEE responds to the user input.
  • the application in 302 runs in the TEE. If the TEE in 309 determines that the user input is responded by the TEE, the application in the TEE responds to the user input. For example, if the user input is text information input in the APP, the application will compare the input text information with The pre-stored text information is compared, and if the comparison is consistent, the corresponding processing flow is executed, such as unlocking the electronic device, logging in to the application, transferring money, and paying.
  • the electronic device displays the TUI.
  • the TUI is the display interface after the fusion of the first graphic drawn by the TEE and the second graphic drawn by the REE.
  • TUI is a dull problem in traditional technology.
  • TEE displays the first graphic on layer 1
  • REE displays the second graphic on layer 2-layer N
  • the first graphic and second graphic are merged to obtain the final display interface .
  • the application in the REE receives a notification message
  • the notification message can be added to the second graphic.
  • the notification message can be displayed on the display interface obtained after the fusion of the first graphic and the updated second graphic. . Therefore, in the prior art, when the electronic device displays the TUI, the TUI cannot display the notification message received by the application in the REE, and the display richness of the TUI is improved.
  • the display method provided by the embodiment of the present application can be implemented through the TEE+REE architecture shown in FIG. 8.
  • a trusted environment system command execution agent agent for command from REE, Agent_ree
  • the Agent_ree can be used as a communication module, and the REE can be received through Agent_ree.
  • User input sent by TEE is also added to the TEE, which is used to send user input to the REE.
  • some modules in the architecture have added corresponding functions.
  • TEE and REE share display templates; for example, when TA is running in TEE, TEE is on the display screen.
  • Middle layer 1 displays the first graphic
  • REE displays the second graphic on layer 2-layer N on the display; for another example, the TUI service in the TEE adds the function of judging whether the user operation is responded by the TEE.
  • REE includes: CA, framework application programming interface (Framework API), trusted client application programming interface (TEE client application programming interface, TEE Client API), rich execution environment operating system kernel layer (REE OS kernel) including: display screen Driver (display screen driver, DSD), touch screen driver (touch screen driver, TSD), Agent_ree.
  • framework application programming interface Framework API
  • trusted client application programming interface TEE client application programming interface
  • TEE Client API rich execution environment operating system kernel layer
  • REE OS kernel including: display screen Driver (display screen driver, DSD), touch screen driver (touch screen driver, TSD), Agent_ree.
  • the CA When the CA is running in the REE, the CA calls the Display Screen Driver in the REE OS kernel through the Framework API to control the display screen to display the display interface provided by the CA.
  • the Touch Screen Driver sends the user input to the CA through the Framework API, and the CA responds to the user input.
  • TEE includes: TA, trusted application programming interface (TEE internal application programming interface, TEE Internal API), trusted execution environment application programming development interface (TEE OS kernel) including: trusted user interface service component (trusted user interface service) , TUI Service), Display Screen Driver, Touch Screen Driver, communication module.
  • TEE trusted application programming interface
  • TEE OS kernel trusted execution environment application programming development interface
  • trusted user interface service component trusted user interface service
  • TUI Service TUI Service
  • Display Screen Driver Touch Screen Driver
  • communication module can be used to send user input to Agent_ree in REE.
  • the communication module may also be integrated in other modules without independent settings.
  • the function of the communication module is integrated in the TUI Service, that is, the TUI Service sends user input to the REE. Therefore, the communication module in Figure 8 is represented by a dashed line.
  • TA runs in TEE
  • TUI service calls TUI service through TEE Internal API
  • TUI service controls layer 1 in the display to display the first graph through the display module
  • REE controls layer 2-layer in the display through the display module in REE OS kernel N shows the second graph.
  • the first graphic and the second graphic are merged (or overlapped) to obtain the TUI.
  • the touch module sends the input operation to the TUI service
  • the TUI service determines whether the TEE responds to the input operation. If it is, the input operation is sent to TA for response. If not, the TUI service sends the input operation through the communication module.
  • Agent_ree in REE
  • Agent_ree sends the input operation to CA to respond.
  • FIG. 9 it is a flow chart of implementing the display method provided by the embodiment of the present application through the TEE+REE architecture shown in FIG. 8.
  • the process may include:
  • CA calls TA.
  • the CA in the REE calls the TA of the TEE through the TEE Client API.
  • TUI Service draws the first graphic according to the display template, and uses the Display Screen Driver in the TEE OS kernel to place the first graphic on the display screen layer 1.
  • the TUI Service in 903 can determine a suitable display template according to the current scene, and then draw the first graphic according to the display template, and the specific process will not be repeated.
  • CA draws the second graphic according to the display template, and displays the second graphic on layer 2-layer N of the display screen through the Display Screen Driver in the REE OS kernel.
  • the display modules used in 903 and 904 are the same display template. It should be noted that the execution sequence between 904 and 901-903 is not limited in the embodiment of the present application.
  • the Touch Screen Driver in the TEE detects user input and sends TUI Service.
  • TUI Service determines the type of user input; if it is not an in-app input, execute 907-909; if it is an in-app input, execute 910.
  • TUI Service exits the TUI service. Exiting the TUI service may include clearing the display content of layer 1, and transferring the control of layer 1 to REE.
  • TUI Service sends the user input to Agent_ree.
  • Agent_ree sends the user input to the CA, and the CA processes the user input.
  • TUI Service determines whether the TUI meets the exit condition or the TUI input is completed, if yes, execute 911, and if not, execute 905.
  • satisfying the TUI exit conditions may include: the user clicks the cancel button in the TUI to trigger the TUI exit, or an abnormal event triggers the TUI exit, including operations such as clicking the power button, incoming call, screenshots, etc., or TA crash (Crash) triggering the TUI exit, Or, display the TUI for a preset time period, such as 120s, and automatically exit the TUI without any operation.
  • the completion of the TUI input may include inputting text information, for example, the number of digital information reaches a preset number, or the user clicks the completion button in the TUI.
  • the TUI Service sends the input information to the TA, and the TA performs processing (for example, security verification), and obtains the processing result (for example, whether the verification is passed or not).
  • processing for example, security verification
  • processing result for example, whether the verification is passed or not.
  • the TA sends the processing result to the CA, and the CA performs subsequent processing.
  • the CA responds to the processing result and displays the main interface on the display screen.
  • FIG. 10 it is a schematic diagram of a logical structure of an electronic device 1000 provided by an embodiment of this application.
  • the electronic device 1000 may be a mobile phone, a tablet computer, or the like.
  • the electronic device 1000 includes a TEE and a REE, where the TEE includes a first processing module 1001 and a first display module 1002, and the REE includes a second processing module 1003 and a second display module 1004.
  • the first processing module 1001 is used to draw a first graph
  • the first display module 1002 is configured to display the first graphic on the first layer of the display screen of the electronic device;
  • the second processing module 1003 is used to draw a second graph
  • the second display module 1004 is configured to display the second graphic on a second layer in the display screen, wherein the first layer is located above the second layer;
  • the first graphic includes a transparent area and a non-transparent area
  • the non-transparent area includes the content drawn by the TEE
  • the display area on the second graphic corresponding to the transparent area includes the content drawn by the REE.
  • the content when the first layer and the second layer are overlapped and displayed, the transparent area does not block the content drawn by the REE.
  • the touch screen in the electronic device is located on the upper layer of the display screen, the display screen includes a plurality of layers, and the first layer is one or more layers close to the touch screen.
  • the second layer is one or more layers away from the touch screen.
  • the first layer is a layer closest to the touch screen in the display screen
  • the second layer is another image in the display screen except the first layer.
  • the first processing module 1001 is further configured to: determine a display template corresponding to the current scene from one or more display templates according to the current scene; the first processing module 1001 is specifically configured to: The template draws the first graphic, and the second processing module 1003 is specifically configured to: draw the second graphic according to the display template.
  • the display template is a graphic, and the graphic includes at least one display area, each of which is configured as the TEE drawing content or the REE drawing content; or, the display template includes At least one graphic, each graphic corresponds to a layer, and each graphic is configured to be drawn by the TEE or the REE.
  • the content drawn by the TEE includes; information related to password input.
  • the first processing module 1001 is further configured to: detect user input and determine whether the user input requires the TEE response; when the user input does not require the TEE response, pass the first TEE response A communication module sends the user input to the REE, and the REE executes a corresponding processing flow in response to the user input.
  • the first processing module 1001 when used to determine whether the user input requires the TEE response, it is specifically configured to: when it is determined that the user input is an input in an application currently running in the TEE, determine The user input requires the TEE response; when it is determined that the user input is not an input in the application, it is determined that the user input does not require the TEE response.
  • the input in the application includes: the input located in the non-transparent area, and/or the input whose operation type meets the preset operation type; the input that is not in the application includes: the input located in the non-transparent area; The input outside the transparent area, and/or the operation type is not the input of the preset operation type.
  • the first processing module 1001 is further configured to: when the application in the REE receives the notification message, update the second graphic, and the updated second graphic corresponds to the transparent area in the first graphic
  • the notification message is displayed in the display area of; the second display module is further configured to display the updated second graphic in the second layer.
  • the unit/module in the above electronic device may be one or more integrated circuits configured to implement the above method, for example: one or more application specific integrated circuits (ASIC), or one Or multiple microprocessors (digital singnal processors, DSP), or, one or more field programmable gate arrays (FPGA), or a combination of at least two of these integrated circuit forms.
  • ASIC application specific integrated circuits
  • DSP digital singnal processors
  • FPGA field programmable gate arrays
  • a unit/module in an electronic device can be implemented in the form of a processing element scheduler.
  • the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call programs.
  • CPU central processing unit
  • these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).
  • the first processing module 1001 and the second processing module 1003 may be different physical devices (here, the physical devices may include chips, chip systems, circuits, circuit systems, etc.), or the same physical devices, such as both It is a processor; the processor includes a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor ( image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU), etc. .
  • the physical device can be called by different computer programs to implement different functions.
  • first display module 1002 and the second display module 1004 may be display screens in an electronic device, and the display screen may include a display panel.
  • the display panel can adopt liquid crystal display (LCD), organic light-emitting diode (OLED), active matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • active matrix organic light-emitting diode active-matrix organic light-emitting diode
  • active-matrix organic light-emitting diode active-matrix organic light-emitting diode
  • AMOLED flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (QLED), etc.
  • an embodiment of the present application provides an electronic device.
  • the electronic device 1100 is equipped with REE and TEE (not shown in the figure); it also includes: a display screen 1130; one or more processors 1110; a memory 1120; among them, one or more processors 1110 can run in REE or TEE
  • a display screen 1130 when one or more processors are running on the TEE, the steps performed by the TEE can be implemented, and when one or more processors are running on the REE, the steps performed by the REE can be implemented.
  • One or more processors 1110 can switch between TEE and REE by means of computer program scheduling.
  • the memory 1120 may store one or more computer programs, the one or more computer programs are configured to be executed by the one or more processors 1110, the one or more computer programs include instructions, and the above instructions may be used To perform each step in the foregoing embodiment.
  • the embodiment of the present application also provides a computer storage medium.
  • the computer storage medium stores computer-executable instructions. When called by a computer, the computer-executable instructions cause the computer to Perform the specific process of each of the method embodiments provided above.
  • the computer-readable storage medium is not limited. For example, it may be RAM (random-access memory, random access memory), ROM (read-only memory, read-only memory), and the like.
  • the embodiment of the present application also provides a computer program product.
  • the computer program product stores instructions, which when running on a computer, causes the computer to execute any of the above possible designs. Method provided in.
  • this application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.
  • this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
  • These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Human Computer Interaction (AREA)
  • Computer Security & Cryptography (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种图像显示方法与电子设备，该方法适用于包括可信执行环境TEE和富执行环境REE的电子设备。其中，TEE绘制第一图形，并在电子设备的显示屏中的第一图层显示第一图形；REE绘制第二图形，并在显示屏中的第二图层显示第二图形，第一图层位于第二图层之上；其中，第一图形上包括透明区域和非透明区域，非透明区域包括TEE绘制的内容，第二图形上与透明区域对应的显示区域包括REE绘制的内容，当第一图层和第二图层重叠显示时，透明区域不遮挡REE绘制的内容。也就是说，借助REE强大的绘图能力，提升可信用户界面(trusted user interface，TUI)的丰富性。

Description

一种图像显示方法与电子设备
相关申请的交叉引用
本申请要求在2019年10月24日提交中国专利局、申请号为201911016209.6、申请名称为“一种图像显示方法与电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中；本申请要求在2019年11月07日提交中国专利局、申请号为201911082843.X、申请名称为“一种图像显示方法与电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及安全验证技术领域，尤其涉及一种图像显示方法与电子设备。
背景技术
电子设备的功能越来越强大，安装的应用(application，app)越来越多，其中涉及到用户隐私信息的应用越来越多，例如，各类手机银行app等。
早期，电子设备仅支持富执行环境(rich execution environment，REE)架构，例如安卓架构。电子设备中的所有应用都在REE中运行，但是REE的安全等级较低，用户隐私信息容易被窃取。为了防止用户隐私信息的泄露，业界提出可信执行环境(trusted execution environment，TEE)，在硬件的支持下，TEE可以为应用提供安全的运行空间。因此，REE+TEE架构式的电子设备应用而生，当应用在TEE中运行时，安全性相对较高，当应用在REE中运行时，安全性较低。因此，涉及到用户隐私信息的应用或业务转移到TEE中运行，以保证数据安全性。
现有的REE+TEE架构，存在两种安全运行机制：
机制一：当应用在TEE运行时，REE控制显示界面，用户在该显示界面中输入口令等信息，REE将检测到的口令发送给TEE进行安全认证。
机制二：当应用在TEE运作时，TEE控制显示界面，用户在该显示界面中输入的口令等被TEE检测然后进行安全认证，口令输入不经过REE。其中，由TEE控制的显示界面被称为可信用户界面(trusted user interface，TUI)。
上述机制一，基于REE强大的绘图能力，所以当应用在TEE运行时，电子设备呈现的显示界面比较丰富，但是由于REE的安全性较低，REE检测到口令之后容易被窃取，安全性较低。上述机制二，虽然解决了机制一中安全性低的问题，但是存在如下缺陷：
1)受限于TEE的处理能力，一方面，TEE控制的TUI通常比较呆板；另一方面，TEE中运行的应用或业务相对较少，电子设备控制TUI时，在REE中运行的应用或业务的相关信息无法在TUI中显示。例如，电子设备控制TUI时，若REE中的应用接收到新消息或通知时，TEE无法在TUI中显示该新消息和通知，所以TUI视觉交互体验较差。
2)TUI上的用户交互操作灵敏性较差。例如，仅能识别有限的操作类型(例如按键点击)，无法识别并处理全面屏手势、上滑/下拉菜单等输入操作。因此，目前TUI给用户触控响应不灵活的感受，用户体验较差。
因此，上述两种机制均存在不可避免的缺陷，都无法满足用户的需求。
发明内容
本申请的目的在于提供了一种图像显示方法与电子设备，用于提升TUI的视觉交互体验。
上述目标和其他目标将通过独立权利要求中的特征来达成。进一步的实现方式在从属权利要求、说明书和附图中体现。
第一方面，提供了一种图像显示方法，应用于包括可信执行环境TEE和富执行环境REE的电子设备，例如手机、平板电脑等。该方法包括：电子设备中TEE绘制第一图形，并在电子设备的显示屏中的第一图层显示第一图形；REE绘制第二图形，并在显示屏中的第二图层显示第二图形，其中，第一图层位于第二图层之上；第一图形上包括透明区域和非透明区域，非透明区域包括TEE绘制的内容，第二图形上与透明区域对应的显示区域包括REE绘制的内容，当第一图层和第二图层重叠显示时，透明区域不遮挡所述REE绘制的内容。
因此，当第一图层和第二图层重叠显示时，最终呈现的显示效果为，显示界面上既包括TEE提供的内容，也包括REE提供的内容。也就是说，借助REE的强大的绘图能力以及丰富的显示资源，第二图形中可以包括较为丰富的信息，例如状态栏、背景等，所以最终得到的显示界面即TUI较为丰富，解决了现有机制中，TEE控制显示屏时，TUI呆板的问题。
其中，第一图层和第二图层可以有多种方式确定，例如如下的方式1和方式2。
方式1，电子设备中触摸屏位于显示屏的上层，显示屏中包括多个图层，第一图层为靠近触摸屏的一个或多个图层，第二图层为远离触摸屏的一个或多个图层。
可以理解的是，显示屏上每个图层重叠显示，位于上层的图层会遮挡位于下层的图层。其中，位于上层的图层即靠近触摸屏的图层，位于下层即远离触摸屏的图层。因此，本申请实施例中，TEE在显示屏中靠近触摸屏的一个或多个图层显示第一图形，REE在显示屏中远离触摸屏的一个或多个图层显示第二图形，由于第一图形中包括透明区域，当第一图形和第二图形重叠显示时，第一图形中透明区域不遮挡第二图形中REE绘制的内容，借助REE强大的绘图能力，提升TUI的丰富性。
方式2，第一图层为显示屏中距离触摸屏最近的一个图层，第二图层为显示屏中除去第一图层之外的其它图层。
假设显示屏包括图层1-图层N，图层1距离触摸屏最近，图层N距离触摸屏最远；TEE在显示屏上图层1显示第一图形，REE在图层2-图层N中的一个或多个图层显示第二图形。可以理解的是，受限于全球标准平台(global platform，GP)的标准规范,TEE的处理能力有限，所以TEE仅控制图层1的话，可以缓解TEE的图形绘制压力，而且能够实现丰富TUI的效果。
在一种可能的设计中，TEE可以根据当前场景从一个或多个显示模板中确定与当前场景对应的显示模板；然后TEE根据显示模板绘制第一图形，REE根据显示模板绘制第二图形。
其中，当前场景可以是TEE内当前运行的应用，或，TEE内当前运行的应用内的业务。TEE根据当前场景确定显示模板之后，REE使用相同的显示模块绘制图形，具体可以使用如下方式1-方式3实现。
方式1，TEE根据当前场景确定合适的显示模板之后，向REE发送该显示模板的标识， 然后REE根据该标识对应的显示模板绘制图形。
方式2，TEE根据当前场景确定合适的显示模板之后，向REE发送显示模板，REE使用该显示模板绘制图形。
方式3，TEE还可以和REE共同确定显示模板，具体的，二者可通过信息发送来协商确定。
应理解，上述3种方式仅是举例，不是限定，其它的能够使得TEE和REE使用相同显示模板的方式也是可以的。使用相同显示模板时，TEE绘制的第一图形中透明区域与第二图形上REE绘制的内容相对应，提升显示界面的丰富性。
其中，显示模板可以有多种实现方式，例如如下方式A和方式B。
方式A，显示模板为一个图形，该图形包括至少一个显示区域，其中的每个显示区域被配置为TEE绘制内容或者REE绘制内容。应理解，方式A中TEE和REE可以约定显示模块上哪个区域由TEE绘制，哪个区域由REE绘制。TEE根据显示模板绘制第一图形时，在显示模块上属于TEE绘制的区域内绘制内容，其它区域设置为透明区域。REE根据显示模板绘制第二图形时，在显示模板上属于REE绘制的区域内绘制内容。这样的话，当第一图形和第二图形重叠显示时，第一图形中透明区域不遮挡第二图形中REE绘制的内容。
方式B，显示模板包括至少一个图形，每个图形对应显示屏上的一个图层，每个图形被配置为TEE绘制或REE绘制。应理解，假设TEE控制图层1、REE绘制图层2-图层N，TEE根据图层1对应的图形绘制第一图形，REE根据图层2-图层N对应的图形绘制第二图形，无需约定哪些区域由TEE绘制，哪些区域由REE绘制。
在一种可能的设计中，TEE绘制的内容可以包括；与口令输入相关的信息。应理解，TEE绘制越少的内容，越可以缓解TEE的图形绘制压力，所以本申请实施例中，TEE可以仅绘制与口令输入相关的信息，例如数字按键值等；其它内容可以借助REE来绘制。
在一种可能的设计中，TEE还可以检测用户输入，并判断用户输入是否需要TEE响应；当用户输入不需要TEE响应时，将用户输入发送给REE，REE响应于用户输入执行相应的处理流程。
如前文所述，电子设备的显示界面上既包括TEE提供的内容，又包括REE提供的内容。用户可能在显示界面执行用户输入，该用户输入可能是针对TEE提供的内容的，也可能是针对REE提供的内容的。因此，本申请实施例中，当TEE检测到用户输入时，可以判断用户输入是否需要TEE响应，若不需要，则将用户输入发送给REE，由REE响应该用户输入，提升用户在显示界面(即TUI)上交互操作的多样性、灵敏性。
其中，电子设备判断用户输入是否需要TEE响应的方式可以是：当TEE确定用户输入是TEE内当前运行的应用内的输入时，确定用户输入需要TEE响应；当TEE确定用户输入不是应用内的输入时，确定用户输入不需要TEE响应。
在本申请实施例中，当应用在TEE中运行时，显示界面中包括TEE和REE提供的内容；TEE检测到用户输入之后，若确定该用户输入属于所述应用内的输入(例如口令输入)时，TEE响应该用户输入，以保证安全性；当TEE确定该用户输入不属于所述应用内的输入时，TEE将该用户输入发送REE，由REE响应该用户输入，提升用户在显示界面(即TUI)上交互操作的多样性、灵敏性。
其中，TEE判断用户输入是应用内输入还是非应用内输入的方式可以是如下方式中至少一种：
方式一，TEE绘制的第一图形包括透明区域和非透明区域，非透明区域包括TEE绘制的信息。因此，TEE判断用户输入的触控位置是否位于非透明区域内，若是，则确定用户输入是应用内输入；若否，则确定用户输入是非应用内输入。也就是说，TEE仅响应触控位置位于非透明区域内的用户输入。
方式二，TEE判断用户输入的操作类型是否是预设操作类型，预设操作类型包括点击操作，长按操作等；若是，则确定用户输入是应用内输入；若否(例如是上滑、下滑等操作)，则确定用户输入是非应用内输入。也就是说，TEE仅响应操作类型是预设操作类型的用户输入。可以理解的是，上滑/下滑等手势操作通常不是用于输入口令的操作，所以TEE将上滑/下滑等手势操作发送REE进行响应。
需要说明的是，上述方式一和方式二可以单独使用，或者，为了提升准确性，方式一和方式二可以结合使用，具体的，应用内的输入包括：位于非透明区域内的输入，和/或，操作类型满足预设操作类型的输入；非应用内的输入，包括：位于非透明区域外的输入，和/或，操作类型不是预设操作类型的输入。
在一种可能的设计中，REE中的应用接收到通知消息时，REE更新第二图形，更新后的第二图形中与第一图形中透明区域对应的显示区域中显示通知消息；REE在第二图层中显示更新后的第二图形。
需要说明的是，现有的TEE+REE架构中，应用在TEE中运行时显示TUI，当REE中的应用收到通知消息时，TEE无法获取该通知消息，而REE不参与TUI的绘制，所以TUI中无法显示所述通知消息。但是，本申请实施例中，电子设备显示TUI时，当REE中的应用收到通知消息时，REE可以更新第二图像，更新后的第二图形中与第一图像上透明区域对应的显示区域包括通知消息，所以，TUI中可以显示该通知消息，提升TUI的丰富性。
第二方面，还提供一种电子设备，该电子设备包括可信执行环境TEE和富执行环境REE；其中，TEE中包括第一处理模块和第一显示模块，REE中包括第二处理模块和第二显示模块：第一处理模块，用于绘制第一图形；第一显示模块，用于在电子设备的显示屏中的第一图层显示第一图形；第二处理模块，用于绘制第二图形；第二显示模块，用于在显示屏中的第二图层显示第二图形，其中，第一图层位于第二图层之上；其中，所述第一图形上包括透明区域和非透明区域，所述非透明区域包括所述TEE绘制的内容，所述第二图形上与所述透明区域对应的显示区域包括所述REE绘制的内容，当所述第一图层和所述第二图层重叠显示时，所述透明区域不遮挡所述REE绘制的内容。
在一种可能的设计中，所述电子设备中触摸屏位于所述显示屏的上层，所述显示屏中包括多个图层，所述第一图层为靠近所述触摸屏的一个或多个图层，所述第二图层为远离所述触摸屏的一个或多个图层。
在一种可能的设计中，所述第一图层为所述显示屏中距离所述触摸屏最近的一个图层，所述第二图层为所述显示屏中除去所述第一图层之外的其它图层。
在一种可能的设计中，所述第一处理模块还用于：根据所述当前场景从一个或多个显示模板中确定与所述当前场景对应的显示模板；所述第一处理模块具体用于：根据所述显示模板绘制所述第一图形，所述第二处理模块具体用于：根据所述显示模板绘制所述第二图形。
在一种可能的设计中，所述显示模板为一个图形，所述图形上包括至少一个显示区域， 其中的每个显示区域被配置为所述TEE绘制内容或者所述REE绘制内容；或者，所述显示模板包括至少一个图形，每个图形对应一个图层，每个图形被配置为所述TEE绘制或所述REE绘制。
在一种可能的设计中，所述TEE绘制的内容，包括；与口令输入相关的信息。
在一种可能的设计中，所述第一处理模块还用于：检测用户输入，并判断所述用户输入是否需要所述TEE响应；当所述用户输入不需要所述TEE响应时，通过所述TEE中的第一通信模块将所述用户输入发送给所述REE，所述REE响应于所述用户输入执行相应的处理流程。
在一种可能的设计中，所述第一处理模块在用于判断所述用户输入是否需要所述TEE响应时，具体用于：当确定所述用户输入是所述TEE内当前运行的应用内的输入时，确定所述用户输入需要所述TEE响应；当确定所述用户输入不是所述应用内的输入时，确定所述用户输入不需要所述TEE响应。
在一种可能的设计中，所述应用内的输入包括：位于所述非透明区域内的输入，和/或，操作类型满足预设操作类型的输入；非所述应用内的输入，包括：位于所述非透明区域外的输入，和/或，操作类型不是所述预设操作类型的输入。
在一种可能的设计中，所述第一处理模块还用于：当所述REE中的应用接收到通知消息时，更新所述第二图形，更新后的第二图形中与所述第一图形中透明区域对应的显示区域中显示所述通知消息；所述第二显示模块还用于，在所述第二图层中显示所述更新后的第二图形。
第三方面，还提供一种电子设备，该电子设备部署有可信执行环境TEE和富执行环境REE，还包括显示屏、处理器和存储器；其中，所述存储器存储有计算机程序，所述计算机程序包括指令，当所述指令被所述处理器执行时，使得所述电子设备实现上述第一方面提供的图像显示方法。
第四方面，还提供一种计算机存储介质，所述计算机存储介质中存储有计算机程序，当计算机程序在部署有可信执行环境TEE和富执行环境REE的电子设备上运行时，使得所述电子设备实现上述第一方面提供的图像显示方法。
第五方面，还提供一种计算机程序，包括指令，当所述指令在部署有可信执行环境TEE和富执行环境REE的计算机上运行时，使得所述计算机实现上述第一方面提供的图像显示方法。
第六方面，还提供一种芯片，所述芯片能够运行在可信执行环境TEE和富执行环境REE，所述芯片用于读取存储器中存储的计算机程序，实现上述第一方面提供的图像显示方法。
第七方面，还提供一种电子设备上的图形用户界面，所述电子设备部署有可信执行环境TEE、富执行环境REE，还包括显示屏、一个或多个存储器、以及一个或多个处理器，所述一个或多个处理器用于执行存储在所述一个或多个存储器中的一个或多个计算机程序，所述图形用户界面包括所述电子设备上述第一方面提供的图像显示方法时显示的图形用户界面。
附图说明
图1为现有的TEE+REE架构的示意图；
图2为显示屏的逻辑结构示意图；
图3为本申请一实施例提供的图像显示方法的流程示意图；
图4为本申请一实施例提供的两种显示模板的示意图；
图5为本申请一实施例提供的第一图形、第二图形、以及融合之后的图形的示意图；
图6为本申请一实施例提供的显示屏的逻辑结构示意图；
图7为本申请一实施例提供的REE中应用收到通知消息时，显示界面的示意图；
图8为本申请一实施例提供的TEE+REE架构的示意图；
图9为本申请一实施例提供的图像显示方法的流程示意图；
图10为本申请一实施例提供的电子设备的逻辑结构示意图；
图11为本申请一实施例提供的电子设备的结构示意图。
具体实施方式
下面将结合本申请以下实施例中的附图，对本申请实施例中的技术方案进行详尽描述。
以下实施例中所使用的术语只是为了描述特定实施例的目的，而并非旨在作为对本申请的限制。如在本申请的说明书和所附权利要求书中所使用的那样，单数表达形式“一个”、“一种”、“所述”、“上述”、“该”和“这一”旨在也包括例如“一个或多个”这种表达形式，除非其上下文中明确地有相反指示。还应当理解，在本申请实施例中，“一个或多个”是指一个、两个或两个以上；“和/或”，描述关联对象的关联关系，表示可以存在三种关系；例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A、B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。
在本说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
以下，先对本申请实施例中涉及的部分用语进行解释说明，以便于本领域技术人员容易理解。
1)、本申请实施例涉及的电子设备，可以是任何支持REE+TEE架构的电子设备，例如手机(mobile phone)、平板电脑、笔记本电脑、掌上电脑、移动互联网设备(mobile internet device，MID)、可穿戴设备(例如，手表、手环、智能头盔等)，虚拟现实(virtual reality，VR)设备、增强现实(augmented reality，AR)设备、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital assistant，PDA)，工业控制(industrial control)中的无线终端、无人驾驶(self driving)中的无线终端、远程手术(remote medical surgery)中的无线终端、智能电网(smart grid)中的无线终端、运输安全(transportation safety)中的无线终端、智慧城市(smart city)中的无线终端、智慧家庭(smart home)中的无线终端等。
需要说明的是，本申请实施例涉及的电子设备还可以是任意的、支持两种或两种以上运行环境的电子设备，本文中，以支持REE和TEE的电子设备为例，但是这并不构成对其的限定，支持其他的两种运行环境的电子设备也是可以的。
2)、富执行环境(rich execution environment，REE)，是指电子设备中不具备特定安全功能的运行环境。例如，Android操作系统是一种富执行环境。
需要说明的是，REE除了被称为“富执行环境”之外，还可以被称为“不可信执行环境”、“普通执行环境”、“不安全执行环境”等等，本申请实施例对此不作限定。
3)、可信执行环境(trusted execution environment，TEE)，是具有更高安全能力的运行环境。TEE通过硬件的支撑，具有安全能力并且能够满足一定的安全需求。TEE定义了严格的保护措施，因此比REE的安全级别更高，能够保护TEE中的资产(如数据，软件等)免受软件攻击，抵抗特定类型的安全威胁。
4)、REE+TEE架构，是指通过TEE与REE结合共同为应用提供服务的架构。也就是说，TEE与REE共同存在于电子设备中。示例性的，TEE通过硬件的支撑，可实现与REE相隔离的运行机制。TEE有自身的运行空间，比REE的安全级别更高，能够保护TEE中的资产(如数据，软件等)免受软件攻击。只有授权的安全软件才能在TEE中执行，同时它也保护了安全软件的资源和数据的机密性。相比REE，由于其隔离和权限控制等保护机制，TEE能够更好的保护数据和资源的安全性。
5)、客户端应用(client application，CA)，通常是指运行在REE中的应用。CA在一定条件下，可以调用TA；例如，CA中的涉及用户信息的业务被触发，则CA调用TA。例如，CA可以通过客户应用程序编程接口(application programming interface，API)对TA进行调用。当CA调用TA之后，TA在TEE中运行，执行相关的安全操作。
6)、可信应用(trusted application，TA)，是指运行在TEE中的应用，能够为运行在TEE之外的客户端应用(client application，CA)提供安全相关的服务。可以理解的是，一个CA对应一个TA(例如，一个TA为对应的CA提供不同的业务)，或者一个CA可以对应多个TA(例如，一个CA中不同业务对应不同的TA)，等等，本申请实施例不作限定。
7)、可信用户界面(trusted user interface，TUI)，是TEE操作系统提供的一项安全功能，可以理解为是由TEE控制的安全显示界面，能够为在TEE中运行的各TA提供可信的用户交互界面，可确保TA在TEE中运行时，显示内容无法被REE获取，可提升业务的安全性。全球平台组织(global platform)对TUI的常用API进行了标准化，方便TA开发者利用这些API进行TUI的开发。通过TUI，可以实现安全显示、安全输入、安全指示标记三项最基础的安全用户交互功能。比如在金融领域，央行的手机盾规范中，就明确要求通过TUI保障输密、签名内容显示界面的安全性。
目前，由于TEE的处理能力有限，例如，绘图能力较弱，所以TEE控制的TUI比较呆板，无法呈现丰富的内容。
8)、手机盾，是一种用手机实现银行U盾(usbkey)功能的产品或者应用程序。手机盾可以在支持TEE+REE架构的手机中实现相关业务，手机中的TEE可以为手机盾提供密码运算和CA数字证书等全部功能所需的高安全环境。手机盾为移动互联网应用提供了密码运算支撑能力，用于身份认证、电子签名、数据保护等。
下面以手机盾业务为例，介绍现有TEE+REE安全架构下的业务执行流程。在TEE+REE架构下，手机盾签名密钥被保存在TEE中，所有签名操作都在TEE内执行，进而确保签名的过程达到金融级安全。用户可以在手机上执行一些大额交易或其他高敏感操作，当一笔业务需要手机盾签名才能进行时，待签名信息被发送到TEE，通过TUI显示并由用户进行确认，用户确认无误后输入手机盾个人通用身份标识码(personal identification number， PIN)，该PIN被发送到TEE内的签名小程序进行验证，验证通过后，用户确认过的交易信息被发送到小程序进行签名，通过数字签名技术确保交易不可抵赖。
在上述手机盾业务执行流程中，TUI有两个重要作用：1、确保用户输入PIN的安全性，仅有手机盾才能访问，REE侧的应用无法通过读屏，读键盘等方式获取用户输入，确保PIN不会泄露，同时也能保证PIN是实时输入的。2、保证“所见即所签”(或者反过来说“所签即所见”)，即，通过TUI保证送入手机盾进行签名的交易信息，不会被篡改或替换，这样，在用户确认交易信息无误后，送入到TEE内进行签名的信息就是用户确认过的信息。目前，业界普遍认为REE侧是不安全的，各种攻击方式都可能实现，如果没有TEE的引入，交易信息、用户PIN等都有可能被恶意程序获取、篡改，进而恶意程序/黑客能够跳过用户进行交易，或者是修改交易内容进行欺诈。显然，TUI的引入，对REE侧可能的恶意攻击行为起到明显的防御作用。但是，如前文所述，由于TEE的绘图能力较弱，所以TUI比较呆板，无法呈现丰富的内容。本申请实施例的显示方法旨在提升TUI的丰富性。
9)、本申请实施例涉及的应用程序(application，app)，可以简称应用，为能够实现某项或多项特定功能的软件程序。通常，电子设备中可以安装多个应用，例如，即时通讯类应用、视频类应用、音频类应用、图像拍摄类应用、支付类应用等等。其中，即时通信类应用，例如可以包括短信应用、微信(WeChat)、WhatsApp Messenger、连我(Line)、照片分享(instagram)、Kakao Talk、钉钉等。图像拍摄类应用，例如可以包括相机应用(系统相机或第三方相机应用)。视频类应用，例如可以包括Youtube、Twitter、抖音、爱奇艺，腾讯视频等等。音频类应用，例如可以包括酷狗音乐、虾米、QQ音乐等等。支付类应用，例如支付宝、微信、各类银行app等。本申请实施例中提到的应用，可以是电子设备出厂时已安装的应用，也可以是用户在使用电子设备的过程中从网络下载或其他电子设备获取的应用。
10)、本申请实施例涉及的至少一个，包括一个或者多个；其中，多个是指大于或者等于两个。另外，需要理解的是，在本申请的描述中，“第一”、“第二”等词汇，仅用于区分描述的目的，而不能理解为指示或暗示相对重要性，也不能理解为指示或暗示顺序。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。
参阅图1所示，为现有的TEE+REE架构的示意图。如图1所示，该架构包括TEE和REE；其中，REE中可运行CA，TEE中可运行TA。
该架构可以实现上述的机制二，当CA在REE中运行时，REE控制显示屏以及触摸屏；当TA在TEE中运行时，TEE控制显示屏以及触摸屏。具体的，CA可以通过富执行环境操作系统内核层(rich execution environment operation system kernel，REE OS kernel)中的显示屏驱动(display screen driver，DSD)控制显示屏，通过REE OS kernel中的触摸屏驱动(touch screen driver，TSD)控制触摸屏。TA可以通过可信执行环境操作系统内核层(trusted execution environment operation system kernel，TEE OS kernel)中的DSD控制显示屏，通过TEE OS kernel中的TSD控制触摸屏。也就是说，TA在TEE中运行时，显示屏显示TEE提供/绘制的显示内容，而且TEE检测触摸屏上的用户操作(例如，口令的输入等)。
1)目前，当TA在TEE中运行时，TEE控制显示屏的方式为：仅控制显示屏中的第 一个图层。具体而言，参见图2所示，为显示屏的逻辑结构示意图。显示屏包括多个图层(display screen)，例如，图层1-图层N，N为大于等于2的整数；触摸屏在图层1的上层。现有机制中，当CA在REE中运行时，REE独占显示屏，即REE控制显示屏中的所有图层。当TA在TEE中运行时，TEE控制显示屏中的图层1。即TEE仅绘制第一图形，然后将该第一图形显示于图层1上，由于图层1的第一图形会遮挡下层即图层2-图层N的图形，所以最终呈现出的TUI仅包括图层1上的内容，即TEE绘制的显示内容。但是，受限于全球标准平台(global platform，GP)的标准规范，TEE的处理能力有限，所以TUI通常比较呆板，仅能显示一些文字、按键、文本框等内容，其它内容例如状态栏(电量信息、运营商信息、时间信息等)等无法显示。
2)目前，当TA在TEE中运行时，TEE控制触摸屏。用户可能触发各类用户操作，例如，全面屏手势、上滑/下拉菜单等操作。这些操作与REE中的业务相关，与TEE中的业务不相关，TEE无法将这些操作发送给REE，TEE自身又无法响应这些操作，所以导致用户在TUI上触发各类操作，但TUI仅能识别有限的操作类型(例如按键点击)，无法识别并处理其它操作，进而导致现有的TUI无法提供丰富的人机交互体验。
3)目前，当TA在TEE中运行时，REE中的CA的相关业务无法传递到TEE，例如，电子设备控制显示屏显示TUI时，REE中运行的应用接收到新消息或通知时，该新消息或通知无法传递到TEE，所以TEE无法在TUI中显示该新消息和通知，所以目前TUI的交互体验较差。
基于上述存在的问题，本申请实施例提供一种图像显示方法，该方法可以应用于部署有TEE和REE的电子设备。该显示方法为TEE和REE融合显示机制。具体的，当TA在TEE中运行时，TEE控制显示屏中的图层1，REE控制显示屏中的图层2-图层N。TEE绘制第一图形，并将第一图形显示于图层1上，REE绘制第二图形，并将第二图形显示在图层2-图层N中的一个或多个图层上；第一图形和第二图形融合(或称叠加、重叠等)得到TUI。其中，第一图形包括非透明区域和透明区域，非透明区域包括TEE绘制的内容例如口令输入信息，第一图形上的透明区域不会遮挡第二图形上与透明区域对应的区域的显示内容，因此，第一图形和第二图形融合显示之后的显示效果为：显示界面即TUI中既包括TEE绘制的内容，也包括REE绘制的内容。也就是说，借助REE的强大的绘图能力以及丰富的显示资源，第二图形中可以包括较为丰富的信息，例如状态栏、背景等，所以最终得到的显示界面较为丰富，解决了上述1)中TUI呆板的问题。
此外，当电子设备显示TUI时，REE可以更新第二图形，例如，当REE中的应用收到通知消息时，REE可以在第二图形上与第一图形中透明区域对应的区域中添加该通知消息，所以图层1上的第一图形不会遮挡该通知消息，所以最终呈现的显示效果为TUI中显示该通知消息。因此，解决了上述3)中TUI中交互体验差的问题。
进一步的，本申请实施例还提供REE和TEE融合输入机制：REE和TEE之间建立用于传输用户输入的通道，例如，REE中包括通信模块1，该通信模块1用于接收TEE发送的信息；TEE中包括通信模块2，该通信模块2用于接收向REE发送信息。当TA在TEE中运行时，TEE控制触摸屏，TEE检测到用户在TUI上的输入之后，判断用户输入由TEE处理还是由REE处理，若由REE处理，则将该用户输入通过所述通道发送给REE；若由TEE处理，则TEE内应用处理该用户输入。因此，解决了上述2)中TUI中人机交互体验差的问题。上述通信模块1和通信模块2可以是核间通信(interprocess communication，IPC) 模块，也可以称为进程间通信模块，或者其它底层硬件通信模块。
下面对上述TEE和REE融合显示机制和融合输入机制进行详细介绍。参见图3所示，为本申请实施例提供的支持TEE+REE架构的电子设备的显示方法的流程示意图，如图3所示，该方法的流程包括：
301，构建一个或多个显示模板。
可以理解的是，显示模板即绘制图形时的参考模板。换言之，显示模板中包括显示区域的布局(位置、尺寸等)、每个区域中的显示内容等。方式A，一个显示模板即一个图形模板，该图形模板上划分不同的区域，每个区域用于显示对应的内容。TEE和REE可以约定显示模板中哪些区域的内容由TEE提供，哪些区域中的内容由REE提供。为了安全起见，TEE和REE中可以约定显示模板中与口令输入相关的内容由TEE提供，其它内容由REE提供。方式1中，显示模板相对简单。
方式B，一个显示模板包括多个图形模板，一个图形模板对应显示屏中的一个图层。例如，显示模板中包括2个图形模板，图形模板1对应显示屏上的图层1，图形模板2对应显示屏上的图层2。TEE控制显示屏的图层1时，可以参照图形模板1绘制第一图形，REE控制显示屏上的图层2时，可以参照图形模板2绘制第二图形。方式2中，TEE和REE无需约定绘制内容，提升效率。
显示模板也可以不需要，即每次绘制时都根据应用或操作系统提供的绘制逻辑来绘制。REE和TEE需要约定的部分可以通过预先设置的绘制逻辑来控制。
需要说明的是，301可以由REE执行，也可以由TEE执行。假设REE构建显示模板，则构建完成后，可以将显示模板发送给TEE。假设TEE构建显示模板，则构建完成后，可以将显示模板发送给REE，总之，实现TEE和REE共享显示模板的即可。
作为一种示例，显示模板与应用无关，TEE或REE构建一个或多个显示模板后，形成共享池，该共享池中包括所述一个或多个显示模块，任何应用在TEE中运行时，TEE都可以从所述共享池中选择一个显示模块使用，例如，可以根据当前场景(例如当前运行的应用，或应用内的细分场景)从共享池中选择合适的显示模板，具体在后文介绍。应理解的是，REE和TEE需要使用同一显示模板。
作为另一种示例，显示模板与应用相关。例如，电子设备下载应用时，一并下载该应用对应的显示模板，例如，显示模块可以是所述应用对应的提供商所提供。REE和TEE可以共享每个应用对应的显示模板，当某个应用在TEE中运行时，使用该应用对应的显示模板即可。
可以理解的是，显示模板可以有一个或多个，不同的场景下可以对应不同的显示模板。其中，示例性的，场景可以包括：
场景1，U盾场景，即电子设备运行U盾以实现相关业务的场景，该场景可以对应一个显示模板。
场景2，锁屏场景，即电子设备处于锁屏且暗屏状态，当电子设备检测到输入操作后，点亮电子设备且显示锁屏界面，该锁屏界面可以对应一个显示模板。
场景3，账号登录场景，例如，电子设备检测到输入操作，响应于该输入操作启动第一应用例如某个银行APP，显示账号登录界面，该账号登录界面可以对应一个显示模板。
场景4，转账场景，电子设备运行第一应用例如某个银行APP，当检测到输入操作，响应于该输入操作启动第一应用中的与安全相关的业务(例如，转账业务)时，显示密码 输入界面，该密码输入界面可以对应一个显示模板。
需要说明的是，上述4个场景仅是举例，而非限定，只要涉及口令输入的任何场景都可以适用，本申请实施例不一一列举。
可选的，多个场景也可以对应同一显示模板。例如，上述场景1-场景4中至少两个场景对应同一显示模板。例如，上述场景3和场景4对应同一显示模板。
示例性的，以方式A为例，参见图4所示，为本申请实施例提供的两种显示模板的示意图。图4中的(a)为锁屏场景即场景2对应的显示模板一的示意图，图4中的(b)为登录银行app场景即场景3对应的显示模板二的示意图。以图4中的(a)所示的显示模板一为例，显示模板一中的部分区域显示TEE提供的内容，另一部分区域显示REE提供的内容。
302，应用在TEE中运行。
302中的应用可以是经过TEE授权的软件，例如支付宝、银行客户端、锁屏应用(screen lock App)等。
302可能存在多种情况，例如，情况1，电子设备显示主界面(home screen)，该主界面中包括一个或多个应用的图标。假设某个应用设置有应用锁，当电子设备检测到针对该应用的图标的操作时，该应用在TEE中运行，显示所述应用的应用锁界面，其中包括口令输入区域。情况2，应用正在REE中运行，当检测到特定输入操作，所述特定输入操作用于启动所述应用中特定业务，例如涉及用户隐私信息的业务，如转账、支付等；响应于该特定输入操作，该应用切换到TEE中运行。其中，应用从REE切换到TEE中运行，即CA在REE中运行，然后CA调用TEE中与所述CA对应的TA，然后TA在TEE中运行。
303，TEE根据当前场景，确定合适的显示模板。
方式1，一个应用即一个场景，一个场景对应一个显示模板，所以，303中TEE根据运行的应用，确定与该应用对应的显示模板。举例来说，电子设备在TEE中运行招商银行APP时，可以使用招商银行APP提供的显示模板进行图形绘制。
方式2，一个应用中不同业务对应不同场景，例如，一个应用中包括转账业务、支付业务等等，不同业务对应不同场景。TEE可以根据当前场景确定对应的显示模板。示例性的，假设TEE确定当前处于锁屏场景，则选择图4中的(a)所示的显示模板一；若TEE确定当前处于转账场景，则选择图4中的(b)所示的显示模板二。
一个CA对应一个或多个TA，每个TA用于为该CA提供不同的与安全相关的业务。因此，当REE中的CA调用TEE中与该CA对应的TA时，TEE可以根据被调用的TA确定当前运行的应用或当前运行的应用内的业务，进而确定合适的显示模板。
举例来说，对于上述情况1，电子设备检测到针对应用1的图标的操作时，确定应用1设置有应用锁，CA调用TEE中用于处理应用1中与应用锁相关的业务的TA，然后，该TA在TEE中运行。这样的话，TEE根据被调用的TA，确定当前运行的应用为应用1，然后确定合适的显示模板。再例如，对于上述情况2，应用正在REE中运行，当检测到特定输入操作，确定该特定输入操作用于启动所述应用内的特定业务，调用TEE中与所述特定业务对应的TA，该TA在TEE中运行。因此，TEE可以根据所述TA，确定当前运行的应用，以及应用内运行的业务，然后确定合适的显示模板。
在其他一些实施例中，TEE还可以和REE共同确定显示模板，具体的，二者可通过信息发送来协商确定。在另外一些实施例中，TEE还可以接受REE确定的显示模板。
304，TEE根据确定出的显示模板，绘制第一图形，第一图形中包括透明区域和非透明区域，其中非透明区域为TEE绘制的内容。
前面介绍显示模板的两种方式，即方式A和方式B。
对于方式A，TEE与REE约定显示模板中哪些区域是TEE提供，哪些区域是REE提供。示例性的，假设TEE确定的显示模板是图4中的(a)所示的显示模板一，并约定显示模板一中口令输入区域的内容由TTE提供，所以TEE可以绘制第一图形，该第一图形上与口令输入区域对应的区域中包括口令输入信息，例如数字按键值等，第一图形上的其它区域为透明区域。示例性的，TEE绘制的第一图形如图5中的(a)所示，第一图形上除去数字按键值之外都是透明区域。
对于上述方式B，由于显示模板中包括每个图层对应的图形模板。假设第一个图层上显示TEE绘制的图形，第二个图层到第N个图层上显示REE绘制的图形，则TEE仅需根据第一个图层对应的图形模板绘制第一图形即可。
305，TEE在显示屏中第一图层显示第一图形。
可以理解的是，电子设备中触摸屏位于显示屏的上层，而显示屏包括多个图层，所以显示屏中第一图层可以是靠近触摸屏的一个或多个图层，例如，第一个图层，或者，第一个图层以及第二个图层。参见图6所示，显示屏包括多个图层，显示屏上的最终的TUI是由各个图层上的图形融合(或称叠加)得到的。TEE可以在第一图层例如图层1上显示第一图形。
306，TEE向REE发送显示模板的标识。
需要说明的是，本申请实施例不限定306与304、305之间的执行顺序。
可以理解的是，TEE和REE共享显示模块，当TEE将该显示模块的标识发送给REE之后，REE可以基于该标识确定对应的显示模板。
307，REE根据所述显示模板，绘制第二图形。
对于上述方式A，TEE与REE约定显示模板中哪些区域需要TEE提供，哪些区域需要REE提供。示例性的，假设TEE确定的显示模板是图4中的(a)所示的显示模板一，并约定显示模板一中仅“口令输入区域”的内容由TTE提供，其它区域的内容由REE提供，所以REE根据显示模板绘制第二图形。较为简单的方式为，REE可以仅绘制第二图形上与第一图形上的透明区域对应的区域，不绘制与非透明区域对应的区域；当然，REE也可以绘制全部的第二图形。
对于上述方式B，由于显示模板中包括每个图层对应的图形模板，假设第一个图层上显示TEE绘制的图形，第二个图层到第N个图层上显示REE绘制的图形，则REE根据第二个图层到第N个图层对应的图形模板绘制第二图形即可。
308，REE在显示屏中第二图层显示第二图形。
需要说明的是，307-308可以在303、304或305之前执行，也就是说，REE先绘制第二图形，然后TEE绘制第一图形；或者，TEE先绘制第一图像，然后REE绘制第二图形都可以，本申请实施例不作限定。当307在303之前执行时，REE可以根据当前场景确定合适的显示模板，然后，将显示模板的标识发送给TEE，只要TEE和REE使用的显示模板相同即可。
电子设备中触摸屏位于显示屏的上层，显示屏包括多个图层，所以显示屏中第二图层可以是远离触摸屏的一个或多个图层。示例性的，305中“第一图层”可以是第一个图层， 308中“第二图层”可以是第二个图层到第N个图层；或者，“第一图层”可以是第一个图层到第3个图层，“第二图层”可以是第4个图层到第N个图层，等等，本申请实施例不作限定。
参见图6所示，图层1上显示第一图形，图层2-图层N显示第二图形。假设REE生成的第二图形如图5中的(b)所示，TEE绘制的第一图形为如图5中的(a)所示，在图层1上显示第一图形，第一图形中的透明区域无法遮挡第二图形上对应区域的内容，第一图形和第二图形融合得到最终的显示界面。例如，最终融合得到的显示界面为图5中的(c)所示的界面。因此，当应用在TEE中运行时，显示界面中不仅包括TEE提供的内容，还包括REE提供的内容，例如包括状态栏、背景等等，显示内容较为丰富。
需要说明的是，以锁屏场景为例，现有的TEE+REE架构中，锁屏界面是REE生成的界面，由于REE具有强大的绘图能力，所以锁屏界面中包括丰富的信息，例如状态栏(间、运营商、电量等信息)，但是，由于REE安全性较低，当用户在锁屏界面中输入解锁密码时，容易泄露密码。本申请实施例提供的显示方法中，图5中的(c)中的锁屏界面是融合TEE绘制的第一图形和REE绘制的第二图形之后得到的TUI，该显示界面中关键信息例如口令输入信息是TEE绘制的，其它信息是REE绘制的。因此，REE无法获知口令输入信息，提升安全性，而且，借助REE的绘图能力，增加了TUI的丰富性。
需要说明的是，应用在TEE中运行的同时，REE中的应用可能会收到通知消息，此时REE可以更新第二图形，例如在第二图形上与第一图形中的透明区域对应的区域中增加所述通知消息，因此，第一图形和更新后的第二图形融合/重叠之后，显示所述通知消息。结合图5中的(c)和图7，图5中的(c)为TEE绘制的第一图形和REE绘制的第二图形融合之后的界面。此时，若REE中的短信收到通知消息，REE在第二图形上与第一图形中透明区域对应的区域(例如，状态栏区域)中绘制该通知消息，所以第一图形和更新后的第二图形融合之后，显示通知消息，如图7所示。
需要说明的是，现有的TEE+REE架构中，应用在TEE中运行时显示TUI，当REE中的应用收到通知消息时，TEE无法获取该通知消息，而REE不参与TUI的绘制，所以TUI中无法显示所述通知消息。但是，本申请实施例中，电子设备显示TUI时，当REE中的应用收到通知消息时，TUI中可以显示该通知消息，提升TUI的丰富性。
可以理解的是，到308为止，电子设备显示屏上的TUI为融合TEE绘制的第一图形和REE绘制的第二图形之后的界面，用户可以在TUI上执行用户输入，本申请实施例采用TEE和REE融合输入机制，具体通过如下步骤309-311实现：
309、TEE检测用户输入。
需要说明的是，在302之后，TEE控制触摸屏，即TEE检测触摸屏上的用户输入。示例性的，假设电子设备显示如图5中的(c)所示的TUI，该TUI中部分内容是TEE提供的，部分内容是REE提供的，用户可能在该TUI上的任意位置执行任意操作例如上滑/下滑等操作。因此，TEE检测到用户输入之后，可以判断该用户操作由TEE响应还是REE响应，即310。
310，TEE判断该用户输入由TEE响应还是由REE响应。
可以理解的是，应用在TEE中运行时，TEE根据显示模板绘制的第一图形，该第一图形中包括与所述应用相关的信息，例如口令输入信息。因此，TEE可以判断用户输入是所述应用内输入(下文中将应用内输入简称为APP内输入)还是非所述应用内输入(下文中 将非应用内输入简称非APP输入)。其中，APP内输入包括APP内控制信息输入和APP内文本信息输入；APP内文本信息输入例如在TUI中输入的口令，其他非APP内文本信息输入比如点击TUI上的返回按键或者点击用于调整键盘输入法的按键等输入属于APP内控制信息输入。非APP内输入例如包括：全面屏手势操作、下拉菜单操作、屏幕底部虚拟按键操作等不是在所述应用内的输入。
方式一，TEE将非APP内输入发送给REE处理，将APP内输入发送TEE内部应用处理。可以理解的是，TEE向REE发送非APP内输入之前，TUI Service退出，释放触摸屏和显示屏图层1，并清除在图层1的内容，REE收到非APP内输入后，通过图层1-图层N输出响应。该方法中，应用在TEE中运行时，所有APP内输入均由TEE处理，不经过REE，安全性较高。
方式二，TEE将非APP内输入、APP内控制信息输入发送给REE处理，将APP内文本信息输入发送TEE内部应用处理。该方法中，应用在TEE中运行时，仅APP内文本信息输入(例如，口令输入)由TEE处理，其它输入通过REE处理，缓解TEE的处理压力。
以图5中的(c)为例，锁屏应用(screen lock App)在TEE中运行，TEE绘制的第一图形中仅包括数字按键值。TEE检测到用户输入口令后，判断用户输入是APP内文本信息输入，则将用户输入发送TEE中锁屏应用处理，若判断用户输入是APP内控制信息输入，例如点击返回按键，可以将用户发送输入REE，也可以将用户输入发送TEE内锁屏应用。
TEE判断用户输入是APP内输入还是非APP内输入的方式可以是如下方式中至少一种：
方式一，304中，TEE绘制的第一图形包括透明区域和非透明区域，非透明区域包括TEE绘制的信息。因此，TEE判断用户输入是应用内输入还是非应用内输入的方式可以是，判断用户输入的触控位置是否位于非透明区域内，若是，则确定用户输入是应用内输入；若否，则确定用户输入是非应用内输入。
方式二，TEE判断用户输入是应用内输入还是非应用内输入的方式还可以是，判断用户输入的操作类型是否是预设操作类型，预设操作类型包括点击操作，长按操作等；若是，则确定用户输入是应用内输入；若否(例如是上滑、下滑等操作)，则确定用户输入是非应用内输入。可以理解的是，上滑/下滑等手势操作通常不是用于输入口令的操作，所以TEE可以根据用户操作的操作类型判断用户操作是否是APP内输入操作。
需要说明的是，上述方式一和方式二可以单独使用，或者，为了提升准确性，方式一和方式二可以结合使用，例如，TEE判断用户输入是否在非透明区域内，若是，继续判断用户输入的操作类型是否是预设操作类型，若是，则TEE内的应用响应用户输入；若用户输入不在非透明区域或者操作类型不是预设操作类型，则将用户输入发送REE处理。
311，TEE将用户输入发送给REE。
可以理解的是，TEE与REE之间可以建立用于传输用户输入的通道。如果309中TEE判断用户输入由REE处理，则将该用户输入通过所述通道发送给REE。REE接收到用户输入之后，响应用户输入。例如，以图5中的(c)为例，TEE检测到用户操作，TEE判断用户操作位于状态栏，且该用户操作的操作类型是向下滑动操作，TEE将用户操作发送给REE，REE响应用户操作，显示下拉菜单。
312，TEE内所述应用响应该用户输入。
302中应用在TEE中运行，如果309中TEE判断用户输入由TEE响应，则TEE中所 述应用响应用户输入，例如，用户输入为APP内文本信息输入，则所述应用将输入的文本信息与预存的文本信息进行比对，若比对一致，则执行相应的处理流程，例如解锁电子设备、登录应用、转账、支付等流程。
需要说明的是，上述步骤301-308中，当应用在TEE中运行时，电子设备显示TUI，该TUI是TEE绘制的第一图形与REE绘制的第二图形的融合之后的显示界面，解决了传统技术中TUI呆板的问题。具体而言，当应用在TEE中运行时，TEE在图层1上显示第一图形，REE在图层2-图层N显示第二图形，第一图形和第二图形融合得到最终的显示界面。此时，若REE中的应用收到通知消息，可以在第二图形中添加该通知消息，这样的话，第一图形和更新后的第二图形融合之后得到的显示界面中可以显示所述通知消息。因此，解决了现有技术中，当电子设备显示TUI时，TUI中无法显示REE中应用收到通知消息的问题，提升了TUI的显示丰富性。
上述步骤309-311中，TEE检测到用户输入之后，若用户输入是APP内文本信息输入，则TEE内应用响应该用户输入，若用户输入不是APP被文本信息输入，则TEE将用户输入发送给REE，由REE进行处理。因此，解决了现有技术中TUI中用户操作灵敏性较差的问题。
本申请实施例提供的显示方法可以通过图8所示的TEE+REE架构实现。与图1所示的现有架构不同的是，该架构中，REE中增加了可信环境系统命令执行代理(agent for command from REE，Agent_ree)，该Agent_ree可以作为通信模块，REE可通过Agent_ree接收TEE发送的用户输入。对应的，TEE中也增加通信模块，该通信模块用于向REE发送用户输入。此外，相较于图1所示的现有架构，该架构中的部分模块新增相应的功能，例如，TEE与REE共享显示模板；又例如，当TA在TEE中运行时，TEE在显示屏中图层1显示第一图形，REE在显示屏上图层2-图层N显示第二图形；又例如，TEE中的TUI service增加判断用户操作是否由TEE响应的功能。
以下对图8所示的TEE+REE架构中各个模块进行详细介绍。
REE中包括：CA、框架应用编程接口(Framework API)、可信客户端应用编程接口(TEE client application programming interface，TEE Client API)、富执行环境操作系统内核层(REE OS kernel)包括：显示屏驱动(display screen driver，DSD)、触摸屏驱动(touch screen driver，TSD)、Agent_ree。
当CA在REE中运行时，CA通过Framework API调用REE OS kernel中的Display Screen Driver，进而控制显示屏显示CA提供的显示界面。当用户在触摸屏上用户输入，Touch Screen Driver将该用户输入通过Framework API发送CA，CA响应该用户输入。
TEE中包括：TA、可信应用编程开发接口(TEE internal application programming interface，TEE Internal API)、可信执行环境中应用编程开发接口(TEE OS kernel)包括：安全用户接口服务组件(trusted User interface service，TUI Service)、Display Screen Driver、Touch Screen Driver、通信模块。该通信模块可以用于向REE中的Agent_ree发送用户输入。应理解，通信模块也可以集成于其它模块中，不需独立设置，例如通信模块的功能集成于TUI Service中，即由TUI Service向REE发送用户输入。因此，图8中通信模块以虚线表示。
TA在TEE中运行，TA通过TEE Internal API调用TUI service，TUI service通过display模块控制显示屏中图层1显示第一图形，REE通过REE OS kernel中display模块控制显示 屏中图层2-图层N显示第二图形。第一图形和第二图形融合(或称重叠)之后得到TUI。当用户在触摸屏上输入操作，touch模块将输入操作发送TUI service，TUI service判断TEE是否响应输入操作，若是，则将输入操作发送TA进行响应，若否，则TUI service通过通信模块将输入操作发送REE中的Agent_ree，Agent_ree将输入操作发送CA进行响应。
参见图9所示，为通过图8所示的TEE+REE架构实现本申请实施例提供的显示方法的流程图，所述流程可以包括：
901，CA调用TA。
具体的，REE中的CA通过TEE Client API调用TEE的TA。
902：TA调用TUI Service。
903：TUI Service根据显示模板绘制第一图形，并通过TEE OS kernel中Display Screen Driver在显示屏图层1上第一图形。
需要说明的是，903中TUI Service可以根据当前场景确定合适的显示模板，然后根据显示模板绘制第一图形，具体过程不再重复赘述。
904：CA根据显示模板绘制第二图形，并通过REE OS kernel中的Display Screen Driver在显示屏的图层2-图层N显示第二图形。
可以理解的是，903和904中使用的显示模块是同一个显示模板。需要说明的是，904与901-903之间的执行顺序，本申请实施例不作限定。
905，TEE中的Touch Screen Driver检测到用户输入，并发送TUI Service。
906，TUI Service判定用户输入的类型；若是非APP内输入，则执行907-909，若是APP内输入，则执行910。
907、TUI Service退出TUI服务。退出TUI服务可以包括清空图层1的显示内容，将图层1的控制权交付REE等。
908，TUI Service将用户输入发送Agent_ree。
909，Agent_ree将用户输入发送CA，CA处理用户输入。
需要说明的是，本申请实施例不限定907-909之间的执行顺序。
910，TUI Service判定TUI是否满足退出条件或TUI输入完成，若是，则执行911，若否，则执行905。其中，满足TUI退出条件可以包括：用户在TUI中点击取消按键触发TUI退出，或者，异常事件触发TUI退出，包括点击power键、来电、截屏等操作，或者，TA崩溃(Crash)触发TUI退出，或者，显示TUI预设时长例如120s无任何操作自动退出TUI。其中，TUI输入完成可以包括输入文本信息例如数字信息的个数达到预设个数，或者，用户在TUI中点击完成按键等。
911，TUI Service将输入信息发送TA，TA进行处理(例如进行安全验证)，得到处理结果(例如验证通过与否)。
912，TA将处理结果发送CA，CA进行后续处理。
示例性的，以锁屏界面为例，若处理结果是用户输入的口令验证通过，则CA响应于该处理结果，通过显示屏显示主界面。
如图10所示，为本申请实施例提供的一种电子设备1000的逻辑结构示意图。电子设备1000可以是手机、平板电脑等。如图10所示，该电子设备1000包括TEE和REE，其中，TEE包括第一处理模块1001和第一显示模块1002，REE包括第二处理模块1003和第二显示模块1004。
其中，第一处理模块1001，用于绘制第一图形；
第一显示模块1002，用于在所述电子设备的显示屏中的第一图层显示所述第一图形；
第二处理模块1003，用于绘制第二图形；
第二显示模块1004，用于在所述显示屏中的第二图层显示所述第二图形，其中，所述第一图层位于所述第二图层之上；
其中，所述第一图形上包括透明区域和非透明区域，所述非透明区域包括所述TEE绘制的内容，所述第二图形上与所述透明区域对应的显示区域包括所述REE绘制的内容，当所述第一图层和所述第二图层重叠显示时，所述透明区域不遮挡所述REE绘制的内容。
可选的，所述电子设备中触摸屏位于所述显示屏的上层，所述显示屏中包括多个图层，所述第一图层为靠近所述触摸屏的一个或多个图层，所述第二图层为远离所述触摸屏的一个或多个图层。
可选的，所述第一图层为所述显示屏中距离所述触摸屏最近的一个图层，所述第二图层为所述显示屏中除去所述第一图层之外的其它图层。
可选的，第一处理模块1001还用于：根据所述当前场景从一个或多个显示模板中确定与所述当前场景对应的显示模板；第一处理模块1001具体用于：根据所述显示模板绘制所述第一图形，第二处理模块1003具体用于：根据所述显示模板绘制所述第二图形。
可选的，所述显示模板为一个图形，所述图形上包括至少一个显示区域，其中的每个显示区域被配置为所述TEE绘制内容或者所述REE绘制内容；或者，所述显示模板包括至少一个图形，每个图形对应一个图层，每个图形被配置为所述TEE绘制或所述REE绘制。
可选的，所述TEE绘制的内容，包括；与口令输入相关的信息。
可选的，第一处理模块1001还用于：检测用户输入，并判断所述用户输入是否需要所述TEE响应；当所述用户输入不需要所述TEE响应时，通过所述TEE中的第一通信模块将所述用户输入发送给所述REE，所述REE响应于所述用户输入执行相应的处理流程。
可选的，第一处理模块1001在用于判断所述用户输入是否需要所述TEE响应时，具体用于：当确定所述用户输入是所述TEE内当前运行的应用内的输入时，确定所述用户输入需要所述TEE响应；当确定所述用户输入不是所述应用内的输入时，确定所述用户输入不需要所述TEE响应。
可选的，所述应用内的输入包括：位于所述非透明区域内的输入，和/或，操作类型满足预设操作类型的输入；非所述应用内的输入，包括：位于所述非透明区域外的输入，和/或，操作类型不是所述预设操作类型的输入。
可选的，第一处理模块1001还用于：当所述REE中的应用接收到通知消息时，更新所述第二图形，更新后的第二图形中与所述第一图形中透明区域对应的显示区域中显示所述通知消息；所述第二显示模块还用于，在所述第二图层中显示所述更新后的第二图形。
在一个例子中，以上电子设备中的单元/模块可以是被配置成实施以上方法的一个或多个集成电路，例如：一个或多个特定集成电路(application specific integrated circuit，ASIC)，或，一个或多个微处理器(digital singnal processor，DSP)，或，一个或者多个现场可编程门阵列(field programmable gate array，FPGA)，或这些集成电路形式中至少两种的组合。再如，电子设备中的单元/模块可以通过处理元件调度程序的形式实现，该处理元件可以是通用处理器，例如中央处理器(central processing unit，CPU)或其它可以调用程序的处理 器。再如，这些单元可以集成在一起，以片上系统(system-on-a-chip，SOC)的形式实现。
需要说明的是，第一处理模块1001、第二处理模块1003可以是不同的物理器件(这里，物理器件可以包括芯片、芯片系统、电路、电路系统等)，也可以是同一物理器件，例如都是处理器；处理器包括中央处理单元(central processing unit，CPU)、应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，存储器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。当第一处理模块1001和第二处理模块1003是同一物理器件时，可以通过不同的计算机程序调用该物理器件以实现不同的功能。
需要说明的是，第一显示模块1002和第二显示模块1004可以是电子设备中的显示屏，显示屏可以包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。
如图11所示，本申请实施例提供一种电子设备。该电子设备1100部署有REE和TEE(图中未示出)；还包括：显示屏1130；一个或多个处理器1110；存储器1120；其中，一个或多个处理器1110可以运行在REE或TEE中，当一个或多个处理器运行在TEE时，可以实现上述TEE执行的步骤，当一个或多个处理器运行在REE时，可以实现上述REE执行的步骤。一个或多个处理器1110可以通过计算机程序调度的方式在TEE和REE中切换运行。
其中，存储器1120中可以存储一个或多个计算机程序，所述一个或多个计算机程序被配置为被该一个或多个处理器1110执行，该一个或多个计算机程序包括指令，上述指令可以用于执行前述实施例中的各个步骤。
基于与上述方法实施例相同构思，本申请实施例中还提供一种计算机存储介质，所述计算机存储介质存储有计算机可执行指令，所述计算机可执行指令在被计算机调用时，使所述计算机执行上述提供的方法实施例中的各个实施例的具体过程。本申请实施例中，对所述计算机可读存储介质不做限定，例如，可以是RAM(random-access memory，随机存取存储器)、ROM(read-only memory，只读存储器)等。
基于与上述方法实施例相同构思，本申请实施例中还提供一种计算机程序产品，所述计算机程序产品中存储有指令，当其在计算机上运行时，使得计算机执行上述任意一种可能的设计中提供的方法。
本领域内的技术人员应明白，本申请实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机 程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本申请中一些可能的实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括本申请实施例以及落入本申请范围的所有变更和修改。
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (25)

  1. 一种图像显示方法，其特征在于，应用于包括可信执行环境TEE和富执行环境REE的电子设备，包括：
    所述TEE绘制第一图形，并在所述电子设备的显示屏中的第一图层显示所述第一图形；
    所述REE绘制第二图形，并在所述显示屏中的第二图层显示所述第二图形，其中，所述第一图层位于所述第二图层之上；
    其中，所述第一图形上包括透明区域和非透明区域，所述非透明区域包括所述TEE绘制的内容，所述第二图形上与所述透明区域对应的显示区域包括所述REE绘制的内容，当所述第一图层和所述第二图层重叠显示时，所述透明区域不遮挡所述REE绘制的内容。
  2. 如权利要求1所述的方法，其特征在于，所述电子设备中触摸屏位于所述显示屏的上层，所述显示屏中包括多个图层，所述第一图层为靠近所述触摸屏的一个或多个图层，所述第二图层为远离所述触摸屏的一个或多个图层。
  3. 如权利要求2所述的方法，其特征在于，所述第一图层为所述显示屏中距离所述触摸屏最近的一个图层，所述第二图层为所述显示屏中除去所述第一图层之外的其它图层。
  4. 如权利要求1-3任一所述的方法，其特征在于，所述方法还包括：
    所述TEE根据所述当前场景从一个或多个显示模板中确定与所述当前场景对应的显示模板；
    所述TEE绘制第一图形包括：所述TEE根据所述显示模板绘制所述第一图形，
    所述REE绘制第二图形包括：所述REE根据所述显示模板绘制所述第二图形。
  5. 如权利要求4所述的方法，其特征在于，所述显示模板为一个图形，所述图形包括至少一个显示区域，其中的每个显示区域被配置为所述TEE绘制内容或者所述REE绘制内容；
    或者，所述显示模板包括至少一个图形，每个图形对应显示屏上的一个图层，所述每个图形被配置为所述TEE绘制或所述REE绘制。
  6. 如权利要求1-5任一所述的方法，其特征在于，所述TEE绘制的内容，包括；与口令输入相关的信息。
  7. 如权利要求1-6任一所述的方法，其特征在于，所述方法还包括：
    所述TEE检测用户输入，并判断所述用户输入是否需要所述TEE响应；
    当所述用户输入不需要所述TEE响应时，将所述用户输入发送给所述REE，所述REE响应于所述用户输入执行相应的处理流程。
  8. 如权利要求7所述的方法，其特征在于，所述判断所述用户输入是否需要所述TEE响应，包括：
    当所述TEE确定所述用户输入是所述TEE内当前运行的应用内的输入时，确定所述用户输入需要所述TEE响应；
    当所述TEE确定所述用户输入不是所述应用内的输入时，确定所述用户输入不需要所述TEE响应。
  9. 如权利要求8所述的方法，其特征在于，所述应用内的输入包括：位于所述非透明区域内的输入，和/或，操作类型满足预设操作类型的输入；
    非所述应用内的输入，包括：位于所述非透明区域外的输入，和/或，操作类型不是所 述预设操作类型的输入。
  10. 如权利要求1-9任一所述的方法，其特征在于，所述方法还包括：
    所述REE中的应用接收到通知消息；
    所述REE更新所述第二图形，更新后的第二图形中与所述第一图形中透明区域对应的显示区域中显示所述通知消息；
    所述REE在所述第二图层中显示所述更新后的第二图形。
  11. 一种电子设备，其特征在于，包括可信执行环境TEE和富执行环境REE；其中，所述TEE中包括第一处理模块和第一显示模块，所述REE中包括第二处理模块和第二显示模块：
    所述第一处理模块，用于绘制第一图形；
    所述第一显示模块，用于在所述电子设备的显示屏中的第一图层显示所述第一图形；
    所述第二处理模块，用于绘制第二图形；
    所述第二显示模块，用于在所述显示屏中的第二图层显示所述第二图形，其中，所述第一图层位于所述第二图层之上；
    其中，所述第一图形上包括透明区域和非透明区域，所述非透明区域包括所述TEE绘制的内容，所述第二图形上与所述透明区域对应的显示区域包括所述REE绘制的内容，当所述第一图层和所述第二图层重叠显示时，所述透明区域不遮挡所述REE绘制的内容。
  12. 如权利要求11所述的电子设备，其特征在于，所述电子设备中触摸屏位于所述显示屏的上层，所述显示屏中包括多个图层，所述第一图层为靠近所述触摸屏的一个或多个图层，所述第二图层为远离所述触摸屏的一个或多个图层。
  13. 如权利要求12所述的电子设备，其特征在于，所述第一图层为所述显示屏中距离所述触摸屏最近的一个图层，所述第二图层为所述显示屏中除去所述第一图层之外的其它图层。
  14. 如权利要求11-13任一所述的电子设备，其特征在于，所述第一处理模块还用于：
    根据所述当前场景从一个或多个显示模板中确定与所述当前场景对应的显示模板；
    所述第一处理模块具体用于：根据所述显示模板绘制所述第一图形，
    所述第二处理模块具体用于：根据所述显示模板绘制所述第二图形。
  15. 如权利要求14所述的电子设备，其特征在于，所述显示模板为一个图形，所述图形上包括至少一个显示区域，其中的每个显示区域被配置为所述TEE绘制内容或者所述REE绘制内容；
    或者，所述显示模板包括至少一个图形，每个图形对应一个图层，每个图形被配置为所述TEE绘制或所述REE绘制。
  16. 如权利要求11-15任一所述的电子设备，其特征在于，所述TEE绘制的内容，包括；与口令输入相关的信息。
  17. 如权利要求11-16任一所述的电子设备，其特征在于，所述第一处理模块还用于：
    检测用户输入，并判断所述用户输入是否需要所述TEE响应；
    当所述用户输入不需要所述TEE响应时，通过所述TEE中的第一通信模块将所述用户输入发送给所述REE，所述REE响应于所述用户输入执行相应的处理流程。
  18. 如权利要求17所述的电子设备，其特征在于，所述第一处理模块在用于判断所述用户输入是否需要所述TEE响应时，具体用于：
    当确定所述用户输入是所述TEE内当前运行的应用内的输入时，确定所述用户输入需要所述TEE响应；
    当确定所述用户输入不是所述应用内的输入时，确定所述用户输入不需要所述TEE响应。
  19. 如权利要求18所述的电子设备，其特征在于，所述应用内的输入包括：位于所述非透明区域内的输入，和/或，操作类型满足预设操作类型的输入；
    非所述应用内的输入，包括：位于所述非透明区域外的输入，和/或，操作类型不是所述预设操作类型的输入。
  20. 如权利要求11-19任一所述的电子设备，其特征在于，所述第一处理模块还用于：
    当所述REE中的应用接收到通知消息时，更新所述第二图形，更新后的第二图形中与所述第一图形中透明区域对应的显示区域中显示所述通知消息；
    所述第二显示模块还用于，在所述第二图层中显示所述更新后的第二图形。
  21. 一种电子设备，其特征在于，所述电子设备部署有可信执行环境TEE和富执行环境REE，还包括显示屏、处理器和存储器；其中，所述存储器存储有计算机程序，当所述计算机程序被所述处理器执行时，使得所述电子设备实现如权利要求1-10任一项所述的方法。
  22. 一种计算机存储介质，其特征在于，所述计算机存储介质中存储有计算机程序，当计算机程序在部署有可信执行环境TEE和富执行环境REE的电子设备上运行时，使得所述电子设备实现如权利要求1-10任一项所述的方法。
  23. 一种计算机程序，其特征在于，包括指令，当所述指令在部署有可信执行环境TEE和富执行环境REE的计算机上运行时，使得所述计算机实现如权利要求1-10任一项所述的方法。
  24. 一种芯片，其特征在于，所述芯片能够运行在可信执行环境TEE和富执行环境REE，所述芯片用于读取存储器中存储的计算机程序，实现如权利要求1-10任一项所述的方法。
  25. 一种电子设备上的图形用户界面，其特征在于，所述电子设备部署有可信执行环境TEE、富执行环境REE，还包括显示屏、一个或多个存储器、以及一个或多个处理器，所述一个或多个处理器用于执行存储在所述一个或多个存储器中的一个或多个计算机程序，所述图形用户界面包括所述电子设备执行如权利要求1至10中任意一项所述的方法时显示的图形用户界面。
PCT/CN2020/110337 2019-10-24 2020-08-20 一种图像显示方法与电子设备 WO2021077882A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/727,148 US11948233B2 (en) 2019-10-24 2022-04-22 Image display method and electronic device

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201911016209 2019-10-24
CN201911016209.6 2019-10-24
CN201911082843.X 2019-11-07
CN201911082843.XA CN112711452B (zh) 2019-10-24 2019-11-07 一种图像显示方法与电子设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/727,148 Continuation US11948233B2 (en) 2019-10-24 2022-04-22 Image display method and electronic device

Publications (1)

Publication Number Publication Date
WO2021077882A1 true WO2021077882A1 (zh) 2021-04-29

Family

ID=75541137

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/110337 WO2021077882A1 (zh) 2019-10-24 2020-08-20 一种图像显示方法与电子设备

Country Status (3)

Country Link
US (1) US11948233B2 (zh)
CN (1) CN112711452B (zh)
WO (1) WO2021077882A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210166227A1 (en) * 2019-11-28 2021-06-03 Qualcomm Incorporated Secure User Interface With Improved User Experience
CN114036015A (zh) * 2021-10-07 2022-02-11 中国航空工业集团公司洛阳电光设备研究所 一种基于fpga的高安全性图形生成装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104820573A (zh) * 2015-05-27 2015-08-05 南京芯度电子科技有限公司 一种安全人机交互接口的系统及其实现方法
CN105683981A (zh) * 2014-08-21 2016-06-15 华为技术有限公司 一种安全交互方法和设备
US20160364723A1 (en) * 2015-06-15 2016-12-15 Kenneth W. Reese Virtual pos terminal method and apparatus
CN108781211A (zh) * 2016-03-01 2018-11-09 高通股份有限公司 用于装置的tee执行的用户界面
WO2019168315A1 (en) * 2018-02-27 2019-09-06 Samsung Electronics Co., Ltd. Trustzone graphic rendering method and display device using the same

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9183373B2 (en) * 2011-05-27 2015-11-10 Qualcomm Incorporated Secure input via a touchscreen
KR101925806B1 (ko) * 2011-12-02 2018-12-07 삼성전자 주식회사 보안 방법 및 장치
US8751800B1 (en) * 2011-12-12 2014-06-10 Google Inc. DRM provider interoperability
KR20130101629A (ko) * 2012-02-16 2013-09-16 삼성전자주식회사 보안 실행 환경 지원 휴대단말에서 컨텐츠 출력 방법 및 장치
US9344275B2 (en) * 2012-05-08 2016-05-17 Arm Technologies Israel Ltd. System, device, and method of secure entry and handling of passwords
US10002255B2 (en) * 2014-04-11 2018-06-19 Samsung Electronics Co., Ltd. Method and device for controlling security screen in electronic device
WO2016172944A1 (zh) * 2015-04-30 2016-11-03 华为技术有限公司 终端的界面显示方法和终端
CN104809413A (zh) * 2015-05-13 2015-07-29 上海瓶钵信息科技有限公司 基于TrustZone技术的移动平台可信用户界面框架
CN110059500A (zh) 2015-11-30 2019-07-26 华为技术有限公司 用户界面切换方法和终端
WO2017156784A1 (zh) * 2016-03-18 2017-09-21 华为技术有限公司 一种通知消息处理方法、装置及终端
CN107315970B (zh) * 2016-04-26 2020-03-20 展讯通信（上海）有限公司 一种敏感数据的交互方法及装置
CN106201479A (zh) * 2016-06-30 2016-12-07 深圳市金立通信设备有限公司 一种通讯消息的显示方法以及终端
CN106325802B (zh) 2016-08-16 2018-03-27 华为技术有限公司 一种显示方法、装置及操作图形用户界面的方法
CN106845285B (zh) * 2016-12-28 2023-04-07 北京握奇智能科技有限公司 一种tee系统与ree系统配合以实现服务的方法及终端设备
CN109766152B (zh) * 2018-11-01 2022-07-12 华为终端有限公司 一种交互方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105683981A (zh) * 2014-08-21 2016-06-15 华为技术有限公司 一种安全交互方法和设备
CN104820573A (zh) * 2015-05-27 2015-08-05 南京芯度电子科技有限公司 一种安全人机交互接口的系统及其实现方法
US20160364723A1 (en) * 2015-06-15 2016-12-15 Kenneth W. Reese Virtual pos terminal method and apparatus
CN108781211A (zh) * 2016-03-01 2018-11-09 高通股份有限公司 用于装置的tee执行的用户界面
WO2019168315A1 (en) * 2018-02-27 2019-09-06 Samsung Electronics Co., Ltd. Trustzone graphic rendering method and display device using the same

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210166227A1 (en) * 2019-11-28 2021-06-03 Qualcomm Incorporated Secure User Interface With Improved User Experience
CN114036015A (zh) * 2021-10-07 2022-02-11 中国航空工业集团公司洛阳电光设备研究所 一种基于fpga的高安全性图形生成装置
CN114036015B (zh) * 2021-10-07 2024-04-19 中国航空工业集团公司洛阳电光设备研究所 一种基于fpga的高安全性图形生成装置

Also Published As

Publication number Publication date
US11948233B2 (en) 2024-04-02
CN112711452B (zh) 2023-11-03
US20220245878A1 (en) 2022-08-04
CN112711452A (zh) 2021-04-27

Similar Documents

Publication Publication Date Title
US11874903B2 (en) User interface switching method and terminal
US9355282B2 (en) Using multiple display servers to protect data
TWI665623B (zh) 授權登錄方法及裝置
US7661126B2 (en) Systems and methods for authenticating a user interface to a computer user
RU2635224C2 (ru) Способ и аппарат для безопасного сенсорного ввода
US7721094B2 (en) Systems and methods for determining if applications executing on a computer system are trusted
US20130145475A1 (en) Method and apparatus for securing touch input
EP2831801B1 (en) Display authentication
WO2019127467A1 (zh) 一种数据访问方法及装置
US11948233B2 (en) Image display method and electronic device
WO2017157192A1 (zh) 一种数据输入方法、装置及用户设备
EP3678021B1 (en) User interface display method and terminal device
US20220155819A1 (en) Trusted User Interface Display Method And Electronic Device
WO2016172944A1 (zh) 终端的界面显示方法和终端
US20220100874A1 (en) Data loss protection on secondary displays
US20210303718A1 (en) Context based data leak prevention of sensitive information
WO2023134376A1 (zh) 信息处理方法和装置
EP3044721B1 (en) Automatic pairing of io devices with hardware secure elements
CN113467656B (zh) 一种屏幕触摸事件的通知方法和车机
KR20230104948A (ko) 가려진 구성 요소와의 상호 작용을 차단하여 사용자 상호 작용 하이재킹 공격 방지
US11245694B2 (en) User terminal apparatus and control method thereof
CN103971057A (zh) 一种移动通信智能终端的可信路径实现方法及系统
CN115277072B (zh) 账号打通方法、装置、存储介质以及计算机设备
US20190087580A1 (en) Secure launch for a hypervisor
TW201439882A (zh) 觸控事件處理方法和實現該方法的可攜式裝置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20879592

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20879592

Country of ref document: EP

Kind code of ref document: A1