CN114647487A - Trusted execution environment architecture based on AMD SEV and trusted execution system - Google Patents

Abstract

Embodiments of the present description provide an AMD SEV based trusted execution environment architecture and trusted execution system. The trusted execution environment architecture includes: the instruction emulator is used for hooking the illegal opcode trap handler in the kernels of the Enclave virtual machine and the application program virtual machine to simulate and execute the SGX instruction; the Enclave manager is deployed in the Enclave virtual machine and used for creating and managing an Enclave application program; the memory manager is used for managing the internal memory of the Enclave virtual machine so as to allow an Enclave application program to access the internal memory of the Enclave virtual machine and the external memory of the Enclave virtual machine, and simultaneously forbidding an external component of the Enclave virtual machine to access the internal memory of the Enclave virtual machine; and a cross-virtual machine communicator which realizes cross-virtual machine communication between the Enclave virtual machine and the application program virtual machine.

Description

Trusted execution environment architecture based on AMD SEV and trusted execution system

Technical Field

Embodiments of the present specification relate generally to the field of trusted execution, and more particularly, to an AMD SEV based trusted execution environment architecture and trusted execution system.

Background

In recent years, the use of Trusted Execution Environment (TEE) such as Intel SGX (Software Guard Execution) and AMD SEV (Secure Encrypted Virtualization) has been rapidly advancing. The TEE may protect the confidentiality and integrity of program code and data from attacks by malicious system software, making it valuable to deploy the TEE in cloud-based systems where the computing platform is not fully trusted by customers.

SGX, as the mainstream TEE platform, has become a de facto standard for building TEE-based applications. However, the Instruction Set Architecture (ISA) extension of the SGA explicitly separates software applications into trusted and untrusted software applications, such that the trusted software applications are isolated within a protected Enclave region that is accessible only to code executing in Enclave mode. Also, developers need to reconstruct (refactor) existing applications or build new applications from the SGX software specification. And compile the application using the SGX SDK. Thus, applications developed for SGX may only run on the SGX processor, resulting in a vendor lock situation.

Disclosure of Invention

In view of the foregoing, embodiments of the present specification provide an AMD SEV based trusted execution environment architecture and trusted execution system. With the trusted execution environment architecture, TEE applications and underlying TEE hardware may be decoupled such that the trusted execution environment architecture may provide binary code compatibility of partitioned SGX Enclave software and may directly execute the binary code of SGX Enclave software on AMD SEVs.

According to an aspect of embodiments of the present specification, there is provided an AMD SEV based trusted execution environment architecture, comprising: the instruction emulator is used for hooking the illegal opcode trap handler in the kernels of the Enclave virtual machine and the application program virtual machine to simulate and execute the SGX instruction; the Enclave manager is deployed in the Enclave virtual machine and used for creating and managing an Enclave application program; the memory manager is used for managing the internal memory of the Enclave virtual machine so as to allow the Enclave application program to access the internal memory of the Enclave virtual machine and the external memory of the Enclave virtual machine, and forbid an external component of the Enclave virtual machine from accessing the internal memory of the Enclave virtual machine; and a cross-virtual machine communicator which realizes cross-virtual machine communication between the Enclave virtual machine and the application program virtual machine.

Optionally, in an example of the above aspect, the trusted execution environment architecture may further include: and the remote authenticator is deployed in the hardware structure of the AMD SEV at the bottom layer and the kernel of the Enclave virtual machine and is used for remotely authenticating the Enclave application program based on a remote authentication framework of the AMD SEV.

Optionally, in one example of the above aspect, the instruction emulator is configured to: simulating execution of all ENCLS instructions on a boundary between the Enclave virtual machine and the application virtual machine; simulating and executing an ENCLU instruction except an EENTER instruction, an EEXIT instruction and an ERESUM instruction in the Enclave virtual machine according to hardware specifications; and executing the EENTER instruction, the EEXIT instruction, and the ERESUME instruction by cross-virtual machine emulation.

Optionally, in one example of the above aspect, the instruction emulator is configured to emulate executing all ENCLS instructions in the following manner: packing the parameters of the ENCLS instruction and necessary data into a request data packet and sending the request data packet to the Enclave virtual machine for execution; and after the Enclave virtual machine completes the execution of the simulation instruction, returning an execution result to the application program virtual machine as a response.

Optionally, in an example of the above aspect, when the ENCLS instruction is not an EWB instruction, the execution result includes an error code and a register value that needs to be updated, and when the ENCLS instruction is an EWB instruction, the execution result includes an encrypted Enclave page and metadata thereof.

Optionally, in an example of the above aspect, when the ENCLS instruction is an EWB instruction and the EWB instruction fails to execute, the response does not include a page payload and metadata.

Optionally, in one example of the above aspect, the instruction emulator is configured to emulate the EENTER instruction, the EEXIT instruction, and the eresum instruction as follows: intercepting, via a first illegal opcode trap handler at a kernel of the application virtual machine, an EENTER instruction, preparing a corresponding EENTER instruction parameter, generating an EENTER request and sending the EENTER request to an EENTER handler at the Enclave virtual machine, the EENTER request including the EENTER instruction parameter, and suspending a thread of execution of an application at the application virtual machine to wait for a corresponding EEXIT; in response to receiving the EENTER request, acquiring a corresponding Enclave thread for processing through an Enclave management program at the Enclave virtual machine; when executing an EEXIT instruction at the Enclave virtual machine, intercepting the EEXIT instruction via a second illegal opcode trap handler at a kernel of the Enclave virtual machine via a hook, generating an EEXIT request, and sending the EEXIT request to a first illegal opcode trap handler at the application virtual machine, in response to receiving the EEXIT request, resuming execution of an execution thread of the application via the EEXIT handler at the application virtual machine.

Optionally, in an example of the above aspect, the instruction emulator is further configured to return, via the EENTER handler, a result of the processing of the EENTER request as an EENTER response to a first illegal opcode trap handler at the application virtual machine

Optionally, in an example of the above aspect, the instruction emulator is further configured to terminate execution of the Enclave thread in response to the interception of the EEXIT instruction by the second illegal opcode trap handler.

Optionally, in one example of the above aspect, the instruction emulator is configured to trigger an asynchronous Enclave exit via an error in the following manner: checking, via a second illegal opcode trap handler hooked at a kernel of the Enclave virtual machine, whether an error is triggered by an Enclave thread; in response to the error being triggered by the Enclave thread, calling an asynchronous Enclave exit handler via the second illegal opcode trap handler and causing the Enclave thread to enter a sleep mode; generating a synthetic register state via the asynchronous envelope exit handler and sending the synthetic register state to a first illegal opcode trap handler at the application virtual machine; waking up a thread of execution of an application at the application virtual machine via the first illegal opcode trap handler to perform error handling using the synthetic register state call corresponding error handlers within an illegal opcode trap context of a previous EENTER instruction; in response to completing error handling, re-executing the Enclave thread at the Enclave virtual machine using the execution thread re-execution process pointed to by the asynchronous exit pointer via the first illegal opcode trap handler, the re-execution handler, and the second illegal opcode trap handler.

Optionally, in an example of the above aspect, the Enclave manager is configured to: creating an Enclave context dedicated to Enclave execution, said Enclave context being separate and isolated from the manager context of the Enclave manager; configuring a memory synchronization thread and a distribution thread, and enabling the memory synchronization thread and the distribution thread to enter an Enclave context to execute actions through a kernel code trapped in the Enclave virtual machine; registering the self as an idle Enable manager to a kernel to wait for an EENTER request for creating an Enable thread; creating an Enclave thread for executing Enclave code in response to receiving the EENTER request; and switching the context of the Enclave thread to the Enclave context after the Enclave thread is correctly set and leaves the manager thread.

Optionally, in an example of the above aspect, the kernel interface of the Enclave virtual machine is arranged such that any system call instruction or software interrupt is not available to the Enclave thread and the Enclave context is not mapped to any system call instruction.

Optionally, in an example of the above aspect, the memory manager includes a first memory manager deployed in a kernel of the Enclave virtual machine and a second memory manager deployed in a kernel of the application virtual machine. The memory manager is configured to: managing virtual address mapping and access permissions for each EPC page using a software Enclave Page Cache (EPC) mapping table, each entry of the software EPC mapping table storing a virtual address mapping and access permissions for each EPC page; enabling an Enclave code running in the Enclave virtual machine to access a memory in the application program virtual machine by using a retrieval and mapping mechanism hooked with a page fault handler of the Enclave virtual machine; and maintaining page synchronization on both sides of the Enclave virtual machine and the application virtual machine by using a non-switching synchronization mechanism.

Optionally, in one example of the above aspect, the memory manager is configured to prohibit fetching and mapping operations to an Enclave page whose virtual address falls within an Enclave linear address range.

Optionally, in one example of the above aspect, the address space of the trusted execution environment architecture includes an EPC address space, a virtual address space, an envelope physical address space, and a manager address space. Wherein the hypervisor address space is managed by a kernel of the Enclave virtual machine, the virtual address space is shared by an application in the application virtual machine and an Enclave thread in the Enclave virtual machine, the EPC address is used as a physical address in the application virtual machine and as an index to an EPCM in the Enclave virtual machine, the Enclave physical address is a backup of an Enclave memory mapped to a virtual address in the Enclave virtual machine and is used to perform addressing.

Optionally, in an example of the above aspect, the cross-virtual machine communicator includes a first cross-virtual machine communicator deployed in a kernel of the Enclave virtual machine, a second cross-virtual machine communicator deployed in a kernel of the application virtual machine, and a hub deployed in a hypervisor. The cross-virtual machine communicator is configured to perform cross-virtual machine communication between the Enclave virtual machine and the application virtual machine using the following cross-virtual machine communication protocol: preparing a data packet to be transmitted in a shared memory through a sender thread in a first virtual machine, and signaling a CPUID processing program in the hub by using a CPUID instruction, wherein a parameter of the CPUID instruction indicates a virtual machine ID of a target second virtual machine; picking up a data packet in the shared memory via the CPUID handler and requesting a send worker thread in the hub to notify a target second virtual machine via an IRQ handler; notifying, via the IRQ handler, a hypervisor using the CPUID instruction and retrieving the data packet to a shared memory of the target second virtual machine; sending the data packet to a dispatcher of the target second virtual machine through a dispatch queue of the target second virtual machine; and decrypting, splicing, and sending the spliced data table to a destination handler at the target second virtual machine, via the distributor.

Optionally, in an example of the above aspect, the remote authenticator emulates a key acquisition instruction of the SGX to acquire a verification key to implement the remote authentication, using a public key provided by a trusted execution environment architecture provider and a root key stored in a kernel of the Enclave virtual machine.

According to another embodiment of the present specification, there is provided a trusted execution system including: the trusted execution environment architecture as described above; an Enclave virtual machine; an application virtual machine; a management program; and the underlying AMD SEV hardware architecture.

According to another embodiment of the present specification, there is provided an AMD SEV based trusted execution environment architecture, comprising: at least one processor, a memory coupled with the at least one processor, and a computer program stored in the memory, the at least one processor executing the computer program to implement: hooking an illegal operation code trap processing program in the kernels of both an Enclave virtual machine and an application program virtual machine to simulate and execute an SGX instruction; creating and managing an Enclave application program; managing an internal memory of the Enclave virtual machine to allow the Enclave application program to access the internal memory of the Enclave virtual machine and an external memory of the Enclave virtual machine, and forbidding an external component of the Enclave virtual machine to access the internal memory of the Enclave virtual machine; and realizing cross-virtual machine communication between the Enclave virtual machine and the application program virtual machine.

According to another embodiment of the present specification, there is provided a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform: hooking an illegal operation code trap processing program in kernels of an Enclave virtual machine and an application program virtual machine to simulate and execute an SGX instruction; creating and managing an Enclave application program; managing an internal memory of the Enclave virtual machine to allow the Enclave application program to access the internal memory of the Enclave virtual machine and an external memory of the Enclave virtual machine, and forbidding an external component of the Enclave virtual machine to access the internal memory of the Enclave virtual machine; and realizing cross-virtual machine communication between the Enclave virtual machine and the application virtual machine.

According to another embodiment of the present specification, there is provided a computer program product comprising a computer program executed by a processor to implement: hooking an illegal operation code trap processing program in kernels of an Enclave virtual machine and an application program virtual machine to simulate and execute an SGX instruction; creating and managing an Enclave application program; managing an internal memory of the Enclave virtual machine to allow the Enclave application program to access the internal memory of the Enclave virtual machine and an external memory of the Enclave virtual machine, and forbidding an external component of the Enclave virtual machine to access the internal memory of the Enclave virtual machine; and realizing cross-virtual machine communication between the Enclave virtual machine and the application program virtual machine.

Drawings

A further understanding of the nature and advantages of the present disclosure may be realized by reference to the following drawings. In the drawings, similar components or features may have the same reference numerals.

FIG. 1 illustrates an example schematic of an AMD SEV-based trusted execution system according to embodiments of the present description.

Fig. 2 illustrates an SGX instruction that may be supported by a trusted execution environment architecture in accordance with embodiments of the present description.

FIG. 3 illustrates an example flow diagram of an ENCLS instruction emulation execution process in accordance with an embodiment of the present description.

FIG. 4 illustrates an example flow diagram for executing an EENTER instruction, an EEXIT instruction, and an ERESMW instruction across VM simulations in accordance with an embodiment of the present description.

Fig. 5 illustrates an example flow diagram of an asynchronous Enclave exit triggering process in accordance with embodiments of the present description.

Fig. 6 illustrates an example flow diagram of an Enclave application management process according to embodiments of the present description.

FIG. 7 illustrates an example schematic diagram for illustrating relationships between address spaces in a trusted execution environment architecture, according to embodiments of the present description.

FIG. 8 illustrates an example flow diagram of a cross-VM communication process in accordance with an embodiment of the specification.

FIG. 9 illustrates an execution flow diagram of a cross-VM communication process according to an embodiment of the present description.

10A-10F illustrate exemplary graphs of benchmark results under a micro-benchmark test according to embodiments of the present description.

Fig. 11 illustrates raw scores of various trusted execution environment architectures under macro-reference testing.

Fig. 12A-12C illustrate normalized score comparison graphs for various trusted execution environment architectures under macro-reference testing.

FIG. 13 illustrates an exemplary graph comparing WOLFSSL performance on a trusted execution environment architecture and Intel SGX according to embodiments of the present description.

14A-14C illustrate example schematic diagrams of performance test results for running a real world application in a trusted execution environment architecture according to embodiments of the present description.

FIG. 15 illustrates an example schematic diagram of a trusted execution environment architecture based on a computer system implementation, according to an embodiment of the present description.

Detailed Description

The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and thereby implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as needed. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with respect to some examples may also be combined in other examples.

As used herein, the term "include" and its variants mean open-ended terms in the sense of "including, but not limited to. The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first," "second," and the like may refer to different or the same object. Other definitions, whether explicit or implicit, may be included below. The definition of a term is consistent throughout the specification unless the context clearly dictates otherwise.

Intel SGX, as the mainstream TEE platform, has become a de facto standard for building TEE-based applications. Intel SGX provides a Trusted Execution Environment (TEE) by isolating the security-critical components of an application in a hardware-protected Enclave. In Intel SGX, everything except Enclave, including the operating system, is untrusted. The Trusted Computing Base (TCB) includes only the underlying hardware and the Enclave itself. In this specification, the term "TEE" refers to a secure area running in the processor, in a separate environment and running in parallel with the operating system. The TEE ensures that confidentiality and integrity of both code and data loaded in a trusted execution environment are protected by protecting the data and the code using both hardware and software. The term "Enclave" refers to a secure zone that is isolated within a trusted execution environment.

For software running in Enclave, Intel SGX provides confidentiality and integrity guarantees. Regarding confidentiality, Intel SGX encrypts, protects, and physically isolates Enclave's Memory in Processor Reserved Memory (PRM), referred to as Enclave Page Cache (EPC). Any software other than Enclave cannot access the code and data in the EPC. In addition, a hardware encryption engine (MEE) is located between the processor and the memory controller, so that all memory accesses to the PRM region are encrypted and decrypted on the fly, i.e., immediately encrypted and decrypted when the memory access is performed. Regarding integrity, when the EPC page is swapped out of memory, the Intel SGX encrypts and authenticates the EPC page using a Message Authentication Code (MAC). Thus, the Intel SGX can prevent direct tampering from external software. Once decrypted and integrity checked, the EPC page can be swapped in. In addition, Intel SGX also provides various platform support, including boot services and remote verification services, so that Enclave and SGX hardware can authenticate itself to a remote party.

However, the Instruction Set Architecture (ISA) extension of the SGA explicitly separates software applications into trusted and untrusted software applications, such that the trusted software applications are isolated within a protected Enclave region that is accessible only to code executing in Enclave mode. Likewise, developers need to reconstruct (refactor) existing applications or build new applications from the SGX software specification. And the application is compiled using the SGX SDK. Thus, applications developed for SGX may only run on the SGX processor, resulting in a vendor lock situation.

AMD SEVs provide VM-based TEE by encrypting a Virtual Machine (VM) without requiring a trusted hypervisor (hereinafter, the hypervisor may also be referred to as a hypervisor) or host (hereinafter, when a Virtual Machine monitor is referred to, hypervisor and host are used interchangeably). Specifically, an AES engine in AMD System On Chip (SOC) is used to protect VM data in memory. When the data of the SEV VM is written into the memory, the data of the SEV VM is automatically encrypted via the AES engine, and when the data of the SEV VM is read from the memory, the data of the SEV VM is automatically decrypted. Each SEV VM has a unique 128-bit VM Encryption Key (VEK). The VEK is stored in an AMD Secure Processor (AMD-SP) and is never exposed to the platform host. The SEV-ES (SEV-Encrypted State) further protects the VM by encrypting its State control block (VMCB) during VMEXIT. The latest version of the SEV-SNP includes an inverted page table so that the integrity of the encrypted memory and nested page tables is protected.

Similar to Intel SGX, everything except the SEV VM (including the co-resident virtual machines and the platform host) is not trusted. AES encryption prevents the hypervisor from compromising the confidentiality and integrity of the client (Guest) VM even though the hypervisor may access the memory of the SEV VM. Unlike Intel SGX, however, SEV supports page-level encryption where the entire Enclave is encrypted, and Guest VMs can control which pages in memory are encrypted. The Guest VM may unregister (unset) the C-bit in the client Page Table Entry (PTE) to modify the encrypted Page to an unencrypted Page to share data with the hypervisor. Another prominent characteristic of AMD SEV is: the SEV VM does not require modification of the application software, even though some modifications to the operating system kernel may be necessary to enable the SEV on both the client side and the host side. Since Linux kernels 4.16 and 5.10, SEV and SEV-ES have been formally patched to support. The SEV also provides a remote authentication framework to protect the integrity and confidentiality of VMs during virtual machine setup.

Unlike SGX, which provides isolated and encrypted memory regions within the application's address space, the security boundary for SEV enforcement is the physical memory of the entire VM. SEV hardware does not provide intra-VM isolation. Thus, to protect the Enclave code and data from untrusted application code, the only securely feasible solution is to run the Enclave in an SEV VM different from the application and handle its instruction execution and communication correctly.

In view of the foregoing, embodiments of the present specification provide a trusted execution environment architecture that is capable of providing binary code compatibility for partitioned SGX Enclave software and of executing directly on AMD SEVs. The trusted execution environment architecture may be viewed as an SGX hardware module that plugs into an SEV machine. In this trusted execution environment architecture, the trusted Enclave of the old SGX application is executed in a separate VM, with the VM protection provided by the SEV. This separate VM is referred to as an Enclave VM (EVM) and the original VM running the untrusted part of the application is referred to as an application VM (avm). Virtualization is achieved by inserting SGX instructions (e.g., EENTER and EEXIT) during execution in the respective VM or cross-VM and implementing corresponding logic in the VM kernel to provide transparency to the Enclave code or application code. Enclave code and data privacy is achieved by using the AMD's Memory Encryption Engine (MEE), while integrity is achieved by building an authentication service with a chain of trust for AMD-based SEV authentication. According to the scheme, when the SGX application program is operated in the trusted execution environment architecture, the security level equivalent to that of Intel SGX is realized, and meanwhile, the unique security provided by AMD SEV for the application program is reserved.

To enforce cross-Enclave isolation, only one Enclave is allowed to occupy the EVM, and the EVM is never reused. By using VM isolation and memory encryption of the SEV, applications and hypervisors cannot access the Enclave memory, and different enclaves are isolated, so that different enclaves cannot access each other's memory.

Furthermore, in order to transparently support Enclave binaries and applications originally built for Intel SGX processors, cross-VM communication mechanisms must be provided to help SGX instructions emulate execution, cross-domain memory access, exception handling, and the like.

There are many technical challenges faced when virtualizing SGX Enclave using AMD SEV. These technical challenges include, for example, (1) instruction simulation; (2) managing a memory; (3) enclave entry and Enclave exit; (4) multiple enclaves and multi-threaded runs; and (5) SGX remote authentication.

Although both Intel and AMD machines are x86-64 Instruction Set Architectures (ISAs), the SGX extension of the x86-64ISA is specific to Intel and is not supported by AMD machines. How to intercept and simulate these extended SGX instructions becomes a problem to be solved by the trusted execution environment architecture of the present specification.

The Intel SGX embeds the Enclave memory in the address space of the application program and allows the Enclave code to access the memory inside and outside the Enclave while prohibiting the external (including another Enclave) access to the Enclave memory. In the case of a multi-VM execution model, the trusted execution environment architecture of the present specification still needs to meet the same requirements as described above.

Furthermore, the semantics of Enclave entry (EENTER) and the semantics of Enclave Exit (EEXIT) of Intel SGX also need to be preserved in the trusted execution environment architecture of this specification. Enclave entry refers to the world transition from untrusted space to trusted space, and Enclave Exit refers to the world transition from trusted space to untrusted space. The control flow of Enclave code must be preserved as in Intel SGX.

In the trusted execution environment architecture of the present specification, it is necessary to support running multiple enclaves simultaneously (e.g., in addition to the Enclave program, like quoted (bidding) enclaves in SGX), while each Enclave should support multithreading. Furthermore, in the trusted execution environment architecture of the present specification, SGX remote authentication needs to be supported.

An AMD SEV based trusted execution environment architecture and trusted execution system according to embodiments of the present specification will be described with reference to the accompanying drawings.

FIG. 1 illustrates an example schematic diagram of an AMD SEV-based trusted execution system 1 according to embodiments of the present description.

As shown in fig. 1, the trusted execution system 1 includes an Enclave Virtual Machine (EVM)10, an Application Virtual Machine (AVM)20, a hypervisor 30, and an underlying AMD SEV hardware architecture 40. Furthermore, the trusted execution system 1 also comprises an AMD SEV based trusted execution environment architecture. In the example of fig. 1, components of the trusted execution system 1 that reside on the EVM10 constitute the Trusted Computing Base (TCB), while components that reside outside the EVM10 are untrusted components. The trusted execution environment architecture comprises an instruction emulator, an Enclose manager, a memory manager, and a cross-virtual machine communicator. For ease of description, the trusted execution environment architecture according to embodiments of the present specification may also be referred to hereinafter as trusted execution environment architecture 1.

The instruction emulator is configured to emulate executing the SGX instruction with an illegal opcode trap (aka # UD trap) handler (hook) in the cores of both the Enclave virtual machine and the application virtual machine. As shown in fig. 1, the instruction emulator may include an instruction emulator 110 disposed in the EVM10 and an instruction emulator 210 disposed in the AVM 20. Using instruction emulator 110 and instruction emulator 210, # UD trap Handler is hooked in the cores of both the AVM and EVM. If the invalid opcode corresponds to one of the ENCLS or ENCLU instructions, the corresponding function is emulated in the core and the associated registers are modified to reflect the results of the execution in the corresponding VM. The SGX instructions supported by the trusted execution environment architecture are listed in fig. 2.

The instruction emulator is configured to emulate execution of all ENCLS instructions on the boundary between the EVM10 and the AVM 20. Fig. 3 illustrates an example flow diagram of an ENCLS instruction emulation execution process 300 in accordance with an embodiment of this specification.

As shown in fig. 3, when the invalid opcode corresponds to an ENCLS instruction, instruction emulator 210 in AVM20 packages the parameters of the ENCLS instruction and the necessary data (e.g., in the case of EADD) into a request packet at 310. Next, at 320, the instruction emulator 210 sends the request packet to the instruction emulator 110 in the EVM10 for execution.

In response to receiving the request packet, at 330, execution is simulated in the EVM10 using the parameters of the ENCLS instruction in the request packet and the necessary data to complete the instruction. Then, at 340, the instruction emulator 110 in the EVM10 returns the execution result as a response (response packet) to the instruction emulator 210 in the AVM 20.

Furthermore, when the instruction emulator simulates execution of the ENCLS instruction, if the ENCLS instruction is not the EWB instruction, the returned execution result contains an error code and a register value that needs to be updated. If the ENCLS instruction is an EWB instruction, since the EWB instruction encrypts the Enclave page and writes it back to the untrusted memory, the instruction emulator processes the EWB by packaging the encrypted Enclave page and its metadata into the response packet, i.e., the returned execution result contains the encrypted Enclave page and its metadata. Further, optionally, in order to reduce overhead, the response packet may not contain a page payload and metadata when the EWB instruction fails simulation.

The instruction emulator is further configured to emulate and execute an ENCLU instruction other than the EENTER instruction, the EEXIT instruction, and the eresum instruction inside the Enclave virtual machine according to a hardware specification (e.g., according to hardware information).

Emulation of most ENCLU instructions (such as ereprt and EGETKEY) can be handled within EVM10 according to hardware specifications, but emulation execution of the EENTER, EEXIT, and eresum instructions involves a Control Flow Transfer (Control Flow Transfer) across the boundary of two VMs. Thus, the instruction emulator is further configured to execute the EENTER instruction, the EEXIT instruction, and the ERESUME instruction by cross-VM emulation. Alternatively, the EVM10 and AVM20 are designed to only execute one cross-VM instruction at a time.

FIG. 4 illustrates an example flow diagram for executing an EENTER instruction, an EEXIT instruction, and an ERESUME instruction across VM simulations in accordance with an embodiment of the present description.

As shown in fig. 4, at step

The # UD Trap Handler (Trap Handler) at the AVM20 intercepts the EENTER instruction, prepares the corresponding parameters, and in step

Generates an EENTER request and sends the EENTER request to an EENTER Handler in the EVM10, thereby passing the execution stream to an Enclose thread (i.e., an Enclose thread) in the EVM10New thread). In this specification, an Enclave thread refers to an execution thread of an Enclave application. The thread of execution of the application at the AVM20 then pauses to wait for the corresponding EEXIT request (step (r)). After the Trap Handler at the AVM20 receives the EEXIT request, at step

The thread of execution of the suspended application program is resumed in the AVM 10.

Upon receiving the EENTER request sent by the EENTER Handler at the AVM20, at step

The EENTER Handler sends a notification to the manager to notify the manager to create an Enclave thread or to pick up an existing Enclave thread (i.e., new thread) for processing (step

). When the EEXIT instruction is executed (step)

) In time, # UD Trap Handler at EVM10 will intercept it, generate an EEXIT request and send the EEXIT request to the Trap Handler in AVM20, passing the execution flow back to AVM20 (step S)

) Further, optionally, the # UD Trap Handler at the EVM10 may terminate execution of the new thread if necessary (step (c)). Further, optionally, the EENTER Handler may also return an EENTER response to the # UD Trap Handler at the AVM 10. In the example of fig. 4, steps that may be performed in parallel are labeled as the same index and are distinguished using filled circles and open circles.

In Intel SGX, asynchronous Enclave eXits (AEX) can be triggered by interrupts or errors. In the trusted execution environment architecture 1 of the present specification, only triggering of AEX by an error (FAULT) is supported. Hardware interrupts in the EVM10 can be handled by the trusted Enclave core without triggering AEX. Fig. 5 illustrates an example flow diagram of an asynchronous Enclave exit triggering process in accordance with embodiments of the present description.

As shown in fig. 5, at step

The # UD Trap Handler in EVM10 is hooked to check if the error was triggered by an Enclave thread. If the error is triggered by an Enclave thread, then in step

# UD Trap Handler would call AEX Handler in EVM10 and then put this Enclave thread into sleep mode, similar to what happens on the AVM20 side for EENTER, thus eliminating the need to save the state of Enclave using XSAVE as with Intel SGX. Then, in step

The AEX Handler will follow the AEX semantics of the Intel SGX and generate synthetic register state (synthetic register state) which is then sent to the # UD Trap Handler at the AVM 10. Subsequently, in step

The execution thread of the application in the AVM20 is then awakened to perform error handling using the synthesis register state call corresponding error handler within the # UD trap context of the previous EENTER instruction. When the error handling is complete, the application crashes or the error has been handled correctly, and in step

The control flow transitions to an Asynchronous Exit Pointer (AEP, which is specified when the application executes the EENTER instruction), which points to the ERESME flow to re-execute the new thread in EVM10 (step S)

To the step

). Specifically, the APP in the AVM20 sends an ERESULE request to the # UD Trap Handler at the AVM20 (step

). The # UD Trap Handler at the AVM20 sends an ERESUME request to the ERESUME Handler at the EVM10 (step

) Eresum request processing is performed and the processed eresum request is sent to the # UD Trap Handler at the EVM 10. # UD Trap Handler returns the processed ERESULE request to the new thread (step

) Thereby re-executing the new thread.

Further, the # UD Trap Handler may wait to receive the ERESME request when a processed ERESME request is not received from the ERESME Handler (step two). In addition, after the eresum Handler at the EVM10 completes the eresum request processing, an eresum response may be returned to the # UD Trap Handler at the AVM20 (step viii).

Further, in the example of fig. 5, steps that may be performed in parallel are labeled as the same index and are distinguished using filled circles and open circles.

Returning to fig. 1, an Enclave manager 120 is deployed in the EVM10, configured to create and manage Enclave applications. The Enclave manager is a user space wrapper process created inside the EVM10 for hosting (host) Enclave. The Enclave binary file is loaded as a shared library in the user space process. When the EVM10 is started, an Enclave manager is created, and then the Enclave manager is paused to wait for the creation of an Enclave. Since only one Enclave is allowed in EVM10, Enclave manager 120 creates only one Enclave (as shown in fig. 1). In this specification, Enclave is an execution environment composed of hardware and software, and Enclave application programs may be executed within the Enclave, and each Enclave program may include a plurality of Enclave threads.

Fig. 6 illustrates an example flow diagram of an Enclave application management process 600 according to embodiments of the present description.

As shown in fig. 6, at 610, an Enclave context is created that is specific to Enclave execution, the created Enclave context being separate and isolated from the manager context of the Enclave manager. The Enclave manager will create after the new EVM is started and pause until the Enclave application is loaded. The Enclave manager will create an isolated address space and corresponding threads for the Enclave application.

At 620, the memory synchronization thread and the dispatch thread are configured and perform their respective actions by trapping the kernel code of the EVM10 so that the memory synchronization thread and the dispatch thread enter the envelope context.

At 630, register itself as an idle Enable manager with the EVM's kernel (Enable kernel) to wait for an EENTER request, such as ECREATE or ELDB/ELDU, to create an Enable thread.

At 640, in response to receiving the EENTER request, an Enclave thread for executing Enclave code is created. Next, at 650, after the Enclave thread is properly set and the hypervisor thread is exited, the context of the Enclave thread is switched to the Enclave context.

Optionally, in one example, to follow the semantics of the Intel SGX not allowing Enclave to access any system services by disabling an instruction such as a system call (SYSCALL), the kernel interface of the EVM10 may be set such that the Enclave thread cannot use any system call instruction or software interrupt, and the Enclave context is not mapped to any system call instruction. Thus, instead of processing the thread in the Enclave context, the interrupt will force the thread to be switched back to the manager context.

The memory manager is configured to manage the internal memory of the EVM10 to allow an Enclave application to access the internal memory of the EVM10 as well as the external memory of the EVM10, while prohibiting external components of the EVM10 from accessing the internal memory of the EVM 10. As shown in fig. 1, the memory managers include a first memory manager 130 deployed in the kernel of the EVM10 and a second memory manager 230 deployed in the kernel of the AVM 20.

In the trusted execution environment architecture 1 provided by the embodiments of the present specification, there are four address spaces: an Enclave Page Cache (EPC) address space, a virtual address space, an Enclave physical address space and a manager address space. The manager address space is managed by the Enclave kernel, and the other three address spaces are used and managed by the trusted execution environment architecture.

FIG. 7 illustrates an example schematic diagram for illustrating relationships between address spaces in a trusted execution environment architecture, according to embodiments of the present description. As shown in fig. 7, the application in the AVM20 and the Enclave in the EVM10 share the same virtual address space. The EPC address is used as a physical address in the AVM20, and as an index to an Envelope Page Cache Map (EPCM) in the EVM10, which is the EPC management architecture of Intel SGX. The Enclave physical address is a true copy of the Enclave memory (backing) mapped to the virtual address in the EVM10 and is used by the CPU to perform addressing. Since an enclo application may use an enclo instruction that uses a Virtual address to negotiate with the EPCM, a Virtual-EPC mapping table may be created inside the enclo to facilitate use of the enclo instruction. Entries in the table are added or deleted when using the ENCLS instruction to add or delete EPC pages.

The memory manager is configured to manage virtual address mapping and access permissions for each EPC page using a software EPC mapping table, each entry of which stores a virtual address mapping and access permissions for each EPC page.

Intel SGX securely manages the virtual address mapping and access permissions of EPC pages using EPCM. To manage EPC pages, the trusted execution environment architecture according to embodiments of the present description implements software EPCM. Similar to the Intel SGX, each entry of the software EPCM stores a mapping and permissions for each EPC page.

Since the SEV processor does not directly use software EPCM, in the trusted execution environment architecture of embodiments of the present specification, when an SGX instruction modifies an EPCM entry (such as the entries listed in FIG. 2), the corresponding page table entry in the EVM10 is also modified. By doing so, the limitations of the EPCM may be reflected in regular memory accesses, such that the page tables in the EVM10 are isolated from the AVM20, such that the page tables in the EVM10 are trusted.

Some instructions, such as ECREATE and EPA, can add pages without providing virtual addresses. The trusted execution system environment architecture 1 allocates these pages in the kernel. ECREATE is Enclave specific, so that allocation of pages added by ECREATE is performed in the corresponding EVM 10. The EPA creates a Version Array (VA) page that is shared between enclaves, but each entry in VA is specific to Enclave. The trusted execution environment architecture 1 creates a VA page for each Enclave, encrypts it by the EVM kernel using an Enclave specific key, and then stores the encrypted VA page in the AVM kernel.

The memory manager is also configured to enable access to memory in the AVM20 by the Enclave code running in the EVM10 using a Fetch-and-Map (Fetch-and-Map) mechanism hooked to the page fault handler of the EVM 10.

In order to allow code running in the EVM10 to access memory outside of Enclave in the AVM20, a Fetch-and-Map mechanism hooked to the page fault (# PF) handler of the EVM10 is designed, with which pages are fetched from the AVM 20. Specifically, when a page fault occurs, the # PF handler sends a request to the AVM20 using the virtual address of the faulting page. If the page is mapped in the AVM20, its data is sent back to the EVM10 so that the # PF handler can map a new page initialized with the received data to the error address of the Enclave execution thread. This process is similar to the demand paging mechanism. The page is mapped as non-executable to ensure that code outside the Enclave cannot be executed in Enclave mode.

Further optionally, the memory manager is configured not to execute the Fetch-and-Map on pages whose virtual addresses fall within the envelope linear address range (ELRANGE). If the virtual address of the page where the page fault occurred falls in ELRANGE, the memory manager will follow the AEX program and notify the AVM to handle the page fault.

The memory manager is also configured to maintain page synchronization on both sides of the EVM10 and AVM20 using a Switchless synchronization (Switchless synchronizing) mechanism.

When the trusted execution environment architecture maps pages between the EVM10 and AVM20, it must maintain page synchronization. Inspired by non-switching OCalls, a similar non-switching synchronization mechanism is designed, and the non-switching synchronization mechanism uses a background working thread to synchronize mapping pages without switching in and out of an Enclave.

Specifically, both the EVM10 and the AVM20 set one thread during initialization, and the set thread is referred to as a no-switch synchronous work thread. When a Fetch-and-Map event occurs, the address and content of the error page is registered to the non-switching synchronization list on both sides. The no-switch synchronous worker thread monitors and synchronizes changes to each page in the list periodically (e.g., every 100 milliseconds).

Furthermore, to avoid overwriting page changes that are not yet synchronized, a 4096-bit bitmap is used for page synchronization. Each bit in the bitmap corresponds to a byte of the page, where a 1-bit indicates that the corresponding byte has changed and a 0-bit indicates that the byte has not changed. In this manner, this bitmap helps mask (mask out) all unchanged bytes so that only those bytes that have changed will be synchronized.

Returning to fig. 1, the cross-virtual machine communicator is configured to enable cross-VM communication between the EVM10 and the AVM 20.

In the trusted execution environment architecture 1 according to embodiments of the present description, cross-VM communication is used to provide services for instruction emulation and memory synchronization. In this description, cross-VM communication may be implemented in a number of ways, for example, using encrypted TCP connections. Optionally, in one example, to obtain better performance, data may be transferred using cross-VM shared pages. Such communication must involve untrusted hypervisors in the threat model. Therefore, a secure cross-VM communication protocol needs to be designed.

The cross-VM communication protocol needs to satisfy the following characteristics.

Arbitrary data size: the trusted execution environment architecture according to embodiments of the present specification may allow for data of arbitrary size by breaking large blocks of data into smaller chunks and encapsulating them into packets. The size of the data packet is the maximum data size that can be transmitted in each round of communication. The packet has a fixed size header containing the total number of packets, the index of the current packet, and the total size of the data. The sender sends the data packets one by one and the receiver concatenates them together to recover the entire data.

Concurrence: multiple senders may send data simultaneously. The trusted execution environment architecture according to embodiments of the present description may ensure that they can send data simultaneously without collisions. To this end, a unique sequential session number is included in each packet header. For large blocks of data, the data flow of these packets is treated as a single session (e.g., call EADD). On the receiver side, the data packets of the respective sessions may be concatenated to reconstruct the data.

Confidentiality: because the hypervisor is not trusted, the trusted execution environment architecture according to embodiments of the present specification may ensure that the hypervisor cannot read communication data. To achieve confidentiality, end-to-end symmetric encryption such as AES-GCM 128 or the like may be used. The trusted execution environment architecture according to embodiments of the present description encrypts the entire data packet, including the header, to ensure that no data is revealed to the hypervisor.

Integrity: malicious hypervisors may alter data during transmission. The data must be guaranteed to reach the destination unmodified. To achieve integrity, the trusted execution environment architecture according to embodiments of the present description appends a key-encrypted Message Authentication Code (MAC) to each data packet. Without the correct key, the hypervisor cannot modify the contents of the data packet.

Multiple targets: since there may be multiple EVMs, the trusted execution environment architecture according to embodiments of the present specification needs to be able to send data packets to a particular VM. To this end, an EVM ID (e.g., a natural number indicating the order in which the EVM registered itself to the hypervisor) is assigned to the EVM. The trusted execution environment architecture according to embodiments of the present specification ensures that the encryption and authentication keys in each EVM are different, so that communication of other enclaves remains secure even if the EVM is hacked.

Retransmission prevention: a malicious hypervisor may retransmit outdated but legitimate packets to the EVM. Thus, the trusted execution environment architecture according to embodiments of the present specification needs to ensure that all data arriving at a destination is new data. The above retransmission prevention has been achieved by using a unique session number for each data packet, which is also encrypted and integrity protected.

Secure key distribution: the trusted execution environment architecture according to embodiments of the present specification assumes that a key is shared between the EVM and its AVM. This may be securely achieved, for example, by using keys pre-embedded in the AVM and EVM that may be configured by the user prior to deployment and use image encryption to prevent hypervisor attacks. Other schemes for securely distributing keys may also be implemented.

Fig. 8 illustrates an example flow diagram of a cross-VM communication protocol according to an embodiment of the specification, and fig. 9 illustrates an execution flow diagram of a cross-VM communication protocol according to an embodiment of the specification.

As shown in fig. 8, the cross-VM communication protocol includes 10 steps, and its execution flowchart is shown in fig. 9. In fig. 8 and 9, each step is marked with a black circle number or a white circle number, where the black circles represent steps within the execution environment and the white circles represent steps between the execution environments.

The cross-VM communication protocol is described below using an example of data transfer from an untrusted component (AVM) to a trusted component (EVM). The data transfer process from a trusted component (EVM) to an untrusted component (AVM) is similar to this process.

In the step of

In step two, the sender thread in the AVM20 prepares data in shared memory and then signals the hypervisor with the CPUID instruction, the parameters of the CPUID instructionThe number indicates the ID of the target VM, as shown in fig. 8. The hub 310 will pick up the packet in the shared memory. Hub 310 is the component in hypervisor 30 responsible for communication across VMs. Next, the CPUID Handler asks for (step)

) Send Worker notifies (step (r)) the target EVM via the IRQ Handler. The IRQ Handler will notify the hypervisor using the CPUID and retrieve the packet to its shared memory (step (c)). The data packet is then transmitted through the distribution queue (step)

To the step

) To a distribution program (distributor) in which the data is decrypted, spliced, and finally transmitted (step s)

) To the Destination handler (Destination). A dispatcher is a kernel thread worker (kernel thread worker) that dispatches data to its destination handler. The distribution program and the destination handler are shown as two distinct modules in fig. 8, but in an implementation, the distribution program may be integrated in the destination handler.

Furthermore, optionally, the trusted execution environment architecture 1 may further include a remote authenticator (not shown in fig. 1) deployed in the kernel of the underlying AMD SEV hardware structure and Enclave virtual machine. The remote authenticator is configured to remotely authenticate the Enclave application based on the remote authentication framework of the AMD SEV.

The trust chain of the trusted execution environment architecture 1 according to embodiments of the present specification relies on the remote validation framework of SEVs to deploy an EVM with an encrypted VM image of the trusted execution environment architecture provider, where any fused secret (mapped secret) and the mapped encryption key inside the EVM are known only to the provider.

The trusted execution environment architecture stores the root secret and the hash value of the trusted execution environment architecture provider's public key in an Enclave kernel that is protected from attacks by the AVM20 and the hypervisor. The root key may be used to derive the private key using EGETKEY following the semantics of Intel SGX. The public key hash value may enable a trusted execution environment architecture provider to launch a provider signed similar function Enclave, such as an Intel signed quote Enclave. The EVM allows vendor signed Enclave to derive the verification key in the Enclave kernel. Thus, the trusted execution environment architecture allows Enclave code to perform remote authentication using routines similar to the Intel SGX application. The Enclave authentication process simulates a key acquisition instruction EGETKEY of an Intel SGX to acquire a verification key so as to realize remote authentication by using a public key of a provider of a trusted execution environment architecture and a root key stored in an Enclave kernel.

In an embodiment according to the present description, the trusted execution environment architecture extends the trust chain of SEVs to Enclave virtual machines.

The root of trust of the AMD processor is a single pair of ARK private/public keys, Kark, which is known only to the Key Distribution Server (KDS) of the AMD. A pair of AMD SEV signing keys Kask, also managed by the KDS, are used to generate a certificate specific to the SEV platform. The public key part in Kask is signed with a private Kark. During the manufacturing process, each SEV platform is equipped with a pair of chip-unique ek (chip endsegment key) keys Kcek, wherein the public key portion of the Keck is signed with a Kask. During the initialization phase of the SEV platform, the SEV firmware generates a platform Diffie-Hellman key Kpdh and a platform EK key Kpek. Kpek is signed with Kcek and Kpdh is signed with Kpek. After initialization, the hypervisor retrieves the credentials for Kpdh and Kpek and the unique platform ID for the SEV firmware. When the client owner attempts to authenticate the platform (before launching the SEV VM thereon), the hypervisor forwards these two certificates and the unique platform ID to the client owner. The client owner may then use the platform ID to obtain the Kcek from the KDS_IDThe signed certificates of Kask and Kark, then platform verification is performed by verifying the following certificate chains: kpdh → Kpek → Kcek_ID→Kask→Kark。

To initiate the EVM, the following steps need to be performed.

In step 1, the client owner sends its DH public key, encrypted Enclave image, encrypted with the disk encryption key Kblk, and an Open Virtual Machine Firmware (OVMF) file to the hypervisor.

At step 2, the hypervisor issues a LAUNCH _ START command passing the DH public key of the client owner to the SEV firmware, thereby establishing a secure channel between the SEV firmware and the client owner encrypted with a transport encryption key, Ktek, derived from the DH.

In step 3, the hypervisor copies the OVMF file to memory and invokes the loop _ UPDATE _ DATA command to perform in-place (in-place) memory encryption of the OVMF file.

At step 4, the hypervisor invokes the LAUNCH _ MEASURE command to instruct the SEV firmware to compute OVMF memory measurements that are sent over the secure channel to the client owner.

In step 5, the client owner verifies the measurement and sends Kblk encrypted using Ktek to the hypervisor. The hypervisor uses the LAUNCH _ SECRET command to configure the encrypted Kblk into the initiating client VM, which is used by the OVMF to decrypt and load the encrypted image of the Enclave VM.

Thus, the following chain of trust is established: VM → Kblk → Ktek → Kpdh.

With the ability to launch multiple enclaves, the trusted execution environment architecture according to embodiments of the present specification may support remote validation of Intel SGX. A Quote Enclave (QE), which is an Enclave signed via a specific trusted execution environment architecture, is used to provide remote authentication. The QE has a unique attribute signed via a particular trusted execution environment architecture that allows the QE to derive a platform's signing key Kp, which is derived using a fused key pre-deployed by the service provider of the trusted execution environment architecture 1. Kp is used to generate a signature that can only be verified by the service provider of the trusted execution environment architecture 1. By confirming that the report is correctly signed with Kp, both the QE and the trusted execution environment architecture image can be verified.

The trusted execution environment architecture 1 according to embodiments of the present specification supports the same remote authentication routine as the Intel SGX.

In step 1, the remote party can trigger remote verification through a challenge, and untrusted applications require Enclave E to be verified_TTo initiate the remote authentication process.

At step 2, the untrusted application now gets the ID (EPID on the intel side) bound to the platform and sends it to the remote party.

At step 3, if the remote party accepts the ID, Enclave executes eroport to generate a report containing the measurement values and metadata of Enclave with a digital signature using the derived signature key K of the trusted execution environment architecture 1_EAnd (6) signing.

In step 4, the report is forwarded to the QE, which then checks whether the report is utilized with the EREPORT Key K_EThe signature is correctly made. If the check passes, then the signature key K of the platform is utilized_PThe report is signed as quote and the quote is sent to the untrusted application.

At step 5, the untrusted application will send a quote to the remote party, which forwards the quote to the service provider of the trusted execution environment architecture 1 (which acts as an authentication service (IAS) on the Intel SGX). The remote party then verifies the correctness of the measurement values and metadata if quote is approved by the service provider of the trusted execution environment architecture 1.

For Enclave E_TThe chain of trust of (c) is established as follows: e_T(→K_E)→K_T→QE(VM)。

It is noted that the remote party may establish a secure channel with Enclave during this process using the DH key exchange algorithm. The data of the DH key exchange may be signed by the QE and packaged in the report. This ensures that the key exchange is secure and reliable.

The AMD SEV based trusted execution environment architecture and trusted execution system according to embodiments of the present description are described above with reference to fig. 1 to 9. The performance of the trusted execution environment architecture according to embodiments of the present specification will be discussed below.

Execution security

With respect to ENCLS and ENCLU instructions

On an Intel processor, the ENCLS and ENCLU instructions are implemented using microcode. The ENCLS instruction represents a kernel permission instruction, and refers to an instruction that needs to be accessed in a kernel mode. Examples of ENCLS instructions include, for example, but are not limited to: EADD, EBLOCK, ECREATE, EDBGRD, EDBGWR, EEXTEND, EINIT, ELDB/ELDU, EPA, EREMOVE, ETRACK, EWB, EAUG, EMODPR, and EMODT. The ENCLU instruction indicates a user authority instruction, which is an instruction that can be accessed in a user mode. Examples of ENCLU instructions include, for example, but are not limited to: EENTER, EEXIT, EGETKEY, EREPORT, ERESME, EACCEPT, EACCEPTCOPY, and EMODPE. Execution of these instructions is protected by the CPU hardware and cannot be intercepted by the software program. Although the trusted execution environment architecture 1 according to embodiments of the present specification cannot use microcode to execute the ENCLS instruction and the ENCLU instruction, the trusted execution environment architecture 1 may implement security of executing the ENCLS instruction and the ENCLU instruction using the following method.

The ENCLS instruction is intended to be managed by Enclave, so only the parameters of the instruction need be included in the request packet and sent to the EVM to perform its function. End-to-end encryption is used when sending requests, so the hypervisor cannot modify the request packets sent. Furthermore, sanity check (sanity check) specified in the SGX reference is also performed inside the EVM so that malicious requests will not succeed. The actual function of the instruction is performed in the EVM so it cannot be intercepted by software running in the AVM or other EVM.

The ENCLU instruction executes primarily within the EVM, except for the EENTER instruction, EEXIT instruction, and eresum instruction. The ENCLU instructions will be checked against the SGX reference to ensure that they are secure. Since ENCLU instructions execute inside the EVM, their execution can be trusted.

Illegal instruction inside Enclave

The Enclave code is prohibited from accessing system resources in intel SGX by not allowing the Enclave code to execute instructions like SYSCALL. If the current thread is an Enclave thread, the trusted execution environment architecture 1 may implement this restriction by performing a check before SYSCALL and software interrupt handler entry. If so, the handler will raise a (throw) # UD error. By doing so, it can be ensured that the Enclave code can never use the system services in the EVM.

On Enclave entry and Enclave exit

In the trusted execution environment architecture 1, the same Enclave entry and Enclave exit semantics as SGX are followed to transfer control flows to an Enclave that protects execution security.

For the EENTER and EEXIT instructions, the trusted execution environment architecture 1 allows control flow to be transferred into Enclave via EENTER and control flow to be transferred out of Enclave via EEXIT. Unlike SGX, the control flow transport in trusted execution environment architecture 1 crosses the boundary of two VMs. In trusted execution environment architecture 1, the EENTER will put the application thread to sleep and launch an Enclave thread in the EVM. After the Enclave thread completes its execution, EEXIT terminates the Enclave thread and wakes up the sleeping application thread. The only potential attack is to wake the sleeping application thread early and block future EEXITs, which may be performed by an attacker with kernel access to the AVM. However, this will only affect the applications in the AVM, not the EVM.

For AEX and eresum, similar to Intel SGX, when Enclave has an error, the control flow is switched back to AVM with the synthesis state, so that Enclave data including register states are never revealed. Unlike Intel SGX which uses XSAVE to save the state of Enclave, the trusted execution environment architecture 1 simply causes the Enclave thread to enter sleep mode, which wakes it up and directly re-executes when Resume arrives. The Enclave thread sleeps within the EVM, protecting it from tampering by any attacker. However, unlike SGX, in trusted execution environment architecture 1 AEX is not triggered by an interrupt or VMEXIT, but by a FAULT. This is because the EVM core is trusted to handle unrelated interrupts and exceptions, such as timer events, and therefore, unlike SGX, a context switch is not necessary.

Memory encryption and isolation

Memory encryption

Both trusted execution environment architecture 1 and SGX prevent software components outside the EVM (Enclave of SGX) from reading encrypted memory in the clear and prevent physical attacks (such as cold boot attacks and DMA attacks) from directly reading secrets in encrypted memory. While the memory encryption of SEV is not verified and thus is slightly weaker than SGX, SEV-SNP does protect memory integrity. Thus, the trusted execution environment architecture 1 achieves security comparable to SGX.

Furthermore, SGX uses a single transient memory encryption key for all encraves, while SEV uses different keys for different VMs. Thus, since the trusted execution environment architecture 1 protects each Enclave in different VMs encrypted with different keys, the trusted execution environment architecture 1 is more secure than SGX.

On performing Enclave memory access rules and isolation

The Intel SGX prevents access to the Enclave memory if (a) the CPU is not in Enclave mode, (b) the corresponding EPCM entry is set to the block flag, (c) the target page is not a PT _ REG page (i.e., a regular Enclave page), (d) the EID of the current Enclave is different from the owner of the page, and (e) the virtual address does not match the record of the EPCM entry. The trusted execution environment architecture 1 implements a similar level of security assurance via VM isolation. The trusted execution environment architecture 1 implements EPCM to maintain metadata for each EPC page, including page type (e.g., PT _ REG), virtual address mapping, access rights, and the like.

Although there are no EPCMs maintained by the lookup software during page table walk, the EVM ensures that its page tables correctly reflect the corresponding EPCM entries: (1) PT _ REG pages (e.g., PT _ SECS) all have user-space mappings, making them inaccessible to the einclave code; (2) when a page transitions to a blocking state, the trusted execution environment architecture sets its access rights to PROT _ NONE, thereby making the page inaccessible; (3) the access permissions and virtual address mapping are correct. Thus, the trusted execution environment architecture 1 maintains the same security level as SGX, since the page tables of Enclave are protected by the Enclave kernel in the SEV VM. Furthermore, since the trusted execution environment architecture 1 implements one Enclave per VM and cannot reuse EVM, with the VM isolation provided by SEV, an Enclave isolation similar to Intel SGX can be achieved.

Restricted non-Enclave memory access in Enclave mode

Intel SGX allows code running in Enclave mode to access non-Enclave memory. However, it does not allow any non-Enclave memory to be mapped to a virtual address within ELRANGE that is reserved for Enclave memory. Furthermore, the entry TLB of a non-Enclave memory page loaded in Enclave mode is forced to set a non-executable (NX) flag in order to ensure that Enclave never executes code outside the Enclave.

The trusted execution environment architecture 1 achieves the same level of security via Fetch-and-Map and no-switch synchronization. First, if the virtual address of a non-Enclave memory falls into ELRANGE, then Fetch-and-Map never maps the Enclave memory to EVM. Thus, any memory access to the morning virtual address in ELRANGE without a valid mapping would directly trigger a page fault. Second, to prevent execution of code in non-Enclave memory, the trusted execution environment architecture 1 also forces the fetched page to be non-executable. Furthermore, since the protected data in Enclave must fall within range, it is never retrieved or synchronized with untrusted memory in AVM, so no switching synchronization will leak the protected data to the outside.

Cross-VM communication

The only interface that the EVM exposes to the outside world is the cross-VM communication interface. In embodiments of the present description, a packet must fall into one of three categories: command emulation packets, no-switch synchronization packets, and Fetch-and-Map packets.

The instruction emulation packets are distributed to the corresponding Enclave specified by the EPC page on which they operate. The authentication is performed in the Enclave's local distributor, verified according to the Intel SGX specification.

The no-switch synchronization packet is first distributed to its corresponding Enclave. Enclave will then check its no-switch synchronization list to see if the page to be synchronized is in the list. If and only if so, the packet is accepted. The correctness of the synchronized page is not an issue because the non-Enclave memory is expected to be incorrect.

The Fetch-and-Map packet is compared to a list of threads waiting on the kernel and a check is made to see if one of the threads in the list is waiting on a particular address. If so, the packet is accepted and the page is mapped to non-Enclave memory in the EVM. If not, the packet is discarded.

Thus, since all packets sent to the EVM are subject to strict scrutiny, an attacker cannot send any packets that are semantically inconsistent with SGX. Furthermore, reordering packets does not raise new security issues. For instruction emulation, there is no concern that the hypervisor may reorder execution, as the trusted execution environment architecture 1 only allows one cross-VM instruction to be executed at a time. Reordering memory-related packets may cause a rewriting problem in untrusted memory. However, since untrusted memory is not protected, this behavior does not introduce security issues in the SGX model.

Discussion of TCB size

In Intel SGX, the only software component inside the TCB is the Enclave binary file. However, microcode implementations of SGX instructions are also part of the TCB because they are firmware running on hardware. In trusted execution environment architecture 1, the TCB contains an Enclave kernel, an Enclave manager and an Enclave binary file. In an implementation of the present solution, 8,840 lines of code are added to the Linux kernel 5.10.20. The Enclave manager is relatively small, having only 250 lines of code. The overall TCB size change in the trusted execution environment architecture 1 is 9,090 lines of code plus a size that minimizes the Linux kernel. Once the formal verification kernel, e.g., seL4, is supported by AMD SEV-ES, it may be used in place of the Linux kernel. This allows the entire additional component that has been added to the Enclave kernel to be fully trustworthy.

The trusted execution environment architecture 1 does not significantly increase the attack surface. Because the only interface that the EVM exposes to the outside is the cross-VM communication interface, a strong attack that an attacker outside the TCB may launch is an attack against the cross-VM communication interface, such as eavesdropping, injecting, dropping, modifying the communication packets. However, as discussed above, these attacks can be prevented by authenticated encryption, replay prevention, and sanity checks performed on the EVM side. Thus, the trusted execution environment architecture 1 may reduce the attack surface of the SEV VM to a level comparable to the SGX.

The AMD SEV based trusted execution environment architecture and trusted execution system according to embodiments of the present specification are described above, and system performance is discussed. Experiments will be used below to evaluate the performance of the trusted execution environment architecture according to embodiments of the present specification.

In one experiment, the trusted execution environment architecture 1 according to embodiments of the present specification has been implemented with 16,167 lines of C language code (LoC) and 121 lines of x86-64 assembly language code. Wherein, 6,377 line LoC and 121 line assembly language codes are used for realizing the AVM module, 8,840 line LoC is used for realizing the Enclave kernel, 250 line LoC is used for realizing the Enclave manager, and 700 line LoC is used for realizing the hypervisor in the KVM module.

The performance overhead problem of the trusted execution environment architecture 1 will be discussed below. To this end, a set of benchmark test programs and real-world applications are designed and selected to analyze the performance overhead at the component level and the application level. Wherein the micro-benchmark test program is used to expose performance instructions and component levels, the macro-benchmark test program is used to reflect overall performance, and the real-world SGX application program is used to reflect compatibility and performance overhead.

In this experiment, the SEV-ES platform was AMD's GIGABYTE MZ31-AR0, which was configured with an EPYC 72518 core processor operating at 2.1 GHz. The SEV-ES platform has 64GiB memories and is loaded with AMD provided Linux kernel 5.10.0 to support SEV-ES host capabilities. In this experiment, virtual machines were configured with 2 SMP cores and 4GiB memory, each with a Linux core 5.10.20. In addition, a control experiment was also performed on an Intel SGX machine, which is a DELL OptiPlex 5060 with an Intel Core i 7-87006 Core processor running at 3.2GHz to compare performance differences. The Intel SGX machine is equipped with 32GiB memory and runs Linux kernel 4.15.0. By default, all performance overhead is measured by running the target benchmark test 10 presentations and then calculating the average.

A. Benchmark test

Micro-benchmark testing

The trusted execution environment architecture 1 has a number of components that are responsible for executing the Enclave program. Enclave (1) needs initialization, (2) executes an ECall/OCall, (3) executes a specific SGX leave instruction, (4) communicates across virtual machines, (5) acquires Enclave external memory, and (6) executes no-switch synchronization if necessary. Therefore, six micro-benchmark test programs were designed to characterize the performance and overhead associated with these executions.

Enclave initialization

First, the overhead of creating and initializing Enclave on the trusted execution environment architecture is measured, which typically involves a set of SGX instructions including, for example, ECREATE, EADD, EEXTEND, and EINIT. Specifically, enclaves with different sizes are launched (e.g., in the number of heap pages) and delay measurements (measurements for trusted execution environment architecture 1) are obtained as shown by the block curves in fig. 10A. It can be observed from fig. 10A that the initialization overhead of Enclave is linear with its size (since ECREATE and EINIT are one-time overheads). In addition, the same experiment was also performed on an Intel SGX machine, the results of which are shown in the triangular curve in fig. 10A. For example, to launch an Enclave of 550 pages, trusted execution environment architecture 1 requires 0.92 seconds on average, 10 times slower (about 92 milliseconds) than Intel SGX. Other data points also show similar slowing down. However, Enclave initialization is not performed often, and the overhead is small throughout the lifecycle of Enclave.

ECall/OCall delay

Essentially, an ECall delay is an EENTER and eext pair and an ECall delay is an eext and EENTER pair, so that only one of them needs to be measured. To do this, the ECall delay is measured by implementing an empty ECall function and measuring how long it takes to execute the EENTER and EEXIT instructions. Note that this empty ECall function also includes code from the Intel SGX SDK. The null ECall function is executed 200 times on the Intel SGX and trusted execution environment architecture 1, with the test results shown in fig. 10B. As can be seen in fig. 10B, on average, the time to call the null ECall function on trusted execution environment architecture 1 is about 1.5 milliseconds, and 9.3 microseconds on Intel SGX, which is 161 times faster than trusted execution environment architecture 1. The 1.5 millisecond overhead means that the maximum throughput ECall of the trusted execution environment architecture 1 is around 650IOPS, which is reasonable for most use cases (e.g., SGX protected cryptographic authentication).

SGX instruction latency

The SGX instruction latency is measured by running the SGX instruction 20 times to measure its total time and then calculating an average. This measurement was then repeated 10 times to estimate the average delay, and the measurement result is shown in fig. 10C. Executing the ENCLS instruction involves 2, 3 or 4 packets. For an ENCLS instruction, it takes approximately 0.9 milliseconds to complete a 2-packet instruction, approximately 1.3 milliseconds to complete a 3-packet instruction, and approximately 1.8 milliseconds to complete a 4-packet instruction. One exception is the ECREATE instruction, which requires about 3 milliseconds because ECREATE also sets the Enclave manager for the new Enclave.

In addition to the EENTER instruction and ERESUME, the ENCLU instruction is typically executed within an Enclave. In the second micro benchmark test procedure, the performance of EENTER, ERESUME, and EEXIT used at Enclave entry and Enclave exit is measured and thus left empty. It can be seen that in most cases, execution of these local ENCLU instructions can be completed within 5 microseconds. Note that EGETKEY and ereprt are relatively slow because they involve encryption operations.

Cross-VM communication overhead

In benchmark testing, overhead of communication across VMs is measured by recording timestamps before and after each step. Since the communication crosses virtual machine boundaries, the Precision Time Protocol (PTP) is used to synchronize the time of the VM and hypervisor to sub-microsecond precision.

Fig. 10D reports the measured delay. Specifically, steps two and four are CPUID events and IRQ events, which take less than 10 μ s, and are less important than the other steps. For the other steps, most of them are delayed under 100 μ s except for the step

And the step of

. This is because of the step

Semaphores are being used to pass data to distribution handlers. A scheduler, whether scheduled or busy, may cause a long delay. The most time-consuming step is the step

It handles the simulated execution of SGX instructions. The time-consuming variation of this step is very large, since simulation routines of different instructions may cause different overheads.

By adding the delays of all steps, it can be seen that if the packet is the first packet with a 3-packet or 4-packet order, it does not involve a step

It takes about 300. mu.s. If the packet is part of an instruction that requires emulation, it takes about 600 mus. In summary, a 2-packet instruction takes approximately 900 μ s, a 3-packet instruction takes approximately 1200 μ s, and a 3-packet instruction takes approximately 1500 μ s.

Memory acquisition overhead

There is memory fetch overhead when Enclave accesses untrusted memory. Therefore, a benchmark program is designed to measure the latency of accessing untrusted memory and compare it with the situation where the page has been fetched, which results as shown in FIG. 10E. Accessing a local page or a fetched page takes about 0.4 mus, while fetching a page results in a 0.9 ms delay. This result also indicates that once a page is fetched, accessing it does not result in a large overhead.

No switching synchronization overhead

The no-handover synchronization delay is measured by measuring the round trip delay and then dividing it by 2. Specifically, the EVM first modifies the non-Enclave pages to trigger no-switch synchronization. When the AVM notices a page modification, it immediately changes it to trigger another synchronization. When this change is noticed by the EVM, one round trip synchronization has been completed, and the measurement results are shown in fig. 10F. As can be seen, the average delay is about 53 milliseconds.

Macro datum test

NBENCH (also known as BYTEMAK) was run on trusted execution environment architecture 1, the ordinary AMD SEV-ES VM and the Intel SGX machine to compare their performance. FIG. 11 shows the raw scores of benchmark programs running on the respective platforms. In addition, the raw scores were also normalized, and the normalized test results are presented in fig. 12A-12C.

First, by comparing trusted execution environment architecture 1 with ordinary SEV-ES, it can be seen that the performance overhead (geometric mean) introduced by trusted execution environment architecture 1 on an SEV-ES machine is 205%. In fig. 12A-12C, it can be seen that most tests show less over 3-fold deceleration, except for strong SORT, BITFIELD, and LU demon, which result in higher overhead. After checking the code, it was found that BITFIELD triggered a large number of ECalls and strong SORTs to randomly access large data objects in non-Enclave memory. There is a 6 times speed reduction of LU demon position due to moderate ECalls. It can thus be seen that for the trusted execution environment architecture 1, there are two factors that severely impact performance: ECall frequency and non-Enclose memory accesses.

Second, the comparison of trusted execution environment architecture 1 with Intel SGX shows a slow down in migration from SGX to trusted execution environment architecture 1. The geometric mean of the overhead is 221%. The scores that are compared directly between different CPU architectures may be used as a reference for how to execute applications when migrating from an SGX machine.

Real world SGX application

Since encryption operations are typical for an Enclave application, while graph is complex enough to expose the functionality of the trusted execution environment architecture 1, running WOLFSSL and graph on the trusted execution environment architecture 1 presents run-out performance.

For the tested application, most of it runs directly without any modification. The exceptions are those applications (e.g., graphics) that use the CPUID instruction, which must bypass the checking process.

WOLFSSL performance

The WOLFSSL performance was measured using a benchmark program named WOLFCRYPT. The benchmark program tests encryption, decryption, digest, and signature verification. The benchmark test program is run on the trusted execution environment architecture 1 and the SGX, the result of which is shown in fig. 13. In fig. 13, the ratio column is the original performance of the Intel SGX divided by the trusted execution environment architecture 1. It can be seen that for most encryption, decryption, and summarization operations, the Intel SGX is approximately 0.5 times faster than the trusted execution environment architecture 1. The trusted execution environment architecture 1 may even defeat the Intel SGX for RSA algorithms and DH key exchange. I/O intensive signature verification and key generation are vulnerabilities of the trusted execution environment architecture 1. The overhead of the geometric mean benchmark test indicates that the Intel SGX is approximately 0.9 times faster than the trusted execution environment architecture 1. This result is acceptable considering that the functionality of the trusted execution environment architecture 1 is being tested in a virtualized environment of the AMD first generation Zen server processor, which is not as powerful as the Intel desktop processor.

Graphene performance

The cURL, GMPBEnch and Nginx were run in Graphere on the trusted execution environment architecture 1 to test their ability to support large Enclave applications. The 256MB size of Graphene boot time is about 5 minutes in trusted execution environment architecture 1 and about 0.5 seconds on Intel SGX. This is because a large amount of EADD and EEXTEND is required to start such a large Enclave.

The time spent by each particular instruction when initiating Graphene is shown in FIG. 14A. As seen in FIG. 14A, EEXTEND incurs 3/4 overhead because each EEXTEND can only hash 256 bytes, so it requires 16 EEXTENDs to hash the entire 4096 byte page.

To measure the performance of the application after Graphene, the crrl and GMPbench were evaluated. The former is an I/O (networking) intensive workload and the latter is a CPU binding. The Grapehen-SGX on the trusted execution environment architecture 1 is compared to the graph-Direct mode that runs the library OS directly outside of Enclave. For the cURL test, it was used to access https:// www.ieee-security. org and measured the latency, the performance of which is shown in FIG. 14B. The Graphene Direct is about 7 times faster than the Graphene-SGX on the trusted execution environment architecture 1. This is because network traffic is handled outside of Enclave, which results in a large number of ocels and access to untrusted memory. In addition, Enclave must copy those caches to its own memory, resulting in additional accesses to untrusted memory. The results of the gmpbunch testing are shown in fig. 14C. As can be seen from fig. 14C, the trusted execution environment architecture 1 does not increase the burden of CPU computation. These results indicate that the trusted execution environment architecture 1 is more suitable for CPU-intensive workloads such as key operations.

An AMD SEV based trusted execution environment architecture and trusted execution system according to embodiments of the present description is described above with reference to fig. 1 through 14C. The above trusted execution environment architecture may be implemented in hardware, or may be implemented in software, or a combination of hardware and software.

FIG. 15 illustrates a schematic diagram of a computer system implementation-based trusted execution environment architecture 1500 in accordance with embodiments of the present description. As shown in fig. 15, the trusted execution environment architecture 1500 may include at least one processor 1510, storage (e.g., non-volatile storage) 1520, memory 1530, and a communication interface 1540, and the at least one processor 1510, storage 1520, memory 1530, and communication interface 1540 are coupled together via a bus 1560. The at least one processor 1510 executes at least one computer-readable instruction (i.e., the elements described above as being implemented in software) stored or encoded in memory.

In one embodiment, computer-executable instructions are stored in the memory that, when executed, cause the at least one processor 1510 to: hooking an illegal operation code trap processing program in kernels of an Enclave virtual machine and an application program virtual machine to simulate and execute an SGX instruction; creating and managing an Enclave application program; managing an internal memory of the Enclave virtual machine to allow an Enclave application program to access the internal memory of the Enclave virtual machine and an external memory of the Enclave virtual machine, and forbidding an external component of the Enclave virtual machine to access the internal memory of the Enclave virtual machine; and realizing cross-virtual machine communication between the Enclave virtual machine and the application virtual machine.

It should be appreciated that the computer-executable instructions stored in the memory, when executed, cause the at least one processor 1510 to perform the various operations and functions described above in connection with fig. 1-14C in the various embodiments of the present description.

According to one embodiment, a program product, such as a machine-readable medium (e.g., a non-transitory machine-readable medium), is provided. A machine-readable medium may have instructions (i.e., elements described above as being implemented in software) that, when executed by a machine, cause the machine to perform various operations and functions described above in connection with fig. 1-14C in the various embodiments of the present specification. Specifically, a system or apparatus may be provided which is provided with a readable storage medium on which software program code implementing the functions of any of the above embodiments is stored, and which causes a computer or processor of the system or apparatus to read out and execute the instructions stored in the readable storage medium.

In this case, the program code itself read from the readable medium can realize the functions of any of the above-described embodiments, and thus the machine-readable code and the readable storage medium storing the machine-readable code form part of the present invention.

Examples of the readable storage medium include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or from the cloud via a communications network.

According to one embodiment, a computer program product is provided that includes a computer program that, when executed by a processor, causes the processor to perform the various operations and functions described above in connection with fig. 1-14C in the various embodiments of the present description.

It will be understood by those skilled in the art that various changes and modifications may be made in the above-disclosed embodiments without departing from the spirit of the invention. Accordingly, the scope of the invention should be determined from the following claims.

It should be noted that not all steps and units in the above flows and system structure diagrams are necessary, and some steps or units may be omitted according to actual needs. The execution order of the steps is not fixed, and can be determined as required. The apparatus structures described in the above embodiments may be physical structures or logical structures, that is, some units may be implemented by the same physical entity, or some units may be implemented by a plurality of physical entities, or some units may be implemented by some components in a plurality of independent devices.

In the above embodiments, the hardware units or modules may be implemented mechanically or electrically. For example, a hardware unit, module or processor may comprise permanently dedicated circuitry or logic (such as a dedicated processor, FPGA or ASIC) to perform the corresponding operations. The hardware elements or processors may also comprise programmable logic or circuitry (e.g., a general-purpose processor or other programmable processor) that may be temporarily configured by software to perform corresponding operations. The specific implementation (mechanical, or dedicated permanent, or temporarily set) may be determined based on cost and time considerations.

The detailed description set forth above in connection with the appended drawings describes exemplary embodiments but does not represent all embodiments that may be practiced or fall within the scope of the claims. The term "exemplary" used throughout this specification means "serving as an example, instance, or illustration," and does not mean "preferred" or "advantageous" over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described technology. However, the techniques may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the described embodiments.

The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (21)

1. An AMD SEV based trusted execution environment architecture comprising:

the instruction emulator is used for hooking the illegal opcode trap processing program into the kernels of the Enable virtual machine and the application program virtual machine to simulate and execute the SGX instruction;

the Enclave manager is deployed in the Enclave virtual machine and used for creating and managing an Enclave application program;

the memory manager is used for managing the internal memory of the Enclave virtual machine so as to allow the Enclave application program to access the internal memory of the Enclave virtual machine and the external memory of the Enclave virtual machine, and forbid an external component of the Enclave virtual machine from accessing the internal memory of the Enclave virtual machine; and

and the cross-virtual machine communicator is used for realizing cross-virtual machine communication between the Enclave virtual machine and the application program virtual machine.

2. The trusted execution environment architecture of claim 1, further comprising:

and the remote authenticator is deployed in a hardware structure of the AMD SEV at the bottom layer and a kernel of the Enclave virtual machine and is used for realizing remote authentication of the Enclave application program based on a remote authentication framework of the AMD SEV.

3. The trusted execution environment architecture of claim 1, wherein the instruction emulator is configured to:

simulating execution of all ENCLS instructions on a boundary between the Enclave virtual machine and the application virtual machine;

simulating and executing an ENCLU instruction except an EENTER instruction, an EEXIT instruction and an ERESUM instruction in the interior of the Enclave virtual machine according to hardware specifications; and

executing the EENTER instruction, the EEXIT instruction, and the ERESUME instruction through cross-virtual machine emulation.

4. The trusted execution environment architecture of claim 3, wherein the instruction emulator is configured to emulate executing an ENCLS instruction by:

packing the parameters of the ENCLS instruction and necessary data into a request data packet and sending the request data packet to the Enclave virtual machine for execution; and

and after the Enclave virtual machine completes the instruction simulation execution, returning an execution result to the application program virtual machine as a response.

5. The trusted execution environment architecture of claim 4 wherein, when the ENCLS instruction is not an EWB instruction, the execution result includes an error code and register values that need to be updated, and when the ENCLS instruction is an EWB instruction, the execution result includes an encrypted Enclave page and its metadata.

6. The trusted execution environment architecture of claim 5 wherein, when the ENCLS instruction is an EWB instruction and the EWB instruction fails to execute, the response does not include a page payload and metadata.

7. The trusted execution environment architecture of claim 3, wherein the instruction emulator is configured to emulate executing the EENTER instruction, the EEXIT instruction, and the ERESUME instruction in a manner that:

intercepting, via a first illegal opcode trap handler at a kernel of the application virtual machine, an EENTER instruction, preparing a corresponding EENTER instruction parameter, generating an EENTER request and sending the EENTER request to an EENTER handler at the Enclave virtual machine, the EENTER request including the EENTER instruction parameter, and suspending a thread of execution of an application at the application virtual machine to wait for a corresponding EEXIT;

in response to receiving the EENTER request, acquiring a corresponding Encalve thread for processing through an Encalve management program at the Encalve virtual machine;

intercepting the EEXIT instruction via a second illegal opcode trap handler hooked at a kernel of the Enclose virtual machine when the EEXIT instruction is executed at the Enclose virtual machine, generating an EEXIT request, and sending the EEXIT request to a first illegal opcode trap handler at the application virtual machine,

in response to receiving the EEXIT request, resuming execution of the thread of execution of the application via an EEXIT handler at the application virtual machine.

8. The trusted execution environment architecture of claim 7, wherein the instruction emulator is further configured to return a result of processing of the EENTER request as an EENTER response to a first illegal opcode trap handler at the application virtual machine via the EENTER handler.

9. The trusted execution environment architecture of claim 7 wherein the instruction emulator is further configured to terminate execution of the envelope thread in response to the interception of the EEXIT instruction by the second illegal opcode trap handler.

10. The trusted execution environment architecture of claim 3, wherein the instruction emulator is configured to trigger an asynchronous Enclave exit via an error in the following manner:

checking, via a second illegal opcode trap handler hooked at a kernel of the Enclave virtual machine, whether an error is triggered by an Enclave thread;

in response to the error being triggered by the Enclave thread, calling an asynchronous Enclave exit handler via the second illegal opcode trap handler and causing the Enclave thread to enter a sleep mode;

generating a synthetic register state via the asynchronous envelope exit handler and sending the synthetic register state to a first illegal opcode trap handler at the application virtual machine;

waking up a thread of execution of an application at the application virtual machine via the first illegal opcode trap handler to perform error handling using the synthetic register state call corresponding error handlers within an illegal opcode trap context of a previous EENTER instruction;

in response to completing error handling, re-executing the execution thread of the Enclave application at the Enclave virtual machine using the thread re-execution process pointed to by the asynchronous exit pointer via the first illegal opcode trap handler, the re-execution handler, and the second illegal opcode trap handler.

11. The trusted execution environment architecture of claim 1, wherein the Enclave manager is configured to:

creating an Enclave context dedicated to Enclave execution, said Enclave context being separate and isolated from a manager context of said Enclave manager;

configuring a memory synchronization thread and a distribution thread, and enabling the memory synchronization thread and the distribution thread to enter an Enclave context to execute actions through a kernel code trapped in the Enclave virtual machine;

registering the self as an idle Enclave manager to a kernel to wait for an EENTER request for creating an Enclave thread;

creating an Enclave thread for executing an Enclave code in response to receiving the EENTER request; and

switching the context of the Enclave thread to an Enclave context after the Enclave thread is properly set and the manager thread is exited.

12. A trusted execution environment as claimed in claim 11, wherein a kernel interface of said Enclave virtual machine is set such that no system call instructions or software interrupts are available to Enclave threads and said Enclave context is not mapped to any system call instructions.

13. The trusted execution environment architecture of claim 1, wherein the memory manager comprises a first memory manager deployed in a kernel of the Enclose virtual machine and a second memory manager deployed in a kernel of the application virtual machine,

the memory manager is configured to:

managing virtual address mapping and access permissions for each EPC page using a software EPC mapping table, each entry of the software EPC mapping table storing a virtual address mapping and access permissions for each EPC page;

enabling an Enclave code running in the Enclave virtual machine to access a memory in the application program virtual machine by using a retrieval and mapping mechanism hooked with a page fault handler of the Enclave virtual machine; and

and maintaining page synchronization of the both sides of the Enclave virtual machine and the application virtual machine by using a non-switching synchronization mechanism.

14. A trusted execution environment architecture as claimed in claim 13, wherein said memory manager is configured to inhibit retrieval and mapping operations from being performed on an Enclave page having a virtual address falling within an Enclave linear address range.

15. The trusted execution environment architecture of claim 13 wherein the address space of the trusted execution environment architecture comprises an EPC address space, a virtual address space, an Enclave physical address space, and a manager address space,

wherein the hypervisor address space is managed by a kernel of the Enclave virtual machine, the virtual address space is shared by an application in the application virtual machine and an Enclave thread in the Enclave virtual machine, the EPC address is used as a physical address in the application virtual machine and as an index to an EPCM in the Enclave virtual machine, the Enclave physical address is a backup of an Enclave memory mapped to a virtual address in the Enclave virtual machine and is used to perform addressing.

16. The trusted execution environment architecture of claim 1, wherein the cross-virtual machine communicator comprises a first cross-virtual machine communicator deployed in a kernel of the Enclave virtual machine, a second cross-virtual machine communicator deployed in a kernel of the application virtual machine, and a hub deployed in a hypervisor,

the cross-virtual machine communicator is configured to perform cross-virtual machine communication between the Enclave virtual machine and the application virtual machine using the following cross-virtual machine communication protocol:

preparing a data packet to be transmitted in a shared memory via a sender thread in a first virtual machine, and signaling a CPUID handler in the hub using a CPUID instruction, a parameter of the CPUID instruction indicating a virtual machine ID of a target second virtual machine;

picking up a data packet in the shared memory via the CPUID handler and requesting a send worker thread in the hub to notify a target second virtual machine via an IRQ handler;

notifying, via the IRQ handler, a hypervisor using the CPUID instruction and retrieving the data packet to a shared memory of the target second virtual machine;

sending the data packet to a dispatcher of the target second virtual machine through a dispatch queue of the target second virtual machine; and

decrypting, splicing, and sending the spliced data table to a destination handler at the target second virtual machine.

17. The trusted execution environment architecture of claim 2, wherein the remote authenticator emulates a key acquisition instruction of the SGX to acquire a validation key to enable remote authentication using a public key provided by a trusted execution environment architecture provider and a root key stored in a kernel of the Enclave virtual machine.

18. A trusted execution system, comprising:

a trusted execution environment architecture as claimed in any one of claims 1 to 17;

an Enclave virtual machine;

an application virtual machine;

a management program; and

the underlying AMD SEV hardware architecture.

19. An AMD SEV based trusted execution environment architecture comprising:

at least one processor for executing a program code for the at least one processor,

a memory coupled to the at least one processor, an

A computer program stored in the memory, the computer program executed by the at least one processor to implement:

hooking an illegal operation code trap processing program in kernels of an Enclave virtual machine and an application program virtual machine to simulate and execute an SGX instruction;

creating and managing an Enclave application program;

managing an internal memory of the Enclave virtual machine to allow the Enclave application program to access the internal memory of the Enclave virtual machine and an external memory of the Enclave virtual machine, and forbidding an external component of the Enclave virtual machine to access the internal memory of the Enclave virtual machine; and

and realizing cross-virtual machine communication between the Enclave virtual machine and the application program virtual machine.

20. A computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform:

hooking an illegal operation code trap processing program in kernels of an Enclave virtual machine and an application program virtual machine to simulate and execute an SGX instruction;

creating and managing an Enclave application program;

managing an internal memory of the Enclave virtual machine to allow the Enclave application program to access the internal memory of the Enclave virtual machine and an external memory of the Enclave virtual machine, and forbidding an external component of the Enclave virtual machine to access the internal memory of the Enclave virtual machine; and

and realizing cross-virtual machine communication between the Enclave virtual machine and the application program virtual machine.

21. A computer program product comprising a computer program that is executed by a processor to implement:

hooking an illegal operation code trap processing program in kernels of an Enclave virtual machine and an application program virtual machine to simulate and execute an SGX instruction;

creating and managing an Enclave application program;

managing an internal memory of the Enclave virtual machine to allow the Enclave application program to access the internal memory of the Enclave virtual machine and an external memory of the Enclave virtual machine, and forbidding an external component of the Enclave virtual machine to access the internal memory of the Enclave virtual machine; and

and realizing cross-virtual machine communication between the Enclave virtual machine and the application program virtual machine.

Images