WO2022121395A1

WO2022121395A1 - Trusted application control method, apparatus, computer storage medium, and terminal

Info

Publication number: WO2022121395A1
Application number: PCT/CN2021/116919
Authority: WO
Inventors: 于文海; 郭伟
Original assignee: 中国银联股份有限公司
Priority date: 2020-12-10
Filing date: 2021-09-07
Publication date: 2022-06-16
Also published as: TW202223704A; CN113051572A

Abstract

A trusted application control method, a trusted application apparatus, a computer storage medium, and an intelligent terminal. A trusted application sends an instruction to a peripheral application associated with the trusted application, such that the peripheral application controls an operating device to write sensitive data into a secure memory; and the trusted application reads the sensitive data from the secure memory, wherein the trusted application is in a trusted execution environment of a first CPU core and the peripheral application is in a common execution environment of the first CPU core, and part of the space in the secure memory is obtained by changing a memory space in the common execution environment.

Description

Control method and device for trusted application, computer storage medium and terminal

technical field

The present invention relates to the field of trusted application control, and more particularly, to a trusted application control method, a trusted application device, a computer storage medium and an intelligent terminal.

Background technique

With the development of computer technology, users' privacy information security has received more and more attention and attention. To this end, a Trusted application (TA) is proposed to process user data with higher security requirements. For example, the trusted application may be a fingerprint authentication application for verifying a user's identity, a face recognition application, and the like. To meet the high security and confidentiality requirements of trusted applications, it is necessary to make trusted applications run in a Trusted Execution Environment (TEE).

At present, trusted applications (such as face recognition applications and fingerprint recognition applications) are gradually being used in the field of financial payment, and the emergence of smart terminals such as face recognition payment terminals has introduced potential security risks to the original payment system. In the absence of targeted security protection, sensitive data such as face recognition algorithms and face image data are vulnerable to external malicious attacks in the ordinary operating systems of existing payment terminals, which lead to severe challenges for users' payment transactions. security threat.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, a method for controlling a trusted application is provided, the method comprising: a trusted application sending an instruction to a peripheral application associated with the trusted application, so that the peripheral application controls an operating device writing sensitive data into a secure memory; and reading the sensitive data from the secure memory by the trusted application, wherein the trusted application is in the trusted execution environment of the first CPU core, and the peripheral application In the common execution environment of the first CPU core, a part of the space in the secure memory is obtained by changing the memory space in the common execution environment.

As a supplement or replacement of the above solution, in the above method, the trusted application is a trusted face recognition application, the peripheral application is a camera control program, and the operating device is a camera.

As a supplement or alternative to the above solution, in the above method, the trusted face recognition application program includes a main thread and more than one sub-thread.

As a supplement or replacement of the above solution, in the above method, sending an instruction by a trusted application to a peripheral application associated with the trusted application includes: the main thread of the trusted face recognition application sending an instruction to the camera control program to start or end an image acquisition job.

As a supplement or replacement of the above solution, in the above method, the trusted application reading the sensitive data from the secure memory includes: the first sub-thread and the second sub-thread of the trusted face recognition application are The image buffer in the secure memory reads image data for face recognition, wherein the first sub-thread runs in the trusted execution environment of the second CPU core, and the second sub-thread runs in the first sub-thread. Trusted Execution Environment with three CPU cores.

As a supplement or alternative to the above solution, in the above method, the first sub-thread and the second sub-thread are started by the main thread of the trusted face recognition application.

As a supplement or alternative to the above solution, the above method further includes: after the trusted application is started, the main thread of the trusted application loads and initializes an operation model.

As a supplement or replacement of the above solution, the above method further includes: the main thread of the trusted application creates a physical address queue and a synchronization lock.

As a supplement or replacement of the above solution, the above method further includes: the main thread of the trusted application receives a notification of completion of collection of sensitive data and an address where the sensitive data is written from the peripheral application.

As a supplement or alternative to the above solution, the above method further includes: the main thread of the trusted application writes the address into the physical address queue.

As a supplement or replacement of the above solution, in the above method, the trusted application reading the sensitive data from the secure memory includes: the child thread of the trusted application acquires the synchronization lock before reading the sensitive data from the The address to which the sensitive data is written is read from the physical address queue.

As a supplement or replacement of the above solution, in the above method, the trusted application reading the sensitive data from the secure memory further includes: a child thread of the trusted application reads from the secure memory according to the address retrieve the sensitive data.

As a supplement or replacement of the above solution, the above method further includes: the sub-thread of the trusted application performs face detection, feature extraction and face comparison through the operation model.

As a supplement or replacement of the above solution, the above method further includes: the sub-thread of the trusted application notifies the main thread of the result of the face comparison.

As a supplement or alternative to the above solution, the above method further includes: the main thread of the trusted application receives a handle from the peripheral application, where the handle corresponds one-to-one with the address where the sensitive data is written.

As a supplement or replacement of the above solution, in the above method, a part of the space in the secure memory is converted back to the memory space in the common execution environment after the use is complete.

According to another aspect of the present invention, a trusted application device is provided, the device includes: a sending unit configured to send an instruction to a peripheral application associated with the trusted application, so that the peripheral application controls operations The device writes sensitive data into a secure memory; and a reading unit is configured to read the sensitive data from the secure memory, wherein the trusted application device is in the trusted execution environment of the first CPU core, and the The peripheral application is in the normal execution environment of the first CPU core, and a part of the space in the secure memory is obtained by changing the memory space in the normal execution environment.

As a supplement or replacement of the above solution, in the above device, the peripheral application is a camera control program, and the operating device is a camera.

As a supplement or alternative to the above solution, in the above device, the sending unit is configured to send an instruction to the camera control program to start or end the image acquisition work.

As a supplement or alternative to the above solution, in the above device, the reading unit includes a first reading sub-unit and a second reading sub-unit, which respectively read image data from the image buffer in the secure memory so as to Perform face recognition, wherein the first reading subunit runs in the trusted execution environment of the second CPU core, and the second reading subunit runs in the trusted execution environment of the third CPU core.

As a supplement or replacement of the above solution, the above device further includes: a loading unit for loading and initializing the operation model.

As a supplement or replacement of the above solution, the above device further includes: a creation unit configured to create a physical address queue and a synchronization lock.

As a supplement or alternative to the above solution, the above device further includes: a receiving unit configured to receive, from the peripheral application, a notification of completion of collection of sensitive data and an address where the sensitive data is written.

As a supplement or alternative to the above solution, the above device further includes: a writing unit, configured to write the address into the physical address queue.

As a supplement or alternative to the above solution, in the above device, the reading unit is configured to read the address to which the sensitive data is written from the physical address queue after acquiring the synchronization lock.

As a supplement or alternative to the above solution, in the above device, the reading unit is further configured to read the sensitive data from the secure memory according to the address.

As a supplement or replacement of the above solution, in the above device, a part of the space in the secure memory is converted back to the memory space in the common execution environment after use.

According to yet another aspect of the present invention, there is provided a computer storage medium comprising instructions that, when executed, perform the method as previously described.

According to yet another aspect of the present invention, an intelligent terminal is provided, and the intelligent terminal includes the aforementioned trusted application device.

The method for controlling a trusted application and the trusted application device according to the embodiments of the present invention communicate with the associated peripheral application in the REE (Rich Execution Environment, common execution environment), so that the associated peripheral application controls the corresponding operating device The sensitive data is written into the secure memory in the TEE, and the trusted application in the TEE reads the sensitive data from the secure memory for subsequent processing, which realizes the secure collection (acquisition) and processing of the sensitive data.

In addition, the trusted application and the associated peripheral application are in the same CPU core, but belong to different execution environments (the trusted application is in the trusted execution environment, while the peripheral application is in the normal execution environment), which can help ensure security It saves the use of CPU cores under the premise of stability, that is, saves hardware overhead.

Finally, a part of the secure memory space in the TEE environment is obtained by changing the memory space in the normal execution environment, which solves the problem of insufficient memory space in the TEE environment.

Description of drawings

The above and other objects and advantages of the present invention will be more fully apparent from the following detailed description taken in conjunction with the accompanying drawings, wherein the same or similar elements are designated by the same reference numerals.

FIG. 1 shows a schematic diagram of a control method for a trusted application according to an embodiment of the present invention;

Fig. 2 shows a schematic structural diagram of a trusted application device according to an embodiment of the present invention;

3 shows a schematic structural diagram of a trusted face recognition system according to an embodiment of the present invention;

FIG. 4 shows an architecture diagram of face collection and recognition according to an embodiment of the present invention;

FIG. 5 shows a memory handover process according to an embodiment of the present invention;

Fig. 6 shows the initialization flow of a trusted face recognition application according to an embodiment of the present invention;

FIG. 7 shows a face data collection process according to an embodiment of the present invention; and

FIG. 8 shows a face recognition flow of a trusted face recognition application according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

In addition, it should be noted that, for the convenience of description, the drawings only show some but not all of the contents related to the present invention. Before discussing the exemplary embodiments in greater detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts various operations (or steps) as a sequential process, many of the operations may be performed in parallel, concurrently, or concurrently. Additionally, the order of operations can be rearranged. The process may be terminated when its operation is complete, but may also have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, subroutines, and the like.

Although example embodiments are described as using multiple units to perform example processes, it should be understood that these example processes may also be performed by one or more modules.

Furthermore, the control logic of the present invention may be embodied on a computer-readable medium as executable program instructions implemented by a processor or the like. Examples of computer-readable media include, but are not limited to, ROM, RAM, optical disks, magnetic tapes, floppy disks, flash drives, smart cards, and optical data storage devices. The computer-readable recording medium can also be distributed over network coupled computer systems so that the computer-readable medium is stored and implemented in a distributed fashion, eg, via an in-vehicle telecommunications service or a controller area network (CAN).

Unless specifically mentioned or obvious from context, as used herein, the term "about" is understood to mean within a range of normal tolerance in the art, eg, within 2 standard deviations of the mean.

Hereinafter, a control scheme of a trusted application according to various exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 shows a schematic diagram of a method 1000 for controlling a trusted application according to an embodiment of the present invention. As shown in Figure 1, method 1000 includes the following steps:

In step S110, the trusted application sends an instruction to a peripheral application associated with the trusted application, so that the peripheral application controls the operating device to write sensitive data into the secure memory; and

In step S120, the trusted application reads the sensitive data from the secure memory, wherein the trusted application is in the trusted execution environment of the first CPU core, and the peripheral application is in the first CPU core In a common execution environment of a CPU core, a part of the space in the secure memory is obtained by changing the memory space in the common execution environment.

In the context of the present invention, "Trusted Execution Environment" in English is TEE (Trusted Execution Environment), which is an execution environment used on the device to run trusted or high-security applications.

A Trusted Execution Environment (TEE) is a secure area within the CPU. It runs in a separate environment and runs in parallel with the operating system. The CPU ensures that both the confidentiality and integrity of the code and data in the TEE are protected. By using both hardware and software to protect data and code, TEEs are more secure than operating systems. Trusted applications running in the TEE have access to the full capabilities of the device's main processor and memory, while hardware isolation protects these components from user-installed applications running in the main operating system. In a word, the code and data running in the TEE are confidential and cannot be tampered with.

In contrast to the "trusted execution environment", REE (ie Rich Execution Environment) represents a common execution environment, which is an execution environment used to run common applications on the device, such as Android, IOS systems, etc.

In one embodiment, the implementation of TEE is based on ARM TrustZone. Its security features are: (1) Protected by hardware mechanisms: TEE is isolated from REE and can only communicate with TEE through a specific entry; (2) Fast communication mechanism: TEE can access REE's memory, REE cannot access hardware-protected TEE memory; (3) can resist some hardware-based attacks.

The term "trusted operating system", also known as Trusted OS or Trusted Operation System, refers to the operating system running on the TEE side. The term "common operating system" is opposite, also known as Rich Operating System or Rich OS, which refers to the operating system running on the REE side.

With the development of mobile devices, the functions of mobile devices become more and more powerful, and mobile devices store users' assets, process payments and other operations. At present, the system operating environment of the mobile terminal is generally REE (Rich Execution Environment), and the operating system in it is Rich OS (Operating System), including Android, IOS and Linux. Rich OS is characterized by powerful functions, openness and scalability. Provide all functions of the device to the upper application, such as camera, touch screen, etc. However, Rich OS has many security risks, such as bugs are often found, OS can obtain all data of the application, it is difficult to verify whether the OS has been tampered with, and it will be subject to various attacks, which have great security for user assets hidden danger.

In order to provide higher security, the TEE provides an environment isolated from the REE to store the user's sensitive information. The TEE can directly obtain the information of the REE, but the REE cannot obtain the information of the TEE. For example, when a user pays, it is verified through the interface provided by TEE to ensure that the payment information will not be tampered with, the password will not be hijacked, and the fingerprint information will not be stolen.

In the context of the present invention, the term "trusted application" may also be referred to as authorized security software, trusted application or TA (Trusted Application), representing an application running on the TEE side. The "trusted applications" can be isolated from each other through cryptographic techniques, and the data of other trusted applications will not be read and operated at will. In addition, in some examples, the trusted application needs to perform integrity verification before execution to ensure that the application has not been tampered with.

The term "peripheral application" is used for applications that control peripherals or operating devices such as touch screens, cameras, fingerprint sensors, etc., which are also called client applications, running on the REE side.

In one embodiment, the trusted application is a trusted face recognition application, the peripheral application is a client application associated with the trusted application, that is, a camera control program, and the operating device is a camera. In another embodiment, the trusted application is a trusted fingerprint identification application, the peripheral application is a fingerprint sensor control program, and the operating device is a fingerprint sensor.

The term "sensitive data" refers to data that may cause serious harm to society or individuals after leakage, and its meaning may vary in different scenarios. For example, in a face recognition application scenario, "sensitive data" may include face recognition-related algorithms, face image data, and other important and undisclosed data. In the application scenario of fingerprint identification, "sensitive data" may include fingerprint identification related algorithms, user's fingerprint data, and other data.

In the context of the present invention, the term "secure memory" refers to a memory space (with higher security) running in a TEE environment, relative to ordinary memory. Ordinary memory/REE memory can be changed to safe memory through MPU (Memory Protection Unit, memory protection unit) hardware.

The term "CPU core", also known as "CPU core", is the core chip in the middle of the CPU, made of single crystal silicon, used to complete calculations, accept/store commands, process data, etc. It is the digital processing core. For example, a quad-core processor has four CPU cores, an octa-core processor has eight CPU cores, and so on.

In step S110, the trusted application communicates with the associated peripheral application in the REE (Rich Execution Environment, common execution environment), so that the associated peripheral application controls the corresponding operating device to write sensitive data into the TEE. Secure memory, and trusted applications in the TEE read sensitive data from the secure memory for subsequent processing, which implements secure collection (acquisition) and processing of sensitive data.

In step S120, a part of the space in the secure memory is obtained by changing the memory space in the common execution environment. By dynamically scheduling the REE memory space for use by the TEE, the problem of insufficient memory space in the TEE environment can be solved.

In one or more embodiments, the trusted application can directly interact with operating devices such as a touch screen, a camera, and a fingerprint sensor, and does not need to use the interface provided by the REE, so the security is further ensured.

In one embodiment, the trusted application is a trusted face recognition application, that is, the work of face recognition (ie, the specific recognition work) is performed in the TEE environment of the device, rather than through a networked server (eg, a cloud server). .

In one or more embodiments, the trusted face recognition application includes a main thread and one or more sub-threads. In one embodiment, step S110 may include: the main thread of the trusted face recognition application program sends an instruction to the camera control program to start or end the image acquisition work. In one embodiment, step S120 may include: the first sub-thread and the second sub-thread of the trusted face recognition application respectively read image data from the image buffer in the secure memory for face recognition, wherein , the first sub-thread runs in the trusted execution environment of the second CPU core, and the second sub-thread runs in the trusted execution environment of the third CPU core. It should be pointed out that the second CPU core and the third CPU core are different CPU cores from the aforementioned first CPU core, and are distinguished here by the ordinal words "first", "second" and "third". Those skilled in the art can understand that the number of CPU cores may not be limited to 3, but may be determined according to actual conditions.

In the foregoing embodiment, the first sub-thread and the second sub-thread are started by the main thread of the trusted face recognition application. For example, the main thread of the trusted face recognition application is responsible for starting or ending the image acquisition work through the camera control program, and starts more than one sub-thread, and requests the trusted operating system (Trusted OS) to bind the sub-thread to a specific CPU core run.

In one embodiment, although not shown in FIG. 1 , the above method 1000 further includes: after the trusted application is started, the main thread of the trusted application loads and initializes an operation model. This operating model may be, for example, a face recognition AI (Artificial Intelligence) model. In one embodiment, the above method 1000 further includes: the main thread of the trusted application creates a physical address queue and a synchronization lock, wherein the physical address queue is used to record the physical address where the collected sensitive data is stored or is related to the physical address. The handle corresponding to the address.

In one embodiment, the above method 1000 may further include: the main thread of the trusted application receives a notification of completion of collection of sensitive data and an address where the sensitive data is written from the peripheral application. The above method 1000 may further include: the main thread of the trusted application writes the address into the physical address queue. For example, after the image data collection is completed, the camera control program notifies the main thread of the trusted face recognition application that the collection of image data has been completed, and transmits the physical address to the main thread of the trusted face recognition application, and finally the trusted face recognition application The application main thread writes the physical address of the image data to the queue.

In one embodiment, step S120 includes: the sub-thread of the trusted application reads the address to which the sensitive data is written from the physical address queue after acquiring the synchronization lock. In one embodiment, step S120 further includes: the sub-thread of the trusted application reads the sensitive data from the secure memory according to the address. Since there is only one queue for storing the physical address of image data in the trusted face recognition application, in order to realize the common reading of multiple sub-threads of the trusted face recognition application, a synchronization lock mechanism is added to ensure that different sub-threads do not Get the same physical address, causing programming logic errors.

As a supplement to the trusted application control method shown in FIG. 1 , the method 1000 may further include: the sub-thread of the trusted application performs face detection, feature extraction and face comparison through the operation model. The method 1000 may further include: judging whether the face comparison is successful, and when successful, the trusted face recognition application sub-thread notifies the main thread that the comparison is successful and ends the process.

In one embodiment, the above-mentioned method 1000 further includes: the sub-thread of the trusted application notifies the main thread of the result of the face comparison.

In addition, in one embodiment, the camera control program and the trusted face recognition application may use a handle instead of a handle in the implementation of transmitting the physical address related to the sensitive data. In one embodiment, the above-mentioned method 1000 further includes: the main thread of the trusted application receives a handle from the peripheral application, where the handle corresponds one-to-one with the address where the sensitive data is written. In this way, the camera control program and the trusted face recognition application can reuse the Handle to query the corresponding memory physical address from the common operating system and the trusted operating system.

In one embodiment, a part of the space in the secure memory is converted back to the memory space in the common execution environment after use, that is, the converted memory space is released in time for the common execution environment to use.

FIG. 2 shows a schematic structural diagram of a trusted application device 2000 according to an embodiment of the present invention. As shown in FIG. 2, the trusted application device 2000 includes: a sending unit 210 and a reading unit 220, wherein the sending unit 210 is configured to send an instruction to a peripheral application associated with the trusted application, so that the peripheral application controls the The operating device writes sensitive data into the secure memory; the reading unit 220 is configured to read the sensitive data from the secure memory, wherein the trusted application device is in the trusted execution environment of the first CPU core, and the The peripheral application is in the normal execution environment of the first CPU core, and a part of the space in the secure memory is obtained by changing the memory space in the normal execution environment.

In one embodiment, the sending unit 210 is configured to communicate with an associated peripheral application in a REE (Rich Execution Environment), so that the associated peripheral application controls the corresponding operating device to write sensitive data in the The secure memory in the TEE, and the reading unit 220 reads sensitive data from the secure memory for subsequent processing, which realizes secure collection (acquisition) and processing of sensitive data.

In addition, a part of the space in the secure memory is obtained by changing the memory space in the normal execution environment. By dynamically scheduling the REE memory space for use by the TEE, the problem of insufficient memory space in the TEE environment can be solved.

In one or more embodiments, in the foregoing apparatus 2000, the peripheral application is a camera control program, and the operating device is a camera.

In one embodiment, the sending unit 210 is configured to send an instruction to the camera control program to start or end the image acquisition work. In one embodiment, the reading unit 220 may include a first reading subunit and a second reading subunit, which respectively read image data from an image buffer in the secure memory for face recognition, wherein, The first reading subunit runs in the trusted execution environment of the second CPU core, and the second reading subunit runs in the trusted execution environment of the third CPU core.

Although not shown in FIG. 2 , in one embodiment, the above-mentioned apparatus 2000 may further include: a loading unit for loading and initializing the operation model. In one embodiment, the above-mentioned apparatus 2000 may further include: a creation unit configured to create a physical address queue and a synchronization lock. In one embodiment, the above-mentioned device 2000 may further include: a receiving unit configured to receive a notification of completion of collection of sensitive data and an address to which the sensitive data is written from the peripheral application. As a supplement to the foregoing solution, in one embodiment, the above-mentioned device 2000 further includes: a writing unit, configured to write the address into the physical address queue.

In one embodiment, the reading unit 220 is configured to read the address to which the sensitive data is written from the physical address queue after acquiring the synchronization lock. The read unit 220 may also be configured to read the sensitive data from the secure memory according to the address.

The trusted application device 2000 of the embodiment shown in FIG. 2 moves the work of face recognition to the TEE environment of the device. In addition, in order to solve the problem of computing power and memory space required for face recognition work, Trusted Application Device 2000 uses the hardware protection capability provided by ARM TrustZone technology to simultaneously schedule multiple CPUs (cores) in the TEE environment to improve In addition, the method of dynamically scheduling REE memory space for TEE to use is also proposed to solve the problem of insufficient memory space in the TEE environment.

Referring to FIG. 3, FIG. 3 shows a schematic structural diagram of a trusted system 3000 according to an embodiment of the present invention. In the face recognition scenario, as shown in FIG. 3 , the trusted face recognition system includes a camera control program 312 in the REE environment 310 , a trusted face recognition application 322 in the TEE environment 320 , and camera hardware 330 .

The camera control program 312 is responsible for receiving commands from the trusted face recognition application 322 and controlling the camera hardware 330, such as turning on the camera, turning off the camera, or capturing images. The camera hardware 330 is responsible for receiving commands from the camera control program 312 , performing image data collection, and writing the image data to a designated address in the secure memory 340 . The main thread of the trusted face recognition application 322 is responsible for notifying the camera control program 312 to start or stop collecting image data and is responsible for creating or ending one or more sub-threads, and the sub-threads are responsible for reading image data from the secure memory 340 and performing face recognition work .

Although the trusted system 3000 shown in FIG. 3 is used for face recognition, those skilled in the art can understand that the trusted system 3000 can be used for other applications, including but not limited to fingerprint recognition, touch screen and the like.

In another embodiment, the trusted system 3000 is a trusted fingerprint identification system. The trusted fingerprint identification system includes a fingerprint sensor control program in a REE environment, a trusted fingerprint identification application in a TEE environment, and a fingerprint sensor. The fingerprint sensor control program is responsible for receiving commands from trusted fingerprint recognition applications and controlling the fingerprint sensor, such as turning on the fingerprint sensor, turning off the fingerprint sensor, or collecting user fingerprints. The fingerprint sensor is responsible for receiving the command of the fingerprint sensor control program, performing fingerprint collection, and writing the fingerprint data into the designated address of the secure memory. The main thread of the trusted fingerprint identification application is responsible for notifying the fingerprint sensor control program to start or stop collecting fingerprint data and is responsible for creating or ending more than one sub-thread, and the sub-thread is responsible for reading the fingerprint data from the secure memory and performing fingerprint identification.

Turning to FIG. 4 , FIG. 4 shows an architecture diagram of a face collection and recognition system 4000 according to an embodiment of the present invention, wherein the face collection and recognition system 4000 adopts a multi-core approach to accelerate face recognition work.

As shown in FIG. 4 , the main thread 420 of the trusted face recognition application is responsible for notifying the camera control program 410 to start or end the image acquisition work, and to open more than one sub-threads 430 and 440 and request the trusted operating system (Trusted OS) to

Threads

430, 440 are bound to run on specific CPU cores. In the embodiment shown in FIG. 4 , the first sub-thread 430 of the trusted face recognition application is bound to the CPU1 core, and the second sub-thread 440 of the trusted face recognition application is bound to the CPU2 core. The main thread 420 of the trusted face recognition application and the camera control program 410 are both located on the CPU0 core, but the main thread 420 of the trusted face recognition application is in the TEE environment, and the camera control program 410 is in the REE environment.

As shown in Figure 4, a trusted face recognition application includes a main thread and more than one sub-thread. The main thread 420 of the trusted face recognition application sends an instruction to the camera control program 410 to start or end the image acquisition work. The camera hardware 470 is responsible for receiving the instructions of the camera control program 410 , collecting image data, and writing the image data into the image buffer 462 . In subsequent operations, the first sub-thread 430 and the second sub-thread 440 of the trusted face recognition application respectively read image data from the image buffer 462 in the secure memory 460 for face recognition.

Those skilled in the art can understand that, in another embodiment, the above-mentioned face collection and recognition system 4000 can be changed to a fingerprint collection and recognition system. Specifically, the fingerprint collection and identification system also adopts a multi-core approach to speed up the fingerprint identification work. The fingerprint collection and identification system includes a fingerprint sensor control program, a trusted fingerprint identification application, a fingerprint sensor, and the like.

In this embodiment, the main thread of the trusted fingerprint identification application is responsible for notifying the fingerprint sensor control program to start or end the fingerprint collection work, and to start more than one sub-thread and request the trusted operating system (Trusted OS) to bind the sub-thread to the run on a specific CPU core. In one embodiment, the first sub-thread of the trusted fingerprinting application is bound on the second CPU core, and the second sub-thread of the trusted fingerprinting application is bound on the third CPU core. The main thread of the trusted fingerprint identification application and the fingerprint sensor control program are both located on the first CPU core, but the main thread of the trusted fingerprint identification application is in the TEE environment, and the fingerprint sensor control program is in the REE environment.

That is, a trusted fingerprinting application may include a main thread and more than one sub-thread. The main thread of the trusted fingerprint identification application sends an instruction to the fingerprint sensor control program to start or end the fingerprint collection. The fingerprint sensor is responsible for receiving the instructions of the fingerprint sensor control program, collecting the fingerprint data, and writing the fingerprint data into the fingerprint buffer. In subsequent operations, the first sub-thread and the second sub-thread of the trusted fingerprint identification application respectively read fingerprint data from the fingerprint buffer in the secure memory for fingerprint identification.

Referring to FIG. 5, FIG. 5 illustrates a memory handover process 5000 according to one embodiment of the present invention. Because the memory space required for face recognition is large, the configuration size of the secure memory space cannot generally meet the requirements of face recognition applications. Therefore, a memory handover process is added in an embodiment of the present application, so that the ordinary memory space can be converted into Safe memory space, and then converted back to normal memory space after use.

In one embodiment, the memory handover process 5000 includes the following steps:

In step S510, a common operating system configures a common memory space, and obtains a physical address;

In step S520, the normal operating system transmits the physical address and memory space size to the trusted operating system; and

In step S530, after obtaining the physical address and the size of the memory space, the trusted operating system changes the memory space to a secure memory through the hardware of the memory protection unit MPU.

FIG. 6 shows an initialization flow 6000 of a trusted face recognition application according to an embodiment of the present invention. In the initialization process 6000, the main thread of the trusted application will create a global image physical address queue, which is used to record the physical address where the image data that has been collected is stored.

In one embodiment, the initialization process 6000 shown in FIG. 6 includes the following steps:

In step S610, the trusted operating system loads and starts the trusted face recognition application;

In step S620, the trusted face recognition application program loads and initializes the face recognition AI model;

In step S630, the main thread of the trusted face recognition application program creates a global image physical address queue and synchronization lock.

FIG. 7 shows a face data collection process 7000 according to an embodiment of the present invention. After the main thread of the trusted face recognition application notifies the camera control program to start collecting image data, the camera control program will transmit the physical address to the camera hardware, and the camera hardware will directly write the collected image data to this physical address. After completing the image data collection, the camera control program informs the main thread of the trusted face recognition application that the collection of image data has been completed and transmits the physical address to the main thread of the trusted face recognition application, and finally the trusted face recognition application The main thread of the program writes the physical address of the image data into the queue.

Specifically, in one embodiment, the face data collection process 7000 includes the following steps:

In step S710, the main thread of the trusted face recognition application program starts to collect images/images through the camera control program;

In step S720, the camera control program is turned on and controls the camera hardware to take a photo, and the camera hardware directly writes the data into the image buffer;

In step S730, the camera control program notifies the main thread of the trusted face recognition application, and transmits the physical address of the image/image data to the main thread of the trusted face recognition application;

In step S740, the camera control program determines whether it has received a notification that the trusted face recognition application stops collecting; if so, execute step S750, otherwise execute step S720;

In step S750, the camera control program controls the camera hardware to stop capturing and closes the camera hardware.

The face data collection process 7000 in the above embodiment can also be applied to fingerprint data collection. In another embodiment, the fingerprint data collection process includes: (1) the main thread of the trusted fingerprint identification application program starts to collect fingerprints through the fingerprint sensor control program; (2) the fingerprint sensor control program starts and controls the fingerprint sensor to acquire a fingerprint, And the fingerprint sensor directly writes the data into the fingerprint buffer; (3) the fingerprint sensor control program notifies the main thread of the trusted fingerprint identification application program, and transmits the physical address of the fingerprint data to the main thread of the trusted fingerprint identification application program; (4) The fingerprint sensor control program judges whether it receives a notification that the trusted fingerprint identification application program stops collecting; if so, execute the next step 5, otherwise execute the aforementioned step 2; (5) The fingerprint sensor control program controls the fingerprint sensor to stop Capture your fingerprint and turn off the fingerprint sensor.

FIG. 8 shows a face recognition process 8000 of a trusted face recognition application according to an embodiment of the present invention. The face recognition process 8000 of FIG. 8 accelerates face recognition work by utilizing multiple CPU cores. In the face recognition process 8000, with the support of ARM TrustZone hardware technology, the method of using multiple CPU cores to run face recognition in TEE. ARM TrustZone hardware technology implements a hardware-level security protection mechanism, so that the entire system is logically divided into two execution environments: REE and TEE. When the CPU core switches to REE, the software running on this CPU core is only allowed to access REE. The environment can access hardware resources. On the contrary, software running on the TEE execution environment can access the hardware resources related to the TEE execution environment.

Each CPU core can switch between REE and TEE. In the embodiment of FIG. 8 , the camera control program and the main thread of the trusted face recognition application run on the same CPU core, and the sub-threads of the trusted face recognition application run on other CPU cores respectively, so that It can achieve the best performance of parallel processing multiple image data.

In addition, there is only one queue for storing the physical address of image data in the trusted face recognition application. Therefore, in order to allow the sub-threads of multiple trusted face recognition applications to read together, a synchronization lock mechanism must be added to ensure that different Child threads will not get the same physical address, causing programming logic errors.

In one embodiment, the face recognition process 8000 of the trusted face recognition application includes the following steps:

In step S810, the main thread of the trusted face recognition application program creates one or more sub-threads and runs on different CPU cores;

In step S820, the child thread of the trusted face recognition application waits for image data;

In step S830, the sub-thread of the trusted face recognition application program determines whether to acquire the synchronization lock; if so, execute step S840, otherwise return to step S820;

In step S840, the sub-thread of the trusted face recognition application program further determines whether there is an address in the queue; if so, executes step S850, otherwise returns to step S820;

In step S850, the sub-thread of the trusted face recognition application program obtains a physical address from the queue, and reads out the image data from the image buffer according to the physical address;

In step S860, the sub-thread of the trusted face recognition application performs face detection, feature extraction and face comparison through the AI model;

In step S870, the sub-thread of the trusted face recognition application program determines whether the face comparison is successful; if so, proceed to step S880, otherwise return to step S820; and

In step S880, the sub-thread of the trusted face recognition application notifies the main thread that the comparison is successful and ends the process.

In another embodiment, the fingerprint identification process of the trusted fingerprint identification application may include the following steps: (1) the main thread of the trusted fingerprint identification application creates one or more sub-threads and runs on different CPU cores; ( 2) The sub-thread of the trusted fingerprint identification application program waits for the fingerprint data; (3) the sub-thread of the trusted fingerprint identification application program determines whether to obtain the synchronization lock; if so, execute step 4, otherwise return to step 2; (4) can The sub-thread of the trusted fingerprint identification application program further determines whether there is an address in the queue; if so, execute step 5, otherwise return to step 2; (5) the sub-thread of the trusted fingerprint identification application program obtains a physical address from the queue, and according to The physical address reads the fingerprint data from the fingerprint buffer; (6) The sub-thread of the trusted fingerprint identification application performs fingerprint detection, feature interception and fingerprint comparison through the AI model; (7) The trusted fingerprint identification application The sub-thread of the program determines whether the fingerprint comparison is successful; if so, continue to execute step 8, otherwise return to step 2; (8) The sub-thread of the trusted fingerprint identification application informs the main thread that the comparison is successful and ends the process.

To sum up, the method for controlling a trusted application and the trusted application device according to the embodiments of the present invention communicate with the associated peripheral application in the REE (Rich Execution Environment, common execution environment), so that the associated peripheral application controls the corresponding peripheral application. The operating device writes sensitive data into the secure memory in the TEE, and the trusted application in the TEE reads the sensitive data from the secure memory for subsequent processing, which realizes the secure collection (acquisition) and processing of sensitive data . In addition, the trusted application and the associated peripheral application are in the same CPU core, but belong to different execution environments (the trusted application is in the trusted execution environment, while the peripheral application is in the normal execution environment), which can help ensure security It saves the use of CPU cores under the premise of stability, that is, saves hardware overhead. Finally, a part of the secure memory space in the TEE environment is obtained by changing the memory space in the normal execution environment, which solves the problem of insufficient memory space in the TEE environment.

Although the foregoing specification describes only some of these embodiments of the present invention, those of ordinary skill in the art will appreciate that the present invention may be embodied in many other forms without departing from the spirit and scope thereof. For example, the camera control program and the trusted face recognition application can use the handle instead of the handle in the implementation of the physical address of the transmitted image data, and the camera control program and the trusted face recognition application can use the handle to send the The conventional operating system (Rich OS) and the trusted operating system (Trusted OS) query the corresponding memory physical address. Accordingly, the examples and embodiments shown are to be regarded as illustrative and not restrictive, and various modifications are possible within the present invention without departing from the spirit and scope of the invention as defined by the appended claims. with replacement.

Claims

A control method for trusted applications, characterized in that the method comprises:

The trusted application sends an instruction to a peripheral application associated with the trusted application, so that the peripheral application controls the operating device to write sensitive data into the secure memory; and

The trusted application reads the sensitive data from the secure memory, wherein the trusted application is in the trusted execution environment of the first CPU core, and the peripheral application is in the normal execution environment of the first CPU core. In an execution environment, a part of the space in the secure memory is obtained by changing the memory space in the normal execution environment.
The method of claim 1, wherein the trusted application is a trusted face recognition application, the peripheral application is a camera control program, and the operating device is a camera.
The method of claim 2, wherein the trusted face recognition application includes a main thread and more than one sub-thread.
The method of claim 3, wherein the trusted application sending an instruction to a peripheral application associated with the trusted application comprises:

The main thread of the trusted face recognition application program sends an instruction to the camera control program to start or end the image acquisition work.
The method of claim 4, wherein the trusted application reading the sensitive data from the secure memory comprises:

The first sub-thread and the second sub-thread of the trusted face recognition application respectively read image data from the image buffer in the secure memory for face recognition, wherein the first sub-thread runs in the second sub-thread. In the trusted execution environment of the CPU core, the second sub-thread runs in the trusted execution environment of the third CPU core.
The method of claim 5, wherein the first sub-thread and the second sub-thread are started by a main thread of the trusted face recognition application.
The method of claim 1, further comprising:

After the trusted application is started, the main thread of the trusted application loads and initializes the operation model.
The method of claim 1 or 7, further comprising:

The main thread of the trusted application creates a physical address queue and a synchronization lock.
The method of claim 8, further comprising:

The main thread of the trusted application receives from the peripheral application a notification of completion of collection of sensitive data and an address where the sensitive data is written.
The method of claim 9, further comprising:

The main thread of the trusted application writes the address into the physical address queue.
The method of claim 10, wherein the trusted application reading the sensitive data from the secure memory comprises:

The child thread of the trusted application reads the address to which the sensitive data is written from the physical address queue after acquiring the synchronization lock.
The method of claim 11, wherein the trusted application reading the sensitive data from the secure memory further comprises:

The child thread of the trusted application reads the sensitive data from the secure memory according to the address.
The method of claim 7, further comprising:

The sub-thread of the trusted application performs face detection, feature extraction and face comparison through the operation model.
The method of claim 13, further comprising:

The sub-thread of the trusted application notifies the main thread of the result of the face comparison.
The method of claim 8, further comprising:

The main thread of the trusted application receives a handle from the peripheral application, and the handle corresponds one-to-one with the address where the sensitive data is written.
The method of claim 1, wherein a part of the space in the secure memory is converted back to the memory space in the normal execution environment after use.
A trusted application device, characterized in that the device includes:

a sending unit, configured to send an instruction to a peripheral application associated with the trusted application, so that the peripheral application controls the operating device to write sensitive data into a secure memory; and

a reading unit, configured to read the sensitive data from the secure memory, wherein the trusted application device is located in the trusted execution environment of the first CPU core, and the peripheral application is located in the first CPU core In the general execution environment, a part of the space in the secure memory is obtained by changing the memory space in the general execution environment.
The device of claim 17, wherein the peripheral application is a camera control program, and the operating device is a camera.
The apparatus of claim 18, wherein the sending unit is configured to send an instruction to the camera control program to start or end the image acquisition work.
20. The apparatus of claim 19, wherein the reading unit includes a first reading subunit and a second reading subunit, which respectively read image data from an image buffer in the secure memory for human Face recognition, wherein the first reading subunit runs in the trusted execution environment of the second CPU core, and the second reading subunit runs in the trusted execution environment of the third CPU core.
The apparatus of claim 17, further comprising:

Loading unit, used to load and initialize the operation model.
The apparatus of claim 17 or 21, further comprising:

Creation unit for creating physical address queues and synchronization locks.
The apparatus of claim 22, further comprising:

A receiving unit, configured to receive a notification of completion of collection of sensitive data and an address to which the sensitive data is written from the peripheral application.
The apparatus of claim 23, further comprising:

a writing unit, configured to write the address into the physical address queue.
25. The apparatus of claim 24, wherein the read unit is configured to read the address to which the sensitive data is written from the physical address queue after the synchronization lock is acquired.
26. The apparatus of claim 25, wherein the read unit is further configured to read the sensitive data from the secure memory according to the address.
18. The apparatus of claim 17, wherein a portion of the secure memory is converted back to memory space in the normal execution environment after use.
A computer storage medium, characterized in that the medium includes instructions that, when executed, perform the method of any one of claims 1 to 16 .
An intelligent terminal, characterized in that, the intelligent terminal includes the trusted application device according to any one of claims 17 to 27.