WO2020034076A1

WO2020034076A1 - Data processing method and device

Info

Publication number: WO2020034076A1
Application number: PCT/CN2018/100305
Authority: WO
Inventors: 文长春; 周君; 陈伟; 张建永
Original assignee: 华为技术有限公司
Priority date: 2018-08-13
Filing date: 2018-08-13
Publication date: 2020-02-20

Abstract

The present application relates to the field of secure computing, and provided thereby are a data processing method and device, which are used to ensure the security of related data operations in AI technology. The device comprises: a general-purpose processor and an artificial intelligence processor. The general-purpose processor is used to configure the artificial intelligence processor to switch from a non-secure mode to a secure mode, and call the artificial intelligence processor to process secure data, the secure data being generated by the general-purpose processor running secure software. The artificial intelligence processor is used to switch from non-secure mode to secure mode in response to the configuration of the general-purpose processor, and process the secure data in response to calling by the general-purpose processor and return a secure data processing result to the general-purpose processor. In secure mode, the artificial intelligence processor cannot process non-secure data generated by the general-purpose processor running non-secure software.

Description

Data processing method and device

Technical field

The present application relates to the field of secure computing, and in particular, to a data processing method and device.

Background technique

With the continuous advancement of technology, the field of artificial intelligence (AI) has been greatly developed. AI is produced by computer technology, but today's artificial intelligence has covered multiple disciplines including linguistics and psychology. Its scope has far exceeded the scope of computer technology, and it has become an independent branch of science. It is a hot field in recent years.

At present, AI technology can be applied to the security features of electronic devices. For example, three-dimensional (3D) face unlocking using AI can be used for mobile phone screen unlocking, mobile phone payment confirmation, and identity authentication. Specifically, a neural-network processing unit (NPU) (also referred to as an NPU processor) can be integrated into an electronic device, and a mobile phone screen unlock, mobile payment confirmation, and identity can be realized through a chip with a built-in NPU processor. Relevant AI data calculation during authentication and other processes.

However, the use of AI technology in the security features of electronic devices requires the security of related data operations in AI technology to prevent user data from being leaked, tampered with, and counterfeited, as well as prevent related computing logic from being tampered with and counterfeited. Problems such as illegal unlocking or illegal payment ensure the safety of users during use and avoid unnecessary losses to users. In the prior art, no solution has been provided for how to ensure the security of related data operations in AI technology.

Summary of the Invention

The embodiments of the present application provide a data processing method and device, which are used to ensure the security of related data operations in AI technology.

To achieve the above purpose, the embodiments of the present application adopt the following technical solutions:

According to a first aspect, a data processing device is provided. The device includes a general-purpose processor and an artificial intelligence processor, wherein the general-purpose processor is used to set the artificial intelligence processor to switch from a non-secure mode to a secure mode, and call the artificial intelligence. The processor processes security data generated by the general-purpose processor running security software; an artificial intelligence processor for switching from a non-secure mode to a secure mode in response to a setting of the general-purpose processor, and in response to the general-purpose processor The call processes the secure data and returns the secure data processing result to the general-purpose processor; in the secure mode, the artificial intelligence processor cannot process the non-secure data generated by the general-purpose processor running non-secure software.

In the above technical solution, when the artificial intelligence processor is set to the secure mode, it cannot process the non-secure data generated by the general-purpose processor running non-secure software, thereby ensuring the security of the artificial intelligence processor when processing the secure data, that is, the guarantee Security of related data operations in AI technology.

In a possible implementation manner of the first aspect, when switching from the non-secure mode to the secure mode, the general-purpose processor is configured to perform at least one of the following: disabling an artificial intelligence processor from processing a non-safety-side interruption function. Security-side interrupts refer to interrupts triggered by non-safety software; or, the function of the artificial intelligence processor to handle security-side interrupts is enabled, and security-side interrupts refer to interrupts triggered by security software; or, setting the memory management unit to safe mode, Non-secure software cannot access the memory corresponding to the memory management unit in safe mode. In the foregoing possible implementation manner, the artificial intelligence processor cannot process non-secure data, can process secure data, and non-secure software cannot access memory in a secure mode, thereby ensuring the security of the secure data and its processing process.

In a possible implementation manner of the first aspect, before the general-purpose processing sets the artificial intelligence processor to switch from the non-secure mode to the secure mode, the general-purpose processor is further configured to set the priority of the secure data to be processed by the artificial intelligence processor. Level is higher than non-secure data, which is generated by general-purpose processors running non-secure software and has not yet been processed by artificial intelligence processors. In the foregoing possible implementation manner, the artificial intelligence processor can be made to process the security data in priority, thereby improving the processing speed of the security data, and further improving the user experience.

In a possible implementation manner of the first aspect, the general-purpose processor is further configured to set the artificial intelligence processor to switch from the secure mode to the non-secure mode, and call the artificial intelligence processor to process non-secure data; the artificial intelligence processor, It is used to switch from the secure mode to the non-secure mode in response to the setting of the general-purpose processor, and to process the non-secure data in response to the call of the general-purpose processor and return the non-secure data processing result to the general-purpose processor. In the foregoing possible implementation manner, when the artificial intelligence processor is set to the non-secure mode, it can process non-secure data, so that the artificial intelligence processor can be compatible with the processing of the secure data and the non-secure data.

In a possible implementation manner of the first aspect, in a non-secure mode, the artificial intelligence processor cannot process secure data. In the foregoing possible implementation manner, the security of the secure data and its processing process can be guaranteed in the non-secure mode.

In a possible implementation manner of the first aspect, when switching from the secure mode to the non-secure mode, the general-purpose processor is configured to perform at least one of the following: disabling the function of the artificial intelligence processor to handle interrupts on the security side, and the security side Interrupts refer to interrupts triggered by secure software; or, enable the artificial intelligence processor to handle non-secure side interrupts, non-secure side interrupts refer to interrupts triggered by non-secure software; or set the memory management unit to non-secure mode In non-secure mode, non-secure software can access the memory corresponding to the memory management unit. In the foregoing possible implementation manner, the artificial intelligence processor cannot process secure data, can process non-secure data, and non-secure software can access memory in the non-secure mode, thereby enabling the artificial intelligence processor to process secure data and non-secure data. Compatible.

In a possible implementation manner of the first aspect, the general-purpose processor is specifically configured to set the artificial intelligence processor to switch from the secure mode to the non-secure mode when no new secure data needs to be processed. In the foregoing possible implementation manner, the problem that the artificial intelligence processor is in a safe mode for a long time and cannot process non-secure data can be avoided, thereby ensuring the efficiency of the artificial intelligence processor in processing non-secure data.

In a possible implementation manner of the first aspect, the general-purpose processor is further configured to set the artificial intelligence processor to a low power consumption state when a duration in which no new non-secure data needs to be processed reaches a preset time. Optionally, setting the artificial intelligence processor to a low power consumption state may include: powering off the artificial intelligence processor, or reducing the power consumption of the artificial intelligence processor, for example, reducing the voltage and current of the artificial intelligence processor Or at least one of the clocks. In the foregoing possible implementation manner, the power consumption of the artificial intelligence processor can be reduced without affecting the data processing process, thereby reducing the power consumption of the data processing device.

In a possible implementation manner of the first aspect, the general-purpose processor is further configured to restore the artificial intelligence processor to a working state when it is required to process new secure data or new non-secure data. Optionally, restoring the artificial intelligence processor to the working state may include: powering on the artificial intelligence processor, or setting the power consumption of the artificial intelligence processor to the working state, for example, changing the artificial intelligence processor ’s The voltage, current, and / or clock are restored to operating values. In the foregoing possible implementation manner, the power consumption of the artificial intelligence processor can be reduced without affecting the data processing process, thereby reducing the power consumption of the data processing device.

In a possible implementation manner of the first aspect, the artificial intelligence processor is a neural network processing unit NPU or an image processing unit (GPU). In the foregoing possible implementation manners, several possible artificial intelligence processors are provided.

In a possible implementation manner of the first aspect, the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general operating system or non-secure application software. In the foregoing possible implementation manners, several possible security software and non-security software are provided.

In a second aspect, a data processing device is provided. The device includes: a secure operating system and a secure artificial intelligence processor driver; wherein the secure operating system is used to notify the secure artificial intelligence processor to drive the artificial intelligence processor into a secure mode; The artificial intelligence processor driver is used to configure the artificial intelligence processor to switch from the non-secure mode to the secure mode. In the secure mode, the artificial intelligence processor cannot process the non-secure data generated by the non-secure software; the secure operating system also uses For sending safety data to the safety artificial intelligence processor driver, the safety data is generated by the safety software; the safety artificial intelligence processor driver is also used to receive the safety data, call the artificial intelligence processor to process the safety data, and return the safety data The data processing results are given to the secure operating system.

In a possible implementation manner of the second aspect, when the artificial intelligence processor is configured to switch from the non-secure mode to the secure mode, the secure artificial intelligence processor driver is configured to perform at least one of the following: enable the secure artificial intelligence processor The driver receives the non-safety-side interruption function. The safety-side interruption refers to the interruption triggered by the safety software; or, the memory management unit is set to the safe mode. In the safe mode, the non-safety software cannot access the memory corresponding to the memory management unit.

In a possible implementation manner of the second aspect, the device further includes: a general operating system and a non-secure artificial intelligence processor driver; wherein the secure operating system is further configured to notify the secure artificial intelligence processor driver before entering the safe mode Sending the first instruction information to the general operating system, where the first instruction information is used to instruct the artificial intelligence processor to enter the safe mode; the general operating system is used to notify the non-secure artificial intelligence processor to drive the human body when the first instruction information is received The intelligent processor enters the secure mode; the non-secure artificial intelligence processor driver is used to configure the artificial intelligence processor to switch from the non-secure mode to the secure mode.

In a possible implementation manner of the second aspect, when the artificial intelligence processor is configured to switch from the non-secure mode to the secure mode, the non-secure artificial intelligence processor driver is configured to perform the following operations: disabling the non-secure artificial intelligence processing The device driver receives the non-safety side interrupt function. The non-safety side interrupt refers to the interrupt triggered by the non-safety software.

In a possible implementation manner of the second aspect, before notifying the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the safe mode, the general operating system is further configured to: set the priority of processing the secure data higher than that of the non-secure artificial intelligence processor. Security data, non-security data is generated by non-security software and has not been processed by the artificial intelligence processor.

In a possible implementation manner of the second aspect, the secure operating system is further configured to notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode; the secure artificial intelligence processor driver is also used to Configure the artificial intelligence processor in non-secure mode; the secure operating system is also used to send second instruction information to the general operating system, the second instruction information is used to instruct the switch from the secure mode to the non-secure mode; the general operating system is also used to When receiving the second instruction information, notify the non-secure artificial intelligence processor to drive the artificial intelligence processor into the non-secure mode; the non-secure artificial intelligence processor driver is also used to configure the artificial intelligence processor to be in the non-secure mode and receive non-secure artificial intelligence processors. Safe data, call the artificial intelligence processor to process the unsafe data, and return the unsafe data processing result to the general operating system.

In a possible implementation manner of the second aspect, when the artificial intelligence processor is configured as the non-secure mode, the secure artificial intelligence processor driver is configured to perform the following operations: disabling the secure artificial intelligence processor driver to receive security Side interrupt function, the safe side interrupt refers to the interrupt triggered by the security software; and, the non-secure artificial intelligence processor driver is used to perform at least one of the following: enable the non-secure artificial intelligence processor driver to receive the non-secure side interrupt function The non-secure side interrupt refers to an interrupt triggered by the non-secure software; or, the memory management unit is set to an unsecure mode, and the non-secure software can access the memory corresponding to the memory management unit in the non-secure mode.

In a possible implementation manner of the second aspect, the secure operating system is specifically configured to notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode when no new secure data needs to be processed.

In a possible implementation manner of the second aspect, the non-secure artificial intelligence processor driver is further configured to: set the artificial intelligence processor to low when there is no new non-secure data that needs to be processed for a preset time. Power status.

In a possible implementation manner of the second aspect, the non-secure artificial intelligence processor driver is further configured to: when new security data or new non-secure data needs to be processed, restore the artificial intelligence processor to a working state.

In a possible implementation manner of the second aspect, the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general operating system or non-secure application software.

In a third aspect, a data processing method is provided. The method includes: the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor into a safe mode; the secure artificial intelligence processor driver configures the artificial intelligence processor to switch from a non-secure mode to Safe mode, in which the artificial intelligence processor cannot process non-secure data generated by non-secure software; the secure operating system sends secure data to the secure artificial intelligence processor driver, and the secure data is generated by the secure software; secure labor The intelligent processor driver receives the safety data, calls the artificial intelligence processor to process the safety data, and returns the safety data processing result to the safety operating system.

In a possible implementation manner of the third aspect, configuring the secure artificial intelligence processor driver to switch the artificial intelligence processor from the non-secure mode to the secure mode includes at least one of the following: enabling the secure artificial intelligence processor driver to receive non-secure The function of the side interrupt, the safe side interrupt refers to the interrupt triggered by the security software; or, the memory management unit is set to the safe mode, and the non-safe software cannot access the memory corresponding to the memory management unit in the safe mode.

In a possible implementation manner of the third aspect, before the secure operating system notifies the secure artificial intelligence processor driver to enter the secure mode, the method further includes: the secure operating system sends the first instruction information to the general operating system, and the first instruction information Used to instruct the artificial intelligence processor to enter the safe mode; when the general operating system receives the first instruction message, it informs the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the safe mode; the non-secure artificial intelligence processor drives to configure the artificial intelligence processing The device switches from non-safe mode to safe mode.

In a possible implementation manner of the third aspect, configuring the non-secure artificial intelligence processor driver to switch the artificial intelligence processor from the non-secure mode to the secure mode includes: disabling the non-secure artificial intelligence processor driver to receive the non-secure side Interrupt function, non-safety side interrupt refers to the interrupt triggered by non-safety software.

In a possible implementation manner of the third aspect, before the general operating system notifies the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the safe mode, the method further includes: the general processing system sets the priority of the secure data to be processed to be high For non-secure data, non-secure data is generated by non-secure software and has not yet been processed by an artificial intelligence processor.

In a possible implementation manner of the third aspect, the method further includes: the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode; the secure artificial intelligence processor drives to configure the artificial intelligence The processor is in a non-secure mode; the secure operating system sends second instruction information to the general operating system, and the second instruction information is used to instruct switching from the secure mode to the non-secure mode; when the general operating system receives the second instruction information, it informs the non- The secure artificial intelligence processor drives the artificial intelligence processor into the non-secure mode; the non-secure artificial intelligence processor drives configure the artificial intelligence processor to the non-secure mode, and receives non-secure data, calls the artificial intelligence processor to process the non-secure data, and returns The results of non-secure data processing are given to the general operating system.

In a possible implementation manner of the third aspect, the safe artificial intelligence processor driver configures the artificial intelligence processor in a non-secure mode, including: disabling the function of the safe artificial intelligence processor driver to receive the safety-side interrupt, and the safety-side interrupt Refers to interrupts triggered by secure software; and, the configuration of an artificial intelligence processor in non-secure mode driven by a non-secure artificial intelligence processor, including at least one of the following: enabling the function of the non-secure artificial intelligence processor driver to receive non-secure side interrupts The non-secure side interrupt refers to the interrupt triggered by the non-secure software; or, the memory management unit is set to the non-secure mode, and the non-secure software can access the memory corresponding to the memory management unit in the non-secure mode.

In a possible implementation manner of the third aspect, the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode, including: when the secure operating system has no new secure data to be processed To notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the safe mode to the non-safe mode.

In a possible implementation manner of the third aspect, the method further includes: when there is no new non-secure data that needs to be processed for a preset time, the non-secure artificial intelligence processor driver sets the artificial intelligence processor to low Power status.

In a possible implementation manner of the third aspect, the method further includes: when new security data or new non-secure data needs to be processed, the non-secure artificial intelligence processor drives and restores the artificial intelligence processor to a working state.

In a possible implementation manner of the third aspect, the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general operating system or non-secure application software.

According to a fourth aspect, a readable storage medium is provided, where the readable storage medium stores instructions, and when the readable storage medium is run on a device, the device is caused to perform the third aspect or any of the third aspects. Data processing methods provided by one possible implementation.

According to a sixth aspect, a computer program product is provided, and when the computer program product runs on a computer or a processor, the computer or processor executes the first aspect or any possible implementation manner of the first aspect. Provided data processing methods.

Understandably, any of the data processing devices, methods, computer storage media, or computer program products provided above is used to perform the functions of the corresponding devices provided above. Therefore, for the beneficial effects that can be achieved, refer to the above. The beneficial effects in the provided corresponding device are not repeated here.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic structural diagram of a system-level data processing device according to an embodiment of the present application;

FIG. 2 is an architecture diagram of software running on a general-purpose processor according to an embodiment of the present application; FIG.

FIG. 3 is an architecture diagram of a universal operating system / secure operating system provided by an embodiment of the present application; FIG.

FIG. 4 is a first schematic flowchart of a data processing method according to an embodiment of the present application; FIG.

5 is a second schematic flowchart of a data processing method according to an embodiment of the present application;

FIG. 6 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present application.

detailed description

In the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association relationship of related objects, and indicates that there can be three kinds of relationships, for example, A and / or B can represent: the case where A exists alone, A and B exist simultaneously, and B alone exists, where A, B can be singular or plural. The character "/" generally indicates that the related objects are an "or" relationship.

FIG. 1 is a schematic structural diagram of a system-level data processing device according to an embodiment of the present application. The data processing device may be a mobile phone, a tablet computer, a computer, a wearable device, a vehicle-mounted device, or a portable device. For convenience of description, the above-mentioned devices may be collectively referred to as a data processing apparatus. Referring to FIG. 1, the data processing apparatus may include a memory 101, a processor 102, a communication interface 103, and a bus 104. The memory 101, the processor 102, and the communication interface 103 are connected to each other through a bus 104.

The memory 101 can be used to store data, software programs, and modules, and mainly includes a program storage area and a data storage area. The storage program area can store an operating system and at least one application required by a function. The storage data area can store the data processing. Data created when the device is used. In the embodiment of the present application, the operating system may include a general-purpose operating system (such as an Android system) and a secure operating system (such as a dedicated operating system such as a Trustzone system); the applications required for the at least one function may include Security application software and non-security application software. Trusted application (TA) software can include secure artificial intelligence processor drivers and other applications with security features (for example, screen unlocking software based on the Trustzone system, payment software, etc.). Security application software is also called client application (CA), which can include non-secure artificial intelligence processor drivers and other applications with non-secure features (for example, Android-based web browsing software, video playback software, etc.); The data stored in the storage data area may include secure data, secure data processing results, non-secure data and / or non-secure data processing results, and the like. In some feasible embodiments, the software module stored in the memory 101 may include a memory management unit, and the storage data area may include memory. The memory management unit may be used to manage the memory. For example, the memory management unit may be used to set access to the memory. Permissions. In other feasible embodiments, there may be one memory or multiple memories; the memory may be a floppy disk, a hard disk such as a built-in hard disk and a mobile hard disk, a magnetic disk, an optical disk, and a magneto optical disk such as a read-only optical disk (compact disc-read- only memory (CD_ROM), digital video disc (read-only memory (DVD_ROM), non-volatile storage devices such as random access memory (RAM), read-only memory (ROM) , Programmable read-only memory (PROM), erasable programmable read-only memory (erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (electrically erasable programmable read-only memory) only memory (EEPROM), flash memory, or any other form of storage medium known in the art.

The processor 102 is configured to control and manage the actions of the data processing device, for example, by running or executing software programs and / or modules stored in the memory 101 and calling data stored in the memory 101 to execute the data processing device. Various functions and processing data. The processor 102 may include at least one of a central processing unit, a digital signal processor, an artificial intelligence (AI) processor, a microprocessor, or a microcontroller, and may optionally include an application specific integrated circuit, a field Programmable gate array or other programmable logic device, transistor logic device, hardware accelerator, or any combination thereof. The processor may also be a combination that implements computing functions. In the embodiment of the present application, the processor 102 may include a combination of a general-purpose processor 1021 and an artificial intelligence processor 1022, and may further optionally include other hardware accelerators, and the combination may be used to support the device to perform data processing functions in the present application. For the specific process of data processing, please refer to the description in the following method embodiments. Optionally, the artificial intelligence processor 1022 may be a neural-network processing unit (NPU), or an image processing unit (graphics processing unit, GPU). The general-purpose processor 1021 may include a processing device running software, such as a central processing unit, a microprocessor, or a microcontroller. The software may include an operating system, application software, or driver software. The data processing device in the above embodiments is a system. Optionally, the processor 102 itself can also be regarded as a data processing device, which functions as a computing core in the system to realize data processing capabilities.

The communication interface 103 is used to support the data processing apparatus to communicate with other equipment. The bus 104 may be a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used in FIG. 1, but it does not mean that there is only one bus or one type of bus.

FIG. 2 is a schematic structural diagram of software running on a general-purpose processor 1021 according to an embodiment of the present application. Among them, the general-purpose processor 1021 can be used to run security software and non-secure software. The data generated by the general-purpose processor 1021 running the security software is called secure data, and the data generated by running the non-secure software is called non-secure data. The general-purpose processor 1021 may include at least one processing unit, such as an ARM, MIPS, or X86 architecture device. Security software has higher security requirements than non-secure software, or the environment formed by secure software is more secure than non-secure software. For example, the isolation between secure and non-secure software prevents non-secure software from randomly accessing the environment created by the secure software. The security software includes at least one of a security operating system (OS) or security application software; a security operating system may refer to an operating system capable of meeting certain security technical requirements (for example, access control, identification, etc.), such as The secure operating system may be an operating system established to ensure the security of related data operations in AI technology, such as a Trustzone system; the security application software may refer to application software that needs to ensure security, and the security application software may Including one or more security applications or security drivers, for example, the security driver may include a security artificial intelligence processor driver (referred to as a security AI driver for driving the artificial intelligence processor 1022 under a secure operating system), and the security application may Including screen unlock software and payment software, it can run on the Trustzone system. Non-secure software includes at least one of a general-purpose operating system and non-secure application software; a general-purpose operating system may refer to an operating system without specific security requirements, for example, a general-purpose operating system may include an Android operating system or a Windows operating system; Non-secure application software can refer to application software that does not require security requirements. Non-secure application software can include one or more non-secure applications or non-secure drivers. For example, non-secure drivers can include non-secure artificial intelligence processors. Drivers (referred to as non-secure AI drivers, used to drive the artificial intelligence processor 1022 under a general operating system), non-secure applications may include web browsing software, video playback software, and the like. Therefore, there is isolation between safety software and non-safety software. For example, the running instructions of both parties can be isolated from each other without affecting the operation of the other party. In the embodiment of the present invention, the security software is more secure than non-secure software. For example, the environment formed by the security software may be called a Trusted Execution Environment (TEE), and the environment formed by the non-secure software may be called a Rich Execution Environment (REE). The environment formed by the security software is more secure than the environment formed by non-secure software. The above software can be stored in the memory 101, and can be stored in isolation to be called and executed by the processor 1021.

In the software architecture shown in FIG. 2, the software running on the general-purpose processor 1021 includes a general-purpose operating system, a secure operating system, a non-secure artificial intelligence processor driver (hereinafter referred to as a non-secure AI driver), and a secure artificial intelligence processor driver ( (Hereinafter referred to as security AI driver) as an example. Meanwhile, in FIG. 2, the software running on the general-purpose processor 1021 also includes non-safety-side applications and safety-side applications as an example for illustration. Among them, the non-secure application can refer to other non-secure application software except for the secure artificial intelligence processor driver, such as web browsing software or video playback software, etc. The secure side application can refer to the security artificial intelligence processor driver. Other security applications, such as screen unlock software or payment software.

In the embodiment of the present application, the security-side application may request the security operating system to process the security data. After receiving the request of the security-side application, the security operating system may call the artificial intelligence processor 1022 through the security AI driver to process the security data and send the security data to the security. The side application returns the result of safe data processing. The non-secure application can request the general operating system to process the non-secure data. After receiving the request of the non-secure side application, the general operating system can call the artificial intelligence processor 1022 through the non-secure AI driver to process the non-secure data and send it to the non-secure side. The application returns non-secure data processing results. For the specific process of data processing, refer to the description in the following method embodiments.

Further, referring to FIG. 3, the security operating system may optionally include a pattern engine layer, a model steward layer, and a hardware adaptation layer, and a general operating system may selectively include a pattern engine layer, a model steward layer, a framework layer, and a hardware adaptation layer. Layer, the pattern engine layer in a secure operating system can communicate with the pattern engine layer in a general operating system.

Among them, in the security operating system, the pattern engine layer is mainly used to receive messages from the security side application and process mode switching; the model housekeeper layer is mainly used to receive messages from the security side application forwarded by the pattern engine layer, and to create corresponding security side applications. The user data model, and transmits the user data model and user input data to the security AI driver layer by layer; the hardware adaptation layer is mainly used under the secure operating system based on different hardware (such as artificial intelligence processors such as NPU or GPU) ) Perform the corresponding adaptation. In addition, in the general operating system, the pattern engine layer is mainly used to receive messages from non-secure side applications and process mode switching; the model housekeeper layer is mainly used to receive messages from non-secure side applications forwarded by the pattern engine layer and create non-secure side Apply the corresponding user data model and transfer the user data model and user input data to the non-secure AI driver layer by layer; the framework layer is mainly used to handle different artificial intelligence frameworks, such as deep open source learning frameworks; hardware adaptation The layer is mainly used for corresponding adaptation based on different hardware (for example, artificial intelligence processors such as NPU or GPU) under a general operating system. More about the general operating system can be found in the existing technology, such as the introduction of the Android system. The introduction of secure operating systems can be found in existing technologies, such as the Trustzone system.

FIG. 4 is a schematic flowchart of a data processing method according to an embodiment of the present application. The method can be applied to the data processing device shown in FIG. 1, and can be specifically processed by the general-purpose processor 1021 shown in FIG. 2 by calling artificial intelligence. The processor 1022 executes the method. Referring to FIG. 4, the method includes the following steps.

S401: The secure operating system sends first instruction information to the general operating system, and the first instruction information is used to instruct the artificial intelligence processor to enter a secure mode. Among them, when the artificial intelligence processor is in a non-secure mode, the general operating system can send non-secure data to the non-secure AI driver, and the non-secure AI driver can call the artificial intelligence processor to process the non-secure data. In this process, when the security operating system receives the request information sent by the security-side application for processing security data, the security operating system may send first instruction information to the general operating system, and the general operating system is instructed by the first instruction information. The artificial intelligence processor enters safe mode.

S402: When receiving the first instruction information, the general operating system notifies the non-secure AI to drive the artificial intelligence processor to enter the safe mode. When the general operating system receives the first instruction information, the general operating system may notify the non-secure AI to drive the artificial intelligence processor to enter the safe mode. At this time, if the non-secure AI driver is in the state of calling the artificial intelligence processor to process the non-secure data, the non-secure AI driver can execute the following S403 after the current non-secure data processing is completed; The state of processing non-secure data. For example, if the non-secure AI driver is in an idle state, the non-secure AI driver can directly execute the following S403.

Further, before the general-purpose operating system notifies the non-secure AI to drive the artificial intelligence processor to enter the safe mode, if there is at least one non-secure data to be processed, the general-purpose operating system may also set the security data to be processed as described in S406 below. The priority is higher than at least one unsecured data to be processed. At least one unsecured data to be processed is generated by unsecured software and has not yet been processed by an artificial intelligence processor. The at least one to-be-processed non-secure data may include one or more to-be-processed non-secure data, and the at least one to-be-processed non-secure data may be located in a to-be-processed queue of the artificial intelligence processor. The general operating system sets the processing priority of the secure data higher than at least one unsecured data to be processed, which may mean that the position of the secure data in the pending queue is before the at least one unsecured data to be processed .

For example, before the general operating system notifies the non-secure AI-driven artificial intelligence processor to enter the safe mode, the non-secure AI driver is calling the artificial intelligence processor to process the non-secure data B1, and the pending queue further includes the unsecure data to be processed. B2, B3, and B4. The processing order is B2-B3-B4. When the security data is A1, the general operating system can insert A1 before B2, B3, and B4, that is, the processing order after insertion is A1-B2. -B3-B4. After that, the general-purpose processor may notify the non-secure AI-driven artificial intelligence processor to enter the safe mode, and then the non-secure AI-drive may execute the following S403 after the processing of the non-secure data B1 is completed.

S403: The non-safety AI-driven configuration artificial intelligence processor switches from the non-safety mode to the safe mode, that is, the occupation of the artificial intelligence processor is abandoned.

When a non-secure AI driver receives a notification from a general-purpose operating system, or when a non-secure AI driver receives a notification from a general-purpose operating system and the current non-secure data processing is completed, the non-secure AI driver can configure the artificial intelligence processor to The safe mode is switched to the safe mode. Specifically, the non-safety AI driver can disable the function of the non-safety AI driver to receive the non-safety side interrupt. The non-safety side interrupt refers to an interrupt triggered by non-safety software. Among them, disabling the function that the non-safety AI driver receives the non-safety-side interrupt can refer to disabling or shielding the function that the non-safety AI driver receives the non-safety-side interrupt, that is, after the function is disabled, the non-safety AI driver cannot receive Non-safety side interrupts, and therefore non-safety AI drivers can no longer call the artificial intelligence processor to process non-secure data. Before the non-safety AI driver disables this function, the non-safety AI driver can receive the non-safety side interrupts, and Non-secure AI drivers can call artificial intelligence processors to process non-secure data.

S404: The security operating system notifies the security AI to drive the artificial intelligence processor into a security mode. Wherein, when a security-side application running on the general-purpose processor 1021 requests the security operating system to process security data, the security-side application may send request information for requesting to enter the security mode to the security operating system. After requesting information from the security side application, it can notify the security AI to drive the artificial intelligence processor into the security mode.

For example, the security side application is screen unlock software. The user unlocks the mobile phone screen through the screen unlock software. When the screen unlock software detects an unlock operation triggered by the user, the screen unlock software can send a request message to the secure operating system to request to enter the safe mode. Therefore, after receiving the request information, the secure operating system can send a notification message to the secure AI driver to notify the secure AI to drive the artificial intelligence processor to enter the secure mode.

S405: The security AI driver configures the artificial intelligence processor to switch from the non-secure mode to the secure mode, that is, the artificial intelligence processor is occupied. In secure mode, artificial intelligence processors cannot process non-secure data generated by non-secure software. Among them, the artificial intelligence processor can be set to a non-secure mode by default, and in the non-secure mode, the artificial intelligence processor can process non-secure data generated by non-secure software. When the secure AI driver receives the notification from the secure operating system, the secure AI driver can configure the artificial intelligence processor to switch from the non-secure mode to the secure mode, so that when the artificial intelligence processor switches to the secure mode, the artificial intelligence processor cannot handle Non-secure data generated by non-secure software.

Optionally, the configuration of the security AI driver to switch the artificial intelligence processor from the non-secure mode to the security mode may include at least one of the following: the function of enabling the security AI driver to receive a security-side interrupt, the security-side interrupt refers to an interrupt triggered by security software Or, set the memory management unit to a safe mode, in which the non-secure software cannot access the memory corresponding to the memory management unit.

Among them, the function of enabling the safe AI driver to receive the interrupt on the safe side can refer to enabling or turning on the function of the safe AI driver to receive the interrupt on the safe side, that is, after the function is enabled, the safe AI driver can receive the interrupt on the safe side. An interrupt is an interrupt triggered by a security-side application when it requests processing of security data. Before the security AI driver enables this function, the security AI driver cannot receive the security-side interrupt. In addition, the memory management unit can be used to manage the access permissions of its corresponding memory. The security AI driver sets the memory management unit to a safe mode, that is, the security AI driver sets the memory access permission of the memory through the memory management unit as follows: the security application software can access the memory. Memory, which cannot be accessed by non-secure software.

S406: The security operating system sends security data to the security AI driver, and the security data is generated by the security software. After the secure AI driver configures the artificial intelligence processor to switch from the non-secure mode to the secure mode, the secure operating system can send secure data to the secure AI driver. The secure data can refer to the data sent by the secure side application to the secure operating system. Refers to the data after the security operating system preprocesses the data sent by the security side application. When the security side application sends data to the secure operating system, it can send the above data to the secure operating system through one transmission or multiple transmissions.

The security data may include user input data and a user data model. The user input data may refer to the currently input user data. The user data model may be a data model previously obtained through model training based on a large amount of user data, such as human labor. Intelligent training model. The specific information of the security data is related to the security side application that sends the request information. The user data model in the security data may be when the user uses the security side application for the first time or updates the user data model, the security side application collects a large number of users. Data and model training.

For example, the data processing device is a mobile phone, and the security side application is screen unlocking software. The security data may include a user image and a user image model currently collected by the mobile phone. The user image model may be a feature combination including multiple image features. The plurality of image features may include one or more of an eye feature, a mouth feature, and a face feature. Alternatively, the security side application is fingerprint payment software, and the security data may include a user fingerprint and a user fingerprint model currently collected by the mobile phone. The user fingerprint model may be a feature combination including multiple fingerprint features, and the multiple fingerprint features may include fingerprints. One or more of a branch point, a fingerprint start point, and a fingerprint end point.

S407: The safety AI driver receives the safety data and calls an artificial intelligence processor to process the safety data. S408: The secure AI driver sends a secure data processing result to the secure operating system. When the safety AI driver receives the safety data, the safety AI driver calls the artificial intelligence processor to process the safety data, that is, the safety AI driver can send the safety data to the artificial intelligence processor for processing, and the artificial intelligence processor receives the safety data. At this time, the safety data can be processed and the safety data processing result can be obtained, and then the artificial intelligence processor sends the safety data processing result to the safety AI driver, so that the safety AI driver returns the safety data processing result to the safety operating system.

Exemplarily, taking the security data including a user input image and a user image model as an example, a specific process for the artificial intelligence processor to process the security data may be: the artificial intelligence processor extracts multiple image features from the user input image, and extracts Multiple image features of the same type as the multiple image features included in the user image model; the artificial intelligence processor matches two image features of the same type; if the two image features of each type are the same, a match can be determined Success. If one or more types of two image features are inconsistent, it can be determined that the matching has failed. After that, the artificial intelligence processor may send the result of the successful match or the failed match to the secure AI driver, so that the secure AI driver returns the result of the successful match or the failed match to the secure operating system. Further, the secure operating system may send the result to the screen unlocking software, and the screen unlocking software may unlock the mobile phone screen when the result is a successful match, and the screen unlocking software may keep the mobile phone screen in an unlocked state when the result is a matching failure.

Further, referring to FIG. 4 and referring to FIG. 5, after S408, the method further includes: S409-S415. The above S401-S408 are not shown in FIG.

S409: The safe operating system notifies the safe AI-driven artificial intelligence processor to switch from the safe mode to the non-safe mode. After the security data processing is completed, or when no new security data needs to be processed, for example, there is no processing task corresponding to the security data in the pending task queue of the artificial intelligence processor, the security operating system may notify the security AI driver The artificial intelligence processor switches from secure mode to non-secure mode. The new security data may be generated by security software and not yet processed by the artificial intelligence processor.

S410: The secure AI driver configures the artificial intelligence processor in a non-secure mode, that is, it renounces the occupation of the artificial intelligence processor. Among them, the security AI driver configures the artificial intelligence processor to a non-secure mode, which can also be referred to as the security AI driver configures the artificial intelligence processor to switch from the secure mode to the non-secure mode, which specifically includes: disabling the secure AI driver to receive interrupts from the security side. Function, the safety-side interrupt is the interrupt triggered by the safety software. In the safe mode, the safety AI driver can receive the safety-side interrupt and process the safety data based on the safety-side interrupt. When the artificial intelligence processor enters the non-safe mode, the safety AI driver can turn off or shield the safety AI driver from receiving the safety-side interrupt. Function, that is, after disabling the function, the safety AI driver cannot receive the safety-side interrupt, and the safety AI driver cannot call the artificial intelligence processor to process safety data.

S411: The secure operating system sends second instruction information to the general operating system, and the second instruction information is used to instruct the artificial intelligence processor to switch from the safe mode to the non-safe mode. S412: When receiving the second instruction information, the general operating system notifies the non-secure AI to drive the artificial intelligence processor to enter the non-secure mode.

S413: The non-secure AI driver configures the artificial intelligence processor in a non-secure mode, that is, the artificial intelligence processor is occupied. The configuration of a non-secure AI driver with an artificial intelligence processor in a non-secure mode may include at least one of the following: enabling a non-secure AI driver to receive a non-secure side interrupt, the non-secure side interrupt refers to an interrupt triggered by non-secure software; or , Setting the memory management unit to a non-secure mode, in which the non-secure software can access the memory corresponding to the memory management unit.

Among them, enabling a non-safety AI driver to receive non-safety-side interrupts can refer to enabling or enabling a non-safety AI driver to receive non-safety-side interrupts. That is, after enabling this function, non-safety AI drivers can receive non-safety Side interrupts, so non-secure AI drivers can also call artificial intelligence processors to process non-secure data. In addition, the memory management unit can be used to manage the access permissions of its corresponding memory. The non-secure AI driver sets the memory management unit to the non-secure mode, that is, the non-secure AI driver sets the memory access permission of the memory through the memory management unit: non-secure applications. Software can access this memory.

S414: The non-secure AI driver receives non-secure data, and calls an artificial intelligence processor to process the non-secure data. S415: The non-secure AI driver sends the result of non-secure data processing to the general operating system. After the non-secure AI driver configures the artificial intelligence processor in the non-secure mode, the general operating system can send non-secure data to the non-secure AI driver. The non-secure data can refer to the data sent by the non-secure side application to the general operating system. It may refer to data after a general operating system preprocesses data sent by a non-security-side application. The non-security side application can send the above data to the general operating system through one transmission or multiple transmissions. Alternatively, the non-secure data may be unprocessed data that was not processed when the artificial intelligence processor was switched to the secure mode last time, that is, the non-secure data was already in the pending queue of the artificial intelligence processor before S413. The AI driver does not need to perform the above steps for receiving non-secure data, but directly calls an artificial intelligence processor to process the non-secure data.

The process of the non-secure AI driver calling the artificial intelligence processor to process the non-secure data may be: the non-secure AI driver sends the non-secure data to the artificial intelligence processor, and when the artificial intelligence processor receives the non-secure data, it can process the non-secure data. The unsecured data obtains the unsecured data processing result, and then the artificial intelligence processor sends the unsecured data processing result to the unsecured AI driver, so that the unsecured AI driver returns the unsecured data processing result to the operating system. Further, when there is already at least one pending non-secure data in the pending queue of the artificial intelligence processor, the artificial intelligence processor may sequentially process the at least one pending non-secure data in the pending queue. At least one unsecured data to be processed, and the unsecured data processing result is returned to the operating system accordingly.

Further, referring to FIG. 5, the method further includes: S416-S417. Among them, S416-S417 and the above S401-S415 may be in no particular order, and FIG. 5 uses S416-S417 after S415 as an example for description.

S416: When there is no new non-secure data that needs to be processed for a preset time, the non-secure AI driver sets the artificial intelligence processor to a low power state. Among them, no new non-secure data needs to be processed can mean that the current non-secure data has been processed, and the non-secure AI driver has not received other non-secure data, or there is no non-secure data in the pending queue of the artificial intelligence processor Safety data.

In addition, setting the artificial intelligence processor to a low power state by a non-secure AI driver may include performing a power-off operation on the artificial intelligence processor or reducing the power consumption of the artificial intelligence processor. Lowering the power consumption of the artificial intelligence processor may include at least one of the following: reducing the voltage of the artificial intelligence processor, reducing the current of the artificial intelligence processor, and reducing the clock of the artificial intelligence processor.

It should be noted that the preset time can be set by those skilled in the art in advance, and the specific numerical value can be set according to the actual situation, which is not specifically limited in the embodiment of the present application.

S417: When new safety data or new non-safety data needs to be processed, the non-safety AI driver restores the artificial intelligence processor to the working state. When the artificial intelligence processor is in a low power consumption state, if the secure AI driver receives new secure data sent by the secure operating system, or the non-secure AI driver receives new unsecured data sent by the general operating system, the non-secure AI driver resumes manual work. The intelligent processor is in a working state.

Specifically, when a non-secure AI driver sets the artificial intelligence processor to a low-power state by performing a power-off operation, the non-secure AI driver may perform a power-on operation on the artificial intelligence processor to restore it to a working state; Alternatively, when a non-secure AI driver sets the artificial intelligence processor's voltage, current, and / or clock to a low power state, the non-secure AI driver can restore the artificial intelligence processor's voltage, current, and / or clock Is working value to restore it to working state.

In the embodiment of the present application, the non-secure AI driver sets the artificial intelligence processor to a low power consumption state when no new non-secure data needs to be processed for a long time, and when it needs to process new secure data or new non-secure data, Restoring the artificial intelligence processor to the working state can reduce the power consumption of the artificial intelligence processor without affecting the data processing process, thereby reducing the power consumption of the data processing device.

The above mainly introduces the data processing method provided by the embodiment of the present application from the perspective of a data processing device. It can be understood that, in order to implement the above functions, the data processing apparatus includes a hardware structure and / or a software module corresponding to each function. Those skilled in the art should easily realize that the present application can be implemented in the form of a combination of hardware and computer software by combining the structures and algorithm steps of the examples described in the embodiments disclosed herein. Whether a certain function is performed by hardware or computer software-driven hardware depends on the specific application of the technical solution and design constraints. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

The data processing device involved in the above embodiment is a system-level device. Its internal software part or hardware part can be independently used to implement data processing functions. FIG. 6 shows a possible structure diagram of another data processing device involved in this embodiment. The data processing device corresponds to the software part of FIG. 2 and can run on the general-purpose processor 1021. The device includes a secure operating system 301 and a secure AI driver 302. The security operating system 301 is used to support the device to execute S401 and S403 in the method embodiment, receive one or more of the steps of the secure data processing result sent by S405, one or more of S406, S409, and S411, and / or implement the foregoing method. Other technical processes described in the examples; the secure AI driver 302 is used to support the device to perform one or more steps in S402, S404, S405, and S410 in the method embodiment, and / or described in the method embodiment above Other technical processes. Further, the data processing device further includes a general operating system 303 and a non-secure AI driver 304. Among them, the universal operating system 303 is used to support receiving the first instruction information sent by S406, S407, receiving the second instruction information sent by S411, S412, and receiving the non-secure data processing results sent by S415 in the method execution embodiment of the device. One or more steps, and / or other technical processes described in the above method embodiments; the non-secure AI driver 304 is used to support the device to execute S408, S413, S414, S415, S416, and S417 in the method embodiment One or more steps, and / or other technical processes described in the above method embodiments.

In a possible implementation manner, in conjunction with FIG. 1, when the secure operating system 301, secure AI driver 302, general operating system 303, and non-secure AI driver 304 are implemented in software, the software programs corresponding to the above functional modules are included in In the memory 101, the general-purpose processor 1021 implements functions corresponding to the above-mentioned function modules by running software programs including the above-mentioned function modules included in the memory 101.

The following describes another data processing device in the embodiment of the present application from the perspective of hardware processing. The another data processing device provided in the embodiment of the present application corresponds to the hardware part of FIG. 2 to provide hardware computing power for data processing functions, including a general-purpose processor 1021 and an artificial intelligence processor 1022. The specific structure can be as shown in FIG. 1 is shown.

In the embodiment of the present application, the general-purpose processor 1021 is configured to set the artificial intelligence processor 1022 to switch from the non-secure mode to the secure mode, and call the artificial intelligence processor 1022 to process the safety data; wherein, the safety data is run by the general-purpose processor 1021. Generated by the security software; the artificial intelligence processor 1022 is used to switch from the non-secure mode to the secure mode in response to the setting of the general-purpose processor 1021, and to process the secure data in response to the call of the general-purpose processor 1021 and return the secure data processing result to General processor 1021; In a secure mode, the artificial intelligence processor 1022 cannot process non-secure data generated by the general processor 1021 running non-secure software.

In a possible implementation manner, when switching from the non-secure mode to the secure mode, the general-purpose processor 1021 is configured to perform at least one of the following: disable the function of the artificial intelligence processor 1022 to handle non-secure side interrupts, and non-secure Side interrupts refer to interrupts triggered by non-secure software; or, enable the artificial intelligence processor 1022 to handle the functions of security side interrupts, and security side interrupts refer to interrupts triggered by security software; or, set the memory management unit to safe mode, Non-secure software cannot access the memory corresponding to the memory management unit in safe mode.

In a possible implementation manner, before the general-purpose processing sets the artificial intelligence processor 1022 to switch from the non-secure mode to the secure mode, the general-purpose processor 1021 is further configured to set a priority for processing the secure data by the artificial intelligence processor 1022. Higher than non-secure data, non-secure data is generated by the general-purpose processor 1021 running non-secure software and has not yet been processed by the artificial intelligence processor 1022.

In a possible implementation manner, the general-purpose processor 1021 is further configured to set the artificial intelligence processor 1022 to switch from a secure mode to a non-secure mode, and call the artificial intelligence processor 1022 to process non-secure data; the artificial intelligence processor 1022, For switching from the secure mode to the non-secure mode in response to the setting of the general-purpose processor 1021, and for processing the non-secure data in response to the call of the general-purpose processor 1021 and returning the non-secure data processing result to the general-purpose processor 1021. Optionally, the general-purpose processor 1021 is specifically configured to set the artificial intelligence processor 1022 to switch from the secure mode to the non-secure mode when no new secure data needs to be processed. Among them, in a non-secure mode, the artificial intelligence processor 1022 cannot process secure data.

In a possible implementation manner, when switching from the secure mode to the non-secure mode, the general-purpose processor 1021 is configured to perform at least one of the following: disable the function of the artificial intelligence processor 1022 to process the security-side interrupt, and the security-side interrupt Refers to interrupts triggered by secure software; or enables the artificial intelligence processor 1022 to handle non-secure side interrupts. Non-secure side interrupts refer to interrupts triggered by non-secure software; or, sets the memory management unit to non-secure mode In non-secure mode, non-secure software can access the memory corresponding to the memory management unit.

In a possible implementation manner, the general-purpose processor 1021 is further configured to set the artificial intelligence processor 1022 to a low power consumption state when there is no new non-secure data that needs to be processed for a preset time. Further, the general-purpose processor 1021 is further configured to restore the artificial intelligence processor 1022 to a working state when new security data or new non-security data needs to be processed.

In a possible implementation manner, the artificial intelligence processor 1022 is a neural network processing unit NPU or an image processing unit GPU.

In a possible implementation manner, the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general-purpose operating system or non-secure application software. Security software forms TEE, while non-security software forms REE.

Each component of the foregoing data processing device provided in the embodiment of the present application is used to implement functions of the corresponding steps of the foregoing data processing method, because in the foregoing embodiment of the data processing method, each step has been described in detail , Will not repeat them here.

In the embodiment of the present application, when the artificial intelligence processor is set to the safe mode, the artificial intelligence processor cannot process the non-secure data generated by the general-purpose processor running non-secure software, and can process the secure data, thereby ensuring the artificial intelligence. Security when the processor processes secure data.

An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores instructions, and when it is run on a device (for example, the device may be a microcontroller, a chip, a computer, or a processor, etc.) , The device is caused to perform one or more steps in the foregoing method embodiments. When each component module of the data processing device is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in the computer-readable storage medium.

Based on this understanding, the embodiments of the present application further provide a computer program product containing instructions. The technical solution of the present application is essentially a part that contributes to the existing technology or all or part of the technical solution may be a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor therein to execute the embodiments of the present application. All or part of the steps of the method.

The above is only a specific implementation of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application. It should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

A data processing device, characterized in that the device includes a general-purpose processor and an artificial intelligence processor, wherein:

The general-purpose processor is configured to set the artificial intelligence processor to switch from a non-secure mode to a secure mode, and call the artificial intelligence processor to process secure data; wherein the secure data is the general-purpose processor's operating security Generated by software;

The artificial intelligence processor is configured to switch from the non-secure mode to the secure mode in response to a setting of the general-purpose processor, and process the secure data and return the secure data in response to a call of the general-purpose processor. The processing result is given to the general-purpose processor; in the secure mode, the artificial intelligence processor cannot process non-secure data generated by the general-purpose processor running non-secure software.
The data processing device according to claim 1, wherein, when switching from the non-secure mode to the secure mode, the general-purpose processor is configured to execute at least one of the following:

Disable the function of the artificial intelligence processor to process non-safety-side interrupts, where the non-safety-side interrupts refer to interrupts triggered by the non-safety software; or,

Enable the function of the artificial intelligence processor to process a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software; or,

The memory management unit is set to a secure mode, in which the non-secure software cannot access a memory corresponding to the memory management unit.
The data processing device according to claim 1 or 2, wherein before the general-purpose processing sets the artificial intelligence processor to switch from a non-secure mode to a secure mode, the general-purpose processor is further configured to:

Setting the security data to be processed by the artificial intelligence processor with a higher priority than the non-secure data, which is generated by the general-purpose processor running non-secure software and has not been processed by the artificial intelligence器处理。 Processing.
The data processing device according to any one of claims 1 to 3, wherein

The general-purpose processor is further configured to set the artificial intelligence processor to switch from the secure mode to the non-secure mode, and call the artificial intelligence processor to process the non-secure data;

The artificial intelligence processor is configured to switch from the secure mode to the non-secure mode in response to a setting of the general-purpose processor, and process the non-secure data and return non-secure data in response to a call of the general-purpose processor. The security data processing result is given to the general-purpose processor.
The data processing device according to claim 4, wherein in the non-secure mode, the artificial intelligence processor cannot process the secure data.
The data processing device according to claim 4 or 5, wherein, when switching from the secure mode to the non-secure mode, the general-purpose processor is configured to execute at least one of the following:

Disabling the function of the artificial intelligence processor to process a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software; or,

Enable the function of the artificial intelligence processor to process non-safety-side interrupts, where the non-safety-side interrupts refer to interrupts triggered by the non-safety software; or,

The memory management unit is set to a non-secure mode, in which the non-secure software can access a memory corresponding to the memory management unit.
The data processing device according to any one of claims 4 to 6, characterized in that the general-purpose processor is specifically configured to set the artificial intelligence processor from the security when no new security data needs to be processed. The mode is switched to the non-safe mode.
The data processing device according to any one of claims 4 to 7, wherein the general-purpose processor is further configured to:

When the duration that no new non-secure data needs to be processed reaches a preset time, the artificial intelligence processor is set to a low power consumption state.
The data processing device according to claim 8, wherein the general-purpose processor is further configured to:

When new safety data or new non-safety data needs to be processed, the artificial intelligence processor is restored to a working state.
The data processing device according to any one of claims 1-9, wherein the artificial intelligence processor is a neural network processing unit NPU or an image processing unit GPU.
The data processing device according to any one of claims 1 to 10, wherein the security software comprises at least one of a secure operating system or secure application software; and the non-secure software comprises a general-purpose operating system or non-secure application At least one item of software.
A data processing device, characterized in that the device includes: a secure operating system and a secure artificial intelligence processor driver;

The secure operating system is used to notify the secure artificial intelligence processor to drive the artificial intelligence processor into a safe mode;

The secure artificial intelligence processor driver is configured to switch the artificial intelligence processor from a non-secure mode to the secure mode, in which the artificial intelligence processor cannot process generated by non-secure software Non-secure data;

The security operating system is further configured to send security data to the security artificial intelligence processor driver, where the security data is generated by security software;

The secure artificial intelligence processor driver is further configured to receive the secure data, call the artificial intelligence processor to process the secure data, and return a secure data processing result to the secure operating system.
The data processing device according to claim 12, wherein when the artificial intelligence processor is configured to switch from a non-secure mode to the secure mode, the secure artificial intelligence processor driver is configured to execute at least one of the following :

Enable the function of the secure artificial intelligence processor to receive a non-safety-side interrupt, the safety-side interrupt refers to an interrupt triggered by the security software; or,

The memory management unit is set to a secure mode, in which the non-secure software cannot access a memory corresponding to the memory management unit.
The data processing device according to claim 12 or 13, wherein the device further comprises: a general-purpose operating system and a non-secure artificial intelligence processor driver, wherein:

The secure operating system is further configured to notify the secure artificial intelligence processor driver to enter the secure mode, and send first instruction information to the general operating system, where the first instruction information is used to instruct the artificial intelligence. The processor enters the security mode;

The universal operating system is configured to, when receiving the first instruction information, notify the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the secure mode;

The non-secure artificial intelligence processor driver is configured to switch the artificial intelligence processor from a non-secure mode to the secure mode.
The data processing device according to claim 14, wherein when the artificial intelligence processor is configured to switch from a non-secure mode to the secure mode, the non-secure artificial intelligence processor driver is configured to perform the following operations:

Disable the function of the non-secure artificial intelligence processor driver to receive non-secure side interrupts, where the non-secure side interrupts refer to interrupts triggered by the non-secure software.
The data processing device according to claim 14 or 15, wherein before the non-secure artificial intelligence processor is notified to drive the artificial intelligence processor to enter the secure mode, the universal operating system is further configured to: :

Setting the security data to be processed has a higher priority than the non-secure data, which is generated by the non-secure software and has not yet been processed by the artificial intelligence processor.
The data processing device according to any one of claims 13 to 16, wherein:

The secure operating system is further configured to notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode;

The secure artificial intelligence processor driver is further configured to configure the artificial intelligence processor to the non-secure mode;

The secure operating system is further configured to send second instruction information to the general operating system, where the second instruction information is used to instruct the artificial intelligence processor to switch from the secure mode to the non-secure mode;

The universal operating system is further configured to notify the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the non-secure mode when the second instruction information is received;

The non-secure artificial intelligence processor driver is further configured to configure the artificial intelligence processor to the non-secure mode, and receive non-secure data, call the artificial intelligence processor to process the non-secure data, and return non-secure data. The result of the secure data processing is given to the general operating system.
The data processing device according to claim 17, wherein, when the artificial intelligence processor is configured to the non-secure mode,

The secure artificial intelligence processor driver is configured to perform the following operations:

Disabling the function of the security artificial intelligence processor driving receiving a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software;

And, the non-secure artificial intelligence processor driver is configured to execute at least one of the following:

Enable the function of the non-safety artificial intelligence processor to receive the non-safety side interrupt, the non-safety side interrupt refers to an interrupt triggered by the non-safety software; or,

The memory management unit is set to a non-secure mode, in which the non-secure software can access a memory corresponding to the memory management unit.
The data processing device according to claim 17 or 18, wherein the secure operating system is specifically configured to notify the secure artificial intelligence processor to drive the artificial intelligence processing when there is no new secure data to be processed The switch from the safe mode to the non-safe mode.
The data processing device according to any one of claims 17 to 19, wherein the non-secure artificial intelligence processor driver is further configured to:

When the duration that no new non-secure data needs to be processed reaches a preset time, the artificial intelligence processor is set to a low power consumption state.
The data processing device according to claim 20, wherein the non-secure artificial intelligence processor driver is further configured to:

When new safety data or new non-safety data needs to be processed, the artificial intelligence processor is restored to a working state.
The data processing device according to any one of claims 12 to 21, wherein the security software comprises at least one of a secure operating system or secure application software; and the non-secure software comprises a general-purpose operating system or non-secure application At least one item of software.
A data processing method, characterized in that the method includes:

The secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor into a safe mode;

The secure artificial intelligence processor driver configures the artificial intelligence processor to switch from a non-secure mode to the secure mode. In the secure mode, the artificial intelligence processor cannot handle non-secure software generated by non-secure software. data;

The security operating system sends security data to the security artificial intelligence processor driver, the security data is generated by security software;

The secure artificial intelligence processor driver receives the secure data, calls the artificial intelligence processor to process the secure data, and returns a secure data processing result to the secure operating system.
The data processing method according to claim 23, wherein the driving of the secure artificial intelligence processor to configure the artificial intelligence processor to switch from a non-secure mode to the secure mode comprises at least one of the following:

Enable the function of the security artificial intelligence processor to receive a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software; or,

The memory management unit is set to a secure mode, in which the non-secure software cannot access a memory corresponding to the memory management unit.
The data processing method according to claim 23 or 24, wherein before the secure operating system notifies a secure artificial intelligence processor driver to enter the secure mode, the method further comprises:

The secure operating system sends first instruction information to a general operating system, where the first instruction information is used to instruct the artificial intelligence processor to enter the secure mode;

When the universal operating system receives the first instruction information, notify the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the secure mode;

The non-secure artificial intelligence processor driver configures the artificial intelligence processor to switch from a non-secure mode to the secure mode.
The data processing method according to claim 25, wherein the driving of the non-secure artificial intelligence processor to configure the artificial intelligence processor to switch from the non-secure mode to the secure mode comprises:

Disable the function of the non-secure artificial intelligence processor driver to receive non-secure side interrupts, where the non-secure side interrupts refer to interrupts triggered by the non-secure software.
The data processing method according to claim 25 or 26, wherein before the general operating system notifies a non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the secure mode, the method further comprises:

The general processing system sets the priority of processing the secure data higher than the non-secure data, which is generated by the non-secure software and has not yet been processed by the artificial intelligence processor.
The data processing method according to any one of claims 24-27, wherein the method further comprises:

The secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode;

The secure artificial intelligence processor driver configures the artificial intelligence processor as the non-secure mode;

Sending, by the secure operating system, the second instruction information to the general operating system, where the second instruction information is used to instruct the artificial intelligence processor to switch from the secure mode to the non-secure mode;

When the universal operating system receives the second instruction information, notify the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the non-secure mode;

The non-secure artificial intelligence processor driver configures the artificial intelligence processor as the non-secure mode, and receives non-secure data, calls the artificial intelligence processor to process the non-secure data, and returns an unsecure data processing result. To the universal operating system.
The data processing method according to claim 28, wherein the configuration of the secure artificial intelligence processor to drive the artificial intelligence processor into the non-secure mode comprises:

Disabling the function of the security artificial intelligence processor driving receiving a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software;

And, the non-secure artificial intelligence processor driver configuring the artificial intelligence processor to be in the non-secure mode includes at least one of the following:

Enable the function of the non-safety artificial intelligence processor to receive the non-safety side interrupt, the non-safety side interrupt refers to an interrupt triggered by the non-safety software; or,

The memory management unit is set to a non-secure mode, in which the non-secure software can access a memory corresponding to the memory management unit.
The data processing method according to claim 28 or 29, wherein the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode, include:

When there is no new secure data to be processed, the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode.
The data processing method according to any one of claims 28 to 30, wherein the method further comprises:

When there is no new non-secure data that needs to be processed for a preset time, the non-secure artificial intelligence processor driver sets the artificial intelligence processor to a low power consumption state.
The data processing method according to claim 31, wherein the method further comprises:

When new secure data or new non-secure data needs to be processed, the non-secure artificial intelligence processor drives and restores the artificial intelligence processor to a working state.
The data processing method according to any one of claims 23 to 32, wherein the security software comprises at least one of a secure operating system or secure application software; and the non-secure software comprises a general operating system or non-secure application At least one item of software.