WO2020034076A1 - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- WO2020034076A1 WO2020034076A1 PCT/CN2018/100305 CN2018100305W WO2020034076A1 WO 2020034076 A1 WO2020034076 A1 WO 2020034076A1 CN 2018100305 W CN2018100305 W CN 2018100305W WO 2020034076 A1 WO2020034076 A1 WO 2020034076A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- secure
- artificial intelligence
- intelligence processor
- data
- processor
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
Definitions
- the present application relates to the field of secure computing, and in particular, to a data processing method and device.
- AI artificial intelligence
- AI technology can be applied to the security features of electronic devices.
- three-dimensional (3D) face unlocking using AI can be used for mobile phone screen unlocking, mobile phone payment confirmation, and identity authentication.
- a neural-network processing unit (NPU) (also referred to as an NPU processor) can be integrated into an electronic device, and a mobile phone screen unlock, mobile payment confirmation, and identity can be realized through a chip with a built-in NPU processor. Relevant AI data calculation during authentication and other processes.
- the embodiments of the present application provide a data processing method and device, which are used to ensure the security of related data operations in AI technology.
- a data processing device includes a general-purpose processor and an artificial intelligence processor, wherein the general-purpose processor is used to set the artificial intelligence processor to switch from a non-secure mode to a secure mode, and call the artificial intelligence.
- the processor processes security data generated by the general-purpose processor running security software; an artificial intelligence processor for switching from a non-secure mode to a secure mode in response to a setting of the general-purpose processor, and in response to the general-purpose processor
- the call processes the secure data and returns the secure data processing result to the general-purpose processor; in the secure mode, the artificial intelligence processor cannot process the non-secure data generated by the general-purpose processor running non-secure software.
- the artificial intelligence processor when the artificial intelligence processor is set to the secure mode, it cannot process the non-secure data generated by the general-purpose processor running non-secure software, thereby ensuring the security of the artificial intelligence processor when processing the secure data, that is, the guarantee Security of related data operations in AI technology.
- the general-purpose processor when switching from the non-secure mode to the secure mode, is configured to perform at least one of the following: disabling an artificial intelligence processor from processing a non-safety-side interruption function.
- Security-side interrupts refer to interrupts triggered by non-safety software; or, the function of the artificial intelligence processor to handle security-side interrupts is enabled, and security-side interrupts refer to interrupts triggered by security software; or, setting the memory management unit to safe mode, Non-secure software cannot access the memory corresponding to the memory management unit in safe mode.
- the artificial intelligence processor cannot process non-secure data, can process secure data, and non-secure software cannot access memory in a secure mode, thereby ensuring the security of the secure data and its processing process.
- the general-purpose processor before the general-purpose processing sets the artificial intelligence processor to switch from the non-secure mode to the secure mode, the general-purpose processor is further configured to set the priority of the secure data to be processed by the artificial intelligence processor. Level is higher than non-secure data, which is generated by general-purpose processors running non-secure software and has not yet been processed by artificial intelligence processors.
- the artificial intelligence processor can be made to process the security data in priority, thereby improving the processing speed of the security data, and further improving the user experience.
- the general-purpose processor is further configured to set the artificial intelligence processor to switch from the secure mode to the non-secure mode, and call the artificial intelligence processor to process non-secure data; the artificial intelligence processor, It is used to switch from the secure mode to the non-secure mode in response to the setting of the general-purpose processor, and to process the non-secure data in response to the call of the general-purpose processor and return the non-secure data processing result to the general-purpose processor.
- the artificial intelligence processor when the artificial intelligence processor is set to the non-secure mode, it can process non-secure data, so that the artificial intelligence processor can be compatible with the processing of the secure data and the non-secure data.
- the artificial intelligence processor in a non-secure mode, cannot process secure data.
- the security of the secure data and its processing process can be guaranteed in the non-secure mode.
- the general-purpose processor when switching from the secure mode to the non-secure mode, is configured to perform at least one of the following: disabling the function of the artificial intelligence processor to handle interrupts on the security side, and the security side Interrupts refer to interrupts triggered by secure software; or, enable the artificial intelligence processor to handle non-secure side interrupts, non-secure side interrupts refer to interrupts triggered by non-secure software; or set the memory management unit to non-secure mode In non-secure mode, non-secure software can access the memory corresponding to the memory management unit.
- the artificial intelligence processor cannot process secure data, can process non-secure data, and non-secure software can access memory in the non-secure mode, thereby enabling the artificial intelligence processor to process secure data and non-secure data.
- the general-purpose processor is specifically configured to set the artificial intelligence processor to switch from the secure mode to the non-secure mode when no new secure data needs to be processed.
- the problem that the artificial intelligence processor is in a safe mode for a long time and cannot process non-secure data can be avoided, thereby ensuring the efficiency of the artificial intelligence processor in processing non-secure data.
- the general-purpose processor is further configured to set the artificial intelligence processor to a low power consumption state when a duration in which no new non-secure data needs to be processed reaches a preset time.
- setting the artificial intelligence processor to a low power consumption state may include: powering off the artificial intelligence processor, or reducing the power consumption of the artificial intelligence processor, for example, reducing the voltage and current of the artificial intelligence processor Or at least one of the clocks.
- the power consumption of the artificial intelligence processor can be reduced without affecting the data processing process, thereby reducing the power consumption of the data processing device.
- the general-purpose processor is further configured to restore the artificial intelligence processor to a working state when it is required to process new secure data or new non-secure data.
- restoring the artificial intelligence processor to the working state may include: powering on the artificial intelligence processor, or setting the power consumption of the artificial intelligence processor to the working state, for example, changing the artificial intelligence processor ’s The voltage, current, and / or clock are restored to operating values.
- the power consumption of the artificial intelligence processor can be reduced without affecting the data processing process, thereby reducing the power consumption of the data processing device.
- the artificial intelligence processor is a neural network processing unit NPU or an image processing unit (GPU).
- NPU neural network processing unit
- GPU image processing unit
- the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general operating system or non-secure application software.
- the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general operating system or non-secure application software.
- a data processing device in a second aspect, includes: a secure operating system and a secure artificial intelligence processor driver; wherein the secure operating system is used to notify the secure artificial intelligence processor to drive the artificial intelligence processor into a secure mode;
- the artificial intelligence processor driver is used to configure the artificial intelligence processor to switch from the non-secure mode to the secure mode.
- the secure mode the artificial intelligence processor cannot process the non-secure data generated by the non-secure software;
- the secure operating system also uses For sending safety data to the safety artificial intelligence processor driver, the safety data is generated by the safety software; the safety artificial intelligence processor driver is also used to receive the safety data, call the artificial intelligence processor to process the safety data, and return the safety data
- the data processing results are given to the secure operating system.
- the secure artificial intelligence processor driver when the artificial intelligence processor is configured to switch from the non-secure mode to the secure mode, the secure artificial intelligence processor driver is configured to perform at least one of the following: enable the secure artificial intelligence processor The driver receives the non-safety-side interruption function.
- the safety-side interruption refers to the interruption triggered by the safety software; or, the memory management unit is set to the safe mode. In the safe mode, the non-safety software cannot access the memory corresponding to the memory management unit.
- the device further includes: a general operating system and a non-secure artificial intelligence processor driver; wherein the secure operating system is further configured to notify the secure artificial intelligence processor driver before entering the safe mode Sending the first instruction information to the general operating system, where the first instruction information is used to instruct the artificial intelligence processor to enter the safe mode; the general operating system is used to notify the non-secure artificial intelligence processor to drive the human body when the first instruction information is received The intelligent processor enters the secure mode; the non-secure artificial intelligence processor driver is used to configure the artificial intelligence processor to switch from the non-secure mode to the secure mode.
- the non-secure artificial intelligence processor driver when the artificial intelligence processor is configured to switch from the non-secure mode to the secure mode, the non-secure artificial intelligence processor driver is configured to perform the following operations: disabling the non-secure artificial intelligence processing
- the device driver receives the non-safety side interrupt function.
- the non-safety side interrupt refers to the interrupt triggered by the non-safety software.
- the general operating system before notifying the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the safe mode, is further configured to: set the priority of processing the secure data higher than that of the non-secure artificial intelligence processor.
- Security data, non-security data is generated by non-security software and has not been processed by the artificial intelligence processor.
- the secure operating system is further configured to notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode; the secure artificial intelligence processor driver is also used to Configure the artificial intelligence processor in non-secure mode; the secure operating system is also used to send second instruction information to the general operating system, the second instruction information is used to instruct the switch from the secure mode to the non-secure mode; the general operating system is also used to When receiving the second instruction information, notify the non-secure artificial intelligence processor to drive the artificial intelligence processor into the non-secure mode; the non-secure artificial intelligence processor driver is also used to configure the artificial intelligence processor to be in the non-secure mode and receive non-secure artificial intelligence processors. Safe data, call the artificial intelligence processor to process the unsafe data, and return the unsafe data processing result to the general operating system.
- the secure artificial intelligence processor driver when the artificial intelligence processor is configured as the non-secure mode, the secure artificial intelligence processor driver is configured to perform the following operations: disabling the secure artificial intelligence processor driver to receive security Side interrupt function, the safe side interrupt refers to the interrupt triggered by the security software; and, the non-secure artificial intelligence processor driver is used to perform at least one of the following: enable the non-secure artificial intelligence processor driver to receive the non-secure side interrupt function
- the non-secure side interrupt refers to an interrupt triggered by the non-secure software; or, the memory management unit is set to an unsecure mode, and the non-secure software can access the memory corresponding to the memory management unit in the non-secure mode.
- the secure operating system is specifically configured to notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode when no new secure data needs to be processed.
- the non-secure artificial intelligence processor driver is further configured to: set the artificial intelligence processor to low when there is no new non-secure data that needs to be processed for a preset time. Power status.
- the non-secure artificial intelligence processor driver is further configured to: when new security data or new non-secure data needs to be processed, restore the artificial intelligence processor to a working state.
- the secure software includes at least one of a secure operating system or secure application software;
- the non-secure software includes at least one of a general operating system or non-secure application software.
- a data processing method includes: the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor into a safe mode; the secure artificial intelligence processor driver configures the artificial intelligence processor to switch from a non-secure mode to Safe mode, in which the artificial intelligence processor cannot process non-secure data generated by non-secure software; the secure operating system sends secure data to the secure artificial intelligence processor driver, and the secure data is generated by the secure software; secure labor
- the intelligent processor driver receives the safety data, calls the artificial intelligence processor to process the safety data, and returns the safety data processing result to the safety operating system.
- configuring the secure artificial intelligence processor driver to switch the artificial intelligence processor from the non-secure mode to the secure mode includes at least one of the following: enabling the secure artificial intelligence processor driver to receive non-secure The function of the side interrupt, the safe side interrupt refers to the interrupt triggered by the security software; or, the memory management unit is set to the safe mode, and the non-safe software cannot access the memory corresponding to the memory management unit in the safe mode.
- the method before the secure operating system notifies the secure artificial intelligence processor driver to enter the secure mode, the method further includes: the secure operating system sends the first instruction information to the general operating system, and the first instruction information Used to instruct the artificial intelligence processor to enter the safe mode; when the general operating system receives the first instruction message, it informs the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the safe mode; the non-secure artificial intelligence processor drives to configure the artificial intelligence processing The device switches from non-safe mode to safe mode.
- configuring the non-secure artificial intelligence processor driver to switch the artificial intelligence processor from the non-secure mode to the secure mode includes: disabling the non-secure artificial intelligence processor driver to receive the non-secure side Interrupt function, non-safety side interrupt refers to the interrupt triggered by non-safety software.
- the method before the general operating system notifies the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the safe mode, the method further includes: the general processing system sets the priority of the secure data to be processed to be high For non-secure data, non-secure data is generated by non-secure software and has not yet been processed by an artificial intelligence processor.
- the method further includes: the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode; the secure artificial intelligence processor drives to configure the artificial intelligence
- the processor is in a non-secure mode; the secure operating system sends second instruction information to the general operating system, and the second instruction information is used to instruct switching from the secure mode to the non-secure mode; when the general operating system receives the second instruction information, it informs the non-
- the secure artificial intelligence processor drives the artificial intelligence processor into the non-secure mode; the non-secure artificial intelligence processor drives configure the artificial intelligence processor to the non-secure mode, and receives non-secure data, calls the artificial intelligence processor to process the non-secure data, and returns The results of non-secure data processing are given to the general operating system.
- the safe artificial intelligence processor driver configures the artificial intelligence processor in a non-secure mode, including: disabling the function of the safe artificial intelligence processor driver to receive the safety-side interrupt, and the safety-side interrupt Refers to interrupts triggered by secure software; and, the configuration of an artificial intelligence processor in non-secure mode driven by a non-secure artificial intelligence processor, including at least one of the following: enabling the function of the non-secure artificial intelligence processor driver to receive non-secure side interrupts
- the non-secure side interrupt refers to the interrupt triggered by the non-secure software; or, the memory management unit is set to the non-secure mode, and the non-secure software can access the memory corresponding to the memory management unit in the non-secure mode.
- the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode, including: when the secure operating system has no new secure data to be processed To notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the safe mode to the non-safe mode.
- the method further includes: when there is no new non-secure data that needs to be processed for a preset time, the non-secure artificial intelligence processor driver sets the artificial intelligence processor to low Power status.
- the method further includes: when new security data or new non-secure data needs to be processed, the non-secure artificial intelligence processor drives and restores the artificial intelligence processor to a working state.
- the secure software includes at least one of a secure operating system or secure application software;
- the non-secure software includes at least one of a general operating system or non-secure application software.
- a readable storage medium stores instructions, and when the readable storage medium is run on a device, the device is caused to perform the third aspect or any of the third aspects.
- Data processing methods provided by one possible implementation.
- a computer program product is provided, and when the computer program product runs on a computer or a processor, the computer or processor executes the first aspect or any possible implementation manner of the first aspect.
- any of the data processing devices, methods, computer storage media, or computer program products provided above is used to perform the functions of the corresponding devices provided above. Therefore, for the beneficial effects that can be achieved, refer to the above. The beneficial effects in the provided corresponding device are not repeated here.
- FIG. 1 is a schematic structural diagram of a system-level data processing device according to an embodiment of the present application
- FIG. 2 is an architecture diagram of software running on a general-purpose processor according to an embodiment of the present application
- FIG. 3 is an architecture diagram of a universal operating system / secure operating system provided by an embodiment of the present application.
- FIG. 4 is a first schematic flowchart of a data processing method according to an embodiment of the present application.
- FIG. 5 is a second schematic flowchart of a data processing method according to an embodiment of the present application.
- FIG. 6 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present application.
- "at least one” means one or more, and “multiple” means two or more.
- "And / or” describes the association relationship of related objects, and indicates that there can be three kinds of relationships, for example, A and / or B can represent: the case where A exists alone, A and B exist simultaneously, and B alone exists, where A, B can be singular or plural.
- the character "/" generally indicates that the related objects are an "or" relationship.
- FIG. 1 is a schematic structural diagram of a system-level data processing device according to an embodiment of the present application.
- the data processing device may be a mobile phone, a tablet computer, a computer, a wearable device, a vehicle-mounted device, or a portable device.
- the above-mentioned devices may be collectively referred to as a data processing apparatus.
- the data processing apparatus may include a memory 101, a processor 102, a communication interface 103, and a bus 104.
- the memory 101, the processor 102, and the communication interface 103 are connected to each other through a bus 104.
- the memory 101 can be used to store data, software programs, and modules, and mainly includes a program storage area and a data storage area.
- the storage program area can store an operating system and at least one application required by a function.
- the storage data area can store the data processing. Data created when the device is used.
- the operating system may include a general-purpose operating system (such as an Android system) and a secure operating system (such as a dedicated operating system such as a Trustzone system); the applications required for the at least one function may include Security application software and non-security application software.
- Trusted application (TA) software can include secure artificial intelligence processor drivers and other applications with security features (for example, screen unlocking software based on the Trustzone system, payment software, etc.).
- Security application software is also called client application (CA), which can include non-secure artificial intelligence processor drivers and other applications with non-secure features (for example, Android-based web browsing software, video playback software, etc.);
- the data stored in the storage data area may include secure data, secure data processing results, non-secure data and / or non-secure data processing results, and the like.
- the software module stored in the memory 101 may include a memory management unit, and the storage data area may include memory.
- the memory management unit may be used to manage the memory. For example, the memory management unit may be used to set access to the memory. Permissions.
- the memory may be a floppy disk, a hard disk such as a built-in hard disk and a mobile hard disk, a magnetic disk, an optical disk, and a magneto optical disk such as a read-only optical disk (compact disc-read- only memory (CD_ROM), digital video disc (read-only memory (DVD_ROM), non-volatile storage devices such as random access memory (RAM), read-only memory (ROM) , Programmable read-only memory (PROM), erasable programmable read-only memory (erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (electrically erasable programmable read-only memory) only memory (EEPROM), flash memory, or any other form of storage medium known in the art.
- CD_ROM compact disc-read- only memory
- DVD_ROM digital video disc
- non-volatile storage devices such as random access memory (RAM), read-only memory (ROM) , Programmable read-only memory (PROM), erasable
- the processor 102 is configured to control and manage the actions of the data processing device, for example, by running or executing software programs and / or modules stored in the memory 101 and calling data stored in the memory 101 to execute the data processing device.
- the processor 102 may include at least one of a central processing unit, a digital signal processor, an artificial intelligence (AI) processor, a microprocessor, or a microcontroller, and may optionally include an application specific integrated circuit, a field Programmable gate array or other programmable logic device, transistor logic device, hardware accelerator, or any combination thereof.
- the processor may also be a combination that implements computing functions.
- the processor 102 may include a combination of a general-purpose processor 1021 and an artificial intelligence processor 1022, and may further optionally include other hardware accelerators, and the combination may be used to support the device to perform data processing functions in the present application.
- the artificial intelligence processor 1022 may be a neural-network processing unit (NPU), or an image processing unit (graphics processing unit, GPU).
- the general-purpose processor 1021 may include a processing device running software, such as a central processing unit, a microprocessor, or a microcontroller.
- the software may include an operating system, application software, or driver software.
- the data processing device in the above embodiments is a system.
- the processor 102 itself can also be regarded as a data processing device, which functions as a computing core in the system to realize data processing capabilities.
- the communication interface 103 is used to support the data processing apparatus to communicate with other equipment.
- the bus 104 may be a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, or the like.
- PCI peripheral component interconnect
- EISA extended industry standard architecture
- the bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used in FIG. 1, but it does not mean that there is only one bus or one type of bus.
- FIG. 2 is a schematic structural diagram of software running on a general-purpose processor 1021 according to an embodiment of the present application.
- the general-purpose processor 1021 can be used to run security software and non-secure software.
- the data generated by the general-purpose processor 1021 running the security software is called secure data, and the data generated by running the non-secure software is called non-secure data.
- the general-purpose processor 1021 may include at least one processing unit, such as an ARM, MIPS, or X86 architecture device.
- Security software has higher security requirements than non-secure software, or the environment formed by secure software is more secure than non-secure software. For example, the isolation between secure and non-secure software prevents non-secure software from randomly accessing the environment created by the secure software.
- the security software includes at least one of a security operating system (OS) or security application software; a security operating system may refer to an operating system capable of meeting certain security technical requirements (for example, access control, identification, etc.), such as
- the secure operating system may be an operating system established to ensure the security of related data operations in AI technology, such as a Trustzone system; the security application software may refer to application software that needs to ensure security, and the security application software may Including one or more security applications or security drivers, for example, the security driver may include a security artificial intelligence processor driver (referred to as a security AI driver for driving the artificial intelligence processor 1022 under a secure operating system), and the security application may Including screen unlock software and payment software, it can run on the Trustzone system.
- a security operating system may refer to an operating system capable of meeting certain security technical requirements (for example, access control, identification, etc.), such as
- the secure operating system may be an operating system established to ensure the security of related data operations in AI technology, such as a Trustzone system
- the security application software may refer to application
- Non-secure software includes at least one of a general-purpose operating system and non-secure application software; a general-purpose operating system may refer to an operating system without specific security requirements, for example, a general-purpose operating system may include an Android operating system or a Windows operating system; Non-secure application software can refer to application software that does not require security requirements.
- Non-secure application software can include one or more non-secure applications or non-secure drivers.
- non-secure drivers can include non-secure artificial intelligence processors.
- Drivers referred to as non-secure AI drivers, used to drive the artificial intelligence processor 1022 under a general operating system
- non-secure applications may include web browsing software, video playback software, and the like. Therefore, there is isolation between safety software and non-safety software.
- the running instructions of both parties can be isolated from each other without affecting the operation of the other party.
- the security software is more secure than non-secure software.
- the environment formed by the security software may be called a Trusted Execution Environment (TEE), and the environment formed by the non-secure software may be called a Rich Execution Environment (REE).
- the environment formed by the security software is more secure than the environment formed by non-secure software.
- the above software can be stored in the memory 101, and can be stored in isolation to be called and executed by the processor 1021.
- the software running on the general-purpose processor 1021 includes a general-purpose operating system, a secure operating system, a non-secure artificial intelligence processor driver (hereinafter referred to as a non-secure AI driver), and a secure artificial intelligence processor driver ( (Hereinafter referred to as security AI driver) as an example.
- the software running on the general-purpose processor 1021 also includes non-safety-side applications and safety-side applications as an example for illustration.
- the non-secure application can refer to other non-secure application software except for the secure artificial intelligence processor driver, such as web browsing software or video playback software, etc.
- the secure side application can refer to the security artificial intelligence processor driver.
- Other security applications such as screen unlock software or payment software.
- the security-side application may request the security operating system to process the security data.
- the security operating system may call the artificial intelligence processor 1022 through the security AI driver to process the security data and send the security data to the security.
- the side application returns the result of safe data processing.
- the non-secure application can request the general operating system to process the non-secure data.
- the general operating system can call the artificial intelligence processor 1022 through the non-secure AI driver to process the non-secure data and send it to the non-secure side.
- the application returns non-secure data processing results.
- the security operating system may optionally include a pattern engine layer, a model steward layer, and a hardware adaptation layer
- a general operating system may selectively include a pattern engine layer, a model steward layer, a framework layer, and a hardware adaptation layer.
- Layer, the pattern engine layer in a secure operating system can communicate with the pattern engine layer in a general operating system.
- the pattern engine layer is mainly used to receive messages from the security side application and process mode switching;
- the model housekeeper layer is mainly used to receive messages from the security side application forwarded by the pattern engine layer, and to create corresponding security side applications.
- the user data model and transmits the user data model and user input data to the security AI driver layer by layer;
- the hardware adaptation layer is mainly used under the secure operating system based on different hardware (such as artificial intelligence processors such as NPU or GPU) ) Perform the corresponding adaptation.
- the pattern engine layer is mainly used to receive messages from non-secure side applications and process mode switching;
- the model housekeeper layer is mainly used to receive messages from non-secure side applications forwarded by the pattern engine layer and create non-secure side Apply the corresponding user data model and transfer the user data model and user input data to the non-secure AI driver layer by layer;
- the framework layer is mainly used to handle different artificial intelligence frameworks, such as deep open source learning frameworks; hardware adaptation
- the layer is mainly used for corresponding adaptation based on different hardware (for example, artificial intelligence processors such as NPU or GPU) under a general operating system. More about the general operating system can be found in the existing technology, such as the introduction of the Android system. The introduction of secure operating systems can be found in existing technologies, such as the Trustzone system.
- FIG. 4 is a schematic flowchart of a data processing method according to an embodiment of the present application.
- the method can be applied to the data processing device shown in FIG. 1, and can be specifically processed by the general-purpose processor 1021 shown in FIG. 2 by calling artificial intelligence.
- the processor 1022 executes the method. Referring to FIG. 4, the method includes the following steps.
- the secure operating system sends first instruction information to the general operating system, and the first instruction information is used to instruct the artificial intelligence processor to enter a secure mode.
- the general operating system can send non-secure data to the non-secure AI driver, and the non-secure AI driver can call the artificial intelligence processor to process the non-secure data.
- the security operating system may send first instruction information to the general operating system, and the general operating system is instructed by the first instruction information.
- the artificial intelligence processor enters safe mode.
- the general operating system When receiving the first instruction information, the general operating system notifies the non-secure AI to drive the artificial intelligence processor to enter the safe mode. When the general operating system receives the first instruction information, the general operating system may notify the non-secure AI to drive the artificial intelligence processor to enter the safe mode. At this time, if the non-secure AI driver is in the state of calling the artificial intelligence processor to process the non-secure data, the non-secure AI driver can execute the following S403 after the current non-secure data processing is completed; The state of processing non-secure data. For example, if the non-secure AI driver is in an idle state, the non-secure AI driver can directly execute the following S403.
- the general-purpose operating system may also set the security data to be processed as described in S406 below.
- the priority is higher than at least one unsecured data to be processed.
- At least one unsecured data to be processed is generated by unsecured software and has not yet been processed by an artificial intelligence processor.
- the at least one to-be-processed non-secure data may include one or more to-be-processed non-secure data, and the at least one to-be-processed non-secure data may be located in a to-be-processed queue of the artificial intelligence processor.
- the general operating system sets the processing priority of the secure data higher than at least one unsecured data to be processed, which may mean that the position of the secure data in the pending queue is before the at least one unsecured data to be processed .
- the non-secure AI driver is calling the artificial intelligence processor to process the non-secure data B1, and the pending queue further includes the unsecure data to be processed.
- the processing order is B2-B3-B4.
- the general operating system can insert A1 before B2, B3, and B4, that is, the processing order after insertion is A1-B2. -B3-B4.
- the general-purpose processor may notify the non-secure AI-driven artificial intelligence processor to enter the safe mode, and then the non-secure AI-drive may execute the following S403 after the processing of the non-secure data B1 is completed.
- S403 The non-safety AI-driven configuration artificial intelligence processor switches from the non-safety mode to the safe mode, that is, the occupation of the artificial intelligence processor is abandoned.
- the non-secure AI driver can configure the artificial intelligence processor to The safe mode is switched to the safe mode.
- the non-safety AI driver can disable the function of the non-safety AI driver to receive the non-safety side interrupt.
- the non-safety side interrupt refers to an interrupt triggered by non-safety software.
- disabling the function that the non-safety AI driver receives the non-safety-side interrupt can refer to disabling or shielding the function that the non-safety AI driver receives the non-safety-side interrupt, that is, after the function is disabled, the non-safety AI driver cannot receive Non-safety side interrupts, and therefore non-safety AI drivers can no longer call the artificial intelligence processor to process non-secure data.
- the non-safety AI driver can receive the non-safety side interrupts, and Non-secure AI drivers can call artificial intelligence processors to process non-secure data.
- S404 The security operating system notifies the security AI to drive the artificial intelligence processor into a security mode.
- a security-side application running on the general-purpose processor 1021 requests the security operating system to process security data
- the security-side application may send request information for requesting to enter the security mode to the security operating system. After requesting information from the security side application, it can notify the security AI to drive the artificial intelligence processor into the security mode.
- the security side application is screen unlock software.
- the user unlocks the mobile phone screen through the screen unlock software.
- the screen unlock software detects an unlock operation triggered by the user, the screen unlock software can send a request message to the secure operating system to request to enter the safe mode. Therefore, after receiving the request information, the secure operating system can send a notification message to the secure AI driver to notify the secure AI to drive the artificial intelligence processor to enter the secure mode.
- the security AI driver configures the artificial intelligence processor to switch from the non-secure mode to the secure mode, that is, the artificial intelligence processor is occupied.
- secure mode artificial intelligence processors cannot process non-secure data generated by non-secure software.
- the artificial intelligence processor can be set to a non-secure mode by default, and in the non-secure mode, the artificial intelligence processor can process non-secure data generated by non-secure software.
- the secure AI driver receives the notification from the secure operating system, the secure AI driver can configure the artificial intelligence processor to switch from the non-secure mode to the secure mode, so that when the artificial intelligence processor switches to the secure mode, the artificial intelligence processor cannot handle Non-secure data generated by non-secure software.
- the configuration of the security AI driver to switch the artificial intelligence processor from the non-secure mode to the security mode may include at least one of the following: the function of enabling the security AI driver to receive a security-side interrupt, the security-side interrupt refers to an interrupt triggered by security software Or, set the memory management unit to a safe mode, in which the non-secure software cannot access the memory corresponding to the memory management unit.
- the function of enabling the safe AI driver to receive the interrupt on the safe side can refer to enabling or turning on the function of the safe AI driver to receive the interrupt on the safe side, that is, after the function is enabled, the safe AI driver can receive the interrupt on the safe side.
- An interrupt is an interrupt triggered by a security-side application when it requests processing of security data. Before the security AI driver enables this function, the security AI driver cannot receive the security-side interrupt.
- the memory management unit can be used to manage the access permissions of its corresponding memory.
- the security AI driver sets the memory management unit to a safe mode, that is, the security AI driver sets the memory access permission of the memory through the memory management unit as follows: the security application software can access the memory. Memory, which cannot be accessed by non-secure software.
- the security operating system sends security data to the security AI driver, and the security data is generated by the security software.
- the secure AI driver configures the artificial intelligence processor to switch from the non-secure mode to the secure mode
- the secure operating system can send secure data to the secure AI driver.
- the secure data can refer to the data sent by the secure side application to the secure operating system. Refers to the data after the security operating system preprocesses the data sent by the security side application.
- the security side application sends data to the secure operating system, it can send the above data to the secure operating system through one transmission or multiple transmissions.
- the security data may include user input data and a user data model.
- the user input data may refer to the currently input user data.
- the user data model may be a data model previously obtained through model training based on a large amount of user data, such as human labor. Intelligent training model.
- the specific information of the security data is related to the security side application that sends the request information.
- the user data model in the security data may be when the user uses the security side application for the first time or updates the user data model, the security side application collects a large number of users. Data and model training.
- the data processing device is a mobile phone
- the security side application is screen unlocking software.
- the security data may include a user image and a user image model currently collected by the mobile phone.
- the user image model may be a feature combination including multiple image features.
- the plurality of image features may include one or more of an eye feature, a mouth feature, and a face feature.
- the security side application is fingerprint payment software
- the security data may include a user fingerprint and a user fingerprint model currently collected by the mobile phone.
- the user fingerprint model may be a feature combination including multiple fingerprint features, and the multiple fingerprint features may include fingerprints. One or more of a branch point, a fingerprint start point, and a fingerprint end point.
- S407 The safety AI driver receives the safety data and calls an artificial intelligence processor to process the safety data.
- the safety AI driver calls the artificial intelligence processor to process the safety data, that is, the safety AI driver can send the safety data to the artificial intelligence processor for processing, and the artificial intelligence processor receives the safety data.
- the safety data can be processed and the safety data processing result can be obtained, and then the artificial intelligence processor sends the safety data processing result to the safety AI driver, so that the safety AI driver returns the safety data processing result to the safety operating system.
- a specific process for the artificial intelligence processor to process the security data may be: the artificial intelligence processor extracts multiple image features from the user input image, and extracts Multiple image features of the same type as the multiple image features included in the user image model; the artificial intelligence processor matches two image features of the same type; if the two image features of each type are the same, a match can be determined Success. If one or more types of two image features are inconsistent, it can be determined that the matching has failed. After that, the artificial intelligence processor may send the result of the successful match or the failed match to the secure AI driver, so that the secure AI driver returns the result of the successful match or the failed match to the secure operating system.
- the secure operating system may send the result to the screen unlocking software, and the screen unlocking software may unlock the mobile phone screen when the result is a successful match, and the screen unlocking software may keep the mobile phone screen in an unlocked state when the result is a matching failure.
- the method further includes: S409-S415.
- S401-S408 are not shown in FIG.
- the safe operating system notifies the safe AI-driven artificial intelligence processor to switch from the safe mode to the non-safe mode.
- the security operating system may notify the security AI driver
- the artificial intelligence processor switches from secure mode to non-secure mode.
- the new security data may be generated by security software and not yet processed by the artificial intelligence processor.
- the secure AI driver configures the artificial intelligence processor in a non-secure mode, that is, it renounces the occupation of the artificial intelligence processor.
- the security AI driver configures the artificial intelligence processor to a non-secure mode, which can also be referred to as the security AI driver configures the artificial intelligence processor to switch from the secure mode to the non-secure mode, which specifically includes: disabling the secure AI driver to receive interrupts from the security side.
- the safety-side interrupt is the interrupt triggered by the safety software.
- the safety AI driver can receive the safety-side interrupt and process the safety data based on the safety-side interrupt.
- the safety AI driver can turn off or shield the safety AI driver from receiving the safety-side interrupt. Function, that is, after disabling the function, the safety AI driver cannot receive the safety-side interrupt, and the safety AI driver cannot call the artificial intelligence processor to process safety data.
- S411 The secure operating system sends second instruction information to the general operating system, and the second instruction information is used to instruct the artificial intelligence processor to switch from the safe mode to the non-safe mode.
- S412 When receiving the second instruction information, the general operating system notifies the non-secure AI to drive the artificial intelligence processor to enter the non-secure mode.
- the non-secure AI driver configures the artificial intelligence processor in a non-secure mode, that is, the artificial intelligence processor is occupied.
- the configuration of a non-secure AI driver with an artificial intelligence processor in a non-secure mode may include at least one of the following: enabling a non-secure AI driver to receive a non-secure side interrupt, the non-secure side interrupt refers to an interrupt triggered by non-secure software; or , Setting the memory management unit to a non-secure mode, in which the non-secure software can access the memory corresponding to the memory management unit.
- enabling a non-safety AI driver to receive non-safety-side interrupts can refer to enabling or enabling a non-safety AI driver to receive non-safety-side interrupts. That is, after enabling this function, non-safety AI drivers can receive non-safety Side interrupts, so non-secure AI drivers can also call artificial intelligence processors to process non-secure data.
- the memory management unit can be used to manage the access permissions of its corresponding memory.
- the non-secure AI driver sets the memory management unit to the non-secure mode, that is, the non-secure AI driver sets the memory access permission of the memory through the memory management unit: non-secure applications. Software can access this memory.
- S414 The non-secure AI driver receives non-secure data, and calls an artificial intelligence processor to process the non-secure data.
- S415 The non-secure AI driver sends the result of non-secure data processing to the general operating system. After the non-secure AI driver configures the artificial intelligence processor in the non-secure mode, the general operating system can send non-secure data to the non-secure AI driver.
- the non-secure data can refer to the data sent by the non-secure side application to the general operating system. It may refer to data after a general operating system preprocesses data sent by a non-security-side application.
- the non-security side application can send the above data to the general operating system through one transmission or multiple transmissions.
- the non-secure data may be unprocessed data that was not processed when the artificial intelligence processor was switched to the secure mode last time, that is, the non-secure data was already in the pending queue of the artificial intelligence processor before S413.
- the AI driver does not need to perform the above steps for receiving non-secure data, but directly calls an artificial intelligence processor to process the non-secure data.
- the process of the non-secure AI driver calling the artificial intelligence processor to process the non-secure data may be: the non-secure AI driver sends the non-secure data to the artificial intelligence processor, and when the artificial intelligence processor receives the non-secure data, it can process the non-secure data.
- the unsecured data obtains the unsecured data processing result, and then the artificial intelligence processor sends the unsecured data processing result to the unsecured AI driver, so that the unsecured AI driver returns the unsecured data processing result to the operating system.
- the artificial intelligence processor may sequentially process the at least one pending non-secure data in the pending queue. At least one unsecured data to be processed, and the unsecured data processing result is returned to the operating system accordingly.
- the method further includes: S416-S417.
- S416-S417 and the above S401-S415 may be in no particular order, and FIG. 5 uses S416-S417 after S415 as an example for description.
- setting the artificial intelligence processor to a low power state by a non-secure AI driver may include performing a power-off operation on the artificial intelligence processor or reducing the power consumption of the artificial intelligence processor.
- Lowering the power consumption of the artificial intelligence processor may include at least one of the following: reducing the voltage of the artificial intelligence processor, reducing the current of the artificial intelligence processor, and reducing the clock of the artificial intelligence processor.
- the preset time can be set by those skilled in the art in advance, and the specific numerical value can be set according to the actual situation, which is not specifically limited in the embodiment of the present application.
- the non-safety AI driver restores the artificial intelligence processor to the working state.
- the artificial intelligence processor is in a low power consumption state, if the secure AI driver receives new secure data sent by the secure operating system, or the non-secure AI driver receives new unsecured data sent by the general operating system, the non-secure AI driver resumes manual work.
- the intelligent processor is in a working state.
- the non-secure AI driver when a non-secure AI driver sets the artificial intelligence processor to a low-power state by performing a power-off operation, the non-secure AI driver may perform a power-on operation on the artificial intelligence processor to restore it to a working state;
- the non-secure AI driver when a non-secure AI driver sets the artificial intelligence processor's voltage, current, and / or clock to a low power state, the non-secure AI driver can restore the artificial intelligence processor's voltage, current, and / or clock Is working value to restore it to working state.
- the non-secure AI driver sets the artificial intelligence processor to a low power consumption state when no new non-secure data needs to be processed for a long time, and when it needs to process new secure data or new non-secure data, Restoring the artificial intelligence processor to the working state can reduce the power consumption of the artificial intelligence processor without affecting the data processing process, thereby reducing the power consumption of the data processing device.
- the data processing apparatus includes a hardware structure and / or a software module corresponding to each function.
- the present application can be implemented in the form of a combination of hardware and computer software by combining the structures and algorithm steps of the examples described in the embodiments disclosed herein. Whether a certain function is performed by hardware or computer software-driven hardware depends on the specific application of the technical solution and design constraints. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
- the data processing device involved in the above embodiment is a system-level device. Its internal software part or hardware part can be independently used to implement data processing functions.
- FIG. 6 shows a possible structure diagram of another data processing device involved in this embodiment.
- the data processing device corresponds to the software part of FIG. 2 and can run on the general-purpose processor 1021.
- the device includes a secure operating system 301 and a secure AI driver 302.
- the security operating system 301 is used to support the device to execute S401 and S403 in the method embodiment, receive one or more of the steps of the secure data processing result sent by S405, one or more of S406, S409, and S411, and / or implement the foregoing method.
- the secure AI driver 302 is used to support the device to perform one or more steps in S402, S404, S405, and S410 in the method embodiment, and / or described in the method embodiment above Other technical processes.
- the data processing device further includes a general operating system 303 and a non-secure AI driver 304.
- the universal operating system 303 is used to support receiving the first instruction information sent by S406, S407, receiving the second instruction information sent by S411, S412, and receiving the non-secure data processing results sent by S415 in the method execution embodiment of the device.
- the non-secure AI driver 304 is used to support the device to execute S408, S413, S414, S415, S416, and S417 in the method embodiment One or more steps, and / or other technical processes described in the above method embodiments.
- the secure operating system 301, secure AI driver 302, general operating system 303, and non-secure AI driver 304 are implemented in software
- the software programs corresponding to the above functional modules are included in In the memory 101
- the general-purpose processor 1021 implements functions corresponding to the above-mentioned function modules by running software programs including the above-mentioned function modules included in the memory 101.
- the following describes another data processing device in the embodiment of the present application from the perspective of hardware processing.
- the another data processing device provided in the embodiment of the present application corresponds to the hardware part of FIG. 2 to provide hardware computing power for data processing functions, including a general-purpose processor 1021 and an artificial intelligence processor 1022.
- the specific structure can be as shown in FIG. 1 is shown.
- the general-purpose processor 1021 is configured to set the artificial intelligence processor 1022 to switch from the non-secure mode to the secure mode, and call the artificial intelligence processor 1022 to process the safety data; wherein, the safety data is run by the general-purpose processor 1021.
- the artificial intelligence processor 1022 is used to switch from the non-secure mode to the secure mode in response to the setting of the general-purpose processor 1021, and to process the secure data in response to the call of the general-purpose processor 1021 and return the secure data processing result to General processor 1021;
- the artificial intelligence processor 1022 cannot process non-secure data generated by the general processor 1021 running non-secure software.
- the general-purpose processor 1021 when switching from the non-secure mode to the secure mode, is configured to perform at least one of the following: disable the function of the artificial intelligence processor 1022 to handle non-secure side interrupts, and non-secure Side interrupts refer to interrupts triggered by non-secure software; or, enable the artificial intelligence processor 1022 to handle the functions of security side interrupts, and security side interrupts refer to interrupts triggered by security software; or, set the memory management unit to safe mode, Non-secure software cannot access the memory corresponding to the memory management unit in safe mode.
- the general-purpose processor 1021 is further configured to set a priority for processing the secure data by the artificial intelligence processor 1022. Higher than non-secure data, non-secure data is generated by the general-purpose processor 1021 running non-secure software and has not yet been processed by the artificial intelligence processor 1022.
- the general-purpose processor 1021 is further configured to set the artificial intelligence processor 1022 to switch from a secure mode to a non-secure mode, and call the artificial intelligence processor 1022 to process non-secure data; the artificial intelligence processor 1022, For switching from the secure mode to the non-secure mode in response to the setting of the general-purpose processor 1021, and for processing the non-secure data in response to the call of the general-purpose processor 1021 and returning the non-secure data processing result to the general-purpose processor 1021.
- the general-purpose processor 1021 is specifically configured to set the artificial intelligence processor 1022 to switch from the secure mode to the non-secure mode when no new secure data needs to be processed. Among them, in a non-secure mode, the artificial intelligence processor 1022 cannot process secure data.
- the general-purpose processor 1021 when switching from the secure mode to the non-secure mode, is configured to perform at least one of the following: disable the function of the artificial intelligence processor 1022 to process the security-side interrupt, and the security-side interrupt Refers to interrupts triggered by secure software; or enables the artificial intelligence processor 1022 to handle non-secure side interrupts.
- Non-secure side interrupts refer to interrupts triggered by non-secure software; or, sets the memory management unit to non-secure mode In non-secure mode, non-secure software can access the memory corresponding to the memory management unit.
- the general-purpose processor 1021 is further configured to set the artificial intelligence processor 1022 to a low power consumption state when there is no new non-secure data that needs to be processed for a preset time. Further, the general-purpose processor 1021 is further configured to restore the artificial intelligence processor 1022 to a working state when new security data or new non-security data needs to be processed.
- the artificial intelligence processor 1022 is a neural network processing unit NPU or an image processing unit GPU.
- the secure software includes at least one of a secure operating system or secure application software;
- the non-secure software includes at least one of a general-purpose operating system or non-secure application software.
- Security software forms TEE, while non-security software forms REE.
- Each component of the foregoing data processing device provided in the embodiment of the present application is used to implement functions of the corresponding steps of the foregoing data processing method, because in the foregoing embodiment of the data processing method, each step has been described in detail , Will not repeat them here.
- the artificial intelligence processor when the artificial intelligence processor is set to the safe mode, the artificial intelligence processor cannot process the non-secure data generated by the general-purpose processor running non-secure software, and can process the secure data, thereby ensuring the artificial intelligence. Security when the processor processes secure data.
- An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores instructions, and when it is run on a device (for example, the device may be a microcontroller, a chip, a computer, or a processor, etc.) , The device is caused to perform one or more steps in the foregoing method embodiments.
- a device for example, the device may be a microcontroller, a chip, a computer, or a processor, etc.
- the device is caused to perform one or more steps in the foregoing method embodiments.
- each component module of the data processing device is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in the computer-readable storage medium.
- the embodiments of the present application further provide a computer program product containing instructions.
- the technical solution of the present application is essentially a part that contributes to the existing technology or all or part of the technical solution may be a software product.
- the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor therein to execute the embodiments of the present application. All or part of the steps of the method.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The present application relates to the field of secure computing, and provided thereby are a data processing method and device, which are used to ensure the security of related data operations in AI technology. The device comprises: a general-purpose processor and an artificial intelligence processor. The general-purpose processor is used to configure the artificial intelligence processor to switch from a non-secure mode to a secure mode, and call the artificial intelligence processor to process secure data, the secure data being generated by the general-purpose processor running secure software. The artificial intelligence processor is used to switch from non-secure mode to secure mode in response to the configuration of the general-purpose processor, and process the secure data in response to calling by the general-purpose processor and return a secure data processing result to the general-purpose processor. In secure mode, the artificial intelligence processor cannot process non-secure data generated by the general-purpose processor running non-secure software.
Description
本申请涉及安全计算领域,尤其涉及一种数据处理方法及装置。The present application relates to the field of secure computing, and in particular, to a data processing method and device.
随着技术的不断进步,人工智能(artificial intelligence,AI)领域得到了长足的发展,AI由计算机技术产生,但现今的人工智能已经涵盖了包括语言学、心理学等在内的多个学科,其范围已经远远超出了计算机技术的范围,成为一个独立的学科分支,是近几年来的热门领域。With the continuous advancement of technology, the field of artificial intelligence (AI) has been greatly developed. AI is produced by computer technology, but today's artificial intelligence has covered multiple disciplines including linguistics and psychology. Its scope has far exceeded the scope of computer technology, and it has become an independent branch of science. It is a hot field in recent years.
目前,AI技术可以被应用于电子设备的安全特性中,比如,使用AI的三维(3D)人脸解锁可用于手机屏幕解锁、手机支付确认、身份认证等用途。具体地,在电子设备中可以集成神经网络处理单元(neural-network processing unit,NPU)(也可以称为NPU处理器),通过内置NPU处理器的芯片来实现手机屏幕解锁、手机支付确认和身份认证等过程中的相关AI数据运算。At present, AI technology can be applied to the security features of electronic devices. For example, three-dimensional (3D) face unlocking using AI can be used for mobile phone screen unlocking, mobile phone payment confirmation, and identity authentication. Specifically, a neural-network processing unit (NPU) (also referred to as an NPU processor) can be integrated into an electronic device, and a mobile phone screen unlock, mobile payment confirmation, and identity can be realized through a chip with a built-in NPU processor. Relevant AI data calculation during authentication and other processes.
但是,将AI技术使用在电子设备的安全特性中,需要保障AI技术中相关数据运算的安全性,以防止用户数据被泄露、篡改和仿冒等,以及防止相关运算逻辑被篡改和仿冒等造成的非法解锁或非法支付等问题,从而保证用户使用过程中的安全性,避免给用户造成不必要的损失。现有技术中,针对如何保障AI技术中相关数据运算的安全性尚未给出解决方案。However, the use of AI technology in the security features of electronic devices requires the security of related data operations in AI technology to prevent user data from being leaked, tampered with, and counterfeited, as well as prevent related computing logic from being tampered with and counterfeited. Problems such as illegal unlocking or illegal payment ensure the safety of users during use and avoid unnecessary losses to users. In the prior art, no solution has been provided for how to ensure the security of related data operations in AI technology.
发明内容Summary of the Invention
本申请的实施例提供一种数据处理方法及装置,用于保障AI技术中相关数据运算的安全性。The embodiments of the present application provide a data processing method and device, which are used to ensure the security of related data operations in AI technology.
为达到上述目的,本申请的实施例采用如下技术方案:To achieve the above purpose, the embodiments of the present application adopt the following technical solutions:
第一方面,提供一种数据处理装置,该装置包括:通用处理器和人工智能处理器;其中,通用处理器,用于设置人工智能处理器从非安全模式切换为安全模式,并调用人工智能处理器处理安全数据,所述安全数据是通用处理器运行安全软件所生成的;人工智能处理器,用于响应于通用处理器的设置从非安全模式切换为安全模式,以及响应于通用处理器的调用处理所述安全数据并返回安全数据处理结果给通用处理器;在所述安全模式下,人工智能处理器无法处理通用处理器运行非安全软件所生成的非安全数据。According to a first aspect, a data processing device is provided. The device includes a general-purpose processor and an artificial intelligence processor, wherein the general-purpose processor is used to set the artificial intelligence processor to switch from a non-secure mode to a secure mode, and call the artificial intelligence. The processor processes security data generated by the general-purpose processor running security software; an artificial intelligence processor for switching from a non-secure mode to a secure mode in response to a setting of the general-purpose processor, and in response to the general-purpose processor The call processes the secure data and returns the secure data processing result to the general-purpose processor; in the secure mode, the artificial intelligence processor cannot process the non-secure data generated by the general-purpose processor running non-secure software.
上述技术方案中,人工智能处理器在被设置为安全模式时,无法处理通用处理器运行非安全软件所生成的非安全数据,从而可以保证人工智能处理器处理安全数据时的安全性,即保证AI技术中相关数据运算的安全性。In the above technical solution, when the artificial intelligence processor is set to the secure mode, it cannot process the non-secure data generated by the general-purpose processor running non-secure software, thereby ensuring the security of the artificial intelligence processor when processing the secure data, that is, the guarantee Security of related data operations in AI technology.
在第一方面的一种可能的实现方式中,在从非安全模式切换为安全模式时,通用处理器用于执行如下至少一项:去使能人工智能处理器处理非安全侧中断的功能,非安全侧中断是指由非安全软件触发的中断;或者,使能人工智能处理器处理安全侧中 断的功能,安全侧中断是指由安全软件触发的中断;或者,设置内存管理单元为安全模式,在安全模式下非安全软件无法访问与内存管理单元对应的内存。上述可能的实现方式中,可以使得人工智能处理器在安全模式下无法处理非安全数据、能够处理安全数据,且非安全软件无法访问内存,从而安全数据及其处理过程的安全性。In a possible implementation manner of the first aspect, when switching from the non-secure mode to the secure mode, the general-purpose processor is configured to perform at least one of the following: disabling an artificial intelligence processor from processing a non-safety-side interruption function. Security-side interrupts refer to interrupts triggered by non-safety software; or, the function of the artificial intelligence processor to handle security-side interrupts is enabled, and security-side interrupts refer to interrupts triggered by security software; or, setting the memory management unit to safe mode, Non-secure software cannot access the memory corresponding to the memory management unit in safe mode. In the foregoing possible implementation manner, the artificial intelligence processor cannot process non-secure data, can process secure data, and non-secure software cannot access memory in a secure mode, thereby ensuring the security of the secure data and its processing process.
在第一方面的一种可能的实现方式中,在通用处理设置人工智能处理器从非安全模式切换为安全模式之前,通用处理器,还用于:设置安全数据被人工智能处理器处理的优先级高于非安全数据,非安全数据是由通用处理器运行非安全软件所生成且尚未被人工智能处理器处理。上述可能的实现方式中,可以使得人工智能处理器优先处理安全数据,从而提高安全数据的处理速度,进而提高用户体验。In a possible implementation manner of the first aspect, before the general-purpose processing sets the artificial intelligence processor to switch from the non-secure mode to the secure mode, the general-purpose processor is further configured to set the priority of the secure data to be processed by the artificial intelligence processor. Level is higher than non-secure data, which is generated by general-purpose processors running non-secure software and has not yet been processed by artificial intelligence processors. In the foregoing possible implementation manner, the artificial intelligence processor can be made to process the security data in priority, thereby improving the processing speed of the security data, and further improving the user experience.
在第一方面的一种可能的实现方式中,通用处理器,还用于设置人工智能处理器从安全模式切换为非安全模式,并调用人工智能处理器处理非安全数据;人工智能处理器,用于响应于通用处理器的设置从安全模式切换为非安全模式,以及响应于通用处理器的调用处理非安全数据并返回非安全数据处理结果给通用处理器。上述可能的实现方式中,人工智能处理器在被设置为非安全模式时,可以处理非安全数据,从而可以实现人工智能处理器处理安全数据和非安全数据的兼容。In a possible implementation manner of the first aspect, the general-purpose processor is further configured to set the artificial intelligence processor to switch from the secure mode to the non-secure mode, and call the artificial intelligence processor to process non-secure data; the artificial intelligence processor, It is used to switch from the secure mode to the non-secure mode in response to the setting of the general-purpose processor, and to process the non-secure data in response to the call of the general-purpose processor and return the non-secure data processing result to the general-purpose processor. In the foregoing possible implementation manner, when the artificial intelligence processor is set to the non-secure mode, it can process non-secure data, so that the artificial intelligence processor can be compatible with the processing of the secure data and the non-secure data.
在第一方面的一种可能的实现方式中,在非安全模式下,人工智能处理器无法处理安全数据。上述可能的实现方式中,可以在非安全模式下能够保证安全数据及其处理过程的安全性。In a possible implementation manner of the first aspect, in a non-secure mode, the artificial intelligence processor cannot process secure data. In the foregoing possible implementation manner, the security of the secure data and its processing process can be guaranteed in the non-secure mode.
在第一方面的一种可能的实现方式中,在从安全模式切换为非安全模式时,通用处理器用于执行如下至少一项:去使能人工智能处理器处理安全侧中断的功能,安全侧中断是指由安全软件触发的中断;或者,使能人工智能处理器处理非安全侧中断的功能,非安全侧中断是指由非安全软件触发的中断;或者,设置内存管理单元为非安全模式,在非安全模式下非安全软件能够访问与内存管理单元对应的内存。上述可能的实现方式中,可以使得人工智能处理器在非安全模式下无法处理安全数据、能够处理非安全数据,且非安全软件能够访问内存,从而实现人工智能处理器处理安全数据和非安全数据的兼容。In a possible implementation manner of the first aspect, when switching from the secure mode to the non-secure mode, the general-purpose processor is configured to perform at least one of the following: disabling the function of the artificial intelligence processor to handle interrupts on the security side, and the security side Interrupts refer to interrupts triggered by secure software; or, enable the artificial intelligence processor to handle non-secure side interrupts, non-secure side interrupts refer to interrupts triggered by non-secure software; or set the memory management unit to non-secure mode In non-secure mode, non-secure software can access the memory corresponding to the memory management unit. In the foregoing possible implementation manner, the artificial intelligence processor cannot process secure data, can process non-secure data, and non-secure software can access memory in the non-secure mode, thereby enabling the artificial intelligence processor to process secure data and non-secure data. Compatible.
在第一方面的一种可能的实现方式中,通用处理器,具体用于在没有新安全数据需要被处理时,设置人工智能处理器从安全模式切换为非安全模式。上述可能的实现方式中,能够避免人工智能处理器长时间处于安全模式,而无法处理非安全数据的问题,从而保证人工智能处理器处理处理非安全数据的效率。In a possible implementation manner of the first aspect, the general-purpose processor is specifically configured to set the artificial intelligence processor to switch from the secure mode to the non-secure mode when no new secure data needs to be processed. In the foregoing possible implementation manner, the problem that the artificial intelligence processor is in a safe mode for a long time and cannot process non-secure data can be avoided, thereby ensuring the efficiency of the artificial intelligence processor in processing non-secure data.
在第一方面的一种可能的实现方式中,通用处理器,还用于:当没有新非安全数据需要被处理的持续时间达到预设时间时,设置人工智能处理器为低功耗状态。可选的,设置人工智能处理器为低功耗状态可以包括:对人工智能处理器进行下电操作,或者调低人工智能处理器的功耗,比如,调低人工智能处理器的电压、电流或者时钟中的至少一种。上述可能的实现方式中,能够在不影响数据处理过程的前提下,降低人工智能处理器的功耗,进而降低数据处理装置的功耗。In a possible implementation manner of the first aspect, the general-purpose processor is further configured to set the artificial intelligence processor to a low power consumption state when a duration in which no new non-secure data needs to be processed reaches a preset time. Optionally, setting the artificial intelligence processor to a low power consumption state may include: powering off the artificial intelligence processor, or reducing the power consumption of the artificial intelligence processor, for example, reducing the voltage and current of the artificial intelligence processor Or at least one of the clocks. In the foregoing possible implementation manner, the power consumption of the artificial intelligence processor can be reduced without affecting the data processing process, thereby reducing the power consumption of the data processing device.
在第一方面的一种可能的实现方式中,通用处理器,还用于:当需要处理新安全数据或者新非安全数据时,恢复人工智能处理器为工作状态。可选的,恢复人工智能处理器为工作状态可以包括:对人工智能处理器进行上电操作,或者设置人工智能处 理器的功耗为工作状态时的功耗,比如,将人工智能处理器的电压、电流和/或时钟恢复为工作值。上述可能的实现方式中,能够在不影响数据处理过程的前提下,降低人工智能处理器的功耗,进而降低数据处理装置的功耗。In a possible implementation manner of the first aspect, the general-purpose processor is further configured to restore the artificial intelligence processor to a working state when it is required to process new secure data or new non-secure data. Optionally, restoring the artificial intelligence processor to the working state may include: powering on the artificial intelligence processor, or setting the power consumption of the artificial intelligence processor to the working state, for example, changing the artificial intelligence processor ’s The voltage, current, and / or clock are restored to operating values. In the foregoing possible implementation manner, the power consumption of the artificial intelligence processor can be reduced without affecting the data processing process, thereby reducing the power consumption of the data processing device.
在第一方面的一种可能的实现方式中,人工智能处理器为神经网络处理单元NPU或者图像处理单元(GPU)。上述可能的实现方式中,提供了几种可能的人工智能处理器。In a possible implementation manner of the first aspect, the artificial intelligence processor is a neural network processing unit NPU or an image processing unit (GPU). In the foregoing possible implementation manners, several possible artificial intelligence processors are provided.
在第一方面的一种可能的实现方式中,安全软件包括安全操作系统或安全应用软件的至少一项;非安全软件包括通用操作系统或非安全应用软件的至少一项。上述可能的实现方式中,提供了几种可能的安全软件和非安全软件。In a possible implementation manner of the first aspect, the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general operating system or non-secure application software. In the foregoing possible implementation manners, several possible security software and non-security software are provided.
第二方面,提供一种数据处理装置,该装置包括:安全操作系统和安全人工智能处理器驱动;其中,安全操作系统,用于通知安全人工智能处理器驱动人工智能处理器进入安全模式;安全人工智能处理器驱动,用于配置人工智能处理器从非安全模式切换为安全模式,在安全模式下,人工智能处理器无法处理由非安全软件所生成的非安全数据;安全操作系统,还用于向安全人工智能处理器驱动发送安全数据,安全数据是由安全软件所生成的;安全人工智能处理器驱动,还用于接收安全数据,调用人工智能处理器处理所述安全数据,并返回安全数据处理结果给安全操作系统。In a second aspect, a data processing device is provided. The device includes: a secure operating system and a secure artificial intelligence processor driver; wherein the secure operating system is used to notify the secure artificial intelligence processor to drive the artificial intelligence processor into a secure mode; The artificial intelligence processor driver is used to configure the artificial intelligence processor to switch from the non-secure mode to the secure mode. In the secure mode, the artificial intelligence processor cannot process the non-secure data generated by the non-secure software; the secure operating system also uses For sending safety data to the safety artificial intelligence processor driver, the safety data is generated by the safety software; the safety artificial intelligence processor driver is also used to receive the safety data, call the artificial intelligence processor to process the safety data, and return the safety data The data processing results are given to the secure operating system.
在第二方面的一种可能的实现方式中,在配置人工智能处理器从非安全模式切换为安全模式时,安全人工智能处理器驱动用于执行如下至少一项:使能安全人工智能处理器驱动接收非安全侧中断的功能,安全侧中断是指由安全软件触发的中断;或者,设置内存管理单元为安全模式,在安全模式下非安全软件无法访问与内存管理单元对应的内存。In a possible implementation manner of the second aspect, when the artificial intelligence processor is configured to switch from the non-secure mode to the secure mode, the secure artificial intelligence processor driver is configured to perform at least one of the following: enable the secure artificial intelligence processor The driver receives the non-safety-side interruption function. The safety-side interruption refers to the interruption triggered by the safety software; or, the memory management unit is set to the safe mode. In the safe mode, the non-safety software cannot access the memory corresponding to the memory management unit.
在第二方面的一种可能的实现方式中,该装置还包括:通用操作系统和非安全人工智能处理器驱动;其中,安全操作系统,还用于通知安全人工智能处理器驱动进入安全模式之前,向通用操作系统发送第一指示信息,第一指示信息用于指示人工智能处理器进入安全模式;通用操作系统,用于在接收到第一指示信息时,通知非安全人工智能处理器驱动人工智能处理器进入安全模式;非安全人工智能处理器驱动,用于配置人工智能处理器从非安全模式切换为安全模式。In a possible implementation manner of the second aspect, the device further includes: a general operating system and a non-secure artificial intelligence processor driver; wherein the secure operating system is further configured to notify the secure artificial intelligence processor driver before entering the safe mode Sending the first instruction information to the general operating system, where the first instruction information is used to instruct the artificial intelligence processor to enter the safe mode; the general operating system is used to notify the non-secure artificial intelligence processor to drive the human body when the first instruction information is received The intelligent processor enters the secure mode; the non-secure artificial intelligence processor driver is used to configure the artificial intelligence processor to switch from the non-secure mode to the secure mode.
在第二方面的一种可能的实现方式中,在配置人工智能处理器从非安全模式切换为安全模式时,非安全人工智能处理器驱动用于执行如下操作:去使能非安全人工智能处理器驱动接收非安全侧中断的功能,非安全侧中断是指由非安全软件触发的中断。In a possible implementation manner of the second aspect, when the artificial intelligence processor is configured to switch from the non-secure mode to the secure mode, the non-secure artificial intelligence processor driver is configured to perform the following operations: disabling the non-secure artificial intelligence processing The device driver receives the non-safety side interrupt function. The non-safety side interrupt refers to the interrupt triggered by the non-safety software.
在第二方面的一种可能的实现方式中,在通知非安全人工智能处理器驱动人工智能处理器进入安全模式之前,通用操作系统,还用于:设置安全数据被处理的优先级高于非安全数据,非安全数据是由非安全软件所生成且尚未被人工智能处理器处理。In a possible implementation manner of the second aspect, before notifying the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the safe mode, the general operating system is further configured to: set the priority of processing the secure data higher than that of the non-secure artificial intelligence processor. Security data, non-security data is generated by non-security software and has not been processed by the artificial intelligence processor.
在第二方面的一种可能的实现方式中,安全操作系统,还用于通知安全人工智能处理器驱动人工智能处理器从安全模式切换为非安全模式;安全人工智能处理器驱动,还用于配置人工智能处理器为非安全模式;安全操作系统,还用于向通用操作系统发送第二指示信息,第二指示信息用于指示从安全模式切换为非安全模式;通用操作系统,还用于在接收到第二指示信息时,通知非安全人工智能处理器驱动人工智能处理器进入非安全模式;非安全人工智能处理器驱动,还用于配置人工智能处理器为非安 全模式,以及接收非安全数据,调用人工智能处理器处理非安全数据,并返回非安全数据处理结果给通用操作系统。In a possible implementation manner of the second aspect, the secure operating system is further configured to notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode; the secure artificial intelligence processor driver is also used to Configure the artificial intelligence processor in non-secure mode; the secure operating system is also used to send second instruction information to the general operating system, the second instruction information is used to instruct the switch from the secure mode to the non-secure mode; the general operating system is also used to When receiving the second instruction information, notify the non-secure artificial intelligence processor to drive the artificial intelligence processor into the non-secure mode; the non-secure artificial intelligence processor driver is also used to configure the artificial intelligence processor to be in the non-secure mode and receive non-secure artificial intelligence processors. Safe data, call the artificial intelligence processor to process the unsafe data, and return the unsafe data processing result to the general operating system.
在第二方面的一种可能的实现方式中,在配置人工智能处理器为所述非安全模式时,安全人工智能处理器驱动用于执行如下操作:去使能安全人工智能处理器驱动接收安全侧中断的功能,安全侧中断是指由安全软件触发的中断;和,非安全人工智能处理器驱动用于执行如下至少一项:使能非安全人工智能处理器驱动接收非安全侧中断的功能,非安全侧中断是指由所述非安全软件触发的中断;或者,设置内存管理单元为非安全模式,在非安全模式下非安全软件能够访问与内存管理单元对应的内存。In a possible implementation manner of the second aspect, when the artificial intelligence processor is configured as the non-secure mode, the secure artificial intelligence processor driver is configured to perform the following operations: disabling the secure artificial intelligence processor driver to receive security Side interrupt function, the safe side interrupt refers to the interrupt triggered by the security software; and, the non-secure artificial intelligence processor driver is used to perform at least one of the following: enable the non-secure artificial intelligence processor driver to receive the non-secure side interrupt function The non-secure side interrupt refers to an interrupt triggered by the non-secure software; or, the memory management unit is set to an unsecure mode, and the non-secure software can access the memory corresponding to the memory management unit in the non-secure mode.
在第二方面的一种可能的实现方式中,安全操作系统,具体用于在没有新安全数据需要被处理时,通知安全人工智能处理器驱动人工智能处理器从安全模式切换为非安全模式。In a possible implementation manner of the second aspect, the secure operating system is specifically configured to notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode when no new secure data needs to be processed.
在第二方面的一种可能的实现方式中,非安全人工智能处理器驱动,还用于:当没有新非安全数据需要被处理的持续时间达到预设时间时,设置人工智能处理器为低功耗状态。In a possible implementation manner of the second aspect, the non-secure artificial intelligence processor driver is further configured to: set the artificial intelligence processor to low when there is no new non-secure data that needs to be processed for a preset time. Power status.
在第二方面的一种可能的实现方式中,非安全人工智能处理器驱动,还用于:当需要处理新安全数据或者新非安全数据时,恢复人工智能处理器为工作状态。In a possible implementation manner of the second aspect, the non-secure artificial intelligence processor driver is further configured to: when new security data or new non-secure data needs to be processed, restore the artificial intelligence processor to a working state.
在第二方面的一种可能的实现方式中,安全软件包括安全操作系统或安全应用软件的至少一项;非安全软件包括通用操作系统或非安全应用软件的至少一项。In a possible implementation manner of the second aspect, the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general operating system or non-secure application software.
第三方面,提供一种数据处理方法,该方法包括:安全操作系统通知安全人工智能处理器驱动人工智能处理器进入安全模式;安全人工智能处理器驱动配置人工智能处理器从非安全模式切换为安全模式,在安全模式下人工智能处理器无法处理由非安全软件所生成的非安全数据;安全操作系统向安全人工智能处理器驱动发送安全数据,安全数据是由安全软件所生成的;安全人工智能处理器驱动接收安全数据,调用人工智能处理器处理该安全数据,并返回安全数据处理结果给安全操作系统。In a third aspect, a data processing method is provided. The method includes: the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor into a safe mode; the secure artificial intelligence processor driver configures the artificial intelligence processor to switch from a non-secure mode to Safe mode, in which the artificial intelligence processor cannot process non-secure data generated by non-secure software; the secure operating system sends secure data to the secure artificial intelligence processor driver, and the secure data is generated by the secure software; secure labor The intelligent processor driver receives the safety data, calls the artificial intelligence processor to process the safety data, and returns the safety data processing result to the safety operating system.
在第三方面的一种可能的实现方式中,安全人工智能处理器驱动配置人工智能处理器从非安全模式切换为安全模式,包括如下至少一项:使能安全人工智能处理器驱动接收非安全侧中断的功能,安全侧中断是指由安全软件触发的中断;或者,设置内存管理单元为安全模式,在安全模式下非安全软件无法访问与内存管理单元对应的内存。In a possible implementation manner of the third aspect, configuring the secure artificial intelligence processor driver to switch the artificial intelligence processor from the non-secure mode to the secure mode includes at least one of the following: enabling the secure artificial intelligence processor driver to receive non-secure The function of the side interrupt, the safe side interrupt refers to the interrupt triggered by the security software; or, the memory management unit is set to the safe mode, and the non-safe software cannot access the memory corresponding to the memory management unit in the safe mode.
在第三方面的一种可能的实现方式中,安全操作系统通知安全人工智能处理器驱动进入安全模式之前,该方法还包括:安全操作系统向通用操作系统发送第一指示信息,第一指示信息用于指示人工智能处理器进入安全模式;通用操作系统在接收到第一指示信息时,通知非安全人工智能处理器驱动人工智能处理器进入安全模式;非安全人工智能处理器驱动配置人工智能处理器从非安全模式切换为安全模式。In a possible implementation manner of the third aspect, before the secure operating system notifies the secure artificial intelligence processor driver to enter the secure mode, the method further includes: the secure operating system sends the first instruction information to the general operating system, and the first instruction information Used to instruct the artificial intelligence processor to enter the safe mode; when the general operating system receives the first instruction message, it informs the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the safe mode; the non-secure artificial intelligence processor drives to configure the artificial intelligence processing The device switches from non-safe mode to safe mode.
在第三方面的一种可能的实现方式中,非安全人工智能处理器驱动配置人工智能处理器从非安全模式切换为安全模式,包括:去使能非安全人工智能处理器驱动接收非安全侧中断的功能,非安全侧中断是指由非安全软件触发的中断。In a possible implementation manner of the third aspect, configuring the non-secure artificial intelligence processor driver to switch the artificial intelligence processor from the non-secure mode to the secure mode includes: disabling the non-secure artificial intelligence processor driver to receive the non-secure side Interrupt function, non-safety side interrupt refers to the interrupt triggered by non-safety software.
在第三方面的一种可能的实现方式中,通用操作系统通知非安全人工智能处理器驱动人工智能处理器进入安全模式之前,该方法还包括:通用处理系统设置安全数据 被处理的优先级高于非安全数据,非安全数据是由非安全软件所生成且尚未被人工智能处理器处理。In a possible implementation manner of the third aspect, before the general operating system notifies the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the safe mode, the method further includes: the general processing system sets the priority of the secure data to be processed to be high For non-secure data, non-secure data is generated by non-secure software and has not yet been processed by an artificial intelligence processor.
在第三方面的一种可能的实现方式中,该方法还包括:安全操作系统通知安全人工智能处理器驱动人工智能处理器从安全模式切换为非安全模式;安全人工智能处理器驱动配置人工智能处理器为非安全模式;安全操作系统向通用操作系统发送第二指示信息,第二指示信息用于指示从安全模式切换为非安全模式;通用操作系统在接收到第二指示信息时,通知非安全人工智能处理器驱动人工智能处理器进入非安全模式;非安全人工智能处理器驱动配置人工智能处理器为非安全模式,以及接收非安全数据,调用人工智能处理器处理非安全数据,并返回非安全数据处理结果给通用操作系统。In a possible implementation manner of the third aspect, the method further includes: the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode; the secure artificial intelligence processor drives to configure the artificial intelligence The processor is in a non-secure mode; the secure operating system sends second instruction information to the general operating system, and the second instruction information is used to instruct switching from the secure mode to the non-secure mode; when the general operating system receives the second instruction information, it informs the non- The secure artificial intelligence processor drives the artificial intelligence processor into the non-secure mode; the non-secure artificial intelligence processor drives configure the artificial intelligence processor to the non-secure mode, and receives non-secure data, calls the artificial intelligence processor to process the non-secure data, and returns The results of non-secure data processing are given to the general operating system.
在第三方面的一种可能的实现方式中,安全人工智能处理器驱动配置人工智能处理器为非安全模式,包括:去使能安全人工智能处理器驱动接收安全侧中断的功能,安全侧中断是指由安全软件触发的中断;和,非安全人工智能处理器驱动配置人工智能处理器为非安全模式,包括如下至少一项:使能非安全人工智能处理器驱动接收非安全侧中断的功能,非安全侧中断是指由非安全软件触发的中断;或者,设置内存管理单元为非安全模式,在非安全模式下非安全软件能够访问与内存管理单元对应的内存。In a possible implementation manner of the third aspect, the safe artificial intelligence processor driver configures the artificial intelligence processor in a non-secure mode, including: disabling the function of the safe artificial intelligence processor driver to receive the safety-side interrupt, and the safety-side interrupt Refers to interrupts triggered by secure software; and, the configuration of an artificial intelligence processor in non-secure mode driven by a non-secure artificial intelligence processor, including at least one of the following: enabling the function of the non-secure artificial intelligence processor driver to receive non-secure side interrupts The non-secure side interrupt refers to the interrupt triggered by the non-secure software; or, the memory management unit is set to the non-secure mode, and the non-secure software can access the memory corresponding to the memory management unit in the non-secure mode.
在第三方面的一种可能的实现方式中,安全操作系统通知安全人工智能处理器驱动人工智能处理器从安全模式切换为非安全模式,包括:安全操作系统在没有新安全数据需要被处理时,通知安全人工智能处理器驱动人工智能处理器从安全模式切换为非安全模式。In a possible implementation manner of the third aspect, the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode, including: when the secure operating system has no new secure data to be processed To notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the safe mode to the non-safe mode.
在第三方面的一种可能的实现方式中,该方法还包括:当没有新非安全数据需要被处理的持续时间达到预设时间时,非安全人工智能处理器驱动设置人工智能处理器为低功耗状态。In a possible implementation manner of the third aspect, the method further includes: when there is no new non-secure data that needs to be processed for a preset time, the non-secure artificial intelligence processor driver sets the artificial intelligence processor to low Power status.
在第三方面的一种可能的实现方式中,该方法还包括:当需要处理新安全数据或者新非安全数据时,非安全人工智能处理器驱动恢复人工智能处理器为工作状态。In a possible implementation manner of the third aspect, the method further includes: when new security data or new non-secure data needs to be processed, the non-secure artificial intelligence processor drives and restores the artificial intelligence processor to a working state.
在第三方面的一种可能的实现方式中,安全软件包括安全操作系统或安全应用软件的至少一项;非安全软件包括通用操作系统或非安全应用软件的至少一项。In a possible implementation manner of the third aspect, the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general operating system or non-secure application software.
第四方面,提供一种可读存储介质,所述可读存储介质中存储有指令,当所述可读存储介质在设备上运行时,使得所述设备执行第三方面或者第三方面的任一项可能的实现方式所提供的数据处理方法。According to a fourth aspect, a readable storage medium is provided, where the readable storage medium stores instructions, and when the readable storage medium is run on a device, the device is caused to perform the third aspect or any of the third aspects. Data processing methods provided by one possible implementation.
第六方面,提供一种计算机程序产品,当所述计算机程序产品在计算机或处理器上运行时,使得所述计算机或处理器执行第一方面或者第一方面的任一项可能的实现方式所提供的数据处理方法。According to a sixth aspect, a computer program product is provided, and when the computer program product runs on a computer or a processor, the computer or processor executes the first aspect or any possible implementation manner of the first aspect. Provided data processing methods.
可以理解地,上述提供的任一种数据处理装置、方法、计算机存储介质或者计算机程序产品均用于执行上文所提供的对应装置的功能,因此,其所能达到的有益效果可参考上文所提供的对应装置中的有益效果,此处不再赘述。Understandably, any of the data processing devices, methods, computer storage media, or computer program products provided above is used to perform the functions of the corresponding devices provided above. Therefore, for the beneficial effects that can be achieved, refer to the above. The beneficial effects in the provided corresponding device are not repeated here.
图1为本申请实施例提供的一种系统级数据处理装置的结构示意图;FIG. 1 is a schematic structural diagram of a system-level data processing device according to an embodiment of the present application;
图2为本申请实施例提供的一种通用处理器上运行软件的架构图;FIG. 2 is an architecture diagram of software running on a general-purpose processor according to an embodiment of the present application; FIG.
图3为本申请实施例提供的一种通用操作系统/安全操作系统的架构图;FIG. 3 is an architecture diagram of a universal operating system / secure operating system provided by an embodiment of the present application; FIG.
图4为本申请实施例提供的一种数据处理方法的流程示意图一;FIG. 4 is a first schematic flowchart of a data processing method according to an embodiment of the present application; FIG.
图5为本申请实施例提供的一种数据处理方法的流程示意图二;5 is a second schematic flowchart of a data processing method according to an embodiment of the present application;
图6为本申请实施例提供的另一种数据处理装置的结构示意图。FIG. 6 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present application.
本申请中,“至少一个”是指一个或者多个,“多个”是指两个或两个以上。“和/或”,描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B的情况,其中A,B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。In the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association relationship of related objects, and indicates that there can be three kinds of relationships, for example, A and / or B can represent: the case where A exists alone, A and B exist simultaneously, and B alone exists, where A, B can be singular or plural. The character "/" generally indicates that the related objects are an "or" relationship.
图1为本申请实施例提供的一种系统级数据处理装置的结构示意图,该数据处理装置可以为手机、平板电脑、计算机、可穿戴设备、车载设备或便携式设备等。为方便描述,可以将上面提到的设备统称为数据处理装置。参见图1,该数据处理装置可以包括存储器101、处理器102、通信接口103和总线104,存储器101、处理器102以及通信接口103通过总线104相互连接。FIG. 1 is a schematic structural diagram of a system-level data processing device according to an embodiment of the present application. The data processing device may be a mobile phone, a tablet computer, a computer, a wearable device, a vehicle-mounted device, or a portable device. For convenience of description, the above-mentioned devices may be collectively referred to as a data processing apparatus. Referring to FIG. 1, the data processing apparatus may include a memory 101, a processor 102, a communication interface 103, and a bus 104. The memory 101, the processor 102, and the communication interface 103 are connected to each other through a bus 104.
其中,存储器101可用于存储数据、软件程序以及模块,主要包括存储程序区和存储数据区,存储程序区可存储操作系统、至少一个功能所需的应用程序等,存储数据区可存储该数据处理装置的使用时所创建的数据等。在本申请实施例中,所述操作系统可以包括通用操作系统(比如,安卓系统)和安全操作系统(比如,专用操作系统,如Trustzone系统);所述至少一个功能所需的应用程序可以包括安全应用软件和非安全应用软件,安全应用(trusted application,TA)软件可以包括安全人工智能处理器驱动和其他具有安全特性的应用(比如,基于Trustzone系统的屏幕解锁软件、支付软件等),非安全应用软件也叫客户端应用(client application,CA),可以包括非安全人工智能处理器驱动和其他具有非安全特性的应用(比如,基于安卓系统的网页浏览软件、视频播放软件等);所述存储数据区存储的数据可以包括安全数据、安全数据处理结果、非安全数据和/或非安全数据处理结果等。在一些可行的实施例中,存储器101中存储的软件模块可以包括内存管理单元,存储数据区中可以包括内存,内存管理单元可用于管理该内存,比如,内存管理单元可用于设置该内存的访问权限。在另一些可行的实施例中,可以有一个存储器,也可以有多个存储器;该存储器可以是软盘,硬盘如内置硬盘和移动硬盘,磁盘,光盘,磁光盘如只读光盘(compact disc read-only memory,CD_ROM)、数字视盘(digital video disc read-only memory,DVD_ROM),非易失性存储设备如随机存取存储器(random access memory,RAM)、只读存储器(read-only memory,ROM)、可编程只读存储器(programmable read-only memory,PROM)、可擦写可编程只读存储器(erasable programmable read-only memory,EPROM)、电可擦写可编程只读存储器(electrically erasable programmable read-only memory,EEPROM)、闪存、或者技术领域内所公知的任意其他形式的存储介质。The memory 101 can be used to store data, software programs, and modules, and mainly includes a program storage area and a data storage area. The storage program area can store an operating system and at least one application required by a function. The storage data area can store the data processing. Data created when the device is used. In the embodiment of the present application, the operating system may include a general-purpose operating system (such as an Android system) and a secure operating system (such as a dedicated operating system such as a Trustzone system); the applications required for the at least one function may include Security application software and non-security application software. Trusted application (TA) software can include secure artificial intelligence processor drivers and other applications with security features (for example, screen unlocking software based on the Trustzone system, payment software, etc.). Security application software is also called client application (CA), which can include non-secure artificial intelligence processor drivers and other applications with non-secure features (for example, Android-based web browsing software, video playback software, etc.); The data stored in the storage data area may include secure data, secure data processing results, non-secure data and / or non-secure data processing results, and the like. In some feasible embodiments, the software module stored in the memory 101 may include a memory management unit, and the storage data area may include memory. The memory management unit may be used to manage the memory. For example, the memory management unit may be used to set access to the memory. Permissions. In other feasible embodiments, there may be one memory or multiple memories; the memory may be a floppy disk, a hard disk such as a built-in hard disk and a mobile hard disk, a magnetic disk, an optical disk, and a magneto optical disk such as a read-only optical disk (compact disc-read- only memory (CD_ROM), digital video disc (read-only memory (DVD_ROM), non-volatile storage devices such as random access memory (RAM), read-only memory (ROM) , Programmable read-only memory (PROM), erasable programmable read-only memory (erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (electrically erasable programmable read-only memory) only memory (EEPROM), flash memory, or any other form of storage medium known in the art.
处理器102用于对该数据处理装置的动作进行控制管理,比如通过运行或执行存储在存储器101内的软件程序和/或模块,以及调用存储在存储器101内的数据,执行该数据处理装置的各种功能和处理数据。其中,处理器102可以包括中央处理单元、数字信号处理器、人工智能(artificial intelligence,AI)处理器、微处理器、或微控制 器等的至少一个,还可选择性包括专用集成电路、现场可编程门阵列或者其他可编程逻辑器件、晶体管逻辑器件、硬件加速器或者其任意组合。所述处理器也可以是实现计算功能的组合。在本申请实施例中,处理器102可以包括通用处理器1021和人工智能处理器的1022组合,并进一步可选择性包括其他硬件加速器,该组合可用于支持该装置执行本申请中数据处理的功能,数据处理的具体过程可以参见下述方法实施例中的描述。可选的,人工智能处理器的1022可以为神经网络处理单元(neural-network processing unit,NPU),或者图像处理单元(graphics processing unit,GPU)。通用处理器1021则可以包括中央处理单元、微处理器、或微控制器等运行软件的处理设备,该软件可包括操作系统、应用软件或驱动软件等。以上实施例中的数据处理装置是一个系统。可选地,处理器102本身也可以视为是个数据处理装置,在系统中作为运算核心实现数据处理能力。The processor 102 is configured to control and manage the actions of the data processing device, for example, by running or executing software programs and / or modules stored in the memory 101 and calling data stored in the memory 101 to execute the data processing device. Various functions and processing data. The processor 102 may include at least one of a central processing unit, a digital signal processor, an artificial intelligence (AI) processor, a microprocessor, or a microcontroller, and may optionally include an application specific integrated circuit, a field Programmable gate array or other programmable logic device, transistor logic device, hardware accelerator, or any combination thereof. The processor may also be a combination that implements computing functions. In the embodiment of the present application, the processor 102 may include a combination of a general-purpose processor 1021 and an artificial intelligence processor 1022, and may further optionally include other hardware accelerators, and the combination may be used to support the device to perform data processing functions in the present application. For the specific process of data processing, please refer to the description in the following method embodiments. Optionally, the artificial intelligence processor 1022 may be a neural-network processing unit (NPU), or an image processing unit (graphics processing unit, GPU). The general-purpose processor 1021 may include a processing device running software, such as a central processing unit, a microprocessor, or a microcontroller. The software may include an operating system, application software, or driver software. The data processing device in the above embodiments is a system. Optionally, the processor 102 itself can also be regarded as a data processing device, which functions as a computing core in the system to realize data processing capabilities.
通信接口103用于支持该数据处理装置与其他设备进行通信。总线104可以是外设部件互连标准(peripheral component interconnect,PCI)总线,或者扩展工业标准结构(extended industry standard architecture,EISA)总线等。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示,图1中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。The communication interface 103 is used to support the data processing apparatus to communicate with other equipment. The bus 104 may be a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used in FIG. 1, but it does not mean that there is only one bus or one type of bus.
图2为本申请实施例提供的一种通用处理器1021上运行的软件的架构示意图。其中,通用处理器1021可用于运行安全软件和非安全软件,通用处理器1021运行安全软件所生成的数据称为安全数据,运行非安全软件所生成的数据称为非安全数据。通用处理器1021可以包括至少一个处理单元,如ARM、MIPS或X86架构设备。安全软件的安全性需求高于非安全软件,或安全软件所形成的环境的安全性高于非安全软件。例如,安全软件和非安全软件间存在隔离,使得非安全软件不得随意访问安全软件所形成的环境。安全软件包括安全操作系统(operating system,OS)、或安全应用软件中的至少一项;安全操作系统可以是指能够满足一定安全技术要求(比如,访问控制、身份识别等)的操作系统,比如,安全操作系统可以是为保障AI技术中相关数据运算的安全性而建立的操作系统,例如可信区域(Trustzone)系统;安全应用软件可以是指需要保障安全性的应用软件,安全应用软件可以包括一个或者多个安全应用程序或安全驱动等,比如,安全驱动可以包括安全人工智能处理器驱动(简称安全AI驱动,用于在安全操作系统下驱动人工智能处理器1022),安全应用程序可以包括屏幕解锁软件和支付软件等,可以运行于Trustzone系统之上。非安全软件包括通用操作系统、非安全应用软件中的至少一项;通用操作系统可以是指没有特定的安全性要求的操作系统,比如,通用操作系统可以包括Android操作系统或Windows操作系统等;非安全应用软件可以是指不需要保障安全性要求的应用软件,非安全应用软件可以包括一个或者多个非安全应程序或非安全驱动等,比如,非安全驱动可以包括非安全人工智能处理器驱动(简称非安全AI驱动,用于在通用操作系统下驱动人工智能处理器1022),非安全应程序可以包括网页浏览软件、视频播放软件等。因此,安全软件和非安全软件之间存在隔离,例如,双方的运行指令可以互相隔离,互不影响对方的运行。在本发明实施例中所述安全软件的安全性高于非安全软件。例如,所述安全软件所形成的环境可以叫可信执行环境(Trusted Execution Environment,TEE),所述非 安全软件所形成的环境可以叫富执行环境(Rich Execut1n Environment,REE)。所述安全软件形成的环境的安全性高于非安全软件形成的环境。以上软件均可存储于存储器101中,并可以被互相隔离的存储,以便被处理器1021调用和执行。FIG. 2 is a schematic structural diagram of software running on a general-purpose processor 1021 according to an embodiment of the present application. Among them, the general-purpose processor 1021 can be used to run security software and non-secure software. The data generated by the general-purpose processor 1021 running the security software is called secure data, and the data generated by running the non-secure software is called non-secure data. The general-purpose processor 1021 may include at least one processing unit, such as an ARM, MIPS, or X86 architecture device. Security software has higher security requirements than non-secure software, or the environment formed by secure software is more secure than non-secure software. For example, the isolation between secure and non-secure software prevents non-secure software from randomly accessing the environment created by the secure software. The security software includes at least one of a security operating system (OS) or security application software; a security operating system may refer to an operating system capable of meeting certain security technical requirements (for example, access control, identification, etc.), such as The secure operating system may be an operating system established to ensure the security of related data operations in AI technology, such as a Trustzone system; the security application software may refer to application software that needs to ensure security, and the security application software may Including one or more security applications or security drivers, for example, the security driver may include a security artificial intelligence processor driver (referred to as a security AI driver for driving the artificial intelligence processor 1022 under a secure operating system), and the security application may Including screen unlock software and payment software, it can run on the Trustzone system. Non-secure software includes at least one of a general-purpose operating system and non-secure application software; a general-purpose operating system may refer to an operating system without specific security requirements, for example, a general-purpose operating system may include an Android operating system or a Windows operating system; Non-secure application software can refer to application software that does not require security requirements. Non-secure application software can include one or more non-secure applications or non-secure drivers. For example, non-secure drivers can include non-secure artificial intelligence processors. Drivers (referred to as non-secure AI drivers, used to drive the artificial intelligence processor 1022 under a general operating system), non-secure applications may include web browsing software, video playback software, and the like. Therefore, there is isolation between safety software and non-safety software. For example, the running instructions of both parties can be isolated from each other without affecting the operation of the other party. In the embodiment of the present invention, the security software is more secure than non-secure software. For example, the environment formed by the security software may be called a Trusted Execution Environment (TEE), and the environment formed by the non-secure software may be called a Rich Execution Environment (REE). The environment formed by the security software is more secure than the environment formed by non-secure software. The above software can be stored in the memory 101, and can be stored in isolation to be called and executed by the processor 1021.
在图2所示软件架构中,以通用处理器1021上运行的软件包括通用操作系统、安全操作系统、非安全人工智能处理器驱动(后续简称非安全AI驱动)和安全人工智能处理器驱动(后续简称安全AI驱动)为例进行说明。同时,图2中以通用处理器1021上运行的软件还包括非安全侧应用和安全侧应用为例进行说明。其中,非安全侧应用可以是指除非安全人工智能处理器驱动之外的其他非安全应用软件,比如网页浏览软件或视频播放软件等;安全侧应用可以是指除安全人工智能处理器驱动之外的其他安全应用软件,比如屏幕解锁软件或支付软件等。In the software architecture shown in FIG. 2, the software running on the general-purpose processor 1021 includes a general-purpose operating system, a secure operating system, a non-secure artificial intelligence processor driver (hereinafter referred to as a non-secure AI driver), and a secure artificial intelligence processor driver ( (Hereinafter referred to as security AI driver) as an example. Meanwhile, in FIG. 2, the software running on the general-purpose processor 1021 also includes non-safety-side applications and safety-side applications as an example for illustration. Among them, the non-secure application can refer to other non-secure application software except for the secure artificial intelligence processor driver, such as web browsing software or video playback software, etc. The secure side application can refer to the security artificial intelligence processor driver. Other security applications, such as screen unlock software or payment software.
在本申请实施例中,安全侧应用可以请求安全操作系统处理安全数据,安全操作系统在接收到安全侧应用的请求后,可以通过安全AI驱动调用人工智能处理器1022处理安全数据,并向安全侧应用返回安全数据处理结果。非安全侧应用可以请求通用操作系统处理非安全数据,通用操作系统在接收到非安全侧应用的请求后,可以通过非安全AI驱动调用人工智能处理器1022处理非安全数据,并向非安全侧应用返回非安全数据处理结果。数据处理的具体过程可以参见下述方法实施例中的描述。In the embodiment of the present application, the security-side application may request the security operating system to process the security data. After receiving the request of the security-side application, the security operating system may call the artificial intelligence processor 1022 through the security AI driver to process the security data and send the security data to the security. The side application returns the result of safe data processing. The non-secure application can request the general operating system to process the non-secure data. After receiving the request of the non-secure side application, the general operating system can call the artificial intelligence processor 1022 through the non-secure AI driver to process the non-secure data and send it to the non-secure side. The application returns non-secure data processing results. For the specific process of data processing, refer to the description in the following method embodiments.
进一步的,参见图3,安全操作系统可以选择性地包括模式引擎层、模型管家层和硬件适配层,通用操作系统可以选择性地包括模式引擎层、模型管家层、框架层和硬件适配层,安全操作系统中的模式引擎层可以与通用操作系统中的模式引擎层通信。Further, referring to FIG. 3, the security operating system may optionally include a pattern engine layer, a model steward layer, and a hardware adaptation layer, and a general operating system may selectively include a pattern engine layer, a model steward layer, a framework layer, and a hardware adaptation layer. Layer, the pattern engine layer in a secure operating system can communicate with the pattern engine layer in a general operating system.
其中,在安全操作系统中,模式引擎层主要用于接收安全侧应用的消息、处理模式切换等;模型管家层主要用于接收模式引擎层转发的安全侧应用的消息、创建安全侧应用对应的用户数据模型,并将用户数据模型和用户输入的数据等逐层传输给安全AI驱动;硬件适配层主要用于在安全操作系统下基于不同的硬件(比如,NPU或GPU等人工智能处理器)进行相应的适配。另外,在通用操作系统中,模式引擎层主要用于接收非安全侧应用的消息、处理模式切换等;模型管家层主要用于接收模式引擎层转发的非安全侧应用的消息、创建非安全侧应用对应的用户数据模型,并将用户数据模型和用户输入的数据等逐层传输给非安全AI驱动;框架层主要用于处理不同的人工智能框架,比如,深度开源学习框架等;硬件适配层主要用于在通用操作系统下基于不同的硬件(比如,NPU或GPU等人工智能处理器)进行相应的适配。关于通用操作系统更多可见于现有技术,如安卓系统的介绍。关于安全操作系统可见于现有技术,如Trustzone系统的介绍。Among them, in the security operating system, the pattern engine layer is mainly used to receive messages from the security side application and process mode switching; the model housekeeper layer is mainly used to receive messages from the security side application forwarded by the pattern engine layer, and to create corresponding security side applications. The user data model, and transmits the user data model and user input data to the security AI driver layer by layer; the hardware adaptation layer is mainly used under the secure operating system based on different hardware (such as artificial intelligence processors such as NPU or GPU) ) Perform the corresponding adaptation. In addition, in the general operating system, the pattern engine layer is mainly used to receive messages from non-secure side applications and process mode switching; the model housekeeper layer is mainly used to receive messages from non-secure side applications forwarded by the pattern engine layer and create non-secure side Apply the corresponding user data model and transfer the user data model and user input data to the non-secure AI driver layer by layer; the framework layer is mainly used to handle different artificial intelligence frameworks, such as deep open source learning frameworks; hardware adaptation The layer is mainly used for corresponding adaptation based on different hardware (for example, artificial intelligence processors such as NPU or GPU) under a general operating system. More about the general operating system can be found in the existing technology, such as the introduction of the Android system. The introduction of secure operating systems can be found in existing technologies, such as the Trustzone system.
图4为本申请实施例提供的一种数据处理方法的流程示意图,该方法可应用于图1所示的数据处理装置中,具体可以由图2所示的通用处理器1021通过调用人工智能处理器1022来执行,参见图4,该方法包括以下几个步骤。FIG. 4 is a schematic flowchart of a data processing method according to an embodiment of the present application. The method can be applied to the data processing device shown in FIG. 1, and can be specifically processed by the general-purpose processor 1021 shown in FIG. 2 by calling artificial intelligence. The processor 1022 executes the method. Referring to FIG. 4, the method includes the following steps.
S401:安全操作系统向通用操作系统发送第一指示信息,第一指示信息用于指示人工智能处理器进入安全模式。其中,在人工智能处理器处于非安全模式时,通用操作系统可以向非安全AI驱动发送非安全数据,非安全AI驱动可以调用人工智能处理器处理该非安全数据。在此过程中,当安全操作系统接收到安全侧应用发送的用于请求处理安全数据的请求信息时,安全操作系统可以向通用操作系统发送第一指示信息, 通过第一指示信息指示通用操作系统人工智能处理器进入安全模式。S401: The secure operating system sends first instruction information to the general operating system, and the first instruction information is used to instruct the artificial intelligence processor to enter a secure mode. Among them, when the artificial intelligence processor is in a non-secure mode, the general operating system can send non-secure data to the non-secure AI driver, and the non-secure AI driver can call the artificial intelligence processor to process the non-secure data. In this process, when the security operating system receives the request information sent by the security-side application for processing security data, the security operating system may send first instruction information to the general operating system, and the general operating system is instructed by the first instruction information. The artificial intelligence processor enters safe mode.
S402:通用操作系统在接收到第一指示信息时,通知非安全AI驱动人工智能处理器进入安全模式。当通用操作系统接收到第一指示信息时,通用操作系统可以通知非安全AI驱动人工智能处理器进入安全模式。此时,若非安全AI驱动处于调用人工智能处理器处理非安全数据的状态,则非安全AI驱动可以在当前非安全数据处理完成之后执行下述S403;若非安全AI驱动不处于调用人工智能处理器处理非安全数据的状态,比如,非安全AI驱动处于空闲状态,则非安全AI驱动可以直接执行下述S403。S402: When receiving the first instruction information, the general operating system notifies the non-secure AI to drive the artificial intelligence processor to enter the safe mode. When the general operating system receives the first instruction information, the general operating system may notify the non-secure AI to drive the artificial intelligence processor to enter the safe mode. At this time, if the non-secure AI driver is in the state of calling the artificial intelligence processor to process the non-secure data, the non-secure AI driver can execute the following S403 after the current non-secure data processing is completed; The state of processing non-secure data. For example, if the non-secure AI driver is in an idle state, the non-secure AI driver can directly execute the following S403.
进一步的,通用操作系统在通知非安全AI驱动人工智能处理器进入安全模式之前,若存在至少一个待处理的非安全数据,通用操作系统还可以设置下述S406中所述的安全数据被处理的优先级高于至少一个待处理的非安全数据,至少一个待处理的非安全数据是由非安全软件所生成且尚未被人工智能处理器处理。其中,至少一个待处理的非安全数据可以包括一个或者多个待处理的非安全数据,且至少一个待处理的非安全数据可以位于人工智能处理器的待处理队列中。通用操作系统设置所述安全数据被处理的优先级高于至少一个待处理的非安全数据,可以是指所述安全数据在所述待处理队列中的位置位于至少一个待处理的非安全数据之前。Further, before the general-purpose operating system notifies the non-secure AI to drive the artificial intelligence processor to enter the safe mode, if there is at least one non-secure data to be processed, the general-purpose operating system may also set the security data to be processed as described in S406 below. The priority is higher than at least one unsecured data to be processed. At least one unsecured data to be processed is generated by unsecured software and has not yet been processed by an artificial intelligence processor. The at least one to-be-processed non-secure data may include one or more to-be-processed non-secure data, and the at least one to-be-processed non-secure data may be located in a to-be-processed queue of the artificial intelligence processor. The general operating system sets the processing priority of the secure data higher than at least one unsecured data to be processed, which may mean that the position of the secure data in the pending queue is before the at least one unsecured data to be processed .
比如,通用操作系统在通知非安全AI驱动人工智能处理器进入安全模式之前,非安全AI驱动正在调用人工智能处理器处理非安全数据B1,所述待处理队列中还包括待处理的非安全数据B2、B3和B4,其处理顺序为B2-B3-B4,当所述安全数据为A1时,通用操作系统可以将A1插入到B2、B3和B4之前,即插入后的处理顺序为A1-B2-B3-B4。之后,通用处理器可以通知非安全AI驱动人工智能处理器进入安全模式,则非安全AI驱动可以在非安全数据B1处理完成后再执行下述S403。For example, before the general operating system notifies the non-secure AI-driven artificial intelligence processor to enter the safe mode, the non-secure AI driver is calling the artificial intelligence processor to process the non-secure data B1, and the pending queue further includes the unsecure data to be processed. B2, B3, and B4. The processing order is B2-B3-B4. When the security data is A1, the general operating system can insert A1 before B2, B3, and B4, that is, the processing order after insertion is A1-B2. -B3-B4. After that, the general-purpose processor may notify the non-secure AI-driven artificial intelligence processor to enter the safe mode, and then the non-secure AI-drive may execute the following S403 after the processing of the non-secure data B1 is completed.
S403:非安全AI驱动配置人工智能处理器从非安全模式切换为安全模式,即放弃对人工智能处理器的占用。S403: The non-safety AI-driven configuration artificial intelligence processor switches from the non-safety mode to the safe mode, that is, the occupation of the artificial intelligence processor is abandoned.
当非安全AI驱动在接收到通用操作系统的通知后、或者当非安全AI驱动接收到通用操作系统的通知且当前的非安全数据处理完成后,非安全AI驱动可以配置人工智能处理器从非安全模式切换为安全模式。具体的,非安全AI驱动可以去使能非安全AI驱动接收非安全侧中断的功能,该非安全侧中断是指由非安全软件触发的中断。其中,去使能非安全AI驱动接收非安全侧中断的功能可以是指关闭或者屏蔽非安全AI驱动接收非安全侧中断的功能,即在去使能该功能之后,非安全AI驱动不能够接收非安全侧中断,进而非安全AI驱动也不能够再调用人工智能处理器处理非安全数据,而在非安全AI驱动去使能该功能之前,非安全AI驱动能够接收该非安全侧中断,进而非安全AI驱动能够调用人工智能处理器处理非安全数据。When a non-secure AI driver receives a notification from a general-purpose operating system, or when a non-secure AI driver receives a notification from a general-purpose operating system and the current non-secure data processing is completed, the non-secure AI driver can configure the artificial intelligence processor to The safe mode is switched to the safe mode. Specifically, the non-safety AI driver can disable the function of the non-safety AI driver to receive the non-safety side interrupt. The non-safety side interrupt refers to an interrupt triggered by non-safety software. Among them, disabling the function that the non-safety AI driver receives the non-safety-side interrupt can refer to disabling or shielding the function that the non-safety AI driver receives the non-safety-side interrupt, that is, after the function is disabled, the non-safety AI driver cannot receive Non-safety side interrupts, and therefore non-safety AI drivers can no longer call the artificial intelligence processor to process non-secure data. Before the non-safety AI driver disables this function, the non-safety AI driver can receive the non-safety side interrupts, and Non-secure AI drivers can call artificial intelligence processors to process non-secure data.
S404:安全操作系统通知安全AI驱动人工智能处理器进入安全模式。其中,当运行在通用处理器1021上的安全侧应用请求安全操作系统处理安全数据时,该安全侧应用可以向安全操作系统发送用于请求进入安全模式的请求信息,安全操作系统在接收到来自安全侧应用的请求信息后,可以通知安全AI驱动人工智能处理器进入安全模式。S404: The security operating system notifies the security AI to drive the artificial intelligence processor into a security mode. Wherein, when a security-side application running on the general-purpose processor 1021 requests the security operating system to process security data, the security-side application may send request information for requesting to enter the security mode to the security operating system. After requesting information from the security side application, it can notify the security AI to drive the artificial intelligence processor into the security mode.
比如,安全侧应用为屏幕解锁软件,用户通过屏幕解锁软件进行手机屏幕解锁,当屏幕解锁软检测到用户触发的解锁操作时,屏幕解锁软件可以向安全操作系统发送 请求信息以请求进入安全模式,从而安全操作系统在接收到该请求信息后,可以向安全AI驱动发送通知消息,以通知安全AI驱动人工智能处理器进入安全模式。For example, the security side application is screen unlock software. The user unlocks the mobile phone screen through the screen unlock software. When the screen unlock software detects an unlock operation triggered by the user, the screen unlock software can send a request message to the secure operating system to request to enter the safe mode. Therefore, after receiving the request information, the secure operating system can send a notification message to the secure AI driver to notify the secure AI to drive the artificial intelligence processor to enter the secure mode.
S405:安全AI驱动配置人工智能处理器从非安全模式切换为安全模式,即占用人工智能处理器。在安全模式下,人工智能处理器无法处理由非安全软件所生成的非安全数据。其中,人工智能处理器默认可以设置为非安全模式,且在非安全模式下,人工智能处理器能够处理由非安全软件所生成的非安全数据。当安全AI驱动接收到安全操作系统的通知后,安全AI驱动可以配置人工智能处理器从非安全模式切换为安全模式,从而在人工智能处理器切换为安全模式时,人工智能处理器无法处理由非安全软件所生成的非安全数据。S405: The security AI driver configures the artificial intelligence processor to switch from the non-secure mode to the secure mode, that is, the artificial intelligence processor is occupied. In secure mode, artificial intelligence processors cannot process non-secure data generated by non-secure software. Among them, the artificial intelligence processor can be set to a non-secure mode by default, and in the non-secure mode, the artificial intelligence processor can process non-secure data generated by non-secure software. When the secure AI driver receives the notification from the secure operating system, the secure AI driver can configure the artificial intelligence processor to switch from the non-secure mode to the secure mode, so that when the artificial intelligence processor switches to the secure mode, the artificial intelligence processor cannot handle Non-secure data generated by non-secure software.
可选的,安全AI驱动配置人工智能处理器从非安全模式切换为安全模式可以包括如下至少一项:使能安全AI驱动接收安全侧中断的功能,安全侧中断是指由安全软件触发的中断;或者,设置内存管理单元为安全模式,在安全模式下非安全软件无法访问与该内存管理单元对应的内存。Optionally, the configuration of the security AI driver to switch the artificial intelligence processor from the non-secure mode to the security mode may include at least one of the following: the function of enabling the security AI driver to receive a security-side interrupt, the security-side interrupt refers to an interrupt triggered by security software Or, set the memory management unit to a safe mode, in which the non-secure software cannot access the memory corresponding to the memory management unit.
其中,使能安全AI驱动接收安全侧中断的功能可以是指开启或者打开安全AI驱动接收安全侧中断的功能,即在使能该功能之后,安全AI驱动可以接收到安全侧中断,该安全侧中断是安全侧应用在请求处理安全数据时触发的中断,而在安全AI驱动使能该功能之前,安全AI驱动无法接收该安全侧中断。另外,内存管理单元可以用于管理其对应的内存的访问权限,安全AI驱动设置内存管理单元为安全模式,即安全AI驱动通过内存管理单元设置该内存的访问权限为:安全应用软件能够访问该内存,非安全软件无法访问该内存。Among them, the function of enabling the safe AI driver to receive the interrupt on the safe side can refer to enabling or turning on the function of the safe AI driver to receive the interrupt on the safe side, that is, after the function is enabled, the safe AI driver can receive the interrupt on the safe side. An interrupt is an interrupt triggered by a security-side application when it requests processing of security data. Before the security AI driver enables this function, the security AI driver cannot receive the security-side interrupt. In addition, the memory management unit can be used to manage the access permissions of its corresponding memory. The security AI driver sets the memory management unit to a safe mode, that is, the security AI driver sets the memory access permission of the memory through the memory management unit as follows: the security application software can access the memory. Memory, which cannot be accessed by non-secure software.
S406:安全操作系统向安全AI驱动发送安全数据,该安全数据是由安全软件所生成的。在安全AI驱动配置人工智能处理器从非安全模式切换为安全模式之后,安全操作系统可以向安全AI驱动发送安全数据,该安全数据可以是指安全侧应用发送给安全操作系统的数据,也可以是指安全操作系统对安全侧应用发送的数据进行预处理之后的数据。安全侧应用可以在向安全操作系统发送数据时,可以通过一次传输或者多次传输将上述数据发送给安全操作系统。S406: The security operating system sends security data to the security AI driver, and the security data is generated by the security software. After the secure AI driver configures the artificial intelligence processor to switch from the non-secure mode to the secure mode, the secure operating system can send secure data to the secure AI driver. The secure data can refer to the data sent by the secure side application to the secure operating system. Refers to the data after the security operating system preprocesses the data sent by the security side application. When the security side application sends data to the secure operating system, it can send the above data to the secure operating system through one transmission or multiple transmissions.
其中,该安全数据可以包括用户输入数据和用户数据模型等,该用户输入数据可以是指当前输入的用户数据,用户数据模型可以是之前根据大量的用户数据通过模型训练得到的数据模型,如人工智能训练模型。该安全数据的具体信息与发送请求信息的安全侧应用相关,该安全数据中的用户数据模型可以是在用户首次使用该安全侧应用或者更新用户数据模型时,该安全侧应用通过采集大量的用户数据并进行模型训练得到的。The security data may include user input data and a user data model. The user input data may refer to the currently input user data. The user data model may be a data model previously obtained through model training based on a large amount of user data, such as human labor. Intelligent training model. The specific information of the security data is related to the security side application that sends the request information. The user data model in the security data may be when the user uses the security side application for the first time or updates the user data model, the security side application collects a large number of users. Data and model training.
比如,该数据处理装置为手机,该安全侧应用为屏幕解锁软件,该安全数据可以包括手机当前采集的用户图像和用户图像模型,该用户图像模型可以是包括多个图像特征的特征组合,该多个图像特征可以包括眼部特征、嘴部特征和脸部特征中的一种或者多种。或者,该安全侧应用为指纹支付软件,该安全数据可以包括手机当前采集的用户指纹和用户指纹模型,该用户指纹模型可以是包括多个指纹特征的特征组合,该多个指纹特征可以包括指纹分支点、指纹起始点和指纹终止点中的一种或者多种。For example, the data processing device is a mobile phone, and the security side application is screen unlocking software. The security data may include a user image and a user image model currently collected by the mobile phone. The user image model may be a feature combination including multiple image features. The plurality of image features may include one or more of an eye feature, a mouth feature, and a face feature. Alternatively, the security side application is fingerprint payment software, and the security data may include a user fingerprint and a user fingerprint model currently collected by the mobile phone. The user fingerprint model may be a feature combination including multiple fingerprint features, and the multiple fingerprint features may include fingerprints. One or more of a branch point, a fingerprint start point, and a fingerprint end point.
S407:安全AI驱动接收该安全数据,调用人工智能处理器处理该安全数据。S408: 安全AI驱动向安全操作系统发送安全数据处理结果。当安全AI驱动接收该安全数据时,安全AI驱动调用人工智能处理器处理该安全数据,即安全AI驱动可以将该安全数据发送给人工智能处理器处理,人工智能处理器在接收到该安全数据时,可以处理该安全数据并得到安全数据处理结果,之后人工智能处理器将安全数据处理结果发送给安全AI驱动,以使安全AI驱动向安全操作系统返回该安全数据处理结果。S407: The safety AI driver receives the safety data and calls an artificial intelligence processor to process the safety data. S408: The secure AI driver sends a secure data processing result to the secure operating system. When the safety AI driver receives the safety data, the safety AI driver calls the artificial intelligence processor to process the safety data, that is, the safety AI driver can send the safety data to the artificial intelligence processor for processing, and the artificial intelligence processor receives the safety data. At this time, the safety data can be processed and the safety data processing result can be obtained, and then the artificial intelligence processor sends the safety data processing result to the safety AI driver, so that the safety AI driver returns the safety data processing result to the safety operating system.
示例性的,以该安全数据包括用户输入图像和用户图像模型为例,则人工智能处理器处理该安全数据的具体过程可以为:人工智能处理器从用户输入图像中提取多个图像特征,提取的多个图像特征与用户图像模型包括的多个图像特征的类型一致;人工智能处理器将类型一致的两个图像特征进行匹配;若每个类型的两个图像特征均一致,则可以确定匹配成功,若存在一种或者多种类型的两个图像特征不一致,则可以确定匹配失败。之后,人工智能处理器可以将匹配成功或者匹配失败的结果发送给安全AI驱动,以使安全AI驱动向安全操作系统返回匹配成功或者匹配失败的结果。进一步的,安全操作系统可以将该结果发送给屏幕解锁软件,当该结果为匹配成功时屏幕解锁软件可以解锁手机屏幕,当该结果为匹配失败时屏幕解锁软件可以保持手机屏幕处于解锁状态。Exemplarily, taking the security data including a user input image and a user image model as an example, a specific process for the artificial intelligence processor to process the security data may be: the artificial intelligence processor extracts multiple image features from the user input image, and extracts Multiple image features of the same type as the multiple image features included in the user image model; the artificial intelligence processor matches two image features of the same type; if the two image features of each type are the same, a match can be determined Success. If one or more types of two image features are inconsistent, it can be determined that the matching has failed. After that, the artificial intelligence processor may send the result of the successful match or the failed match to the secure AI driver, so that the secure AI driver returns the result of the successful match or the failed match to the secure operating system. Further, the secure operating system may send the result to the screen unlocking software, and the screen unlocking software may unlock the mobile phone screen when the result is a successful match, and the screen unlocking software may keep the mobile phone screen in an unlocked state when the result is a matching failure.
进一步的,结合图4,参见图5,在S408之后,该方法还包括:S409-S415。图5中未示出上述S401-S408。Further, referring to FIG. 4 and referring to FIG. 5, after S408, the method further includes: S409-S415. The above S401-S408 are not shown in FIG.
S409:安全操作系统通知安全AI驱动人工智能处理器从安全模式切换为非安全模式。在所述安全数据处理完成之后、或者在没有新的安全数据需要被处理时,比如人工智能处理器的待处理任务队列中不存在任何安全数据对应的处理任务,安全操作系统可以通知安全AI驱动人工智能处理器从安全模式切换为非安全模式。其中,新的安全数据可以是由安全软件所生成且尚未被人工智能处理器处理。S409: The safe operating system notifies the safe AI-driven artificial intelligence processor to switch from the safe mode to the non-safe mode. After the security data processing is completed, or when no new security data needs to be processed, for example, there is no processing task corresponding to the security data in the pending task queue of the artificial intelligence processor, the security operating system may notify the security AI driver The artificial intelligence processor switches from secure mode to non-secure mode. The new security data may be generated by security software and not yet processed by the artificial intelligence processor.
S410:安全AI驱动配置人工智能处理器为非安全模式,即放弃对人工智能处理器的占用。其中,安全AI驱动配置人工智能处理器为非安全模式,也可以称为安全AI驱动配置人工智能处理器从安全模式切换为非安全模式,具体包括:去使能安全AI驱动接收安全侧中断的功能,该安全侧中断是指由安全软件触发的中断。在安全模式下,安全AI驱动可以接收安全侧中断,并基于该安全侧中断处理安全数据,当人工智能处理器进入非安全模式时,安全AI驱动可以关闭或者屏蔽安全AI驱动接收安全侧中断的功能,即在去使能该功能之后,安全AI驱动不能够接收安全侧中断,进而安全AI驱动也无法调用人工智能处理器处理安全数据。S410: The secure AI driver configures the artificial intelligence processor in a non-secure mode, that is, it renounces the occupation of the artificial intelligence processor. Among them, the security AI driver configures the artificial intelligence processor to a non-secure mode, which can also be referred to as the security AI driver configures the artificial intelligence processor to switch from the secure mode to the non-secure mode, which specifically includes: disabling the secure AI driver to receive interrupts from the security side. Function, the safety-side interrupt is the interrupt triggered by the safety software. In the safe mode, the safety AI driver can receive the safety-side interrupt and process the safety data based on the safety-side interrupt. When the artificial intelligence processor enters the non-safe mode, the safety AI driver can turn off or shield the safety AI driver from receiving the safety-side interrupt. Function, that is, after disabling the function, the safety AI driver cannot receive the safety-side interrupt, and the safety AI driver cannot call the artificial intelligence processor to process safety data.
S411:安全操作系统向通用操作系统发送第二指示信息,第二指示信息用于指示人工智能处理器从安全模式切换为非安全模式。S412:通用操作系统在接收到第二指示信息时,通知非安全AI驱动人工智能处理器进入非安全模式。S411: The secure operating system sends second instruction information to the general operating system, and the second instruction information is used to instruct the artificial intelligence processor to switch from the safe mode to the non-safe mode. S412: When receiving the second instruction information, the general operating system notifies the non-secure AI to drive the artificial intelligence processor to enter the non-secure mode.
S413:非安全AI驱动配置人工智能处理器为非安全模式,即占用人工智能处理器。非安全AI驱动配置人工智能处理器为非安全模式可以包括以下至少一项:使能非安全AI驱动接收非安全侧中断的功能,该非安全侧中断是指由非安全软件触发的中断;或者,设置内存管理单元为非安全模式,在非安全模式下非安全软件能够访问与该内存管理单元对应的内存。S413: The non-secure AI driver configures the artificial intelligence processor in a non-secure mode, that is, the artificial intelligence processor is occupied. The configuration of a non-secure AI driver with an artificial intelligence processor in a non-secure mode may include at least one of the following: enabling a non-secure AI driver to receive a non-secure side interrupt, the non-secure side interrupt refers to an interrupt triggered by non-secure software; or , Setting the memory management unit to a non-secure mode, in which the non-secure software can access the memory corresponding to the memory management unit.
其中,使能非安全AI驱动接收非安全侧中断的功能可以是指开启或者打开非安全 AI驱动接收非安全侧中断的功能,即在使能该功能之后,非安全AI驱动可以接收到非安全侧中断,从而非安全AI驱动也能够调用人工智能处理器处理非安全数据。另外,内存管理单元可以用于管理其对应的内存的访问权限,非安全AI驱动设置内存管理单元为非安全模式,即非安全AI驱动通过内存管理单元设置该内存的访问权限为:非安全应用软件能够访问该内存。Among them, enabling a non-safety AI driver to receive non-safety-side interrupts can refer to enabling or enabling a non-safety AI driver to receive non-safety-side interrupts. That is, after enabling this function, non-safety AI drivers can receive non-safety Side interrupts, so non-secure AI drivers can also call artificial intelligence processors to process non-secure data. In addition, the memory management unit can be used to manage the access permissions of its corresponding memory. The non-secure AI driver sets the memory management unit to the non-secure mode, that is, the non-secure AI driver sets the memory access permission of the memory through the memory management unit: non-secure applications. Software can access this memory.
S414:非安全AI驱动接收非安全数据,调用人工智能处理器处理非安全数据。S415:非安全AI驱动向通用操作系统发送非安全数据处理结果。在非安全AI驱动配置人工智能处理器为非安全模式之后,通用操作系统可以向非安全AI驱动发送非安全数据,该非安全数据可以是指非安全侧应用发送给通用操作系统的数据,也可以是指通用操作系统对非安全侧应用发送的数据进行预处理之后的数据。非安全侧应用可以通过一次传输或者多次传输将上述数据发送给通用操作系统。或者,该非安全数据可以是上一次人工智能处理器切换为安全模式时未处理的非安全数据,即在S413之前该非安全数据已位于人工智能处理器的待处理队列中,此时非安全AI驱动无需执行上述接收非安全数据的步骤,而是直接调用人工智能处理器处理该非安全数据。S414: The non-secure AI driver receives non-secure data, and calls an artificial intelligence processor to process the non-secure data. S415: The non-secure AI driver sends the result of non-secure data processing to the general operating system. After the non-secure AI driver configures the artificial intelligence processor in the non-secure mode, the general operating system can send non-secure data to the non-secure AI driver. The non-secure data can refer to the data sent by the non-secure side application to the general operating system. It may refer to data after a general operating system preprocesses data sent by a non-security-side application. The non-security side application can send the above data to the general operating system through one transmission or multiple transmissions. Alternatively, the non-secure data may be unprocessed data that was not processed when the artificial intelligence processor was switched to the secure mode last time, that is, the non-secure data was already in the pending queue of the artificial intelligence processor before S413. The AI driver does not need to perform the above steps for receiving non-secure data, but directly calls an artificial intelligence processor to process the non-secure data.
非安全AI驱动调用人工智能处理器处理该非安全数据的过程可以为:非安全AI驱动向人工智能处理器发送该非安全数据,人工智能处理器在接收到该非安全数据时,可以处理该非安全数据并得到非安全数据处理结果,之后人工智能处理器将非安全数据处理结果发送给非安全AI驱动,以使非安全AI驱动向通过操作系统返回该非安全数据处理结果。进一步地,当人工智能处理器的待处理队列中已经存在至少一个待处理的非安全数据时,人工智能处理器可以按照至少一个待处理的非安全数据在待处理队列中的排列顺序,依次处理至少一个待处理的非安全数据,并相应的向通过操作系统返回非安全数据处理结果。The process of the non-secure AI driver calling the artificial intelligence processor to process the non-secure data may be: the non-secure AI driver sends the non-secure data to the artificial intelligence processor, and when the artificial intelligence processor receives the non-secure data, it can process the non-secure data. The unsecured data obtains the unsecured data processing result, and then the artificial intelligence processor sends the unsecured data processing result to the unsecured AI driver, so that the unsecured AI driver returns the unsecured data processing result to the operating system. Further, when there is already at least one pending non-secure data in the pending queue of the artificial intelligence processor, the artificial intelligence processor may sequentially process the at least one pending non-secure data in the pending queue. At least one unsecured data to be processed, and the unsecured data processing result is returned to the operating system accordingly.
进一步的,参见图5,该方法还包括:S416-S417。其中,S416-S417与上述S401-S415可以不分先后顺序,图5中以S416-S417位于S415之后为例进行说明。Further, referring to FIG. 5, the method further includes: S416-S417. Among them, S416-S417 and the above S401-S415 may be in no particular order, and FIG. 5 uses S416-S417 after S415 as an example for description.
S416:当没有新非安全数据需要被处理的持续时间达到预设时间时,非安全AI驱动设置人工智能处理器为低功耗状态。其中,没有新非安全数据需要被处理可以是指当前的非安全数据已处理完成,且非安全AI驱动未接收到其他的非安全数据、或者人工智能处理器的待处理队列中不存在任何非安全数据。S416: When there is no new non-secure data that needs to be processed for a preset time, the non-secure AI driver sets the artificial intelligence processor to a low power state. Among them, no new non-secure data needs to be processed can mean that the current non-secure data has been processed, and the non-secure AI driver has not received other non-secure data, or there is no non-secure data in the pending queue of the artificial intelligence processor Safety data.
另外,非安全AI驱动设置人工智能处理器为低功耗状态可以包括对人工智能处理器执行下电操作或者调低将人工智能处理器的功耗。调低将人工智能处理器的功耗可以包括以下至少一项:调低人工智能处理器的电压、调低人工智能处理器的电流、调低人工智能处理器的时钟。In addition, setting the artificial intelligence processor to a low power state by a non-secure AI driver may include performing a power-off operation on the artificial intelligence processor or reducing the power consumption of the artificial intelligence processor. Lowering the power consumption of the artificial intelligence processor may include at least one of the following: reducing the voltage of the artificial intelligence processor, reducing the current of the artificial intelligence processor, and reducing the clock of the artificial intelligence processor.
需要说明的是,预设时间可以事先由本领域技术人员设置,且具体的数值大小可以根据实际情况进行设定,本申请实施例对此不做具体限定。It should be noted that the preset time can be set by those skilled in the art in advance, and the specific numerical value can be set according to the actual situation, which is not specifically limited in the embodiment of the present application.
S417:当需要处理新安全数据或者新非安全数据时,非安全AI驱动恢复人工智能处理器为工作状态。在人工智能处理器处于低功耗状态时,若安全AI驱动接收到安全操作系统发送的新安全数据、或者非安全AI驱动接收到通用操作系统发送的新非安全数据,非安全AI驱动恢复人工智能处理器为工作状态。S417: When new safety data or new non-safety data needs to be processed, the non-safety AI driver restores the artificial intelligence processor to the working state. When the artificial intelligence processor is in a low power consumption state, if the secure AI driver receives new secure data sent by the secure operating system, or the non-secure AI driver receives new unsecured data sent by the general operating system, the non-secure AI driver resumes manual work. The intelligent processor is in a working state.
具体的,当非安全AI驱动通过对人工智能处理器执行下电操作将其设置为低功耗 状态时,非安全AI驱动可以对人工智能处理器执行上电操作以将其恢复为工作状态;或者,当非安全AI驱动通过调低人工智能处理器的电压、电流和/或时钟将其设置为低功耗状态时,非安全AI驱动可以恢复人工智能处理器的电压、电流和/或时钟为工作值以将其恢复为工作状态。Specifically, when a non-secure AI driver sets the artificial intelligence processor to a low-power state by performing a power-off operation, the non-secure AI driver may perform a power-on operation on the artificial intelligence processor to restore it to a working state; Alternatively, when a non-secure AI driver sets the artificial intelligence processor's voltage, current, and / or clock to a low power state, the non-secure AI driver can restore the artificial intelligence processor's voltage, current, and / or clock Is working value to restore it to working state.
在本申请实施例中,非安全AI驱动通过在长时间没有新非安全数据需要被处理时,设置人工智能处理器为低功耗状态,以及在需要处理新安全数据或者新非安全数据时,恢复人工智能处理器为工作状态,可以在不影响数据处理过程的前提下,降低人工智能处理器的功耗,进而降低数据处理装置的功耗。In the embodiment of the present application, the non-secure AI driver sets the artificial intelligence processor to a low power consumption state when no new non-secure data needs to be processed for a long time, and when it needs to process new secure data or new non-secure data, Restoring the artificial intelligence processor to the working state can reduce the power consumption of the artificial intelligence processor without affecting the data processing process, thereby reducing the power consumption of the data processing device.
上述主要从数据处理装置的角度对本申请实施例提供的数据处理方法进行了介绍。可以理解的是,该数据处理装置为了实现上述功能,其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到,结合本文中所公开的实施例描述的各示例的结构及算法步骤,本申请能够以硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。The above mainly introduces the data processing method provided by the embodiment of the present application from the perspective of a data processing device. It can be understood that, in order to implement the above functions, the data processing apparatus includes a hardware structure and / or a software module corresponding to each function. Those skilled in the art should easily realize that the present application can be implemented in the form of a combination of hardware and computer software by combining the structures and algorithm steps of the examples described in the embodiments disclosed herein. Whether a certain function is performed by hardware or computer software-driven hardware depends on the specific application of the technical solution and design constraints. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
以上实施例涉及的数据处理装置是一个系统级装置。其内部的软件部分或硬件部分可以独立的分别具体用于实现数据处理的功能。图6示出了本实施例中所涉及的另一种数据处理装置的一种可能的结构示意图。该数据处理装置对应于图2的软件部分,可以运行于通用处理器1021上,该装置包括:安全操作系统301和安全AI驱动302。其中,安全操作系统301用于支持该装置执行方法实施例中的S401、S403、接收S405发送的安全数据处理结果的步骤、S406、S409以及S411中的一个或者多个,和/或上述方法实施例中所描述的其他技术过程;安全AI驱动302用于支持该装置执行方法实施例中的S402、S404、S405和S410中的一个或者多个步骤,和/或上述方法实施例中所描述的其他技术过程。进一步的,该数据处理装置还包括:通用操作系统303和非安全AI驱动304。其中,通用操作系统303用于支持该装置执行方法实施例中的接收S406发送的第一指示信息、S407、接收S411发送的第二指示信息、S412、以及接收S415发送的非安全数据处理结果中的一个或者多个步骤,和/或上述方法实施例中所描述的其他技术过程;非安全AI驱动304用于支持该装置执行方法实施例中的S408、S413、S414、S415、S416以及S417中的一个或者多个步骤,和/或上述方法实施例中所描述的其他技术过程。The data processing device involved in the above embodiment is a system-level device. Its internal software part or hardware part can be independently used to implement data processing functions. FIG. 6 shows a possible structure diagram of another data processing device involved in this embodiment. The data processing device corresponds to the software part of FIG. 2 and can run on the general-purpose processor 1021. The device includes a secure operating system 301 and a secure AI driver 302. The security operating system 301 is used to support the device to execute S401 and S403 in the method embodiment, receive one or more of the steps of the secure data processing result sent by S405, one or more of S406, S409, and S411, and / or implement the foregoing method. Other technical processes described in the examples; the secure AI driver 302 is used to support the device to perform one or more steps in S402, S404, S405, and S410 in the method embodiment, and / or described in the method embodiment above Other technical processes. Further, the data processing device further includes a general operating system 303 and a non-secure AI driver 304. Among them, the universal operating system 303 is used to support receiving the first instruction information sent by S406, S407, receiving the second instruction information sent by S411, S412, and receiving the non-secure data processing results sent by S415 in the method execution embodiment of the device. One or more steps, and / or other technical processes described in the above method embodiments; the non-secure AI driver 304 is used to support the device to execute S408, S413, S414, S415, S416, and S417 in the method embodiment One or more steps, and / or other technical processes described in the above method embodiments.
在一种可能的实现方式中,结合图1,当安全操作系统301、安全AI驱动302、通用操作系统303和非安全AI驱动304通过软件形式实现时,上述功能模块所对应的软件程序包括在存储器101中,通用处理器1021通过运行存储器101中包括的上述功能模块在内的软件程序以实现上述功能模块对应的功能。In a possible implementation manner, in conjunction with FIG. 1, when the secure operating system 301, secure AI driver 302, general operating system 303, and non-secure AI driver 304 are implemented in software, the software programs corresponding to the above functional modules are included in In the memory 101, the general-purpose processor 1021 implements functions corresponding to the above-mentioned function modules by running software programs including the above-mentioned function modules included in the memory 101.
下面从硬件处理的角度对本申请实施例中的另一种数据处理装置进行描述。本申请实施例还提供的该另一种数据处理装置,对应于图2的硬件部分,来为数据处理功能提供硬件算力,包括通用处理器1021和人工智能处理器1022,具体结构可以如图1所示。The following describes another data processing device in the embodiment of the present application from the perspective of hardware processing. The another data processing device provided in the embodiment of the present application corresponds to the hardware part of FIG. 2 to provide hardware computing power for data processing functions, including a general-purpose processor 1021 and an artificial intelligence processor 1022. The specific structure can be as shown in FIG. 1 is shown.
在本申请实施例中,通用处理器1021,用于设置人工智能处理器1022从非安全 模式切换为安全模式,并调用人工智能处理器1022处理安全数据;其中,安全数据是通用处理器1021运行安全软件所生成的;人工智能处理器1022,用于响应于通用处理器1021的设置从非安全模式切换为安全模式,以及响应于通用处理器1021的调用处理安全数据并返回安全数据处理结果给通用处理器1021;在安全模式下,人工智能处理器1022无法处理通用处理器1021运行非安全软件所生成的非安全数据。In the embodiment of the present application, the general-purpose processor 1021 is configured to set the artificial intelligence processor 1022 to switch from the non-secure mode to the secure mode, and call the artificial intelligence processor 1022 to process the safety data; wherein, the safety data is run by the general-purpose processor 1021. Generated by the security software; the artificial intelligence processor 1022 is used to switch from the non-secure mode to the secure mode in response to the setting of the general-purpose processor 1021, and to process the secure data in response to the call of the general-purpose processor 1021 and return the secure data processing result to General processor 1021; In a secure mode, the artificial intelligence processor 1022 cannot process non-secure data generated by the general processor 1021 running non-secure software.
在一种可能的实现方式中,在从非安全模式切换为安全模式时,通用处理器1021用于执行如下至少一项:去使能人工智能处理器1022处理非安全侧中断的功能,非安全侧中断是指由非安全软件触发的中断;或者,使能人工智能处理器1022处理安全侧中断的功能,安全侧中断是指由安全软件触发的中断;或者,设置内存管理单元为安全模式,在安全模式下非安全软件无法访问与内存管理单元对应的内存。In a possible implementation manner, when switching from the non-secure mode to the secure mode, the general-purpose processor 1021 is configured to perform at least one of the following: disable the function of the artificial intelligence processor 1022 to handle non-secure side interrupts, and non-secure Side interrupts refer to interrupts triggered by non-secure software; or, enable the artificial intelligence processor 1022 to handle the functions of security side interrupts, and security side interrupts refer to interrupts triggered by security software; or, set the memory management unit to safe mode, Non-secure software cannot access the memory corresponding to the memory management unit in safe mode.
在一种可能的实现方式中,在通用处理设置人工智能处理器1022从非安全模式切换为安全模式之前,通用处理器1021,还用于:设置安全数据被人工智能处理器1022处理的优先级高于非安全数据,非安全数据是由通用处理器1021运行非安全软件所生成且尚未被人工智能处理器1022处理。In a possible implementation manner, before the general-purpose processing sets the artificial intelligence processor 1022 to switch from the non-secure mode to the secure mode, the general-purpose processor 1021 is further configured to set a priority for processing the secure data by the artificial intelligence processor 1022. Higher than non-secure data, non-secure data is generated by the general-purpose processor 1021 running non-secure software and has not yet been processed by the artificial intelligence processor 1022.
在一种可能的实现方式中,通用处理器1021,还用于设置人工智能处理器1022从安全模式切换为非安全模式,并调用人工智能处理器1022处理非安全数据;人工智能处理器1022,用于响应于通用处理器1021的设置从安全模式切换为非安全模式,以及响应于通用处理器1021的调用处理非安全数据并返回非安全数据处理结果给通用处理器1021。可选的,通用处理器1021,具体用于在没有新安全数据需要被处理时,设置人工智能处理器1022从安全模式切换为非安全模式。其中,在非安全模式下,人工智能处理器1022无法处理安全数据。In a possible implementation manner, the general-purpose processor 1021 is further configured to set the artificial intelligence processor 1022 to switch from a secure mode to a non-secure mode, and call the artificial intelligence processor 1022 to process non-secure data; the artificial intelligence processor 1022, For switching from the secure mode to the non-secure mode in response to the setting of the general-purpose processor 1021, and for processing the non-secure data in response to the call of the general-purpose processor 1021 and returning the non-secure data processing result to the general-purpose processor 1021. Optionally, the general-purpose processor 1021 is specifically configured to set the artificial intelligence processor 1022 to switch from the secure mode to the non-secure mode when no new secure data needs to be processed. Among them, in a non-secure mode, the artificial intelligence processor 1022 cannot process secure data.
在一种可能的实现方式中,在从安全模式切换为非安全模式时,通用处理器1021用于执行如下至少一项:去使能人工智能处理器1022处理安全侧中断的功能,安全侧中断是指由安全软件触发的中断;或者,使能人工智能处理器1022处理非安全侧中断的功能,非安全侧中断是指由非安全软件触发的中断;或者,设置内存管理单元为非安全模式,在非安全模式下非安全软件能够访问与内存管理单元对应的内存。In a possible implementation manner, when switching from the secure mode to the non-secure mode, the general-purpose processor 1021 is configured to perform at least one of the following: disable the function of the artificial intelligence processor 1022 to process the security-side interrupt, and the security-side interrupt Refers to interrupts triggered by secure software; or enables the artificial intelligence processor 1022 to handle non-secure side interrupts. Non-secure side interrupts refer to interrupts triggered by non-secure software; or, sets the memory management unit to non-secure mode In non-secure mode, non-secure software can access the memory corresponding to the memory management unit.
在一种可能的实现方式中,通用处理器1021,还用于:当没有新非安全数据需要被处理的持续时间达到预设时间时,设置人工智能处理器1022为低功耗状态。进一步的,通用处理器1021,还用于:当需要处理新安全数据或者新非安全数据时,恢复人工智能处理器1022为工作状态。In a possible implementation manner, the general-purpose processor 1021 is further configured to set the artificial intelligence processor 1022 to a low power consumption state when there is no new non-secure data that needs to be processed for a preset time. Further, the general-purpose processor 1021 is further configured to restore the artificial intelligence processor 1022 to a working state when new security data or new non-security data needs to be processed.
在一种可能的实现方式中,人工智能处理器1022为神经网络处理单元NPU或者图像处理单元GPU。In a possible implementation manner, the artificial intelligence processor 1022 is a neural network processing unit NPU or an image processing unit GPU.
在一种可能的实现方式中,安全软件包括安全操作系统或安全应用软件的至少一项;非安全软件包括通用操作系统或非安全应用软件的至少一项。安全软件形成TEE,而非安全软件形成REE。In a possible implementation manner, the secure software includes at least one of a secure operating system or secure application software; the non-secure software includes at least one of a general-purpose operating system or non-secure application software. Security software forms TEE, while non-security software forms REE.
本申请实施例提供的上述数据处理装置的各组成部分分别用于实现相对应的前述数据处理方法的各步骤的功能,由于在前述的数据处理方法实施例中,已经对各步骤进行了详细说明,在此不再赘述。Each component of the foregoing data processing device provided in the embodiment of the present application is used to implement functions of the corresponding steps of the foregoing data processing method, because in the foregoing embodiment of the data processing method, each step has been described in detail , Will not repeat them here.
在本申请实施例中,人工智能处理器在被设置为安全模式时,人工智能处理器无 法处理通用处理器运行非安全软件所生成的非安全数据,而能够处理安全数据,从而保证了人工智能处理器处理安全数据时的安全性。In the embodiment of the present application, when the artificial intelligence processor is set to the safe mode, the artificial intelligence processor cannot process the non-secure data generated by the general-purpose processor running non-secure software, and can process the secure data, thereby ensuring the artificial intelligence. Security when the processor processes secure data.
本申请实施例还提供了一种计算机可读存储介质,该计算机可读存储介质中存储有指令,当其在一个设备(比如,该设备可以是单片机,芯片、计算机或处理器等)上运行时,使得该设备执行上述方法实施例中的一个或多个步骤。上述数据处理装置的各组成模块如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在所述计算机可读取存储介质中。An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores instructions, and when it is run on a device (for example, the device may be a microcontroller, a chip, a computer, or a processor, etc.) , The device is caused to perform one or more steps in the foregoing method embodiments. When each component module of the data processing device is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in the computer-readable storage medium.
基于这样的理解,本申请实施例还提供一种包含指令的计算机程序产品,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)或其中的处理器执行本申请各个实施例所述方法的全部或部分步骤。Based on this understanding, the embodiments of the present application further provide a computer program product containing instructions. The technical solution of the present application is essentially a part that contributes to the existing technology or all or part of the technical solution may be a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor therein to execute the embodiments of the present application. All or part of the steps of the method.
以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以所述权利要求的保护范围为准。The above is only a specific implementation of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application. It should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.
Claims (33)
- 一种数据处理装置,其特征在于,所述装置包括:通用处理器和人工智能处理器,其中,A data processing device, characterized in that the device includes a general-purpose processor and an artificial intelligence processor, wherein:所述通用处理器,用于设置所述人工智能处理器从非安全模式切换为安全模式,并调用所述人工智能处理器处理安全数据;其中,所述安全数据是所述通用处理器运行安全软件所生成的;The general-purpose processor is configured to set the artificial intelligence processor to switch from a non-secure mode to a secure mode, and call the artificial intelligence processor to process secure data; wherein the secure data is the general-purpose processor's operating security Generated by software;所述人工智能处理器,用于响应于所述通用处理器的设置从所述非安全模式切换为所述安全模式,以及响应于所述通用处理器的调用处理所述安全数据并返回安全数据处理结果给所述通用处理器;在所述安全模式下,所述人工智能处理器无法处理所述通用处理器运行非安全软件所生成的非安全数据。The artificial intelligence processor is configured to switch from the non-secure mode to the secure mode in response to a setting of the general-purpose processor, and process the secure data and return the secure data in response to a call of the general-purpose processor. The processing result is given to the general-purpose processor; in the secure mode, the artificial intelligence processor cannot process non-secure data generated by the general-purpose processor running non-secure software.
- 根据权利要求1所述的数据处理装置,其特征在于,在从所述非安全模式切换为所述安全模式时,所述通用处理器用于执行如下至少一项:The data processing device according to claim 1, wherein, when switching from the non-secure mode to the secure mode, the general-purpose processor is configured to execute at least one of the following:去使能所述人工智能处理器处理非安全侧中断的功能,所述非安全侧中断是指由所述非安全软件触发的中断;或者,Disable the function of the artificial intelligence processor to process non-safety-side interrupts, where the non-safety-side interrupts refer to interrupts triggered by the non-safety software; or,使能所述人工智能处理器处理安全侧中断的功能,所述安全侧中断是指由所述安全软件触发的中断;或者,Enable the function of the artificial intelligence processor to process a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software; or,设置内存管理单元为安全模式,在所述安全模式下所述非安全软件无法访问与所述内存管理单元对应的内存。The memory management unit is set to a secure mode, in which the non-secure software cannot access a memory corresponding to the memory management unit.
- 根据权利要求1或2所述的数据处理装置,其特征在于,在所述通用处理设置所述人工智能处理器从非安全模式切换为安全模式之前,所述通用处理器,还用于:The data processing device according to claim 1 or 2, wherein before the general-purpose processing sets the artificial intelligence processor to switch from a non-secure mode to a secure mode, the general-purpose processor is further configured to:设置所述安全数据被所述人工智能处理器处理的优先级高于所述非安全数据,所述非安全数据是由所述通用处理器运行非安全软件所生成且尚未被所述人工智能处理器处理。Setting the security data to be processed by the artificial intelligence processor with a higher priority than the non-secure data, which is generated by the general-purpose processor running non-secure software and has not been processed by the artificial intelligence器 处理。 Processing.
- 根据权利要求1-3任一项所述的数据处理装置,其特征在于,The data processing device according to any one of claims 1 to 3, wherein所述通用处理器,还用于设置所述人工智能处理器从所述安全模式切换为所述非安全模式,并调用所述人工智能处理器处理所述非安全数据;The general-purpose processor is further configured to set the artificial intelligence processor to switch from the secure mode to the non-secure mode, and call the artificial intelligence processor to process the non-secure data;所述人工智能处理器,用于响应于所述通用处理器的设置从所述安全模式切换为所述非安全模式,以及响应于所述通用处理器的调用处理所述非安全数据并返回非安全数据处理结果给所述通用处理器。The artificial intelligence processor is configured to switch from the secure mode to the non-secure mode in response to a setting of the general-purpose processor, and process the non-secure data and return non-secure data in response to a call of the general-purpose processor. The security data processing result is given to the general-purpose processor.
- 根据权利要求4所述的数据处理装置,其特征在于,在所述非安全模式下,所述人工智能处理器无法处理所述安全数据。The data processing device according to claim 4, wherein in the non-secure mode, the artificial intelligence processor cannot process the secure data.
- 根据权利要求4或5所述的数据处理装置,其特征在于,在从所述安全模式切换为所述非安全模式时,所述通用处理器用于执行如下至少一项:The data processing device according to claim 4 or 5, wherein, when switching from the secure mode to the non-secure mode, the general-purpose processor is configured to execute at least one of the following:去使能所述人工智能处理器处理安全侧中断的功能,所述安全侧中断是指由所述安全软件触发的中断;或者,Disabling the function of the artificial intelligence processor to process a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software; or,使能所述人工智能处理器处理非安全侧中断的功能,所述非安全侧中断是指由所述非安全软件触发的中断;或者,Enable the function of the artificial intelligence processor to process non-safety-side interrupts, where the non-safety-side interrupts refer to interrupts triggered by the non-safety software; or,设置内存管理单元为非安全模式,在所述非安全模式下所述非安全软件能够访问 与所述内存管理单元对应的内存。The memory management unit is set to a non-secure mode, in which the non-secure software can access a memory corresponding to the memory management unit.
- 根据权利要求4-6任一项所述的数据处理装置,其特征在于,所述通用处理器,具体用于在没有新安全数据需要被处理时,设置所述人工智能处理器从所述安全模式切换为所述非安全模式。The data processing device according to any one of claims 4 to 6, characterized in that the general-purpose processor is specifically configured to set the artificial intelligence processor from the security when no new security data needs to be processed. The mode is switched to the non-safe mode.
- 根据权利要求4-7任一项所述的数据处理装置,其特征在于,所述通用处理器,还用于:The data processing device according to any one of claims 4 to 7, wherein the general-purpose processor is further configured to:当没有新非安全数据需要被处理的持续时间达到预设时间时,设置所述人工智能处理器为低功耗状态。When the duration that no new non-secure data needs to be processed reaches a preset time, the artificial intelligence processor is set to a low power consumption state.
- 根据权利要求8所述的数据处理装置,其特征在于,所述通用处理器,还用于:The data processing device according to claim 8, wherein the general-purpose processor is further configured to:当需要处理新安全数据或者新非安全数据时,恢复所述人工智能处理器为工作状态。When new safety data or new non-safety data needs to be processed, the artificial intelligence processor is restored to a working state.
- 根据权利要求1-9任一项所述的数据处理装置,其特征在于,所述人工智能处理器为神经网络处理单元NPU或者图像处理单元GPU。The data processing device according to any one of claims 1-9, wherein the artificial intelligence processor is a neural network processing unit NPU or an image processing unit GPU.
- 根据权利要求1-10任一项所述的数据处理装置,其特征在于,所述安全软件包括安全操作系统或安全应用软件的至少一项;所述非安全软件包括通用操作系统或非安全应用软件的至少一项。The data processing device according to any one of claims 1 to 10, wherein the security software comprises at least one of a secure operating system or secure application software; and the non-secure software comprises a general-purpose operating system or non-secure application At least one item of software.
- 一种数据处理装置,其特征在于,所述装置包括:安全操作系统和安全人工智能处理器驱动;其中,A data processing device, characterized in that the device includes: a secure operating system and a secure artificial intelligence processor driver;所述安全操作系统,用于通知所述安全人工智能处理器驱动人工智能处理器进入安全模式;The secure operating system is used to notify the secure artificial intelligence processor to drive the artificial intelligence processor into a safe mode;所述安全人工智能处理器驱动,用于配置所述人工智能处理器从非安全模式切换为所述安全模式,在所述安全模式下,所述人工智能处理器无法处理由非安全软件所生成的非安全数据;The secure artificial intelligence processor driver is configured to switch the artificial intelligence processor from a non-secure mode to the secure mode, in which the artificial intelligence processor cannot process generated by non-secure software Non-secure data;所述安全操作系统,还用于向所述安全人工智能处理器驱动发送安全数据,所述安全数据是由安全软件所生成的;The security operating system is further configured to send security data to the security artificial intelligence processor driver, where the security data is generated by security software;所述安全人工智能处理器驱动,还用于接收所述安全数据,调用所述人工智能处理器处理所述安全数据,并返回安全数据处理结果给所述安全操作系统。The secure artificial intelligence processor driver is further configured to receive the secure data, call the artificial intelligence processor to process the secure data, and return a secure data processing result to the secure operating system.
- 根据权利要求12所述的数据处理装置,其特征在于,在配置所述人工智能处理器从非安全模式切换为所述安全模式时,所述安全人工智能处理器驱动用于执行如下至少一项:The data processing device according to claim 12, wherein when the artificial intelligence processor is configured to switch from a non-secure mode to the secure mode, the secure artificial intelligence processor driver is configured to execute at least one of the following :使能所述安全人工智能处理器驱动接收非安全侧中断的功能,所述安全侧中断是指由所述安全软件触发的中断;或者,Enable the function of the secure artificial intelligence processor to receive a non-safety-side interrupt, the safety-side interrupt refers to an interrupt triggered by the security software; or,设置内存管理单元为安全模式,在所述安全模式下所述非安全软件无法访问与所述内存管理单元对应的内存。The memory management unit is set to a secure mode, in which the non-secure software cannot access a memory corresponding to the memory management unit.
- 根据权利要求12或13所述的数据处理装置,其特征在于,所述装置还包括:通用操作系统和非安全人工智能处理器驱动,其中,The data processing device according to claim 12 or 13, wherein the device further comprises: a general-purpose operating system and a non-secure artificial intelligence processor driver, wherein:所述安全操作系统,还用于通知所述安全人工智能处理器驱动进入所述安全模式之前,向所述通用操作系统发送第一指示信息,所述第一指示信息用于指示所述人工智能处理器进入所述安全模式;The secure operating system is further configured to notify the secure artificial intelligence processor driver to enter the secure mode, and send first instruction information to the general operating system, where the first instruction information is used to instruct the artificial intelligence. The processor enters the security mode;所述通用操作系统,用于在接收到所述第一指示信息时,通知所述非安全人工智能处理器驱动所述人工智能处理器进入所述安全模式;The universal operating system is configured to, when receiving the first instruction information, notify the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the secure mode;所述非安全人工智能处理器驱动,用于配置所述人工智能处理器从非安全模式切换为所述安全模式。The non-secure artificial intelligence processor driver is configured to switch the artificial intelligence processor from a non-secure mode to the secure mode.
- 根据权利要求14所述的数据处理装置,其特征在于,在配置所述人工智能处理器从非安全模式切换为所述安全模式时,所述非安全人工智能处理器驱动用于执行如下操作:The data processing device according to claim 14, wherein when the artificial intelligence processor is configured to switch from a non-secure mode to the secure mode, the non-secure artificial intelligence processor driver is configured to perform the following operations:去使能所述非安全人工智能处理器驱动接收非安全侧中断的功能,所述非安全侧中断是指由所述非安全软件触发的中断。Disable the function of the non-secure artificial intelligence processor driver to receive non-secure side interrupts, where the non-secure side interrupts refer to interrupts triggered by the non-secure software.
- 根据权利要求14或15所述的数据处理装置,其特征在于,在通知所述非安全人工智能处理器驱动所述人工智能处理器进入所述安全模式之前,所述通用操作系统,还用于:The data processing device according to claim 14 or 15, wherein before the non-secure artificial intelligence processor is notified to drive the artificial intelligence processor to enter the secure mode, the universal operating system is further configured to: :设置所述安全数据被处理的优先级高于所述非安全数据,所述非安全数据是由所述非安全软件所生成且尚未被所述人工智能处理器处理。Setting the security data to be processed has a higher priority than the non-secure data, which is generated by the non-secure software and has not yet been processed by the artificial intelligence processor.
- 根据权利要求13-16任一项所述的数据处理装置,其特征在于,The data processing device according to any one of claims 13 to 16, wherein:所述安全操作系统,还用于通知所述安全人工智能处理器驱动所述人工智能处理器从所述安全模式切换为所述非安全模式;The secure operating system is further configured to notify the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode;所述安全人工智能处理器驱动,还用于配置所述人工智能处理器为所述非安全模式;The secure artificial intelligence processor driver is further configured to configure the artificial intelligence processor to the non-secure mode;所述安全操作系统,还用于向所述通用操作系统发送第二指示信息,所述第二指示信息用于指示所述人工智能处理器从所述安全模式切换为所述非安全模式;The secure operating system is further configured to send second instruction information to the general operating system, where the second instruction information is used to instruct the artificial intelligence processor to switch from the secure mode to the non-secure mode;所述通用操作系统,还用于在接收到所述第二指示信息时,通知所述非安全人工智能处理器驱动所述人工智能处理器进入所述非安全模式;The universal operating system is further configured to notify the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the non-secure mode when the second instruction information is received;所述非安全人工智能处理器驱动,还用于配置所述人工智能处理器为所述非安全模式,以及接收非安全数据,调用所述人工智能处理器处理所述非安全数据,并返回非安全数据处理结果给所述通用操作系统。The non-secure artificial intelligence processor driver is further configured to configure the artificial intelligence processor to the non-secure mode, and receive non-secure data, call the artificial intelligence processor to process the non-secure data, and return non-secure data. The result of the secure data processing is given to the general operating system.
- 根据权利要求17所述的数据处理装置,其特征在于,在配置所述人工智能处理器为所述非安全模式时,The data processing device according to claim 17, wherein, when the artificial intelligence processor is configured to the non-secure mode,所述安全人工智能处理器驱动用于执行如下操作:The secure artificial intelligence processor driver is configured to perform the following operations:去使能所述安全人工智能处理器驱动接收安全侧中断的功能,所述安全侧中断是指由所述安全软件触发的中断;Disabling the function of the security artificial intelligence processor driving receiving a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software;和,所述非安全人工智能处理器驱动用于执行如下至少一项:And, the non-secure artificial intelligence processor driver is configured to execute at least one of the following:使能所述非安全人工智能处理器驱动接收非安全侧中断的功能,所述非安全侧中断是指由所述非安全软件触发的中断;或者,Enable the function of the non-safety artificial intelligence processor to receive the non-safety side interrupt, the non-safety side interrupt refers to an interrupt triggered by the non-safety software; or,设置内存管理单元为非安全模式,在所述非安全模式下所述非安全软件能够访问与所述内存管理单元对应的内存。The memory management unit is set to a non-secure mode, in which the non-secure software can access a memory corresponding to the memory management unit.
- 根据权利要求17或18所述的数据处理装置,其特征在于,所述安全操作系统,具体用于在没有新安全数据需要被处理时,通知所述安全人工智能处理器驱动所述人工智能处理器从所述安全模式切换为所述非安全模式。The data processing device according to claim 17 or 18, wherein the secure operating system is specifically configured to notify the secure artificial intelligence processor to drive the artificial intelligence processing when there is no new secure data to be processed The switch from the safe mode to the non-safe mode.
- 根据权利要求17-19任一项所述的数据处理装置,其特征在于,所述非安全人工智能处理器驱动,还用于:The data processing device according to any one of claims 17 to 19, wherein the non-secure artificial intelligence processor driver is further configured to:当没有新非安全数据需要被处理的持续时间达到预设时间时,设置所述人工智能处理器为低功耗状态。When the duration that no new non-secure data needs to be processed reaches a preset time, the artificial intelligence processor is set to a low power consumption state.
- 根据权利要求20所述的数据处理装置,其特征在于,所述非安全人工智能处理器驱动,还用于:The data processing device according to claim 20, wherein the non-secure artificial intelligence processor driver is further configured to:当需要处理新安全数据或者新非安全数据时,恢复所述人工智能处理器为工作状态。When new safety data or new non-safety data needs to be processed, the artificial intelligence processor is restored to a working state.
- 根据权利要求12-21任一项所述的数据处理装置,其特征在于,所述安全软件包括安全操作系统或安全应用软件的至少一项;所述非安全软件包括通用操作系统或非安全应用软件的至少一项。The data processing device according to any one of claims 12 to 21, wherein the security software comprises at least one of a secure operating system or secure application software; and the non-secure software comprises a general-purpose operating system or non-secure application At least one item of software.
- 一种数据处理方法,其特征在于,所述方法包括:A data processing method, characterized in that the method includes:安全操作系统通知安全人工智能处理器驱动人工智能处理器进入安全模式;The secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor into a safe mode;所述安全人工智能处理器驱动配置所述人工智能处理器从非安全模式切换为所述安全模式,在所述安全模式下,所述人工智能处理器无法处理由非安全软件所生成的非安全数据;The secure artificial intelligence processor driver configures the artificial intelligence processor to switch from a non-secure mode to the secure mode. In the secure mode, the artificial intelligence processor cannot handle non-secure software generated by non-secure software. data;所述安全操作系统向所述安全人工智能处理器驱动发送安全数据,所述安全数据是由安全软件所生成的;The security operating system sends security data to the security artificial intelligence processor driver, the security data is generated by security software;所述安全人工智能处理器驱动接收所述安全数据,调用所述人工智能处理器处理所述安全数据,并返回安全数据处理结果给所述安全操作系统。The secure artificial intelligence processor driver receives the secure data, calls the artificial intelligence processor to process the secure data, and returns a secure data processing result to the secure operating system.
- 根据权利要求23所述的数据处理方法,其特征在于,所述安全人工智能处理器驱动配置所述人工智能处理器从非安全模式切换为所述安全模式,包括如下至少一项:The data processing method according to claim 23, wherein the driving of the secure artificial intelligence processor to configure the artificial intelligence processor to switch from a non-secure mode to the secure mode comprises at least one of the following:使能所述安全人工智能处理器驱动接收安全侧中断的功能,所述安全侧中断是指由所述安全软件触发的中断;或者,Enable the function of the security artificial intelligence processor to receive a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software; or,设置内存管理单元为安全模式,在所述安全模式下所述非安全软件无法访问与所述内存管理单元对应的内存。The memory management unit is set to a secure mode, in which the non-secure software cannot access a memory corresponding to the memory management unit.
- 根据权利要求23或24所述的数据处理方法,其特征在于,所述安全操作系统通知安全人工智能处理器驱动进入所述安全模式之前,所述方法还包括:The data processing method according to claim 23 or 24, wherein before the secure operating system notifies a secure artificial intelligence processor driver to enter the secure mode, the method further comprises:所述安全操作系统向通用操作系统发送第一指示信息,所述第一指示信息用于指示所述人工智能处理器进入所述安全模式;The secure operating system sends first instruction information to a general operating system, where the first instruction information is used to instruct the artificial intelligence processor to enter the secure mode;所述通用操作系统在接收到所述第一指示信息时,通知非安全人工智能处理器驱动所述人工智能处理器进入所述安全模式;When the universal operating system receives the first instruction information, notify the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the secure mode;所述非安全人工智能处理器驱动配置所述人工智能处理器从非安全模式切换为所述安全模式。The non-secure artificial intelligence processor driver configures the artificial intelligence processor to switch from a non-secure mode to the secure mode.
- 根据权利要求25所述的数据处理方法,其特征在于,所述非安全人工智能处理器驱动配置所述人工智能处理器从非安全模式切换为所述安全模式,包括:The data processing method according to claim 25, wherein the driving of the non-secure artificial intelligence processor to configure the artificial intelligence processor to switch from the non-secure mode to the secure mode comprises:去使能所述非安全人工智能处理器驱动接收非安全侧中断的功能,所述非安全侧中断是指由所述非安全软件触发的中断。Disable the function of the non-secure artificial intelligence processor driver to receive non-secure side interrupts, where the non-secure side interrupts refer to interrupts triggered by the non-secure software.
- 根据权利要求25或26所述的数据处理方法,其特征在于,所述通用操作系统通知非安全人工智能处理器驱动所述人工智能处理器进入所述安全模式之前,所述方法还包括:The data processing method according to claim 25 or 26, wherein before the general operating system notifies a non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the secure mode, the method further comprises:所述通用处理系统设置所述安全数据被处理的优先级高于所述非安全数据,所述非安全数据是由所述非安全软件所生成且尚未被所述人工智能处理器处理。The general processing system sets the priority of processing the secure data higher than the non-secure data, which is generated by the non-secure software and has not yet been processed by the artificial intelligence processor.
- 根据权利要求24-27任一项所述的数据处理方法,其特征在于,所述方法还包括:The data processing method according to any one of claims 24-27, wherein the method further comprises:所述安全操作系统通知所述安全人工智能处理器驱动所述人工智能处理器从所述安全模式切换为所述非安全模式;The secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode;所述安全人工智能处理器驱动配置所述人工智能处理器为所述非安全模式;The secure artificial intelligence processor driver configures the artificial intelligence processor as the non-secure mode;所述安全操作系统向所述通用操作系统发送第二指示信息,所述第二指示信息用于指示所述人工智能处理器从所述安全模式切换为所述非安全模式;Sending, by the secure operating system, the second instruction information to the general operating system, where the second instruction information is used to instruct the artificial intelligence processor to switch from the secure mode to the non-secure mode;所述通用操作系统在接收到所述第二指示信息时,通知所述非安全人工智能处理器驱动所述人工智能处理器进入所述非安全模式;When the universal operating system receives the second instruction information, notify the non-secure artificial intelligence processor to drive the artificial intelligence processor to enter the non-secure mode;所述非安全人工智能处理器驱动配置所述人工智能处理器为所述非安全模式,以及接收非安全数据,调用所述人工智能处理器处理所述非安全数据,并返回非安全数据处理结果给所述通用操作系统。The non-secure artificial intelligence processor driver configures the artificial intelligence processor as the non-secure mode, and receives non-secure data, calls the artificial intelligence processor to process the non-secure data, and returns an unsecure data processing result. To the universal operating system.
- 根据权利要求28所述的数据处理方法,其特征在于,所述安全人工智能处理器驱动配置所述人工智能处理器为所述非安全模式,包括:The data processing method according to claim 28, wherein the configuration of the secure artificial intelligence processor to drive the artificial intelligence processor into the non-secure mode comprises:去使能所述安全人工智能处理器驱动接收安全侧中断的功能,所述安全侧中断是指由所述安全软件触发的中断;Disabling the function of the security artificial intelligence processor driving receiving a security-side interrupt, where the security-side interrupt refers to an interrupt triggered by the security software;和,所述非安全人工智能处理器驱动配置所述人工智能处理器为所述非安全模式,包括如下至少一项:And, the non-secure artificial intelligence processor driver configuring the artificial intelligence processor to be in the non-secure mode includes at least one of the following:使能所述非安全人工智能处理器驱动接收非安全侧中断的功能,所述非安全侧中断是指由所述非安全软件触发的中断;或者,Enable the function of the non-safety artificial intelligence processor to receive the non-safety side interrupt, the non-safety side interrupt refers to an interrupt triggered by the non-safety software; or,设置内存管理单元为非安全模式,在所述非安全模式下所述非安全软件能够访问与所述内存管理单元对应的内存。The memory management unit is set to a non-secure mode, in which the non-secure software can access a memory corresponding to the memory management unit.
- 根据权利要求28或29所述的数据处理方法,其特征在于,所述安全操作系统通知所述安全人工智能处理器驱动所述人工智能处理器从所述安全模式切换为所述非安全模式,包括:The data processing method according to claim 28 or 29, wherein the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode, include:所述安全操作系统在没有新安全数据需要被处理时,通知所述安全人工智能处理器驱动所述人工智能处理器从所述安全模式切换为所述非安全模式。When there is no new secure data to be processed, the secure operating system notifies the secure artificial intelligence processor to drive the artificial intelligence processor to switch from the secure mode to the non-secure mode.
- 根据权利要求28-30任一项所述的数据处理方法,其特征在于,所述方法还包括:The data processing method according to any one of claims 28 to 30, wherein the method further comprises:当没有新非安全数据需要被处理的持续时间达到预设时间时,所述非安全人工智能处理器驱动设置所述人工智能处理器为低功耗状态。When there is no new non-secure data that needs to be processed for a preset time, the non-secure artificial intelligence processor driver sets the artificial intelligence processor to a low power consumption state.
- 根据权利要求31所述的数据处理方法,其特征在于,所述方法还包括:The data processing method according to claim 31, wherein the method further comprises:当需要处理新安全数据或者新非安全数据时,所述非安全人工智能处理器驱动恢复人工智能处理器为工作状态。When new secure data or new non-secure data needs to be processed, the non-secure artificial intelligence processor drives and restores the artificial intelligence processor to a working state.
- 根据权利要求23-32任一项所述的数据处理方法,其特征在于,所述安全软件包括安全操作系统或安全应用软件的至少一项;所述非安全软件包括通用操作系统或非安全应用软件的至少一项。The data processing method according to any one of claims 23 to 32, wherein the security software comprises at least one of a secure operating system or secure application software; and the non-secure software comprises a general operating system or non-secure application At least one item of software.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2018/100305 WO2020034076A1 (en) | 2018-08-13 | 2018-08-13 | Data processing method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2018/100305 WO2020034076A1 (en) | 2018-08-13 | 2018-08-13 | Data processing method and device |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020034076A1 true WO2020034076A1 (en) | 2020-02-20 |
Family
ID=69524939
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2018/100305 WO2020034076A1 (en) | 2018-08-13 | 2018-08-13 | Data processing method and device |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2020034076A1 (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106027257A (en) * | 2016-05-05 | 2016-10-12 | 北京元心科技有限公司 | Method and system for securely performing identity authentication |
US20170325088A1 (en) * | 2016-05-05 | 2017-11-09 | Qualcomm Incorporated | Securing sensor status by leveraging always-on processor and host-based trusted execution |
CN107980134A (en) * | 2017-08-10 | 2018-05-01 | 福建联迪商用设备有限公司 | The method and its system of information security of intelligent terminal input |
-
2018
- 2018-08-13 WO PCT/CN2018/100305 patent/WO2020034076A1/en active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106027257A (en) * | 2016-05-05 | 2016-10-12 | 北京元心科技有限公司 | Method and system for securely performing identity authentication |
US20170325088A1 (en) * | 2016-05-05 | 2017-11-09 | Qualcomm Incorporated | Securing sensor status by leveraging always-on processor and host-based trusted execution |
CN107980134A (en) * | 2017-08-10 | 2018-05-01 | 福建联迪商用设备有限公司 | The method and its system of information security of intelligent terminal input |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20220091998A1 (en) | Technologies for secure device configuration and management | |
CN105683981B (en) | A kind of safety interacting method and equipment | |
EP3326104B1 (en) | Technologies for secure trusted i/o access control | |
TWI648651B (en) | Equipment for processing safety information, single-chip system and method for realizing safety operating system switching | |
US10591975B2 (en) | Memory access management for low-power use cases of a system on chip via secure non-volatile random access memory | |
JP5153887B2 (en) | Method and apparatus for transfer of secure operating mode access privileges from a processor to a peripheral device | |
WO2019104988A1 (en) | Plc security processing unit and bus arbitration method thereof | |
TW200836064A (en) | Protection agents and privilege modes | |
CN107017014B (en) | Dynamic containerized system memory protection for low energy MCU | |
US20210089684A1 (en) | Controlled access to data stored in a secure partition | |
EP2746981A1 (en) | Trusted execution environment access control rules derivation | |
CN108154032B (en) | Computer system trust root construction method with memory integrity guarantee function | |
US20140096270A1 (en) | Secure data containers and data access control | |
US20180285578A1 (en) | Temporally isolating data accessed by a computing device | |
US20180046399A1 (en) | Storage Partitioning Method and Terminal | |
CN103457974A (en) | Safety control method and device for virtual machine mirror images | |
WO2017161569A1 (en) | Access control method, apparatus and system | |
IL266078A (en) | A system and method for securing electronic devices | |
CN112417470A (en) | Method and device for realizing GPU data security access, electronic equipment and storage medium | |
DE102018126136A1 (en) | Biometric authentication technologies before booting | |
CN112749397A (en) | System and method | |
CN111949596A (en) | MPK technology-based communication method, system and medium between microkernel modules | |
CN103037153A (en) | Monitoring method based on camera and electronic device with camera | |
WO2020034076A1 (en) | Data processing method and device | |
CN110765471B (en) | Working method of microcontroller-based access capability embedded platform |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18930253 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18930253 Country of ref document: EP Kind code of ref document: A1 |