WO2023082860A1 - 数据处理芯片、模组、终端及电源管理方法 - Google Patents

数据处理芯片、模组、终端及电源管理方法 Download PDF

Info

Publication number
WO2023082860A1
WO2023082860A1 PCT/CN2022/120651 CN2022120651W WO2023082860A1 WO 2023082860 A1 WO2023082860 A1 WO 2023082860A1 CN 2022120651 W CN2022120651 W CN 2022120651W WO 2023082860 A1 WO2023082860 A1 WO 2023082860A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
unit
data processing
cache unit
state
Prior art date
Application number
PCT/CN2022/120651
Other languages
English (en)
French (fr)
Inventor
祝叶华
姜坤
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2023082860A1 publication Critical patent/WO2023082860A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3234—Power saving characterised by the action undertaken
    • G06F1/325—Power saving in peripheral device
    • G06F1/3275—Power saving in memory, e.g. RAM, cache
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/26—Power supply means, e.g. regulation thereof
    • G06F1/32—Means for saving power
    • G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
    • G06F1/3234—Power saving characterised by the action undertaken
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the embodiments of the present application relate to the technical field of artificial intelligence, and in particular to a data processing chip, a module, a terminal, and a power management method.
  • the terminal can run in a low power consumption mode when the user does not use the terminal.
  • the terminal powers off some unused modules, and maintains power supply for some modules that require an online mode (Always-On, AON).
  • the terminal maintains power supply to modules such as a Wireless Fidelity (Wi-Fi) module and an identity verification module, but cuts off power to modules such as a video playback module and a screen brightness module.
  • Wi-Fi Wireless Fidelity
  • Embodiments of the present application provide a data processing chip, a module, a terminal, and a power management method, which can reduce energy consumption of the terminal. Described technical scheme is as follows:
  • a data processing chip including: a power management unit, a first cache unit, a second cache unit, and a data processing unit;
  • the power management unit is used to control power supply to the first cache unit and the second cache unit;
  • the first cache unit and the second cache unit are used to cache data
  • the power management unit is configured to control power supply to the first cache unit in response to the first state of the data processing unit, not to supply power to the second cache unit, and respond to the second state of the data processing unit , controlling to supply power to the first cache unit and to supply power to the second cache unit.
  • a data processing module in another aspect, includes the data processing chip as described in the above aspect.
  • a terminal in another aspect, includes the data processing module as described in the above aspect.
  • a power management method is provided, which is applied to a data processing chip, and the data processing chip includes a first cache unit, a second cache unit, and a data processing unit, and the first cache unit and the second cache The unit is used to cache data, and the power management method includes:
  • a computer-readable storage medium stores at least one piece of program code, and the at least one piece of program code is used to be executed by a processor to implement power management as described in the above aspect method.
  • a computer program product stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the power management method as described in the above aspect.
  • the power management unit in both the first state and the second state of the data processing unit, maintains power supply to the first cache unit, and only in the second state of the data processing unit, it supplies power to the second cache unit.
  • Unit power supply so that in the AON mode, only the first buffer unit needs to be powered when it is in the first state, thereby reducing the number of power supply units and reducing the power consumption in the AON mode.
  • FIG. 1 shows a schematic structural diagram of a data processing chip provided by an exemplary embodiment of the present application
  • Fig. 2 shows a schematic structural diagram of a data processing chip provided by an exemplary embodiment of the present application
  • Fig. 3 shows a schematic structural diagram of a data processing chip provided by an exemplary embodiment of the present application
  • FIG. 4 shows a schematic structural diagram of a data packet provided by an exemplary embodiment of the present application
  • Fig. 5 shows a schematic structural diagram of a data processing chip provided by an exemplary embodiment of the present application
  • FIG. 6 shows a flowchart of a power management method provided by an exemplary embodiment of the present application
  • Fig. 7 shows a structural block diagram of a terminal provided by an embodiment of the present application.
  • the "plurality” mentioned herein means two or more.
  • “And/or” describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently.
  • the character “/” generally indicates that the contextual objects are an "or” relationship.
  • the image data and other data involved in this application are data authorized by the user or fully authorized by all parties.
  • a data processing chip including: a power management unit, a first cache unit, a second cache unit, and a data processing unit;
  • the power management unit is used to control power supply to the first cache unit and the second cache unit;
  • the first cache unit and the second cache unit are used to cache data
  • the power management unit is used to control the power supply to the first cache unit in response to the first state of the data processing unit, not to supply power to the second cache unit, and control the power supply to the second cache unit in response to the second state of the data processing unit. Power is supplied to a cache unit and power is supplied to the second cache unit.
  • the first state of the data processing unit includes a state of not processing data
  • the second state of the data processing unit includes a state of processing data
  • the data processing unit is used to process data using the neural network model in the second state
  • the first cache unit is used to store the data of the neural network model
  • the second cache unit is used to store the data of the neural network model. Data to be processed, intermediate processed data and processed result data of the data processing unit.
  • the data processing chip further includes a data transmission interface for receiving data
  • the data processing unit is configured to switch from the first state to the second state in response to the data received by the data transmission interface containing specific information.
  • the specific information includes at least one of the following: motion detection information, humanoid detection information.
  • the first cache unit and the second cache unit are different units of the same volatile memory.
  • the first cache unit and the second cache unit are different volatile memories.
  • the first state includes a state of performing model training on the neural network model; the second state includes a state of performing data processing based on the neural network model.
  • the data processing chip further includes a control unit, a neural network processing subunit, and a data processing subunit, and the control unit is connected to the neural network processing subunit and the data processing subunit respectively;
  • the control unit is configured to monitor the data volume of the intermediate processing data generated by the data processing subunit, and trigger the neural network processing subunit in response to the data volume matching the target data volume of the neural network processing subunit Perform data processing.
  • a data processing module is further provided, and the data processing module includes the data processing chip as described in the above aspects.
  • a terminal is further provided, and the terminal includes the data processing module as described in the above aspect.
  • a power management method is also provided, the method is applied to a data processing chip, the data processing chip includes a first cache unit, a second cache unit and a data processing unit, the first cache unit and the second The cache unit is used to cache data, and the power management method includes:
  • the first state of the data processing unit includes a state of not processing data
  • the second state of the data processing unit includes a state of processing data
  • the data processing unit is used to process data using the neural network model in the second state
  • the first cache unit is used to store the data of the neural network model
  • the second cache unit is used to store the data of the neural network model. Data to be processed, intermediate processed data and processed result data of the data processing unit.
  • the data processing system further includes a data transmission interface configured to receive data
  • the data processing unit is configured to switch from the first state to the second state in response to the data received by the data transmission interface containing specific information.
  • the specific information includes at least one of the following: motion detection information, humanoid detection information.
  • the first cache unit and the second cache unit are different units of the same volatile memory.
  • the first cache unit and the second cache unit are different volatile memories.
  • the power management unit in both the first state and the second state of the data processing unit, maintains power supply to the first cache unit, and only in the second state of the data processing unit, it supplies power to the second cache unit.
  • Unit power supply so that in the AON mode, only the first buffer unit needs to be powered when it is in the first state, thereby reducing the number of power supply units and reducing the power consumption in the AON mode.
  • the terminal can run in a low power consumption mode when the user does not use the terminal.
  • the terminal powers off some unused modules, and maintains power supply for some modules that require an online mode (Always-On, AON).
  • the terminal maintains power supply to modules such as a Wireless Fidelity (Wi-Fi) module and an identity verification module, but cuts off power to modules such as a video playback module and a screen brightness module.
  • Wi-Fi Wireless Fidelity
  • the terminal is a mobile phone, and when the user does not use the mobile phone, the screen of the mobile phone can be locked, and the screen locked state of the mobile phone is a low power consumption state.
  • mobile phones can collect face images or fingerprint information in the lock screen state, and then process the collected face images or fingerprint information through the identity verification module, and verify based on the processing result data, so as to face image or fingerprint information to unlock the phone. Therefore, it is necessary to keep the information collection module and the identity verification module in a power supply state when the mobile phone is locked, that is, the information collection module and the identity verification module are in the AON mode.
  • the power supply to the above-mentioned modules is always maintained, but the above-mentioned modules are not always in the working state in the AON mode.
  • the identity verification module does not need to perform operations such as data processing. Therefore, in AON mode, it is difficult to supply power to the units involved in the data processing process to meet the requirements of low power consumption in AON mode. Require.
  • the terminal in order to further save the power consumption of the terminal in AON mode, the terminal further refines the working unit in AON mode through a power management unit (Power Management Unit, PMU), so that in AON mode, the terminal can The power supply is provided according to the usage state of the working unit, which further saves the power consumption of the terminal in the AON mode and meets the requirement of low power consumption in the AON mode.
  • PMU Power Management Unit
  • FIG. 1 shows a schematic structural diagram of a data processing chip provided by an exemplary embodiment of the present application.
  • the data processing chip includes: a power management unit 11, a first cache unit 12, a second cache unit 13 and a data processing unit 14; the power management unit 11 is used to control the first cache unit 12 and the second The power supply of the cache unit 13; the first cache unit 12 and the second cache unit 13 are used to cache data; the power management unit 11 is used to control the first cache unit in response to the first state of the data processing unit 14 12 to supply power to the second cache unit 13, and to control the power supply to the first cache unit 12 and the second cache unit 13 in response to the second state of the data processing unit 14.
  • the power management unit 11 keeps working in the AON mode, so that the power management unit 11 can control the power supply of other working units as required.
  • the power management unit 11 is used for controlling power supply to the first cache unit 12 and the second cache unit 13 under different states of the data processing unit 14 .
  • the power management unit 11 is connected to the first cache unit 12 and the second cache unit 13 respectively; the first cache unit 12 and the second cache unit 13 are connected to the data processing unit 14 respectively. In some embodiments, the power management unit 11 is also connected to the data processing unit 14 . Correspondingly, the power management unit 11 is configured to not supply power to the data processing unit 14 in response to not performing data processing, and to supply power to the data processing unit 14 in response to performing data processing. Alternatively, the data processing unit 14 is kept powered.
  • the power management unit 11 is also connected to a power supply module of the terminal, and the power supply module is connected to the first cache unit 12 and the second cache unit 13 respectively.
  • the power management unit 11 implements power supply to the first cache unit 12 or the second cache unit 13 by controlling the power supply module.
  • the first state of the data processing unit 14 includes a state of not processing data
  • the second state of the data processing unit 14 includes a state of processing data
  • the data processing chip is a chip for image data processing in the identity verification module, and correspondingly, the data processing chip is used for image data processing of collected face images or fingerprints. Then the data processing unit 14 considers that the first state is the state when the terminal is in the screen-locked state and no unlocking operation is detected, and the second state is the state when the terminal is in the screen-locked state and an unlocking operation is detected.
  • the first state includes a state of performing model training on the neural network model.
  • the second state includes a state of performing data processing based on the neural network model.
  • the first cache unit 12 is used to cache model parameters of the neural network model.
  • model training is performed on the neural network model, and model parameters of the neural network model obtained from the model training are stored in the first cache unit 12 .
  • the terminal performs data processing through the cached model parameters in the first cache unit 12 .
  • the power management unit 11 keeps supplying power to the first cache unit 12, so that in the AON mode, even in the second state When other working units are powered off, it can also ensure that the first cache unit 12 will not be cleared due to power failure. In this way, in the AON mode, only part of the storage units are powered, which reduces the number of powered working units and AON power consumption in mode.
  • the first cache unit 12 and the second cache unit 13 are cache units in the memory for storing data.
  • the size of the storage space of the first cache unit 12 and the second cache unit 13 can be set as required, and the sizes of the storage spaces of the first cache unit 12 and the second cache unit 13 are the same or different.
  • the memory is any type of volatile memory.
  • the memory is a static memory (Static Random Access Memory, SRAM).
  • the first cache unit 12 and the second cache unit 13 are different units of the same volatile memory.
  • the storage units of the volatile memory are divided, and the power management unit 11 supplies power to different units of the volatile memory respectively.
  • the first cache unit 12 and the second cache unit 13 are different units of different volatile memories.
  • the first cache unit 12 is a storage unit of a volatile memory in the data processing unit 14
  • the second cache unit 13 is a storage unit in a volatile memory other than the data processing unit 14 .
  • a volatile memory is set inside the data processing unit 14, and the storage medium of the volatile memory inside the data processing unit 14 is determined as the first cache unit 12, so that when the data processing unit 14 needs to read When fetching the data in the first cache unit 12, it only needs to be read from the internal volatile memory, thereby saving the transfer time between internal data and reducing the power consumption of reading data.
  • the data processing unit 14 is used to process data using the neural network model in the second state
  • the first cache unit 12 is used to store the data of the neural network model
  • the second cache unit 13 uses It is used to store the data to be processed, intermediate processed data and processed result data of the data processing unit 14.
  • the data of the neural network model includes model parameters of the neural network model.
  • the data processing unit 14 is used to perform neural network calculations based on the data to be processed and intermediate processed data stored in the first cache unit 12 and the second cache unit 13 to obtain the processing result data, and write the processing result data into the second cache unit 13.
  • the power management unit 11 keeps supplying power to the first cache unit 12, so that in the AON mode, even in the second state When other working units are powered off, it can also ensure that the first cache unit 12 will not be cleared due to power failure. In this way, in the AON mode, only part of the storage units are powered, which reduces the number of powered working units and AON power consumption in mode.
  • the data processing unit 14 includes a neural network processing subunit 141 and a data processing subunit 142 .
  • the neural network processing subunit 141 and the data processing subunit 142 are both connected to the second cache unit 13; and the neural network processing subunit 142, and in the first state, at least one of the data processing subunit 141 and the neural network processing subunit 142 is not powered.
  • the data processing subunit 141 is used to process the image data to obtain intermediate processed data, and write the intermediate processed data into the second cache unit 13;
  • the neural network processing subunit 141 is used to obtain the intermediate processed data from the second Read the intermediate processing data in the cache unit 13, and read the data of the neural network model from the first cache unit 12, perform neural network calculation on the intermediate processing data based on the data of the neural network model, and obtain the intermediate
  • the processing result data corresponding to the processing data is written into the second cache unit 13 .
  • the data processing subunit 142 is a unit capable of processing image data.
  • the data processing subunit 142 is an image signal processor (Image Sensor Processing, ISP).
  • ISP Image Sensor Processing
  • the data processing subunit 142 reads the image data and processes the image data.
  • the neural network processing subunit 141 is a processing unit capable of deploying a neural network.
  • the neural network processing subunit 141 is a neural network processor (Network Processing Unit, NPU).
  • NPU Network Processing Unit
  • the number of the neural network processing subunit 141 is one or more, which is not specifically limited in this embodiment of the present application.
  • the number of the neural network processing subunit 141 is 1, 2 or 4, etc.
  • different neural networks are respectively deployed in the multiple neural network processing subunits 141, and correspondingly, the network parameters of multiple neural networks are stored in the same first cache Unit 12.
  • each neural network processing subunit 141 corresponds to a first cache unit 12 , and the network parameters of the neural network deployed in each neural network processing subunit 141 are respectively stored in the corresponding first cache unit 12 .
  • the data read by the data processing subunit 142 is the original image data, or the data read by the data processing subunit 142 is the image data processed by the neural network processing subunit 141 .
  • the neural network processing subunit 141 is used to implement an intermediate process in the entire data processing process.
  • the data processing subunit 142 can continue to process the processing result data processed by the neural network processing subunit 141 .
  • the neural network processing subunit 141 is used to perform denoising processing on image data.
  • the data processing subunit 142 preprocesses the image data to obtain intermediate processed data, performs denoising processing on the intermediate processing data through the neural network processing subunit 141, and rewrites the image processing result obtained by the denoising processing into the second In the second buffer unit 13, the data processing sub-unit 142 continues to process the denoising processed image processing result.
  • the second cache unit 13 includes a first sub-cache unit and a second sub-cache unit
  • the data processing sub-unit 142 is used to write the intermediate processing data into the first sub-cache unit
  • the network processing subunit 141 is configured to write the processing result data into the second sub-cache unit.
  • the first sub-cache unit is used to store the intermediate processing data written by the data processing sub-unit 142
  • the second sub-cache unit is used to store the processing result data written by the neural network processing sub-unit 141 .
  • the division of the second cache unit 13 is made finer, and then the power supply to the second cache unit 13 can be realized, further saving the second cache unit 13. energy consumption.
  • the first cache unit 12 and the second cache unit 13 are storage units in different static memories
  • the first cache unit 12 is a static memory integrated in the neural network processing subunit 141 storage unit.
  • the data processing chip also includes a control unit 15; the control unit 15 is connected to the neural network processing subunit 141 and the data processing subunit 142 respectively; the unit management unit is also connected to the control unit connected, the power management unit 11 is also used to supply power to the control unit.
  • the control unit 15 is configured to monitor the data volume of the intermediate processing data generated by the data processing subunit 142, and trigger the neural network processing subunit 141 in response to the data volume matching the target data volume of the neural network processing subunit 141 Perform data processing.
  • the data amount represents the amount of intermediate processing data.
  • the target data volume represents the minimum data volume read by the neural network processing subunit 141 each time.
  • the data volume of the intermediate processing data generated by the control unit 15 is monitored so that when the data volume of the intermediate processing data reaches target data volume, the neural network processing subunit 141 can be called to process the intermediate processing data, so that the neural network processing subunit 141 can start neural network calculation without waiting for the data processing subunit 142 to finish processing a frame of image data, Therefore, the waiting time of the neural network processing subunit 141 is reduced, and the processing speed of the image data processing system is improved, thereby reducing the image processing time of the image data processing system and reducing the power consumption of the image data processing system.
  • the control unit 15 can be any control unit capable of controlling the data processing subunit 142 and the neural network processing subunit 141 .
  • the control unit 15 is a microcontroller unit (Microcontroller Unit, MCU), and correspondingly, the neural network processing subunit 141 and the data processing subunit 142 are called by the microcontroller unit.
  • the control unit 15 is a mailbox mechanism (Mailbox, MB) control unit, correspondingly, the neural network processing subunit 141 and the data processing subunit 142 are carried out by the mechanism of data packets through the MB control unit.
  • Data transmission implements the scheduling of the neural network processing subunit 141 and the data processing subunit 142 .
  • the MB control unit can schedule the neural network processing subunit 141 and the data image processing subunit in the form of data packets.
  • the MB control unit monitors the data volume of the intermediate processing data generated by the data processing subunit 142, and generates a data packet based on the data information of the intermediate processing data, and the data information includes the data volume of the intermediate processing data, the storage address of the intermediate processing data, etc. .
  • the target field in the data packet is encoded to obtain a data packet representing the data information of the intermediate processing data.
  • the data packet includes a plurality of fields, for example, a data source field (ISP_NPU), a data volume field (Date Size), a data address field (Data Addr), a data attribute field (CMD) and the like.
  • ISP-NPU indicates whether the source of the data is the ISP or the NPU. For example, if the code of ISP_NPU is 0001, it means that the intermediate processing data is data from ISP, or if the code is 0010, it means that the intermediate processing data is data from NPU. It should be noted that the correspondence between the encoded content and the data source is set as required, which is not specifically limited in this embodiment of the present application.
  • Date Size indicates the data volume of the intermediate processing data. For example, if the code of Date Size is 0010, it means that the data volume of the intermediate processing data is 8 bits. It should be noted that the correspondence between the encoded content and the amount of data is set according to needs, which is not specifically limited in this embodiment of the present application.
  • Data Addr represents the storage address of the intermediate processing data. For example, if the code of Data Addr is 0010, it means that the intermediate processing data is stored in the first sub-cache unit.
  • CMD represents the data attribute of the intermediate processing data, for example, the code of CMD is 011, indicating that the intermediate processing data is readable and writable data, and the like.
  • the data packet may also include other fields, for example, an end field (End Of File, EOF) is used to indicate that the content of the data packet ends here.
  • End Of File EOF
  • the control unit 15 records the data information of the intermediate processing data through the data packet, so that the data information such as the data volume of the intermediate processing data can be represented only by setting different fields in the data packet. Therefore, in the data processing When the subunit 142 and the neural network processing subunit 141 perform data interaction with the control unit 15 respectively, they can perform data interaction based on data packets, thereby reducing data transmission between the control unit 15 and the data processing subunit 142 and the neural network processing subunit 141. Occupied resources further reduce the energy consumption of data interaction between the control unit 15 and the data processing sub-unit 142 and the neural network processing sub-unit 141 .
  • control unit 15 is used to monitor the processing speed of the neural network processing subunit 141; in response to the processing speed of the neural network processing subunit 141 being lower than the processing speed of the data processing subunit 142, control the data The processing subunit 142 reduces the processing speed; the data processing subunit 142 is configured to process the image data based on the reduced processing speed.
  • control unit 15 determines the processing speeds of the two according to the amount of data read by the neural network processing subunit 141 and the data processing subunit 142 respectively; The amount of data generated determines the processing speed of both.
  • the control unit 15 monitors the amount of data read by the neural network processing subunit 141 for each data processing, and determines the value of the neural network processing subunit 141 based on the amount of data read by the neural network processing subunit 141 per unit time. processing speed.
  • the control unit 15 monitors the amount of intermediate processing data generated by the data processing subunit 142 per unit time, and determines the processing speed of the data processing subunit 142 based on the amount of intermediate processing data generated by the data processing subunit 142 .
  • the control unit 15 controls the data processing subunit 142 to reduce the processing speed. In some embodiments, in response to the processing speed of the neural network processing subunit 141 being lower than the processing speed of the data processing subunit 142 , the control unit 15 determines the unprocessed intermediate processing data in the second cache unit 13, and in response to the second The amount of intermediate processing data in the second buffer unit 13 is greater than the first preset data amount, and the control unit 15 controls the data processing sub-unit 142 to reduce the processing speed.
  • control unit 15 determines the remaining storage space of the second cache unit 13, and in response to the storage capacity of the remaining storage space If the amount of data is not greater than the second preset amount of data, the control unit controls the data processing unit 14 to reduce the processing speed.
  • control unit 15 reversely controls the processing speed of the data processing sub-unit 142 according to the processing speed of the neural network processing sub-unit 141, thus indirectly realizing the control of the second cache unit 13, thereby preventing The storage space of the second cache unit 13 is insufficient, resulting in the inability to store the intermediate processing data, which ensures that the image data processing process can be carried out normally.
  • the data processing chip further includes a data transmission interface 16, the data transmission interface 16 is used to receive data; the data processing unit 14 is used to respond to the data received by the data transmission interface containing Specific information is switched from the first state to the second state.
  • the specific information includes at least one of the following: motion detection information and humanoid detection information.
  • the data transmission interface 16 keeps working in the AON mode, so as to ensure timely data transmission to the data processing unit 14 .
  • the data transmission interface 16 is used to transmit the image data collected by the image collection module to the data processing unit 14 .
  • the data transmission interface 16 is used to transmit data collected by other modules in the terminal to the data processing unit 14 .
  • the data transmission interface 16 is a Mobile Industry Processor Interface (MIPI).
  • MIPI Mobile Industry Processor Interface
  • the power management unit 11 is used to control the power supply voltage of the data processing unit 14 according to the data processing speed of the data processing unit 14; wherein, the power supply voltage is positively correlated with the data processing speed.
  • the power management unit 11 controls the power supply voltage of the data processing unit 14 based on the Dynamic Voltage and Frequency Scaling (DVFS) technology.
  • DVFS Dynamic Voltage and Frequency Scaling
  • the NPU design used in the non-AON mode can be reused.
  • DVFS technology is adopted, and different voltage strategies are formulated according to the NPU hardware configuration and the computing power of the AON mode, so that different AON modes can run at suitable voltage levels. , to achieve optimal power consumption in the scenario.
  • the power management unit in both the first state and the second state of the data processing unit, maintains power supply to the first cache unit, and only in the second state of the data processing unit, it supplies power to the second cache unit.
  • Unit power supply so that in the AON mode, only the first buffer unit needs to be powered when it is in the first state, thereby reducing the number of power supply units and reducing the power consumption in the AON mode.
  • FIG. 6 shows a flowchart of a power management method provided by an exemplary embodiment of the present application.
  • the method is applied to a data processing chip, the data processing chip includes a first cache unit, a second cache unit and a data processing unit, the first cache unit and the second cache unit are used for caching data, and the method includes the following steps:
  • Step S601 In response to the first state of the data processing unit, the data processing chip supplies power to the first cache unit and does not supply power to the second cache unit;
  • Step S602 In response to the second state of the data processing unit, the data processing chip supplies power to the first cache unit and to the second cache unit.
  • the first state of the data processing unit includes a state of not processing data
  • the second state of the data processing unit includes a state of processing data
  • the data processing unit is used to process data using the neural network model in the second state
  • the first cache unit is used to store the data of the neural network model
  • the second cache unit is used to store the data of the neural network model. Data to be processed, intermediate processed data and processed result data of the data processing unit.
  • the data processing system further includes a data transmission interface configured to receive data
  • the data processing unit is configured to switch from the first state to the second state in response to the data received by the data transmission interface containing specific information.
  • the specific information includes at least one of the following: motion detection information, humanoid detection information.
  • the first cache unit and the second cache unit are different units of the same volatile memory.
  • the first cache unit and the second cache unit are different volatile memories.
  • the power management unit in both the first state and the second state of the data processing unit, maintains power supply to the first cache unit, and only in the second state of the data processing unit, it supplies power to the second cache unit.
  • Unit power supply so that in the AON mode, only the first buffer unit needs to be powered when it is in the first state, thereby reducing the number of power supply units and reducing the power consumption in the AON mode.
  • FIG. 7 shows a structural block diagram of a terminal 700 provided by an exemplary embodiment of the present application.
  • the terminal 700 may be a terminal capable of deploying a neural network, such as a smart phone, a computer, and a tablet computer.
  • the terminal 700 in this application includes the data processing module 710 in the above embodiment, and the data processing module includes the data processing chip shown in the embodiment of this application.
  • the terminal 700 further includes one or more of the following components: a processor 720 and a memory 730 .
  • Processor 720 may include one or more processing cores.
  • the processor 720 uses various interfaces and lines to connect various parts of the entire terminal 700, by running or executing the program code, program, code set or program code set stored in the memory 730, and calling the data stored in the memory 730, Various functions of the terminal 700 are executed and data is processed.
  • the processor 720 may adopt at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in the form of hardware.
  • DSP Digital Signal Processing
  • FPGA Field-Programmable Gate Array
  • PLA Programmable Logic Array
  • the processor 720 may integrate one or more of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a neural network processor (Neural-network Processing Unit, NPU) and a modem, etc.
  • a central processing unit Central Processing Unit, CPU
  • an image processor Graphics Processing Unit, GPU
  • a neural network processor Neural-network Processing Unit, NPU
  • the CPU mainly handles the operating system, user interface and application programs, etc.
  • the GPU is used to render and draw the content that needs to be displayed on the display screen
  • the NPU is used to realize artificial intelligence (Artificial Intelligence, AI) functions
  • the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 720, but may be realized by a single chip.
  • the memory 730 may include random access memory (Random Access Memory, RAM), and may also include read-only memory (Read-Only Memory).
  • the memory 730 includes a non-transitory computer-readable storage medium.
  • Memory 730 may be used to store program code, programs, codes, sets of codes, or sets of program codes.
  • the memory 730 may include a program storage area and a data storage area, wherein the program storage area may store program codes for implementing an operating system, program codes for at least one function (such as a touch function, a sound playback function, an image playback function, etc. ), program codes for implementing the following method embodiments, etc.; the storage data area can store data created according to the use of the terminal 700 (such as audio data, phone book) and the like.
  • terminal 700 also includes a display screen.
  • a display is a display component for displaying a user interface.
  • the display screen is a display screen with a touch function. Through the touch function, the user can use any suitable object such as a finger or a touch pen to perform a touch operation on the display screen.
  • the display screen is usually set on the front panel of the terminal 700 .
  • the display screen can be designed as full screen, curved screen, special-shaped screen, double-sided screen or folding screen.
  • the display screen can also be designed as a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, etc., which are not limited in this embodiment.
  • the structure of the terminal 700 shown in the above drawings does not constitute a limitation on the terminal 700, and the terminal 700 may include more or less components than those shown in the figure, or combine certain some components, or a different arrangement of components.
  • the terminal 700 also includes components such as a microphone, a speaker, a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (Wi-Fi) module, a power supply, and a bluetooth module, which will not be repeated here.
  • Wi-Fi wireless fidelity
  • the embodiment of the present application also provides a computer-readable medium, the computer-readable medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the power management method shown in the above embodiments.
  • the embodiment of the present application also provides a computer program product, the computer program product stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the power management method shown in the above embodiments.
  • the computer programs involved in the embodiments of the present application can be deployed and executed on one computer device, or executed on multiple computer devices at one location, or distributed in multiple locations and communicated Executed on multiple computer devices interconnected by the network, multiple computer devices distributed in multiple locations and interconnected through a communication network can form a blockchain system.
  • the functions described in the embodiments of the present application may be implemented by hardware, software, firmware or any combination thereof.
  • the functions may be stored on or transmitted over as one or more program code or codes on a computer-readable medium.
  • Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.
  • a storage media may be any available media that can be accessed by a general purpose or special purpose computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Power Sources (AREA)

Abstract

本申请涉及一种数据处理芯片、模组、终端及电源管理方法，涉及人工智能技术领域。数据处理芯片，包括：电源管理单元、第一缓存单元、第二缓存单元和数据处理单元；该电源管理单元用于控制对该第一缓存单元和第二缓存单元的供电；该第一缓存单元和该第二缓存单元用于缓存数据；该电源管理单元用于响应于该数据处理单元的第一状态，控制对该第一缓存单元供电，对该第二缓存单元不供电，响应于该数据处理单元的第二状态，控制对该第一缓存单元供电并对该第二缓存单元供电。通过本方案能够降低终端的功耗。

Description

数据处理芯片、模组、终端及电源管理方法
本申请要求于2021年11月10日提交的申请号为202111329007.4、发明名称为“数据处理芯片、模组、终端及电源管理方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及人工智能技术领域，特别涉及一种数据处理芯片、模组、终端及电源管理方法。
背景技术
为了增加终端的待机时长，在用户不使用终端的期间，终端能够以低功耗模式运行。在低功耗模式下，终端对一些不使用的模块断电，而对一些需要在线模式(Always-On，AON)的模块保持供电。例如，在低功耗模式下终端保持对无线保真(Wireless Fidelity，Wi-Fi)模块、身份验证模块等模块的供电，而对视频播放模块、屏幕亮度模块等模块断电。
发明内容
本申请实施例提供了一种数据处理芯片、模组、终端及电源管理方法，能够降低终端的耗能。所述技术方案如下：
一方面，提供了一种数据处理芯片，包括：电源管理单元、第一缓存单元、第二缓存单元和数据处理单元；
所述电源管理单元用于控制对所述第一缓存单元和第二缓存单元的供电；
所述第一缓存单元和所述第二缓存单元用于缓存数据；
所述电源管理单元用于响应于所述数据处理单元的第一状态，控制对所述第一缓存单元供电，对所述第二缓存单元不供电，响应于所述数据处理单元的第二状态，控制对所述第一缓存单元供电并对所述第二缓存单元供电。
另一方面，提供了一种数据处理模组，所述数据处理模组包括如上述方面所述的数据处理芯片。
另一方面，提供了一种终端，所述终端包括如上述方面所述的数据处理模组。
另一方面，提供了一种电源管理方法，应用于数据处理芯片，所述数据处理芯片包括第一缓存单元、第二缓存单元和数据处理单元，所述第一缓存单元和所述第二缓存单元用于缓存数据，所述电源管理方法包括：
响应于所述数据处理单元的第一状态，对所述第一缓存单元供电，对所述第二缓存单元不供电；
响应于所述数据处理单元的第二状态，对所述第一缓存单元供电并对所述第二缓存单元供电。
另一方面，提供了一种计算机可读存储介质，所述计算机可读存储介质存储有至少一条程序代码，所述至少一条程序代码用于被处理器执行以实现如上述方面所述的电源管理方法。
另一方面，提供了一种计算机程序产品，所述计算机程序产品存储有至少一条程序代码，所述至少一条程序代码由处理器加载并执行以实现如上述方面所述的电源管理方法。
在本申请实施例中，在数据处理单元的第一状态和第二状态下，电源管理单元均保持对第一缓存单元供电，而只有在数据处理单元的第二状态下，才对第二缓存单元供电，这样在AON模式时，处于第一状态时，只需要对第一缓存单元进行供电，进而减少了供电的单元数量，降低了AON模式下的耗电量。
附图说明
图1示出了本申请一个示例性实施例提供的数据处理芯片的结构示意图；
图2示出了本申请一个示例性实施例提供的数据处理芯片的结构示意图；
图3示出了本申请一个示例性实施例提供的数据处理芯片的结构示意图；
图4示出了本申请一个示例性实施例提供的数据包的结构示意图；
图5示出了本申请一个示例性实施例提供的数据处理芯片的结构示意图；
图6示出了本申请一个示例性实施例提供的电源管理方法的流程图；
图7示出了本申请一个实施例提供的终端的结构框图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
在本文中提及的“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。另外，本申请涉及到的图像数据等数据为经用户授权或者经过各方充分授权的数据。
本申请实施例中，提供一种数据处理芯片，包括：电源管理单元、第一缓存单元、第二缓存单元和数据处理单元；
该电源管理单元用于控制对该第一缓存单元和第二缓存单元的供电；
该第一缓存单元和该第二缓存单元用于缓存数据；
该电源管理单元用于响应于该数据处理单元的第一状态，控制对该第一缓存单元供电，对该第二缓存单元不供电，响应于该数据处理单元的第二状态，控制对该第一缓存单元供电并对该第二缓存单元供电。
在一些实施例中，该数据处理单元的第一状态包括不处理数据的状态，该数据处理单元的第二状态包括处理数据的状态。
在一些实施例中，该数据处理单元用于在第二状态下使用神经网络模型对数据进行处理，该第一缓存单元用于存储该神经网络模型的数据，该第二缓存单元用于存储该数据处理单元的待处理数据、中间处理数据和处理结果数据。
在一些实施例中，该数据处理芯片还包括数据传输接口，该数据传输接口用于接收数据；
该数据处理单元用于，响应于该数据传输接口接收到的数据包含特定信息，从该第一状态切换成该第二状态。
在一些实施例中，该特定信息包括以下至少一种：运动侦测信息、人形侦测信息。
在一些实施例中，该第一缓存单元和该第二缓存单元为同一易失性存储器的不同单元。
在一些实施例中，该第一缓存单元和该第二缓存单元为不同的易失性存储器。
在一些实施例中，所述第一状态包括对神经网络模型进行模型训练的状态；所述第二状态包括基于神经网络模型进行数据处理的状态。
在一些实施例中，所述数据处理芯片还包括控制单元、神经网络处理子单元和数据处理子单元，所述控制单元分别与所述神经网络处理子单元和所述数据处理子单元连接；
所述控制单元用于监控所述数据处理子单元生成的中间处理数据的数据量，响应于所述数据量与所述神经网络处理子单元的目标数据量匹配，触发所述神经网络处理子单元进行数据处理。
本申请实施例中，还提供一种数据处理模组，数据处理模组包括如上述方面所述的数据处理芯片。
本申请实施例中，还提供一种终端，终端包括如上述方面所述的数据处理模组。
本申请实施例中，还提供一种电源管理方法，该方法应用于数据处理芯片，该数据处理芯片包括第一缓存单元、第二缓存单元和数据处理单元，该第一缓存单元和该第二缓存单元用于缓存数据，该电源管理方法包括：
响应于该数据处理单元的第一状态，对该第一缓存单元供电，对该第二缓存单元不供电；
响应于该数据处理单元的第二状态，对该第一缓存单元供电并对该第二缓存单元供电。
在一些实施例中，该数据处理单元的第一状态包括不处理数据的状态，该数据处理单元的第二状态包括处理数据的状态。
在一些实施例中，该数据处理单元用于在第二状态下使用神经网络模型对数据进行处理，该第一缓存单元用于存储该神经网络模型的数据，该第二缓存单元用于存储该数据处理单元的待处理数据、中间处理数据和处理结果数据。
在一些实施例中，该数据处理系统还包括数据传输接口，该数据传输接口用于接收数据；
该数据处理单元用于，响应于该数据传输接口接收到的数据包含特定信息，从该第一状态切换成该第二状态。
在一些实施例中，该特定信息包括以下至少一种：运动侦测信息、人形侦测信息。
在一些实施例中，该第一缓存单元和该第二缓存单元为同一易失性存储器的不同单元。
在一些实施例中，该第一缓存单元和该第二缓存单元为不同的易失性存储器。
在本申请实施例中，在数据处理单元的第一状态和第二状态下，电源管理单元均保持对第一缓存单元供电，而只有在数据处理单元的第二状态下，才对第二缓存单元供电，这样在AON模式时，处于第一状态时，只需要对第一缓存单元进行供电，进而减少了供电的单元数量，降低了AON模式下的耗电量。
为了增加终端的待机时长，在用户不使用终端的期间，终端能够以低功耗模式运行。在低功耗模式下，终端对一些不使用的模块断电，而对一些需要在线模式(Always-On，AON)的模块保持供电。例如，在低功耗模式下终端保持对无线保真(Wireless Fidelity，Wi-Fi)模块、身份验证模块等模块的供电，而对视频播放模块、屏幕亮度模块等模块断电。
例如，终端为手机，在用户不使用手机时，可以将手机锁屏，则手机的锁屏状态即为低功耗状态。随着终端技术的发展，手机能够在锁屏状态下采集人脸图像或指纹信息，进而通过身份验证模块对采集的人脸图像或指纹信息进行数据处理，基于处理结果数据进行验证，以便基于人脸图像或指纹信息解锁手机。因此，手机锁屏状态下需要保持信息采集模块和身份验证模块处于供电状态，即信息采集模块和身份验证模块处于AON模式。
相关技术中，在AON模式下，始终保持对上述模块的供电，而上述模块在 AON模式下不一定一直处于工作状态。例如，身份验证模块在用户不进行解锁等操作时，可以不进行数据处理等操作，因此，在AON模式下，对数据处理过程涉及到的单元进行供电，很难满足AON模式下低功耗的要求。
在本申请实施例中，为了进一步节省终端在AON模式下的耗电，终端通过电源管理单元(Power Management Unit，PMU)对AON模式下的工作单元进一步细化，使得在AON模式下，终端能根据工作单元的使用状态进行供电，进一步节省了终端在AON模式下的耗电，满足AON模式下低功耗的需求。
请参考图1，其示出了本申请一个示例性实施例提供的数据处理芯片的结构示意图。参见图1，该数据处理芯片包括：电源管理单元11、第一缓存单元12、第二缓存单元13和数据处理单元14；该电源管理单元11用于控制对该第一缓存单元12和第二缓存单元13的供电；该第一缓存单元12和该第二缓存单元13用于缓存数据；该电源管理单元11用于响应于该数据处理单元14的第一状态，控制对该第一缓存单元12供电，对该第二缓存单元13不供电，响应于该数据处理单元14的第二状态，控制对该第一缓存单元12供电并对该第二缓存单元13供电。
参见图1，该电源管理单元11在AON模式下一直保持工作状态，以使该电源管理单元11能够根据需要控制其他工作单元的供电情况。电源管理单元11用于在数据处理单元14的不同状态下控制对第一缓存单元12和第二缓存单元13供电。
其中，该电源管理单元11分别与该第一缓存单元12和该第二缓存单元13连接；该第一缓存单元12和该第二缓存单元13分别与该数据处理单元14连接。在一些实施例中，该电源管理单元11还与数据处理单元14连接。相应的，该电源管理单元11用于响应于不进行数据处理，对该数据处理单元14不供电，响应于进行数据处理，对该数据处理单元14供电。或者，保持对该数据处理单元14供电。
在一些实施例中，该电源管理单元11还与终端的供电模组连接，该供电模组分别与第一缓存单元12和第二缓存单元13连接。相应的，该电源管理单元11通过控制供电模组实现对第一缓存单元12或第二缓存单元13的供电。
在一些实施例中，该数据处理单元14的第一状态包括不处理数据的状态， 该数据处理单元14的第二状态包括处理数据的状态。
例如，该数据处理芯片为身份验证模块中进行图像数据处理的芯片，相应的，该数据处理芯片用于对采集到的人脸图像或指纹进行图像数据处理。则数据处理单元14为该第一状态为终端处于锁屏状态且未检测到解锁操作时的状态，第二状态为终端处于锁屏状态且检测到解锁操作时的状态。
在一些实施例中，该第一状态包括对神经网络模型进行模型训练的状态。第二状态包括基于神经网络模型进行数据处理的状态。
例如，该第一缓存单元12用用于缓存神经网络模型的模型参数。在第一状态下，对神经网络模型进行模型训练，将模型训练得到的神经网络模型的模型参数存储至第一缓存单元12中。在第二状态下，终端通过第一缓存单元12中的缓存的模型参数进行数据处理。
基于第一缓存单元12和第二缓存单元13的特性，第一缓存单元12和第二缓存单元13在断电后其存储的数据会被清除，为了保证数据处理芯片能够随时调用第一缓存单元12中的数据进行数据处理，需要保证第一缓存单元12中存储的数据不被清除。而在本申请实施例中，在数据处理单元14的第一状态和第二状态下，电源管理单元11均保持对第一缓存单元12供电，这样在AON模式时，就算在第二状态下对其他工作单元断电，也能保证第一缓存单元12不会因为断电而被清除数据，这样在AON模式下，只对部分存储单元进行供电，减少了供电的工作单元的数量，降低了AON模式下的耗电量。
其中，第一缓存单元12和第二缓存单元13为存储器中的缓存单元，用于存储数据。该第一缓存单元12和第二缓存单元13的存储空间大小，根据需要进行设置，且该第一缓存单元12和第二缓存单元13的存储空间大小相同或者不同。在本申请实施例中，对此均不作具体限定。其中，该存储器为任一类型的易失性存储器。例如，该存储器为静态存储器(Static Random Access Memory，SRAM)。
在一些实施例中，该第一缓存单元12和该第二缓存单元13为同一易失性存储器的不同单元。相应的，对易失性存储器的存储单元进行划分，电源管理单元11分别对该易失性存储器的不同单元进行供电。
在一些实施例中，该第一缓存单元12和第二缓存单元13为不同易失性存储器的不同单元。在一些实施例中，该第一缓存单元12为该数据处理单元14 中的易失性存储器的存储单元，该第二缓存单元13为该数据处理单元14以外的易失性存储器中的存储单元。
在本申请实施例中，在数据处理单元14内部设置易失性存储器，将该数据处理单元14内部的易失性存储器的存储介质确定为第一缓存单元12，这样当数据处理单元14需要读取第一缓存单元12中的数据时，只需要从内部的易失性存储器中读取，从而节省了内部数据间的搬运时间，降低了读取数据的功耗。
在一些实施例中，该数据处理单元14用于在第二状态下使用神经网络模型对数据进行处理，该第一缓存单元12用于存储该神经网络模型的数据，该第二缓存单元13用于存储该数据处理单元14的待处理数据、中间处理数据和处理结果数据。
其中，该神经网络模型的数据包括神经网络模型的模型参数。该数据处理单元14用于基于该第一缓存单元12以及该第二缓存单元13所存储的待处理数据和中间处理数据等数据进行神经网络计算，得到该处理结果数据，将该处理结果数据写入至该第二缓存单元13。
在本申请实施例中，为了保证数据处理芯片能够随时调用第一缓存单元12中的神经网络模型的数据进行数据处理，需要保证第一缓存单元12中存储的数据不被清除。而在本申请实施例中，在数据处理单元14的第一状态和第二状态下，电源管理单元11均保持对第一缓存单元12供电，这样在AON模式时，就算在第二状态下对其他工作单元断电，也能保证第一缓存单元12不会因为断电而被清除数据，这样在AON模式下，只对部分存储单元进行供电，减少了供电的工作单元的数量，降低了AON模式下的耗电量。
在一些实施例中，该数据处理单元14包括神经网络处理子单元141和数据处理子单元142。参见图2，该神经网络处理子单元141和该数据处理子单元142均与该第二缓存单元13连接；该电源管理单元11，用于在该第二状态下，对该数据处理子单元141和神经网络处理子单元142供电，在第一状态下，对该数据处理子单元141和神经网络处理子单元142中的至少一个不供电。其中，该数据处理子单元141用于对该图像数据进行处理，得到中间处理数据，将该中间处理数据写入该第二缓存单元13；该神经网络处理子单元141，用于从该第二缓存单元13中读取该中间处理数据，以及，从该第一缓存单元12中读取该神经网络模型的数据，基于该神经网络模型的数据对该中间处理数据进行神经 网络计算，得到该中间处理数据对应的处理结果数据，将该处理结果数据写入该第二缓存单元13。
数据处理子单元142为能够对图像数据进行处理的单元。例如，该数据处理子单元142为图像信号处理器(Image Sensor Processing，ISP)。在对图像数据进行处理时，该数据处理子单元142读取图像数据，对该图像数据进行处理。神经网络处理子单元141为能够部署神经网络的处理单元。例如，该神经网络处理子单元141为神经网络处理器(Network Processing Unit，NPU)。在对图像数据进行处理时，神经网络处理子单元141能够调用网络参数和中间处理数据，基于该网络参数对中间处理数据进行处理。
需要说明的一点是，该神经网络处理子单元141的数量为一个或多个，在本申请实施例中，对此不作具体限定。例如，该神经网络处理子单元141的数量为1个、2个或4个等。在该神经网络处理子单元141的数量为多个的情况下，该多个神经网络处理子单元141中分别部署不同的神经网络，相应的，多个神经网络的网络参数存储在同一第一缓存单元12中。或者，每个神经网络处理子单元141对应一个第一缓存单元12，每个神经网络处理子单元141中部署的神经网络的网络参数分别存储在对应的第一缓存单元12中。
需要说明的另一点是，数据处理子单元142读取的数据为原始图像数据，或者，数据处理子单元142读取的数据为经由神经网络处理子单元141处理后的图像数据。
相应的，该神经网络处理子单元141用于实现整个数据处理过程中的一个中间过程。数据处理子单元142能够对神经网络处理子单元141处理后的处理结果数据继续进行处理。例如，神经网络处理子单元141用于对图像数据进行去噪处理。相应的，数据处理子单元142对图像数据进行预处理，得到中间处理数据，通过神经网络处理子单元141对中间处理数据进行去噪处理，将去噪处理得到的图像处理结果重新写入到第二缓存单元13，由数据处理子单元142继续对去噪处理后的图像处理结果进行处理。
在一些实施例中，该第二缓存单元13包括第一子缓存单元和第二子缓存单元，该数据处理子单元142，用于将该中间处理数据写入该第一子缓存单元；该神经网络处理子单元141，用于将该处理结果数据写入该第二子缓存单元。
也即该第一子缓存单元用于存储数据处理子单元142写入的中间处理数据， 该第二子缓存单元用于存储神经网络处理子单元141写入的处理结果数据。在本申请实施例中，通过对第二缓存单元13再进行划分，使得第二缓存单元13划分更精细，进而能够实现对第二缓存单元13的分区供电，进一步节省了第二缓存单元13的耗能。
需要说明的一点是，在该第一缓存单元12和第二缓存单元13为不同静态存储器中的存储单元的情况下，该第一缓存单元12为神经网络处理子单元141中集成的静态存储器中的存储单元。
在一些实施例中，参见图3，该数据处理芯片还包括控制单元15；该控制单元15分别与该神经网络处理子单元141和该数据处理子单元142连接；单元管理单元还与该控制单元连接，电源管理单元11还用对该控制单元进行供电。该控制单元15，用于监控该数据处理子单元142生成的中间处理数据的数据量，响应于该数据量与该神经网络处理子单元141的目标数据量匹配，触发该神经网络处理子单元141进行数据处理。
其中，该数据量表示中间处理数据的多少。该目标数据量表示神经网络处理子单元141每次读取的最小数据量，在本申请实施例中，通过控制单元15监控生成的中间处理数据的数据量，使得当中间处理数据的数据量达到目标数据量，就能调用神经网络处理子单元141对该中间处理数据进行处理，这样神经网络处理子单元141无需等待数据处理子单元142处理完一帧图像数据，就能开始进行神经网络计算，从而减少了神经网络处理子单元141的等待时间，进而提高了图像数据处理系统的处理速度，从而减少了图像数据处理系统进行图像处理的时间，降低了图像数据处理系统的功耗。
该控制单元15可以为任一能够控制数据处理子单元142和神经网络处理子单元141的控制单元。在一些实施例中，该控制单元15为微控制单元(Microcontroller Unit，MCU)，相应的，通过微控制单元调用神经网络处理子单元141和数据处理子单元142。在一些实施例中，该控制单元15为邮箱机制(Mailbox，MB)控制单元，相应的，通过该MB控制单元以数据包的机制在神经网络处理子单元141和数据处理子单元142之间进行数据传输，实现调度神经网络处理子单元141和数据处理子单元142。
其中，通过MB控制单元控制该神经网络处理子单元141和数据处理子单元142时，MB控制单元能够以数据包的形式调度神经网络处理子单元141和数 据图像处理子单元。相应的，MB控制单元监控数据处理子单元142生成中间处理数据的数据量，基于该中间处理数据的数据信息生成数据包，该数据信息包括中间处理数据的数据量、中间处理数据的存储地址等。基于该数据信息，对数据包中的目标字段进行编码，得到用于表示该中间处理数据的数据信息的数据包。
参见图4，该数据包包括多个字段，例如，数据来源字段(ISP_NPU)、数据量字段(Date Size)、数据地址字段(Data Addr)、数据属性字段(CMD)等。其中，ISP-NPU表示该数据的来源为ISP还是NPU。例如，ISP_NPU的编码为0001，则表示该中间处理数据为来自于ISP的数据，或者，该编码为0010，则表示该中间处理数据为来自于NPU的数据。需要说明的一点是，编码内容与数据来源的对应关系根据需要进行设置，在本申请实施例中，对此不作具体限定。
Date Size表示中间处理数据的数据量，例如，Date Size的编码为0010，则表示该中间处理数据的数据量为8bit。需要说明的一点是，编码内容与数据量的对应关系根据需要进行设置，在本申请实施例中，对此不作具体限定。Data Addr表示中间处理数据的存储地址，例如，Data Addr的编码为0010，则表示该中间处理数据存储在第一子缓存单元。CMD表示该中间处理数据的数据属性，例如，CMD的编码为011，表示该中间处理数据为可读写数据等。该数据包中还可以包括其他字段，例如，结束字段(End Of File，EOF)用于表示该数据包的内容到此结束。
在本申请实施例中，控制单元15通过数据包记录中间处理数据的数据信息，这样可以仅通过设置数据包中的不同字段就能表示中间处理数据的数据量等数据信息，因此，在数据处理子单元142和神经网络处理子单元141分别与控制单元15进行数据交互时，能够基于数据包进行数据交互，从而降低控制单元15与数据处理子单元142和神经网络处理子单元141之间数据传输占用的资源，进而降低了控制单元15与数据处理子单元142和神经网络处理子单元141进行数据交互的耗能。
在一些实施例中，该控制单元15用于监控该神经网络处理子单元141的处理速度；响应于该神经网络处理子单元141的处理速度小于该数据处理子单元142的处理速度，控制该数据处理子单元142降低处理速度；该数据处理子单元142，用于基于降低后的处理速度对该图像数据进行处理。
其中，控制单元15分别根据神经网络处理子单元141和数据处理子单元142读取的数据量确定二者的处理速度；或者，控制单元15分别根据神经网络处理子单元141和数据处理子单元142生成的数据量确定二者的处理速度。在一些实施例中，控制单元15监控神经网络处理子单元141每次数据处理读取的数据量，基于神经网络处理子单元141单位时间内读取的数据量来确定神经网络处理子单元141的处理速度。控制单元15监控数据处理子单元142单位时间内生成的中间处理数据的数据量，基于数据处理子单元142生成的中间处理数据的数据量来确定该数据处理子单元142的处理速度。
在一些实施例中，响应于该神经网络处理子单元141的处理速度小于该数据处理子单元142的处理速度，控制单元15就控制数据处理子单元142降低处理速度。在一些实施例中，响应于该神经网络处理子单元141的处理速度小于该数据处理子单元142的处理速度，控制单元15确定第二缓存单元13中未被处理的中间处理数据，响应于第二缓存单元13中的中间处理数据的数量大于第一预设数据量，控制单元15控制数据处理子单元142降低处理速度。或者，响应于该神经网络处理子单元141的处理速度小于该数据处理子单元142的处理速度，控制单元15确定第二缓存单元13剩余的存储空间，响应于所述剩余的存储空间的能够存储的数据量不大于第二预设数据量，控制单元控制数据处理单元14降低处理速度。
在本申请实施例中，控制单元15根据神经网络处理子单元141的处理速度，反向控制该数据处理子单元142的处理速度，这样间接实现了对第二缓存单元13的控制，从而防止了第二缓存单元13的存储空间不足，导致中间处理数据无法被存储，保证了图像数据处理过程能够正常进行。
在一些实施例中，参见图5，该数据处理芯片还包括数据传输接口16，该数据传输接口16用于接收数据；该数据处理单元14用于，响应于该数据传输接口接收到的数据包含特定信息，从该第一状态切换成该第二状态。其中，该特定信息包括以下至少一种：运动侦测信息、人形侦测信息。
其中，该数据传输接口16在AON模式下一直保持工作状态，从而保证及时将数据传输至数据处理单元14。例如，在身份认证的场景下，该数据传输接口16用于将图像采集模块采集的图像数据传输给数据处理单元14。该数据传输接口16用于将终端中其他模块采集的数据传输给数据处理单元14。例如，该数 据传输接口16为移动行业处理器接口(Mobile Industry Processor Interface，MIPI)。
需要说明的一点是，该电源管理单元11用于根据该数据处理单元14的数据处理速度，控制对该数据处理单元14的供电电压；其中，该供电电压与该数据处理速度正相关。相应的，电源管理单元11基于动态电压频率调节(Dynamic Voltage and Frequency Scaling，DVFS)技术实现对数据处理单元14的供电电压的控制。这样实现了复用非AON模式使用的NPU设计，在AON模式下采用DVFS技术，根据NPU硬件配置和AON模式的运算能力，制定不同的电压策略，使得不同的AON模式运行在适合的电压挡位，实现场景下的功耗最优。
在本申请实施例中，在数据处理单元的第一状态和第二状态下，电源管理单元均保持对第一缓存单元供电，而只有在数据处理单元的第二状态下，才对第二缓存单元供电，这样在AON模式时，处于第一状态时，只需要对第一缓存单元进行供电，进而减少了供电的单元数量，降低了AON模式下的耗电量。
请参考图6，其示出了本申请一个示例性实施例提供的电源管理方法的流程图。该方法应用于数据处理芯片，该数据处理芯片包括第一缓存单元、第二缓存单元和数据处理单元，该第一缓存单元和该第二缓存单元用于缓存数据，该方法包括以下步骤：
步骤S601：响应于该数据处理单元的第一状态，数据处理芯片对该第一缓存单元供电，对该第二缓存单元不供电；
步骤S602：响应于该数据处理单元的第二状态，数据处理芯片对该第一缓存单元供电并对该第二缓存单元供电。
在一些实施例中，该数据处理单元的第一状态包括不处理数据的状态，该数据处理单元的第二状态包括处理数据的状态。
在一些实施例中，该数据处理单元用于在第二状态下使用神经网络模型对数据进行处理，该第一缓存单元用于存储该神经网络模型的数据，该第二缓存单元用于存储该数据处理单元的待处理数据、中间处理数据和处理结果数据。
在一些实施例中，该数据处理系统还包括数据传输接口，该数据传输接口用于接收数据；
该数据处理单元用于，响应于该数据传输接口接收到的数据包含特定信息， 从该第一状态切换成该第二状态。
在一些实施例中，该特定信息包括以下至少一种：运动侦测信息、人形侦测信息。
在一些实施例中，该第一缓存单元和该第二缓存单元为同一易失性存储器的不同单元。
在一些实施例中，该第一缓存单元和该第二缓存单元为不同的易失性存储器。
在本申请实施例中，在数据处理单元的第一状态和第二状态下，电源管理单元均保持对第一缓存单元供电，而只有在数据处理单元的第二状态下，才对第二缓存单元供电，这样在AON模式时，处于第一状态时，只需要对第一缓存单元进行供电，进而减少了供电的单元数量，降低了AON模式下的耗电量。
请参考图7，其示出了本申请一个示例性实施例提供的终端700的结构方框图。终端700可以是智能手机、电脑、平板电脑等具有部署神经网络功能的终端。本申请中的终端700包括上述实施例中的数据处理模组710，该数据处理模组包括本申请实施例中所示的数据处理芯片。
在一些实施例中红，该终端700还包括一个或多个如下部件：处理器720和存储器730。
处理器720可以包括一个或者多个处理核心。处理器720利用各种接口和线路连接整个终端700内的各个部分，通过运行或执行存储在存储器730内的程序代码、程序、代码集或程序代码集，以及调用存储在存储器730内的数据，执行终端700的各种功能和处理数据。可选地，处理器720可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器720可集成中央处理器(Central Processing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)、神经网络处理器(Neural-network Processing Unit，NPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责显示屏所需要显示的内容的渲染和绘制；NPU用于实现人工智能(Artificial Intelligence，AI)功能；调制解调器用于处理无线通信。可以理解的是，上述调 制解调器也可以不集成到处理器720中，单独通过一块芯片进行实现。
存储器730可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。可选地，该存储器730包括非瞬时性计算机可读介质(non-transitory computer-readable storage medium)。存储器730可用于存储程序代码、程序、代码、代码集或程序代码集。存储器730可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的程序代码、用于至少一个功能的程序代码(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的程序代码等；存储数据区可存储根据终端700的使用所创建的数据(比如音频数据、电话本)等。
在一些实施例中，终端700还包括显示频。显示屏是用于显示用户界面的显示组件。可选的，该显示屏为具有触控功能的显示屏，通过触控功能，用户可以使用手指、触摸笔等任何适合的物体在显示屏上进行触控操作。显示屏通常设置在终端700的前面板。显示屏可被设计成为全面屏、曲面屏、异型屏、双面屏或折叠屏。显示屏还可被设计成为全面屏与曲面屏的结合，异型屏与曲面屏的结合等，本实施例对此不加以限定。
除此之外，本领域技术人员可以理解，上述附图所示出的终端700的结构并不构成对终端700的限定，终端700可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。比如，终端700中还包括麦克风、扬声器、射频电路、输入单元、传感器、音频电路、无线保真(Wireless Fidelity，Wi-Fi)模块、电源、蓝牙模块等部件，在此不再赘述。
本申请实施例还提供了一种计算机可读介质，该计算机可读介质存储有至少一条程序代码，该至少一条程序代码由该处理器加载并执行以实现如上各个实施例示出的电源管理方法。
本申请实施例还提供了一种计算机程序产品，该计算机程序产品存储有至少一条程序代码，该至少一条程序代码由该处理器加载并执行以实现如上各个实施例示出的电源管理方法。
在一些实施例中，本申请实施例所涉及的计算机程序可被部署在一个计算机设备上执行，或者在位于一个地点的多个计算机设备上执行，又或者，在分布在多个地点且通过通信网络互连的多个计算机设备上执行，分布在多个地点 且通过通信网络互连的多个计算机设备可以组成区块链系统。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本申请实施例所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个程序代码或代码进行传输。计算机可读介质包括计算机存储介质和通信介质，其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。
以上所述仅为本申请的可选实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (20)

  1. 一种数据处理芯片，包括：电源管理单元、第一缓存单元、第二缓存单元和数据处理单元；
    所述电源管理单元用于控制对所述第一缓存单元和第二缓存单元的供电；
    所述第一缓存单元和所述第二缓存单元用于缓存数据；
    所述电源管理单元用于响应于所述数据处理单元的第一状态，控制对所述第一缓存单元供电，对所述第二缓存单元不供电，响应于所述数据处理单元的第二状态，控制对所述第一缓存单元供电并对所述第二缓存单元供电。
  2. 根据权利要求1所述的数据处理芯片，所述数据处理单元的第一状态包括不处理数据的状态，所述数据处理单元的第二状态包括处理数据的状态。
  3. 根据权利要求2所述的数据处理芯片，所述数据处理单元用于在第二状态下使用神经网络模型对数据进行处理，所述第一缓存单元用于存储所述神经网络模型的数据，所述第二缓存单元用于存储所述数据处理单元的待处理数据、中间处理数据和处理结果数据。
  4. 根据权利要求1-3所述的数据处理芯片，所述数据处理芯片还包括数据传输接口，所述数据传输接口用于接收数据；
    所述数据处理单元用于，响应于所述数据传输接口接收到的数据包含特定信息，从所述第一状态切换成所述第二状态。
  5. 根据权利要求4所述的数据处理芯片，所述特定信息包括以下至少一种：运动侦测信息、人形侦测信息。
  6. 根据权利要求1-3所述的数据处理芯片，所述第一缓存单元和所述第二缓存单元为同一易失性存储器的不同单元。
  7. 根据权利要求1-3所述的数据处理芯片，所述第一缓存单元和所述第二缓 存单元为不同的易失性存储器。
  8. 根据权利要求1-3所述的数据处理芯片，所述第一状态包括对神经网络模型进行模型训练的状态；所述第二状态包括基于神经网络模型进行数据处理的状态。
  9. 根据权利要求1-3所述的数据处理芯片，所述数据处理芯片还包括控制单元、神经网络处理子单元和数据处理子单元，所述控制单元分别与所述神经网络处理子单元和所述数据处理子单元连接；
    所述控制单元用于监控所述数据处理子单元生成的中间处理数据的数据量，响应于所述数据量与所述神经网络处理子单元的目标数据量匹配，触发所述神经网络处理子单元进行数据处理。
  10. 一种数据处理模组，所述数据处理模组包括如权利要求1-7任一项所述的数据处理芯片。
  11. 一种终端，所述终端包括如权利要求10所述的数据处理模组。
  12. 一种电源管理方法，所述方法应用于数据处理芯片，所述数据处理芯片包括第一缓存单元、第二缓存单元和数据处理单元，所述第一缓存单元和所述第二缓存单元用于缓存数据，所述电源管理方法包括：
    响应于所述数据处理单元的第一状态，对所述第一缓存单元供电，对所述第二缓存单元不供电；
    响应于所述数据处理单元的第二状态，对所述第一缓存单元供电并对所述第二缓存单元供电。
  13. 根据权利要求12所述的电源管理方法，所述数据处理单元的第一状态包括不处理数据的状态，所述数据处理单元的第二状态包括处理数据的状态。
  14. 根据权利要求13所述的电源管理方法，所述数据处理单元用于在第二状 态下使用神经网络模型对数据进行处理，所述第一缓存单元用于存储所述神经网络模型的数据，所述第二缓存单元用于存储所述数据处理单元的待处理数据、中间处理数据和处理结果数据。
  15. 根据权利要求12-14所述的电源管理方法，所述数据处理系统还包括数据传输接口，所述数据传输接口用于接收数据；
    所述数据处理单元用于，响应于所述数据传输接口接收到的数据包含特定信息，从所述第一状态切换成所述第二状态。
  16. 根据权利要求15所述的电源管理方法，所述特定信息包括以下至少一种：运动侦测信息、人形侦测信息。
  17. 根据权利要求12-14所述的电源管理方法，所述第一缓存单元和所述第二缓存单元为同一易失性存储器的不同单元。
  18. 根据权利要求12-14所述的电源管理方法，所述第一缓存单元和所述第二缓存单元为不同的易失性存储器。
  19. 一种计算机可读存储介质，所述计算机可读存储介质存储有至少一条程序代码，所述至少一条程序代码用于被处理器执行以实现如权利要求12至18任一所述的电源管理方法。
  20. 一种计算机程序产品，所述计算机程序产品存储有至少一条程序代码，所述至少一条程序代码由处理器加载并执行以实现如权利要求12至18任一项所述的电源管理方法。
PCT/CN2022/120651 2021-11-10 2022-09-22 数据处理芯片、模组、终端及电源管理方法 WO2023082860A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111329007.4A CN116107413A (zh) 2021-11-10 2021-11-10 数据处理芯片、模组、终端及电源管理方法
CN202111329007.4 2021-11-10

Publications (1)

Publication Number Publication Date
WO2023082860A1 true WO2023082860A1 (zh) 2023-05-19

Family

ID=86258469

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/120651 WO2023082860A1 (zh) 2021-11-10 2022-09-22 数据处理芯片、模组、终端及电源管理方法

Country Status (2)

Country Link
CN (1) CN116107413A (zh)
WO (1) WO2023082860A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101980103A (zh) * 2010-10-29 2011-02-23 威盛电子股份有限公司 电源状态管理方法及相关的计算机系统
CN102385562A (zh) * 2010-08-31 2012-03-21 联想(北京)有限公司 计算机和数据交互方法
US20130166935A1 (en) * 2011-12-26 2013-06-27 Yuuki Imaizumi Control device and computer program product for controlling method
US20160378592A1 (en) * 2014-03-20 2016-12-29 Kabushiki Kaisha Toshiba Cache memory and processor system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102385562A (zh) * 2010-08-31 2012-03-21 联想(北京)有限公司 计算机和数据交互方法
CN101980103A (zh) * 2010-10-29 2011-02-23 威盛电子股份有限公司 电源状态管理方法及相关的计算机系统
US20130166935A1 (en) * 2011-12-26 2013-06-27 Yuuki Imaizumi Control device and computer program product for controlling method
US20160378592A1 (en) * 2014-03-20 2016-12-29 Kabushiki Kaisha Toshiba Cache memory and processor system

Also Published As

Publication number Publication date
CN116107413A (zh) 2023-05-12

Similar Documents

Publication Publication Date Title
JP6742465B2 (ja) ブルートゥーススピーカーにおける連続ウェイクアップ遅延低減の方法、装置及びブルートゥーススピーカー
US7734936B2 (en) Method and apparatus for a computing system having an active sleep mode CPU that uses the cache of a normal active mode CPU
TWI512676B (zh) 用於顯示子系統之混和顯示圖框緩衝器
US11586544B2 (en) Data prefetching method and terminal device
CN105706168B (zh) 用于经由数据掩蔽来降低存储器i/o功率的系统、方法和计算机可读介质
US20160203083A1 (en) Systems and methods for providing dynamic cache extension in a multi-cluster heterogeneous processor architecture
US9116698B2 (en) Intelligent selection of operating state upon change of power state
CN108604211B (zh) 用于片上系统中的多区块数据事务的系统和方法
US9632562B2 (en) Systems and methods for reducing volatile memory standby power in a portable computing device
WO2022161227A1 (zh) 图像处理方法、装置、图像处理芯片和电子设备
CN114840450B (zh) 一种存储空间整理方法及电子设备
WO2022156600A1 (zh) 缓冲区容量的调整方法、装置、电子设备及可读存储介质
WO2023082860A1 (zh) 数据处理芯片、模组、终端及电源管理方法
US9489305B2 (en) System and method for managing bandwidth and power consumption through data filtering
CN108681469A (zh) 基于Android系统的页面缓存方法、装置、设备及存储介质
WO2024139279A1 (zh) 访问控制列表规则配置方法、装置、电子设备及存储介质
US12130690B2 (en) Technique for extended idle duration for display to improve power consumption
CN117311628A (zh) 一种固态硬盘的命令处理方法、装置、设备及介质
WO2023241026A1 (zh) 数据传输方法、装置、终端及存储介质
CN116128031A (zh) 数据处理芯片、模组、终端、数据处理方法和装置
US20200167646A1 (en) Data transmission method and calculation apparatus for neural network, electronic apparatus, computer-raedable storage medium and computer program product
CN107209544A (zh) 基于I/O操作特征的SoC空闲功率状态控制的系统和方法
US11630502B2 (en) Hierarchical state save and restore for device with varying power states
WO2023123496A1 (zh) 一种芯片系统及控制方法
WO2024021657A1 (zh) 存储控制方法、装置、设备、存储介质、芯片及存储器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22891662

Country of ref document: EP

Kind code of ref document: A1