CN114510139B

CN114510139B - Frequency modulation method and device and electronic equipment

Info

Publication number: CN114510139B
Application number: CN202011280933.2A
Authority: CN
Inventors: 潘品睿
Original assignee: Oneplus Technology Shenzhen Co Ltd
Current assignee: Oneplus Technology Shenzhen Co Ltd
Priority date: 2020-11-16
Filing date: 2020-11-16
Publication date: 2024-06-04
Anticipated expiration: 2040-11-16
Also published as: CN114510139A

Abstract

The embodiment of the invention relates to the technical field of electronics, and discloses a frequency modulation method, a frequency modulation device and electronic equipment, wherein the method comprises the following steps: determining the predicted load of the current frame according to the load state of at least one frame before the current frame; based on the preset frequency point combination set of at least one module, estimating the running time required by the current thread to run based on each preset frequency point combination of at least one module; determining power consumption generated by the current thread running based on each preset frequency point combination of at least one module according to the running time and at least one power consumption characteristic; and determining an optimal frequency point combination corresponding to at least one module according to the initial time constraint, the running time and the power consumption, and adjusting the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption on the premise of not dropping the frame. By the mode, the invention can save power consumption on the premise of not dropping frames.

Description

Frequency modulation method and device and electronic equipment

Technical Field

The present invention relates to the field of electronic technologies, and in particular, to a frequency modulation method, a frequency modulation device, and an electronic device.

Background

The development of modern technologies is faster and faster, especially the rising of mobile terminals such as smart phones and tablet computers, causes the rapid development of corresponding application programs such as chat programs and game programs, and the power consumption requirements of the application programs are larger and larger, while the battery of the electronic device is difficult to meet the long-time continuous high-power consumption requirements.

In order to reduce the power consumption of an electronic device, this is typically achieved by adjusting the frequency of the individual devices or modules, for example: CPU, DDR, GPU, the prior art generally determines the frequency by voting by a plurality of devices or modules, or determines the load by using a window, and then determines whether to increase or decrease the frequency by the load.

The inventor finds that the frame dropping phenomenon is easy to occur in the prior art scheme in the process of realizing the invention, and the problem of power consumption waste is not well solved.

In view of this, improvements are needed in the art.

Disclosure of Invention

The embodiment of the invention aims to provide a frequency modulation method, a frequency modulation device and electronic equipment, which solve the technical problems of frame dropping and power consumption waste in the current frequency modulation and realize the power consumption saving on the premise of not dropping frames.

In order to solve the technical problems, the embodiment of the invention provides the following technical scheme:

In a first aspect, an embodiment of the present invention provides a frequency modulation method, applied to an electronic device, where the method includes:

Determining the predicted load of the current frame according to the load state of at least one frame before the current frame;

According to the predicted load, based on a preset frequency point combination set of at least one module, estimating the running time required by the current thread to run based on each preset frequency point combination of at least one module;

determining power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic;

Determining an initial time constraint, and determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting a working frequency point of the at least one module for processing a current frame to be the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame.

In some embodiments, the determining the predicted load of the current frame according to the load state of at least one frame before the current frame includes:

and averaging the load state of at least one frame before the current frame, and determining the average value as the predicted load of the current frame.

Dividing the load state into at least two load gears, and establishing a load gear model;

Determining a load gear corresponding to each frame in at least one frame before the current frame;

predicting a load gear corresponding to the predicted load of the current frame according to the load gear corresponding to each frame in at least one frame before the current frame;

And determining the predicted load of the current frame according to the load gear corresponding to the predicted load of the current frame.

In some embodiments, the predicting the load gear corresponding to the predicted load of the current frame according to the load gear corresponding to each frame in at least one frame before the current frame includes:

acquiring the occurrence times of the load gear corresponding to the next frame under the condition that the load gear corresponding to each frame in at least one frame before the current frame in the load gear model is the same;

and determining the load gear with the largest occurrence number as the load gear corresponding to the predicted load of the current frame.

training a load prediction model;

And inputting the load state of at least one frame before the current frame into the load prediction model, and outputting the predicted load of the current frame.

In some embodiments, the estimating, according to the predicted load, the running time required by the current thread to run based on the preset frequency point combination of at least one module includes:

training a performance model offline, wherein the performance model stores the corresponding relation between the preset frequency point combination of at least one module and the running time;

And determining the running time required by the current thread to run based on the preset frequency point combination of the at least one module based on the performance model according to the preset frequency point combination of the at least one module.

In some embodiments, the determining the power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and the at least one power consumption characteristic includes:

offline training a power consumption model, wherein the power consumption model stores the running time and the corresponding relation between at least one power consumption characteristic and power consumption;

and determining the power consumption generated by the current thread running based on each preset frequency point combination of the at least one module based on the power consumption model according to the running time and the at least one power consumption characteristic.

In some embodiments, the determining the initial time constraint includes:

counting the frame rate of each frame in a preset time period before a current frame;

calculating the average frame rate in the preset time period;

If the average frame rate is smaller than the expected frame rate, reducing the time constraint of the previous frame of the current frame to determine an initial time constraint;

If the average frame rate is greater than or equal to the desired frame rate, increasing the time constraint of the previous frame of the current frame to determine an initial time constraint.

In some embodiments, the method further comprises:

dividing a preset time period into a plurality of time slices;

counting the average frame rate of each time segment in the preset time period;

comparing the average frame rate of each time segment with the expected frame rate, determining a time constraint adjustment trend for each time segment, the time constraint adjustment trend comprising increasing the time constraint or decreasing the time constraint;

Counting the number of each time constraint adjustment tendency in a preset time period, and determining the time constraint of the last frame of the current frame to be increased or decreased according to the number of each time constraint adjustment tendency in the preset time period so as to determine the time constraint of the current frame.

In some embodiments, the determining the optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption includes:

dynamically adjusting the initial time constraint, and determining a first frequency point combination set meeting the initial time constraint based on a preset frequency point combination set of at least one module;

searching a frequency point combination meeting preset conditions according to the first frequency point combination set, wherein the preset conditions comprise that the running time is less than or equal to an initial time constraint and the power consumption is minimum;

And determining the frequency point combination meeting the preset condition as the optimal frequency point combination corresponding to the at least one module.

In some embodiments, the method further comprises:

determining an operation result based on the current frame operated by the optimal frequency point combination;

and according to the operation result, adjusting the initial time constraint, and determining an updated initial time constraint so that the next frame of the current frame can determine the corresponding optimal frequency point combination based on the updated initial time constraint.

In some embodiments, the operation results include at least one of frame length, CPU frequency, GPU frequency, DDR frequency, frame drop condition, temperature, current.

In some embodiments, the load state comprises: at least one of the number of instructions required to process the current frame, the number of times a first level data cache miss occurs, the number of times a second level data cache miss occurs, the number of times a third level data cache miss occurs, CPU frequency, CPU usage, GPU frequency, GPU usage, DDR frequency, NPU usage, network speed, temperature;

The at least one module comprises at least one of a central processing unit, a graphics processing unit, a neural network processing unit, and a memory for storing the current frame;

the at least one power consumption characteristic includes at least one of frame length, CPU frequency, GPU frequency, DDR frequency, NPU frequency, temperature, power.

In a second aspect, an embodiment of the present invention provides a frequency modulation apparatus, applied to an electronic device, where the apparatus includes:

The load prediction unit is used for determining the predicted load of the current frame according to the load state of at least one frame before the current frame;

The running time unit is connected with the load prediction unit and used for predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module based on the preset frequency point combination set of at least one module according to the predicted load;

The power consumption unit is connected with the running time unit and used for determining power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic;

the optimal frequency point combination unit is connected with the load prediction unit, the operation time unit and the power consumption unit and is used for determining initial time constraint, determining the optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the operation time and the power consumption, and adjusting the working frequency point of the at least one module for processing the current frame into the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame.

In some embodiments, the load prediction unit comprises:

and the load prediction model is used for acquiring the load state of at least one frame before the current frame and determining the predicted load of the current frame according to the load state of at least one frame before the current frame.

In some embodiments, the runtime unit comprises:

the performance model is used for receiving the predicted load sent by the load prediction unit, and estimating the running time required by the current thread to run based on each preset frequency point combination of at least one module based on the preset frequency point combination set of at least one module according to the predicted load.

In some embodiments, the power consuming unit comprises:

and the power consumption model is used for receiving the running time sent by the performance model and determining the power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic.

In some embodiments, the best frequency point combining unit includes:

A time constraint model for determining an initial time constraint;

And the optimizer is used for determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting the working frequency point of the at least one module for processing the current frame into the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame.

In a third aspect, an embodiment of the present invention provides an electronic device, including:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the frequency modulation method described above.

In a fourth aspect, embodiments of the present invention provide a computer readable storage medium comprising a computer program which, when run on a computer or processor, causes the computer or processor to perform the frequency modulation method described above.

The embodiment of the invention has the beneficial effects that: in contrast to the situation of the prior art, the frequency modulation method provided by the embodiment of the invention is applied to electronic equipment, and the method comprises the following steps: determining the predicted load of the current frame according to the load state of at least one frame before the current frame; according to the predicted load, based on a preset frequency point combination set of at least one module, estimating the running time required by the current thread to run based on each preset frequency point combination of at least one module; determining power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic; determining an initial time constraint, and determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting a working frequency point of the at least one module for processing a current frame to be the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame. Through initial time constraint, running time and power consumption, based on a preset frequency point combination set of at least one module, the optimal frequency point combination corresponding to the at least one module is determined.

Drawings

One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.

Fig. 1 is a schematic structural diagram of a frequency modulation system according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a reinforcement learning model provided by an embodiment of the present invention;

FIG. 3 is a schematic diagram of an improved model based on reinforcement learning according to an embodiment of the present invention;

Fig. 4 is a schematic flow chart of a frequency modulation method according to an embodiment of the present invention;

Fig. 5 is a detailed flowchart of step S10 in fig. 4;

fig. 6 is a detailed flowchart of step S13 in fig. 5;

fig. 7 is a detailed flowchart of step S20 in fig. 4;

Fig. 8 is a detailed flowchart of step S30 in fig. 4;

Fig. 9 is a detailed flowchart of step S40 in fig. 4;

Fig. 10 is a schematic structural diagram of a frequency modulation device according to an embodiment of the present invention;

Fig. 11 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that, if not in conflict, the features of the embodiments of the present invention may be combined with each other, which is within the protection scope of the present invention. In addition, while functional block division is performed in a device diagram and logical order is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. Furthermore, the words "first," "second," "third," and the like as used herein do not limit the order of data and execution, but merely distinguish between identical or similar items that have substantially the same function and effect.

Before explaining the present invention in detail, terms and terminology involved in the embodiments of the present invention are explained, and the terms and terminology involved in the embodiments of the present invention are applicable to the following explanation.

(1) FPS (Frame per Second, FPS): the display frame number per second FPS is a definition in the field of images, and refers to the number of frames per second transmitted by a picture surface, and in colloquial terms, refers to the number of pictures of an animation or video. The FPS measures the amount of information used to save and display the dynamic video. The more frames per second, the smoother the displayed motion. Typically, the minimum to avoid motion dysfluency is 30. Some computer video formats can only provide 15 frames per second.

(2) EFPS (extended FRAMES PER Second, EFPS): the desired frame rate, each game or application, has its own desired FPS (e.g., game "principals glowing" currently has two EFPS settings selectable: 30 and 60). In general, the FPS is equal to the EFPS when the game or application is executed; however, if the current load is too heavy, the FPS may be smaller than the EFPS, for example, when the wander glows to set efps=60, the FPS may oscillate between 56-60.

(3) E-Period (Expected Period, expected FRAME LENGTH, E-Period): the desired frame length is 1/EFPS. For example: 60FPS game, E-Period is about 16.667ms.

(4) Frame length (FRAME LENGTH): the picture update time interval of the game can be roughly estimated as 1/FPS.

(5) Long Frame (Long Frame): meaning that the frame length of a frame is greater than the desired frame length (E-Period).

(6) Short Frame (Short Frame): meaning that the frame length of a frame is less than the desired frame length (E-Period).

(7) Very Long Frame (Very Long Frame): meaning that the frame length of a frame is much longer than E-Period, for example, when the frame length is more than twice as long, then the user will typically feel a clear click.

(8) Dropping frames (JANK): when the FPS is less than the EFPS, it is referred to as dropped frames. When the gap between EFPS and FPS is larger, also called FPS jitter is larger, the user can feel the click obviously.

(9) GPU drawing API: and drawing function libraries for drawing by using the GPU in the Android mobile phone.

(10) Drawing thread (RENDER THREAD): the library of drawing functions is responsible for drawing, and is most easily identified as a drawing thread because of direct calls to the drawing functions.

(11) Control Thread: the main function of the method is to control the frame rate and indirectly control the drawing start time of the drawing thread. This thread is difficult to identify because it does not directly call the library of drawing functions.

(12) Run time (run time): in situations where hardware performance is abundant, or where game play is not highly demanding, runtimes are typically less than or equal to FRAME LENGTH. For example, with the efps=40 game, the frame length of each frame is 25ms, but the game itself may run for only 15ms, and the remaining 10ms may be sleep time. In addition, the drawing thread and the control thread are respectively provided with different run times; but the frame length (FRAME LENGTH) will only be one at the time.

(13) System frequency modulation: the method refers to a built-in CPU and DDR frequency modulation strategy of a mobile phone. The original frequency modulation strategy is general, and some performance margins are usually left, namely: the system will operate at a slightly higher frequency than the CPU, DDR frequency that would have just run out in E-Period.

(14) A window: and counting the time unit for monitoring the thread load.

(15) Offline (Off-line) learning: offline learning refers to the direct analysis of data by a developer at the time of development and training of a model with that data. The benefit of learning at this stage is that the developer can greatly optimize the model. However, if too much offline learning is used in the algorithm, a game or application of a modification is easily caused, and the offline personnel must be readjusted, so that the universality is not high in online learning.

(16) Online (On-line) learning: i.e., online learning, as games or applications run on the electronic device, training and adjustment of the model is done in real time. In this process, the developer cannot stare at the right moment and optimize the learning result. The model can only learn from the originally planned steps and flows of the developer. However, if online learning can achieve good results, the model can be quite popular, so that the model does not need to be returned to offline training due to game or application modification.

In the prior art, the selection of the frequencies of the computing resources (e.g., CPU, DDR, GPU, NPU) are independent of each other. In the frequency selection of the CPU, whether the conventional DVFS or the EAS proposed later, the load is generally determined by a window method, and the load is used to determine whether to increase the current frequency. Under the rule of experience, the current CPU frequency technical scheme selects to prevent the fluctuation of task load with 1.25 times of efficiency.

In an electronic device, such as a smart phone, multiple devices or modules (CPU, GPU, modem) all use Double Data Rate (DDR) at the same time, so that DDR frequency selection depends on each module to vote and take the highest frequency. The CPU and DDR frequency modulation strategies native to smartphones are also typically used for general purpose, leaving some margin in performance. But with performance margins, represents a waste of power consumption. When multiple devices vote on the DDR frequency and take the frequency highest in the system, some devices may exaggerate the DDR frequency required by themselves, so that the voted DDR frequency is higher, and the waste of power consumption is caused.

In addition, the frequency selection of the native DDR and the frequency selection of the CPU are independent of each other. But in fact, there is some coupling relationship between DDR frequency and CPU frequency: at some point, the "low frequency DDR and high frequency CPU" and "high frequency DDR and low frequency CPU" may allow the game to not drop frames; however, in comparison with the two, the former may save power consumption (determined by actual measurement), so that simply adjusting the CPU frequency or DDR frequency easily results in waste of power consumption.

Based on the above problems, the embodiment of the invention provides a frequency modulation method, which realizes the optimal frequency point combination of at least one module with the lowest power consumption on the premise of not dropping frames.

Specifically, the embodiment of the invention is specifically described below by taking electronic equipment as an example of a smart phone.

Referring to fig. 1, fig. 1 is a schematic structural diagram of a fm system according to an embodiment of the invention;

As shown in fig. 1, the fm system 100 is applied to an electronic device, such as a smart phone, and the fm system is a hardware system architecture of the electronic device, and the fm system 100 includes: a central processing unit 110, a graphics processing unit 120, a memory 130, a neural network processing unit 140, a bus 150, and a hardware accelerator 160.

It will be appreciated that the illustrated frequency modulation system 100 may be included in one or more chips to form an electronic system. The system may be located in an electronic device, which may be a wireless terminal, a wired terminal, a user device or a connectionless device. For example, the electronic device may be a smart phone, a personal computer, a tablet computer, a game console, or the like.

The central processing unit 110 (central processing unit, CPU) is shown as being coupled to the graphics processing unit 120, the memory 130, the neural network processing unit 140, and the hardware accelerator 160 via a bus 150.

The illustrated graphics processing unit 120, i.e., (graphics processing unit, GPU), is coupled to the central processing unit 110, memory 130, neural network processing unit 140, and hardware accelerator 160 via bus 150.

The memory 130 shown may be, for example, a DDR memory, and the memory 130 may be used to store program code and data for the fm system 100. The memory 130 is connected to the central processing unit 110, the graphic processing unit 120, and the neural network processing unit 140 through a bus 150.

The neural network processing unit 140, i.e., (network process units, NPU), is shown as being coupled to the central processing unit 110, the graphics processing unit 120, the memory 130, and the hardware accelerator 160 via a bus 150.

It is understood that bus 150 may be any type of bus, such as a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus. The bus 150 may be classified into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one line is shown in FIG. 1, but not only one bus or one type of bus.

The cpu 110, the gpu 120, and the neural network processing unit 140 may be configured to process images, and the cpu 110, the gpu 120, and the neural network processing unit 140 may be on a chip and implement corresponding functions by reading software codes stored in the memory 130 through an interface for coupling with the outside of the chip. The software codes are read from the memory 130 into the chip interior via the interface and bus 150 and used by at least one of the central processing unit 110, the graphics processing unit 120, or the neural network processing unit 140.

The hardware accelerator 160 is shown connected to the central processing unit 110, the graphics processing unit 120, the memory 130, and the neural network processing unit 140 via the bus 150. The hardware accelerator 160 may include an application-specific integrated circuit (ASIC)), a field programmable gate array (field programmable GATE ARRAY, FPGA), or other programmable logic device, transistor logic device, hardware component, or any combination thereof.

The following describes the electronic device as a smart phone in detail with reference to the fm system 100 shown in fig. 1.

Referring to fig. 2, fig. 2 is a schematic diagram of a reinforcement learning model according to an embodiment of the invention;

as shown in fig. 2, the reinforcement learning model 20 includes: agent21 and Environment22.

The illustrated Environment22, i.e. the internal Environment of the smart phone, and the illustrated Agent21, i.e. the Agent, is an Agent that has to make an Action, i.e. an Action, by receiving State, i.e. a load State, through a learning model, where the Action is a frequency point combination of at least one module, and in fig. 2, the at least one module includes a CPU and a DDR, and the frequency point combination is a CPU frequency and a DDR frequency, i.e. a CPU frequency and a DDR frequency.

Wherein the State, i.e., the State, in the embodiment of the present invention refers to the load State, and the State in fig. 2 includes the number of instructions (instructions) required for processing the current frame, the number of times of occurrence of the data cache miss (CACHE MISS), and the usage rate (util), where CACHE MISS includes the number of times of occurrence of the first-level data cache miss, the number of times of occurrence of the second-level data cache miss, the number of times of occurrence of the third-level data cache miss, and the util includes the CPU usage rate, the GPU usage rate, and the NPU usage rate.

In the embodiment of the present invention, the load state may further include internal information of the electronic device, such as a CPU frequency, a GPU frequency, a DDR frequency, a network speed, and a temperature, that is, the load state includes at least one of an instruction number required for processing a current frame, a number of occurrences of a primary data cache miss, a number of occurrences of a secondary data cache miss, a number of occurrences of a tertiary data cache miss, a CPU frequency, a CPU usage rate, a GPU frequency, a GPU usage rate, a DDR frequency, an NPU usage rate, a network speed, and a temperature.

As shown in fig. 2, the event sends a state to the Agent, which receives the state to generate an Action, i.e., recommends a set of frequency point combinations for the event. After the system of the electronic device, such as the mobile phone system, runs in the frequency point combination, the result (such as whether the game is dropped in frames and more power consumption is saved than before) is transmitted back to the Agent in the form of Reward (rewards).

Among the ways in which Reward may be designed include: when the game drops frames, -1 is given. When the game is not dropped and the frequency is lower than the frequency recommended by the cell phone system, the difference in frequency is given as Reward (positive prize). When the game does not drop frames, but is equal to the frequency selected by the system, 0 is awarded.

The reinforcement learning Model is called a Model Free Model, wherein agents of the Model Free Model generally need to use Tabular Solution in reinforcement learning, such as Monte Carlo:

a period of time at the beginning must be tested by random trial and error, various combinations of frequency points are tried, and the results are recorded in a first table, wherein the first table is a high-dimension table, and the first table comprises Reward obtained by each State (Si) and adopting different actions (Aj).

During Deploy, i.e. during configuration of the frequency bin combinations, when state=si occurs, the highest rewarded Action (Ai) can be obtained by looking up in the table when Si occurs. The model finally obtains the CPU and DDR frequencies (i, j) that the model considers best, and this approach is called explloit.

During Deploy, the potentially optimal actions may be changed due to the game scene change, so there must be a small probability to try if other actions get higher Reward, this is called random trial and error (explor), e-greedy algorithm (epsilon greedy).

It will be appreciated that since the table is discrete and limited in number, but the state of many systems is continuous, the state of these systems must be shifted. For example: instructions is [0,1e10], different load ranges can be determined according to the number of instructions (instructions) required to process the current frame, to facilitate building the first table:

The Model Free Model can be learned online because of low calculation amount, and is not influenced by an operation chip and a game because of online learning. However, during the learning process, the relation of random trial and error (explore) may cause significant frame dropping and reduce the user experience.

To sum up, to overcome the above drawbacks, the offline learning must be changed. Offline learning must be directed to different computing chips, games, applications, etc., and must have its own model (i.e., high-dimensional tables). And the model must be changed when the game is updated. For cell phone vendors, if the updated information for the game or application is not determined in real time, the user experience may be impacted. And because different models are trained for different games and different operation chips offline, huge manpower resources are required to be spent.

On-line or off-line learning, however, has its limitations. The present invention therefore additionally proposes an improved Model Based on reinforcement learning, namely a Model Based Model, to avoid the above drawbacks.

Referring to fig. 3, fig. 3 is a schematic diagram of an improved model based on reinforcement learning according to an embodiment of the present invention;

as shown in fig. 3, the reinforcement learning-based improved model 30 includes:

agent31, the Agent;

environmental 32, the internal Environment of the smartphone;

And a load prediction model 33, configured to obtain a load state of at least one frame before the current frame, and determine a predicted load of the current frame according to the load state of at least one frame before the current frame.

And the performance model 34 is used for receiving the predicted load sent by the load prediction unit, and estimating the running time Tij required by the running of the current thread based on each preset frequency point combination of at least one module based on the preset frequency point combination set of at least one module according to the predicted load.

And the power consumption model 35 is used for receiving the running time sent by the performance model, and determining the power consumption Pij generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic.

And an Optimizer36, i.e. an Optimizer, where the Optimizer is configured to determine an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjust a working frequency point of the at least one module for processing the current frame to the optimal frequency point combination corresponding to the at least one module, where the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame.

When Deploy is performed, namely frequency point combination is configured, the whole flow is as follows:

The Environment32 transmits the system state, i.e., the load state, of at least one frame preceding the current frame to the Agent31 and the load prediction model 33.

The load prediction model 33 generates a predicted load from the load state, and outputs the predicted load to the performance model. The predicted load comprises at least one of instruction number required by processing the current frame, the occurrence frequency of primary data cache miss, the occurrence frequency of secondary data cache miss, the occurrence frequency of tertiary data cache miss, CPU frequency, CPU utilization rate, GPU frequency, GPU utilization rate, DDR frequency, NPU utilization rate, network speed and temperature;

the performance model 34 obtains the predicted load predicted by the load prediction model 33, and estimates the running time of the current thread, for example, running time runtimes (Tij) of the game threads. Where Tij represents the run time after the current thread is executed by the frequency bin combination of at least one module, for example: and executing the run after the game thread by using the CPU frequency i and the DDR frequency j.

The input of the power consumption model 35 is the running time Tij of the current thread and at least one power consumption feature sent by the performance model 34, wherein the power consumption feature comprises at least one of frame length, CPU frequency, GPU frequency, DDR frequency, NPU frequency, temperature and electric quantity, and the output of the power consumption model is the power consumption Pij generated by the current thread running based on each preset frequency point combination of the at least one module, for example, the power consumption required by executing the game thread by using the CPU frequency i and the DDR frequency j.

The inputs to the Optimizer36 are Tij, pij for all frequency bin combinations of the preset frequency bin combination set of at least one module, for example: tij, pij of all permutations and combinations of CPU, DDR.

The Optimizer obtains a Time Constraint Tc (Tc) from the Agent and calculates an optimal frequency point combination, e.g., CPU, DDR frequency point combination (i, j), corresponding to at least one module with minimal power consumption without dropping the game.

The system of the electronic device, for example, the mobile phone system, executes a game of one frame by using the optimal frequency point combination (i, j), and feeds back the running result (such as the frame length, whether the frame is dropped, the temperature, the current …, and other system states) to the Agent, and enables the Agent to adjust the system state, i.e., the load state, so that the next frame can obtain the reasonable time constraint Tc more accurately.

In the embodiment of the invention, the Agent includes a Time Constraint model (Time Constraint model), wherein the Time Constraint model, an Optimizer (Optimizer) and a mobile phone system form a reinforcement learning mechanism:

The Agent observes the state of the handset (e.g., instructions, CACHE MISS, CPU frequency, DDR frequency, GPU frequency, temperature, current) for the previous frames, taking Action (output Tc, or output after modifying Tc for the previous moment). The Optimizer selects a proper frequency point combination, such as a CPU frequency and a DDR frequency, according to the Tc, and recommends the frequency point combination to the mobile phone system, so that the mobile phone system operates by using the frequency point combination in the current frame.

At the end of the current frame (i.e. at the beginning of the next frame), it can be known whether to run dropped frames with the frequency bin combination and feed back whether dropped frames to the Agent. The Agent receives the feedback, and after adjusting the system state, the steps are repeated, and the process is continuously circulated.

The time constraint model comprises a model constructed by a statistical method such as a Monte Carlo method (Monte Carlo Method), and the like, and also can be a model using an artificial neural network (ARTIFICIAL NEURAL NETWORKS).

Wherein, after obtaining the appropriate Tc, the Optimizer solves the following constraint optimization (Constraint Optimization) by equation (1):

argminP _ij s.t.T_ij Tc formula (1)

Wherein:

i, j represents running the game at CPU frequency i and DDR frequency j;

pij represents the power consumption generated when running the game at CPU frequency i and DDR frequency j;

tij represents the run time Runtime required to run the game at CPU frequency i and DDR frequency j.

It can be understood that the above formula (1) is equivalent to solving the CPU and DDR frequency point combination with smaller power consumption under the condition of meeting the condition of not dropping frames;

As can be seen from the above equation, increasing or decreasing Tc changes the CPU and DDR frequency points that can be selected. When Tc is larger, tij meeting the time constraint is more, CPU and DDR frequency points with lower frequency are more likely to be selected, so that more power consumption can be saved. It can be understood that increasing Tc may suggest that the mobile phone system uses a lower frequency CPU, DDR, which may be considered as suggesting mobile phone system deceleration; conversely, decreasing Tc may be considered to accelerate the handset system.

Referring to fig. 4, fig. 4 is a flow chart of a frequency modulation method according to an embodiment of the invention;

As shown in fig. 4, the frequency modulation method includes:

Step S10: determining the predicted load of the current frame according to the load state of at least one frame before the current frame;

Specifically, the load state includes at least one of an instruction number (instructions) required to process the current frame, a number of times a first level data cache miss occurs (L1 DATA CACHE MISSES), a number of times a second level data cache miss occurs (L2 DATA CACHE MISSES), a number of times a third level data cache miss occurs (L3 DATA CACHE MISSES), a CPU frequency, a CPU utilization, a GPU frequency, a GPU utilization, a DDR frequency, an NPU utilization, a network speed, and a temperature;

Specifically, the determining the predicted load of the current frame according to the load state of at least one frame before the current frame includes:

And averaging the load state of at least one frame before the current frame, and determining the average value as the predicted load of the current frame, wherein the average value is determined as the first predicted load of the current frame.

Specifically, the load state of each frame in at least one frame before the current frame is determined, the load state of each frame in at least one frame before the current frame is averaged, and the average value is determined as the predicted load of the current frame. It will be appreciated that at least one frame preceding the current frame should be a continuous multi-frame to improve the accuracy of the predicted load.

Specifically, referring to fig. 5 again, fig. 5 is a detailed flowchart of step S10 in fig. 4;

As shown in fig. 5, this step S10: determining a predicted load of the current frame according to a load state of at least one frame before the current frame, including:

Step S11: dividing the load state into at least two load gears, and establishing a load gear model;

Specifically, the load state includes the number of instructions (instructions) required to process the current frame, where the value range of the number of instructions instructions is [0,1e10], so the number of instructions (instructions) required to process the current frame can be divided into several load gears, for example: dividing the instruction number into 5 load gears according to the number of the instruction number, wherein the load gears are respectively as follows:

First load gear: [0,2e9]

Second load gear: [2e9,4e9]

Third load gear: [4e9,6e9]

Fourth load gear: [6e9,8e9]

Fifth load gear: [8e9,1e10]

And establishing a load gear model according to the load gear, wherein the load gear model comprises a load gear of at least one frame before the current frame and a load gear of a next frame after the current frame, and the load gear model further comprises the occurrence times of the load gear corresponding to the next frame under the condition that the load gear corresponding to each frame in at least one frame before the current frame is the same, for example: and when the load gear of the first three frames before the current frame is [3,5 and 4] respectively, the current frame is the occurrence frequency of 1. It will be appreciated that the load range model includes [ k,3,5,4, i=1-5 ], where k is the number of occurrences and i is the load range, which may be 1 to 5, representing the load range of the current frame.

Step S12: determining a load gear corresponding to each frame in at least one frame before the current frame;

Specifically, according to the instruction number of each frame in at least one frame before the current frame, determining a load gear corresponding to each frame in at least one frame before the current frame.

Step S13: predicting a load gear corresponding to the predicted load of the current frame according to the load gear corresponding to each frame in at least one frame before the current frame;

specifically, referring to fig. 6 again, fig. 6 is a detailed flowchart of step S13 in fig. 5;

As shown in fig. 6, this step S13: according to the load gear corresponding to each frame in at least one frame before the current frame, predicting the load gear corresponding to the predicted load of the current frame comprises:

Step S131: acquiring the occurrence times of the load gear corresponding to the next frame under the condition that the load gear corresponding to each frame in at least one frame before the current frame in the load gear model is the same;

Specifically, the load gear model includes the occurrence times of the load gear corresponding to each frame in at least one frame before the current frame, for example: and when the load gear of the first three frames before the current frame is [3,5 and 4] respectively, the current frame is the occurrence frequency of 1.

Step S132: and determining the load gear with the largest occurrence number as the load gear corresponding to the predicted load of the current frame.

Specifically, if the load gear of the first three frames before the current frame is [3,5,4] respectively, the number of occurrences that the current frame is 1 is the largest, and the load gear corresponding to the predicted load of the current frame is determined to be 1.

It will be appreciated that after the current frame is processed, the load gear model will be updated, for example: the load gear is updated and the number of occurrences is redetermined.

Step S14: and determining the predicted load of the current frame according to the load gear corresponding to the predicted load of the current frame.

Specifically, according to the load gear corresponding to the predicted load of the current frame, the predicted load of the current frame is determined as the median of the load gear, for example: if the load gear corresponding to the predicted load of the current frame is the first load gear and the value range of the first load gear is [0,2e9], determining that the load gear corresponding to the predicted load of the current frame is the median of the value range of the first load gear, namely 1e9, wherein the predicted load of the current frame is determined according to the load gear corresponding to the predicted load of the current frame, and the predicted load is determined to be the second predicted load.

In an embodiment of the present invention, the determining the predicted load of the current frame according to the load status of at least one frame before the current frame includes:

training a load prediction model;

Specifically, the load state in the preset time period is obtained and used as a training set, a load prediction model is trained, the load prediction model can be a neural network model, for example, a Long Short-Term Memory (LSTM), the load state of at least one frame before the current frame is used as a test set, the load state of at least one frame before the current frame is input into the load prediction model, and the predicted load of the current frame is output. And inputting the load state of at least one frame before the current frame into the load prediction model, and outputting the predicted load of the current frame, wherein the predicted load is a third predicted load.

In an embodiment of the present invention, the method further includes:

and averaging the first predicted load, the second predicted load and the third predicted load according to the first predicted load, the second predicted load and the third predicted load, and taking the average value as a final predicted load.

The method and the device can further improve the accuracy of the predicted load by respectively determining the predicted load in a plurality of modes and taking the predicted load determined in the plurality of modes as the final predicted load.

Step S20: according to the predicted load, based on a preset frequency point combination set of at least one module, estimating the running time required by the current thread to run based on each preset frequency point combination of at least one module;

specifically, referring to fig. 7 again, fig. 7 is a detailed flowchart of step S20 in fig. 4;

as shown in fig. 7, this step S20: according to the predicted load, based on a preset frequency point combination set of at least one module, estimating the running time required by the current thread to run based on each preset frequency point combination of at least one module, including:

step S21: training a performance model offline, wherein the performance model stores the corresponding relation between the preset frequency point combination of at least one module and the running time;

In particular, the performance model may be obtained through supervised learning (Supervise Learning). It will be appreciated that the performance model is only related to the computing chip of the electronic device and not to the running game or application, and thus, the performance model may be accomplished through offline training or offline training. That is, the performance model needs to be retrained only when the computing chip of the electronic device is replaced.

In an embodiment of the invention, the performance Model includes one or more of a machine learning Model, an artificial neural network Model (ARTIFICIAL NEURAL NETWORKS), a deep learning Model (DEEP LEARNING Model), a statistical learning Model, such as a linear regression Model (Linear Regression), a Curve Fitting Model (Curve Fitting).

Step S22: and determining the running time required by the current thread to run based on the preset frequency point combination of the at least one module based on the performance model according to the preset frequency point combination of the at least one module.

Specifically, the input of the performance model is a load state predicted by a load prediction model, including: the output of the performance model is the running time (Runtime) required by the current frame of the current thread to run at the preset frequency point combination of at least one module, for example, the output of the performance model is the running time Tij required by the current frame to run at the CPU frequency i and the DDR frequency j.

Step S30: determining power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic;

specifically, referring to fig. 8 again, fig. 8 is a detailed flowchart of step S30 in fig. 4;

As shown in fig. 8, this step S30: according to the running time and at least one power consumption characteristic, determining the power consumption generated by the current thread running based on each preset frequency point combination of the at least one module, wherein the method comprises the following steps:

step S31: offline training a power consumption model, wherein the power consumption model stores the running time and the corresponding relation between at least one power consumption characteristic and power consumption;

The running time is the running time predicted by the performance model, that is, the running time required by the at least one module to run in a preset frequency point combination, and the power consumption features include at least one of frame length, CPU frequency, GPU frequency, DDR frequency, NPU frequency, temperature, and electric quantity, and the power consumption features are features related to power consumption, and it should be understood that any feature related to power consumption in the electronic device may be regarded as the power consumption features, which falls within the scope of protection of the present invention.

In particular, the power consumption model may be obtained through supervised learning (Supervise Learning). It will be appreciated that the power consumption model is only related to the operational chip of the electronic device and not to the running game or application, and thus, the power consumption model may be accomplished through offline training or offline training. That is, the power consumption model needs to be retrained only when the operation chip of the electronic device is replaced.

In an embodiment of the invention, the power consumption Model includes one or more of a machine learning Model, an artificial neural network Model (ARTIFICIAL NEURAL NETWORKS), a deep learning Model (DEEP LEARNING Model), a statistical learning Model, such as a linear regression Model (Linear Regression), a Curve Fitting Model (Curve Fitting).

Step S32: and determining the power consumption generated by the current thread running based on each preset frequency point combination of the at least one module based on the power consumption model according to the running time and the at least one power consumption characteristic.

Specifically, the input of the power consumption model is the running time and at least one power consumption characteristic, and the output of the power consumption model is the power consumption generated by the current thread running based on each preset frequency point combination of the at least one module.

Step S40: determining an initial time constraint, and determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting a working frequency point of the at least one module for processing a current frame to be the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame.

Specifically, the determining the initial time constraint includes:

calculating the average frame rate in the preset time period;

Specifically, the preset time period may be set manually, for example, to half a second, one second, or the like, or determined by the number of frames, for example: the frame number is 40 frames, the preset time period before the current frame comprises a plurality of frames, the average frame rate of the frames is calculated by counting the frame rate of each frame of the frames, and the average frame rate of the frames is taken as the average frame rate of the preset time period.

Wherein, assuming that the average frame rate is AFPS (AVERAGE FRAMES PER Second, AFPS), the desired frame rate, EFPS (extended FRAMES PER Second, EFPS), decreases the time constraint Tc of the previous frame of the current frame when AFPS < EFPS, and increases the time constraint Tc of the previous frame of the current frame when AFPS is greater than or equal to EFPS.

In an embodiment of the present invention, the method further includes:

When the average frame rate AFPS is approximately equal to the expected frame rate EFPS, that is, the average frame rate AFPS is approximately equal to the expected frequency EFPS, at this time, the time constraint Tc is increased heuristically, and the threads are executed through different parameter configurations, that is, the frequency point combinations, and the parameter configurations with the smallest frame dropping situation and the lowest power consumption are determined, that is, the frequency point combinations with the smallest frame dropping situation and the lowest power consumption are determined, so that the time constraint corresponding to the frequency point combinations with the smallest frame dropping situation and the lowest power consumption is determined, and the initial time constraint is adjusted to the time constraint corresponding to the frequency point combinations with the smallest frame dropping situation and the lowest power consumption.

In an embodiment of the present invention, the method further includes:

dividing a preset time period into a plurality of time slices, and counting the average frame rate of each time slice in the preset time period;

Specifically, dividing the preset time period into a plurality of time slices is equivalent to dividing the preset time period into a plurality of cells, each cell is a time slice, and the lengths of each time slice can be equal or unequal.

Determining a time constraint adjustment trend for each time segment by comparing the average frame rate of each time segment with the expected frame rate, the time constraint adjustment trend comprising increasing the time constraint or decreasing the time constraint, equivalent to voting to decide whether to increase or decrease the time constraint;

determining to increase or decrease the time constraint of the previous frame of the current frame according to the number of the time constraint adjustment tendencies in the preset time period by counting the number of the time constraint adjustment tendencies in the preset time period so as to determine the time constraint of the current frame;

specifically, if the number of the increased time constraints is greater than the number of the decreased time constraints, determining to increase the time constraint of the previous frame of the current frame, and determining the increased time constraint as the time constraint of the current frame; if the number of the increased time constraints is smaller than the number of the decreased time constraints, determining to decrease the time constraint of the previous frame of the current frame, and determining the decreased time constraint as the time constraint of the current frame. Wherein the increased or decreased time constraint value is determined by a preset rule, for example, the preset rule includes: and determining an increased or decreased time constraint value proportionally according to the difference value between the number of increased time constraints and the number of decreased time constraints, wherein the time constraint of the current frame=the time constraint of the previous frame of the current frame + -the increased or decreased time constraint value.

In the embodiment of the invention, the time constraint is heuristically increased or decreased by counting the frame rate change in the preset time period, so that the better time constraint can be determined under the condition that the frame is not dropped greatly, and the frequency point combination with lower power consumption can be determined on the premise that the frame is not dropped is met.

In an embodiment of the present invention, the determining an initial time constraint further includes:

and determining the initial time constraint according to the running time of the drawing thread of at least one frame before the current frame, or determining the initial time constraint according to the experience value of the application program corresponding to the current frame.

Referring to fig. 9 again, fig. 9 is a detailed flowchart of step S40 in fig. 4;

As shown in fig. 9, this step S40: according to the initial time constraint, the running time and the power consumption, determining the optimal frequency point combination corresponding to the at least one module comprises the following steps:

Step S41: dynamically adjusting the initial time constraint, and determining a first frequency point combination set meeting the initial time constraint based on a preset frequency point combination set of at least one module;

Specifically, the meeting the initial time constraint refers to that the running time of running the current frame based on the preset frequency point combination is smaller than the initial time constraint, and the initial time constraint is dynamically adjusted, for example: and increasing or decreasing the initial time constraint, determining an updated initial time constraint, and acquiring a first frequency point combination set meeting the updated initial time constraint from a preset frequency point combination set of the at least one module based on the updated initial time constraint.

Step S42: searching a frequency point combination meeting preset conditions according to the first frequency point combination set, wherein the preset conditions comprise that the running time is less than or equal to an initial time constraint and the power consumption is minimum;

Specifically, the first frequency point combination set is a plurality of frequency point combinations meeting the initial time constraint, and the first frequency point combination set is screened through preset conditions to determine the optimal frequency point combination. The preset condition includes that the running time is smaller than or equal to an initial time constraint and the power consumption is minimum, which is equivalent to screening a frequency point combination with the running time smaller than or equal to the initial time constraint and the power consumption minimum from the first frequency point combination set, and determining the frequency point combination as an optimal frequency point combination.

Step S43: and determining the frequency point combination meeting the preset condition as the optimal frequency point combination corresponding to the at least one module.

Specifically, the screening operation time in the first frequency point combination set is smaller than or equal to the initial time constraint, and the frequency point combination with the minimum power consumption is determined to be the optimal frequency point combination corresponding to the at least one module, and the optimal frequency point combination is recommended to a system of the electronic equipment, such as a mobile phone system, so that the mobile phone system operates the current frame based on the optimal frequency point combination.

In an embodiment of the present invention, the method further includes:

Specifically, the operation result includes at least one of frame length, CPU frequency, GPU frequency, DDR frequency, frame dropping condition, temperature, and current.

Specifically, according to the operation result, determining the load state of the current frame, thereby determining the predicted load of the next frame of the current frame, and according to the predicted load of the next frame of the previous frame, estimating the operation time required by the current thread for operation based on each preset frequency point combination of at least one module based on the preset frequency point combination set of at least one module; determining power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic; and determining an updated initial time constraint so that a next frame of the current frame determines a corresponding optimal frequency point combination based on the updated initial time constraint.

In an embodiment of the present invention, a frequency modulation method is provided and applied to an electronic device, where the method includes: determining the predicted load of the current frame according to the load state of at least one frame before the current frame; according to the predicted load, based on a preset frequency point combination set of at least one module, estimating the running time required by the current thread to run based on each preset frequency point combination of at least one module; determining power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic; determining an initial time constraint, and determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting a working frequency point of the at least one module for processing a current frame to be the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame. Through initial time constraint, running time and power consumption, based on a preset frequency point combination set of at least one module, the optimal frequency point combination corresponding to the at least one module is determined.

Referring to fig. 10 again, fig. 10 is a schematic structural diagram of a frequency modulation device according to an embodiment of the present invention;

as shown in fig. 10, the frequency modulation device 10 is applied to an electronic apparatus, and the frequency modulation device 10 includes:

a load prediction unit 101, configured to determine a predicted load of the current frame according to a load state of at least one frame preceding the current frame;

The running time unit 102 is connected with the load prediction unit and is used for predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module based on the preset frequency point combination set of at least one module according to the predicted load;

A power consumption unit 103, connected to the runtime unit, configured to determine, according to the runtime and at least one power consumption feature, power consumption generated by a current thread running based on each preset frequency point combination of the at least one module;

And the optimal frequency point combination unit 104 is connected with the load prediction unit, the operation time unit and the power consumption unit and is used for determining an initial time constraint, determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the operation time and the power consumption, and adjusting the working frequency point of the at least one module for processing the current frame into the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame.

In an embodiment of the present invention, the load prediction unit includes:

In an embodiment of the present invention, the runtime unit includes:

In an embodiment of the present invention, the power consumption unit includes:

In an embodiment of the present invention, the optimal frequency point combining unit includes:

A time constraint model for determining an initial time constraint;

In an embodiment of the present invention, the load prediction unit is configured to:

In an embodiment of the present invention, the load prediction unit is further configured to:

In an embodiment of the present invention, the predicting a load gear corresponding to a predicted load of a current frame according to a load gear corresponding to each frame in at least one frame before the current frame includes:

training a load prediction model;

In an embodiment of the present invention, the runtime unit is configured to:

In an embodiment of the present invention, the power consumption unit is configured to:

In the embodiment of the present invention, the optimal frequency point combining unit is configured to:

calculating the average frame rate in the preset time period;

dividing a preset time period into a plurality of time slices;

counting the average frame rate of each time segment in the preset time period;

In the embodiment of the present invention, the optimal frequency point combining unit is further configured to:

In the embodiment of the invention, the operation result comprises at least one of frame length, CPU frequency, GPU frequency, DDR frequency, frame dropping condition, temperature and current.

In an embodiment of the present invention, the load state includes: at least one of the number of instructions required to process the current frame, the number of times a first level data cache miss occurs, the number of times a second level data cache miss occurs, the number of times a third level data cache miss occurs, CPU frequency, CPU usage, GPU frequency, GPU usage, DDR frequency, NPU usage, network speed, temperature;

In an embodiment of the present invention, a frequency modulation device is provided and applied to an electronic device, where the device includes: the load prediction unit is used for determining the predicted load of the current frame according to the load state of at least one frame before the current frame; the running time unit is connected with the load prediction unit and used for predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module based on the preset frequency point combination set of at least one module according to the predicted load; the power consumption unit is connected with the running time unit and used for determining power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic; the optimal frequency point combination unit is connected with the load prediction unit, the operation time unit and the power consumption unit and is used for determining initial time constraint, determining the optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the operation time and the power consumption, and adjusting the working frequency point of the at least one module for processing the current frame into the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame. Through initial time constraint, running time and power consumption, based on a preset frequency point combination set of at least one module, the optimal frequency point combination corresponding to the at least one module is determined.

Referring to fig. 11, fig. 11 is a schematic diagram illustrating a hardware structure of an electronic device according to various embodiments of the present invention;

As shown in fig. 11, the electronic device 11 includes, but is not limited to: radio frequency unit 111, network module 112, audio output unit 113, input unit 114, sensor 115, display unit 116, user input unit 117, interface unit 118, memory 119, processor 1110, and power supply 1111, etc., and the electronic device further comprises a camera. It will be appreciated by those skilled in the art that the structure of the electronic device shown in fig. 11 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than illustrated, or may combine certain components, or may have a different arrangement of components. In the embodiment of the invention, the electronic equipment comprises, but is not limited to, a television, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer and the like.

A processor 1110 for determining a predicted load of the current frame according to a load status of at least one frame preceding the current frame; according to the predicted load, based on a preset frequency point combination set of at least one module, estimating the running time required by the current thread to run based on each preset frequency point combination of at least one module; determining power consumption generated by the current thread running based on each preset frequency point combination of the at least one module according to the running time and at least one power consumption characteristic; determining an initial time constraint, and determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting a working frequency point of the at least one module for processing a current frame to be the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame.

In the embodiment of the invention, the optimal frequency point combination corresponding to at least one module is determined based on the preset frequency point combination set of the at least one module through initial time constraint, running time and power consumption.

It should be understood that, in the embodiment of the present invention, the radio frequency unit 111 may be used for receiving and transmitting signals during the process of receiving and transmitting information or communication, specifically, receiving downlink data from a base station, and then processing the downlink data by the processor 1110; and, the uplink data is transmitted to the base station. Typically, the radio frequency unit 111 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 111 may also communicate with networks and other devices through a wireless communication system.

The electronic device 11 provides wireless broadband internet access to the user via the network module 112, such as helping the user to email, browse web pages, access streaming media, and the like.

The audio output unit 113 may convert audio data received by the radio frequency unit 111 or the network module 112 or stored in the memory 119 into an audio signal and output as sound. Also, the audio output unit 113 may also provide audio output (e.g., call signal reception sound, message reception sound, etc.) related to a specific function performed by the electronic device 11. The audio output unit 113 includes a speaker, a buzzer, a receiver, and the like.

The input unit 114 is used to receive an audio or video signal. The input unit 114 may include a graphics processor (Graphics Processing Unit, GPU) 1141 and a microphone 1142, the graphics processor 1141 processing a target image of a still picture or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 116. The image frames processed by the graphics processor 1141 may be stored in memory 119 (or other storage medium) or transmitted via the radio frequency unit 111 or the network module 112. Microphone 1142 may receive sound and may be capable of processing such sound into audio data. The processed audio data may be converted into a format output that can be transmitted to the mobile communication base station via the radio frequency unit 111 in the case of a telephone call mode.

The electronic device 11 further comprises at least one sensor 115, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1161 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 1161 and/or the backlight when the electronic device 11 moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for recognizing the gesture of the electronic equipment (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; the sensor 115 may further include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which are not described herein.

The display unit 116 is used to display information input by a user or information provided to the user. The display unit 116 may include a display panel 1161, and the display panel 1161 may be configured in the form of a Liquid crystal display (Liquid CRYSTAL DISPLAY, LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 117 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the electronic device. Specifically, the user input unit 117 includes a touch panel 1171 and other input devices 1172. The touch panel 1171, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (such as operations of the user on the touch panel 1171 or thereabout using any suitable object or accessory such as a finger, stylus, or the like). The touch panel 1171 may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends the touch point coordinates to the processor 1110, and receives and executes commands sent from the processor 1110. In addition, the touch panel 1171 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1171, the user input unit 117 may also include other input devices 1172. In particular, other input devices 1172 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein.

Further, the touch panel 1171 may be overlaid on the display panel 1161, and when the touch panel 1171 detects a touch operation thereon or thereabout, the touch panel 1171 is transferred to the processor 1110 to determine a type of touch event, and then the processor 1110 provides a corresponding visual output on the display panel 1161 according to the type of touch event. Although in fig. 11, the touch panel 1171 and the display panel 1161 are two independent components for implementing the input and output functions of the electronic device, in some embodiments, the touch panel 1171 may be integrated with the display panel 1161 to implement the input and output functions of the electronic device, which is not limited herein.

The interface unit 118 is an interface for connecting an external device to the electronic apparatus 11. For example, the external devices may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 118 may be used to receive input (e.g., data information, power, etc.) from an external device and to transmit the received input to one or more elements within the electronic apparatus 11 or may be used to transmit data between the electronic apparatus 11 and an external device.

Memory 119 may be used to store software programs as well as various data. The memory 119 may mainly include a storage program area and a storage data area, wherein the storage program area may store an application program 1191 (such as a sound playing function, an image playing function, etc.) required for at least one function, an operating system 1192, etc.; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 119 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

The processor 1110 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 119, and calling data stored in the memory 119, thereby performing overall monitoring of the electronic device. Processor 1110 may include one or more processing units; preferably, the processor 1110 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 1110.

The electronic device 11 may also include a power supply 1111 (e.g., a battery) for powering the various components, and the power supply 1111 may preferably be logically coupled to the processor 1110 by a power management system for performing functions such as managing charging, discharging, and power consumption by the power management system.

In addition, the electronic device 11 includes some functional modules, which are not shown, and will not be described in detail herein.

Preferably, the embodiment of the present invention further provides an electronic device, including a processor 1110, a memory 119, and a computer program stored in the memory 119 and capable of running on the processor 1110, where the computer program when executed by the processor 1110 implements the respective processes of the above embodiment of the battery detection method, and the same technical effects can be achieved, and for avoiding repetition, a detailed description is omitted herein.

The electronic device 11 of the embodiments of the present invention exists in a variety of forms including, but not limited to:

(1) A mobile communication device: such devices are characterized by mobile communication capabilities and are primarily aimed at providing voice, data communications. Such electronic devices include smart phones (e.g., iphones), multimedia phones, functional phones, and low-end phones, among others.

(2) Mobile personal computer equipment, which belongs to the category of personal computers, has the functions of calculation and processing and generally has the characteristic of mobile internet surfing. Such electronic devices include: PDA, MID, and UMPC devices, etc., such as iPad.

(3) Portable entertainment device: such devices can display and play video content, and typically also have mobile internet features. The device comprises: video players, palm game players, smart toys and portable car navigation devices.

(4) And other electronic devices with video playing function and internet surfing function.

The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by one or more processors, implements the processes of the above battery detection method embodiment, and can achieve the same technical effects, so that repetition is avoided, and no further description is given here. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above-described embodiments of the apparatus or device are merely illustrative, in which the unit modules illustrated as separate components may or may not be physically separate, and the components shown as unit modules may or may not be physical units, may be located in one place, or may be distributed over multiple network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising several instructions for causing a terminal (which may be a mobile terminal, a personal computer, a server, or a network device, etc.) to perform the method according to the embodiments or some parts of the embodiments of the present invention.

Finally, it should be noted that: the embodiments described above in connection with the accompanying drawings are only for illustrating the technical aspects of the present application, and the present application is not limited to the above-described embodiments, which are only illustrative, but not restrictive; the technical features of the above embodiments or in the different embodiments may also be combined within the idea of the application, the steps may be implemented in any order, and there are many other variations of the different aspects of the application as described above, which are not provided in detail for the sake of brevity; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.

Claims

1. A frequency modulation method applied to an electronic device, the method comprising:

Determining an initial time constraint, and determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting a working frequency point of the at least one module for processing a current frame to be the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame;

The determining an initial time constraint includes:

calculating the average frame rate in the preset time period;

If the average frame rate is greater than or equal to the expected frame rate, increasing the time constraint of the previous frame of the current frame to determine an initial time constraint;

The determining, according to the initial time constraint, the running time and the power consumption, the optimal frequency point combination corresponding to the at least one module includes:

2. The method of claim 1, wherein determining the predicted load for the current frame based on the load status of at least one frame preceding the current frame comprises:

3. The method of claim 1, wherein determining the predicted load for the current frame based on the load status of at least one frame preceding the current frame comprises:

4. A method according to claim 3, wherein predicting a load gear corresponding to a predicted load of the current frame based on a load gear corresponding to each of at least one frame preceding the current frame comprises:

5. The method of claim 1, wherein determining the predicted load for the current frame based on the load status of at least one frame preceding the current frame comprises:

training a load prediction model;

6. The method of claim 1, wherein predicting the runtime required for the current thread to run based on the preset frequency bin combination of the at least one module according to the predicted load comprises:

7. The method of claim 1, wherein determining the power consumption by the current thread based on each predetermined frequency point combination of the at least one module operation based on the run time and at least one power consumption characteristic comprises:

8. The method according to claim 1, wherein the method further comprises:

dividing a preset time period into a plurality of time slices;

counting the average frame rate of each time segment in the preset time period;

9. The method according to claim 1, wherein the method further comprises:

10. The method of claim 9, wherein the operation result comprises at least one of frame length, CPU frequency, GPU frequency, DDR frequency, frame drop condition, temperature, current.

11. The method according to any one of claims 1-10, wherein the load state comprises: at least one of the number of instructions required to process the current frame, the number of times a first level data cache miss occurs, the number of times a second level data cache miss occurs, the number of times a third level data cache miss occurs, CPU frequency, CPU usage, GPU frequency, GPU usage, DDR frequency, NPU usage, network speed, temperature;

12. A frequency modulation device for use in an electronic device, the device comprising:

The optimal frequency point combination unit is connected with the load prediction unit, the operation time unit and the power consumption unit and is used for determining initial time constraint, determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the operation time and the power consumption, and adjusting the working frequency point of the at least one module for processing the current frame into the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame;

The optimal frequency point combining unit is specifically configured to:

calculating the average frame rate in the preset time period;

the optimal frequency point combining unit is specifically configured to:

13. The apparatus of claim 12, wherein the load prediction unit comprises:

14. The apparatus of claim 13, wherein the runtime unit comprises:

15. The apparatus of claim 14, wherein the power consuming element comprises:

16. The apparatus of claim 15, wherein the best frequency bin combining unit comprises:

A time constraint model for determining an initial time constraint;

17. An electronic device, comprising:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the frequency modulation method of any one of claims 1-11.

18. A computer readable storage medium comprising a computer program which, when run on a computer or processor, causes the computer or processor to perform the frequency modulation method as claimed in any one of claims 1 to 11.