CN114510139A

CN114510139A - Frequency modulation method and device and electronic equipment

Info

Publication number: CN114510139A
Application number: CN202011280933.2A
Authority: CN
Inventors: 潘品睿
Original assignee: Oneplus Technology Shenzhen Co Ltd
Current assignee: Oneplus Technology Shenzhen Co Ltd
Priority date: 2020-11-16
Filing date: 2020-11-16
Publication date: 2022-05-17
Anticipated expiration: 2040-11-16
Also published as: CN114510139B

Abstract

The embodiment of the invention relates to the technical field of electronics, and discloses a frequency modulation method, a frequency modulation device and electronic equipment, wherein the method comprises the following steps: determining the predicted load of the current frame according to the load state of at least one frame before the current frame; based on a preset frequency point combination set of at least one module, estimating the running time required by the running of the current thread based on each preset frequency point combination of at least one module; determining power consumption generated by the current thread based on the combined operation of each preset frequency point of at least one module according to the operation time and at least one power consumption characteristic; determining an optimal frequency point combination corresponding to at least one module according to initial time constraint, running time and power consumption, and adjusting a working frequency point of a current frame processed by at least one module to the optimal frequency point combination corresponding to at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping frames. By the mode, the power consumption can be saved on the premise of not dropping frames.

Description

Frequency modulation method and device and electronic equipment

Technical Field

The present invention relates to the field of electronic technologies, and in particular, to a frequency modulation method and apparatus, and an electronic device.

Background

Modern science and technology is developing more and more rapidly, especially electronic equipment, for example mobile terminals such as smart phones, tablet computers and the like are rising, so that application programs corresponding to the electronic equipment, for example, programs such as chat programs, game programs and the like are rapidly developed, the power consumption requirement of the electronic equipment is greater and greater, and the battery of the electronic equipment cannot meet the long-lasting large power consumption requirement.

In order to reduce the power consumption of an electronic device, this is usually achieved by adjusting the frequency of the individual devices or modules, for example: in the prior art, the frequency is generally determined by voting through a plurality of devices or modules, or the load is determined by using a window, and then the load determines whether to increase or decrease the frequency.

The inventor finds that the scheme in the prior art easily causes the occurrence of the frame dropping phenomenon in the process of implementing the invention, and the problem of power consumption waste is not well solved.

In view of the foregoing, there is a need for improvement in the art.

Disclosure of Invention

The embodiment of the invention aims to provide a frequency modulation method, a frequency modulation device and electronic equipment, which solve the technical problems of frame dropping and power consumption waste in the conventional frequency modulation and realize power consumption saving on the premise of not dropping frames.

In order to solve the above technical problems, embodiments of the present invention provide the following technical solutions:

in a first aspect, an embodiment of the present invention provides a frequency modulation method applied to an electronic device, where the method includes:

determining the predicted load of the current frame according to the load state of at least one frame before the current frame;

according to the predicted load, based on a preset frequency point combination set of at least one module, predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module;

determining the power consumption generated by the current thread based on the combined operation of each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic;

determining initial time constraint, determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, running time and power consumption, and adjusting the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping frames.

In some embodiments, the determining the predicted load of the current frame according to the load status of at least one frame before the current frame includes:

and averaging the load state of at least one frame before the current frame, and determining the average as the predicted load of the current frame.

dividing the load state into at least two load gears, and establishing a load gear model;

determining a load gear corresponding to each frame in at least one frame before the current frame;

predicting a load gear corresponding to the predicted load of the current frame according to the load gear corresponding to each frame in at least one frame before the current frame;

and determining the predicted load of the current frame according to the load gear corresponding to the predicted load of the current frame.

In some embodiments, the predicting a load step corresponding to a predicted load of the current frame according to a load step corresponding to each frame of at least one frame before the current frame includes:

acquiring the occurrence frequency of the load gear corresponding to the next frame under the condition that the load gear corresponding to each frame in at least one frame before the current frame in the load gear model is the same;

and determining the load gear with the largest occurrence frequency as the load gear corresponding to the predicted load of the current frame.

training a load prediction model;

and inputting the load state of at least one frame before the current frame into the load prediction model, and outputting the predicted load of the current frame.

In some embodiments, the predicting, according to the predicted load, a running time required for the current thread to run based on a preset frequency point combination of at least one module includes:

off-line training a performance model, wherein the performance model stores the corresponding relation between the preset frequency point combination of at least one module and the running time;

and determining the running time required by the current thread to run based on the preset frequency point combination of at least one module based on the performance model according to the preset frequency point combination of at least one module.

In some embodiments, the determining, according to the running time and the at least one power consumption characteristic, power consumption generated by the current thread running based on each preset frequency point combination of the at least one module includes:

training a power consumption model offline, wherein the power consumption model stores the corresponding relation between the running time and at least one power consumption characteristic and the power consumption;

and determining the power consumption generated by the current thread based on the combined operation of each preset frequency point of the at least one module based on the power consumption model according to the operation time and the at least one power consumption characteristic.

In some embodiments, the determining an initial time constraint comprises:

in a preset time period before a current frame, counting the frame rate of each frame in the preset time period;

calculating the average frame rate in the preset time period;

if the average frame rate is less than the expected frame rate, reducing the time constraint of the previous frame of the current frame to determine an initial time constraint;

if the average frame rate is greater than or equal to the expected frame rate, increasing the time constraint of the last frame of the current frame to determine the initial time constraint.

In some embodiments, the method further comprises:

dividing a preset time period into a plurality of time segments;

counting the average frame rate of each time segment in the preset time period;

comparing the average frame rate of each time slice with the expected frame rate, and determining the time constraint adjustment tendency of each time slice, wherein the time constraint adjustment tendency comprises increasing time constraint or decreasing time constraint;

and counting the number of each time constraint adjustment trend in the preset time period, and determining to increase or decrease the time constraint of the previous frame of the current frame according to the number of each time constraint adjustment trend in the preset time period so as to determine the time constraint of the current frame.

In some embodiments, the determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time, and the power consumption includes:

dynamically adjusting the initial time constraint, and determining a first frequency point combination set meeting the initial time constraint based on a preset frequency point combination set of at least one module;

searching a frequency point combination meeting a preset condition according to the first frequency point combination set, wherein the preset condition comprises that the running time is less than or equal to an initial time constraint and the power consumption is minimum;

and determining the frequency point combination meeting the preset condition as the optimal frequency point combination corresponding to the at least one module.

In some embodiments, the method further comprises:

operating the current frame based on the optimal frequency point combination, and determining an operation result;

and adjusting the initial time constraint according to the operation result, and determining an updated initial time constraint so as to enable the next frame of the current frame to determine the corresponding optimal frequency point combination based on the updated initial time constraint.

In some embodiments, the operation result includes at least one of a frame length, a CPU frequency, a GPU frequency, a DDR frequency, a dropped frame condition, a temperature, and a current.

In some embodiments, the load condition comprises: processing at least one of the number of instructions required by the current frame, the number of times of occurrence of first-level data cache misses, the number of times of occurrence of second-level data cache misses, the number of times of occurrence of third-level data cache misses, a CPU frequency, a CPU utilization rate, a GPU frequency, a GPU utilization rate, a DDR frequency, an NPU utilization rate, a network speed and a temperature;

the at least one module comprises at least one of a central processing unit, a graphic processing unit, a neural network processing unit and a memory for storing the current frame;

the at least one power consumption characteristic includes at least one of frame length, CPU frequency, GPU frequency, DDR frequency, NPU frequency, temperature, and power.

In a second aspect, an embodiment of the present invention provides a frequency modulation apparatus, which is applied to an electronic device, and the apparatus includes:

the load prediction unit is used for determining the predicted load of the current frame according to the load state of at least one frame before the current frame;

the running time unit is connected with the load prediction unit and used for predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module based on the preset frequency point combination set of at least one module according to the predicted load;

the power consumption unit is connected with the operation time unit and used for determining the power consumption generated by the combined operation of the current thread based on each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic;

and the optimal frequency point combination unit is connected with the load prediction unit, the running time unit and the power consumption unit and used for determining initial time constraint, determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame.

In some embodiments, the load prediction unit comprises:

the load prediction model is used for acquiring the load state of at least one frame before the current frame and determining the predicted load of the current frame according to the load state of at least one frame before the current frame.

In some embodiments, the runtime unit comprises:

and the performance model is used for receiving the predicted load sent by the load prediction unit and predicting the running time of the current thread required by running based on each preset frequency point combination of at least one module based on the preset frequency point combination set of at least one module according to the predicted load.

In some embodiments, the power consuming unit includes:

and the power consumption model is used for receiving the running time sent by the performance model and determining the power consumption generated by the combined running of the current thread based on each preset frequency point of the at least one module according to the running time and the at least one power consumption characteristic.

In some embodiments, the optimal frequency point combining unit includes:

a time constraint model for determining an initial time constraint;

and the optimizer is used for determining the optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame.

In a third aspect, an embodiment of the present invention provides an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the frequency modulation method described above.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which includes a computer program, when the computer program runs on a computer or a processor, the computer or the processor is caused to execute the frequency modulation method described above.

The embodiment of the invention has the beneficial effects that: in contrast to the prior art, an embodiment of the present invention provides a frequency modulation method applied to an electronic device, where the method includes: determining the predicted load of the current frame according to the load state of at least one frame before the current frame; according to the predicted load, based on a preset frequency point combination set of at least one module, predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module; determining the power consumption generated by the current thread based on the combined operation of each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic; determining initial time constraint, determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, running time and power consumption, and adjusting the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping frames. The method and the device for controlling the frequency point combination of the at least one module determine the optimal frequency point combination corresponding to the at least one module based on the preset frequency point combination set of the at least one module through initial time constraint, running time and power consumption.

Drawings

One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.

Fig. 1 is a schematic structural diagram of a frequency modulation system according to an embodiment of the present invention;

FIG. 2 is a diagram of a reinforcement learning model according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an improved model based on reinforcement learning according to an embodiment of the present invention;

fig. 4 is a schematic flow chart of a frequency modulation method according to an embodiment of the present invention;

FIG. 5 is a detailed flowchart of step S10 in FIG. 4;

FIG. 6 is a detailed flowchart of step S13 in FIG. 5;

FIG. 7 is a detailed flowchart of step S20 in FIG. 4;

FIG. 8 is a detailed flowchart of step S30 in FIG. 4;

FIG. 9 is a detailed flowchart of step S40 in FIG. 4;

fig. 10 is a schematic structural diagram of a frequency modulation apparatus according to an embodiment of the present invention;

fig. 11 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that, if not conflicted, the various features of the embodiments of the invention may be combined with each other within the scope of protection of the invention. Additionally, while functional block divisions are performed in apparatus schematics, with logical sequences shown in flowcharts, in some cases, steps shown or described may be performed in sequences other than block divisions in apparatus or flowcharts. The terms "first", "second", "third", and the like used in the present invention do not limit data and execution order, but distinguish the same items or similar items having substantially the same function and action.

Before the present invention is explained in detail, terms and expressions referred to in the embodiments of the present invention are explained, and the terms and expressions referred to in the embodiments of the present invention are applied to the following explanations.

(1) FPS (Frame per Second, FPS): the FPS is the definition in the field of images, and refers to the number of frames transmitted per second of a picture, and refers to the number of pictures of animation or video in popular terms. The FPS measures the amount of information used to store and display the motion video. The greater the number of frames per second, the smoother the displayed motion will be. Typically, a minimum of 30 to avoid motion blindness is used. Some computer video formats can only provide 15 frames per second.

(2) EFPS (extruded Frames per Second, EFPS): the desired frame rate, for each game or application, has its own desired FPS (e.g., game "King Rong" currently has two EFPS settings available: 30 and 60). In general, a game or application, when executed, has an FPS equal to EFPS; however, if the current load is too heavy, the FPS may be smaller than the EFPS, for example, when the major glows to set the EFPS to 60, the FPS may oscillate between 56-60.

(3) E-Period (Expected Period, Expected Frame Length, E-Period): the expected frame length is 1/EFPS. For example: 60FPS, E-Period is about 16.667 ms.

(4) Frame Length (Frame Length): the picture updating time interval of the game can be roughly estimated to be 1/FPS.

(5) Long Frame (Long Frame): meaning that the frame length of a frame is greater than the desired frame length (E-Period).

(6) Short Frame (Short Frame): meaning that the frame length of a frame is less than the desired frame length (E-Period).

(7) Very Long Frame (Very Long Frame): it means that the frame length of a certain frame is much larger than that of the E-Period, for example, the frame length is more than twice as large, which usually makes the user feel stuck obviously.

(8) Dropped frame (JANK): when the FPS is smaller than EFPS, it is called dropped frame. When the difference between the EFPS and the FPS is larger, which is also called as the FPS jitter is larger, the user can obviously feel the jerkiness.

(9) GPU drawing API: the drawing function library utilizes a GPU to draw in the Android mobile phone.

(10) Drawing Thread (Render Thread): the responsible drawing uses the library of drawing functions, which is most easily identified as the drawing thread because of the direct calling of the drawing function.

(11) Control Thread (Control Thread): the system is responsible for waking up the drawing thread and controlling the frame length, and has the main function of controlling the frame rate and indirectly controlling the drawing starting time of the drawing thread. The thread is difficult to identify because it does not directly call the mapping function library.

(12) Run time (Runtime): under the conditions of abundant hardware performance or low performance requirement of the game, the Runtime is usually less than or equal to the Frame Length. For example, in an EFPS-40 game, each frame is 25ms long, but the game itself may only run for 15ms, and the remaining 10ms may be sleep time. In addition, the drawing thread and the control thread respectively have different Runtime; but there will only be one Frame Length (Frame Length) at the moment.

(13) And (3) system frequency modulation: the method refers to a native built-in CPU and DDR frequency modulation strategy of the mobile phone. The original frequency modulation strategy is general, and usually leaves some performance margin, namely: the system can run by using a frequency which is a little higher than the frequency of the CPU and the DDR which can just run out in the E-Period originally.

(14) Window: and the time unit for monitoring the thread load is counted.

(15) Offline (Off-line) learning: i.e., offline learning, refers to a developer directly analyzing data and using the data to train a model at development time. The benefit of learning at this stage is that the developer can optimize the model substantially. However, if too many offline learning methods are used in the algorithm, the game or the application of the game is modified easily, and the offline learning method needs to be readjusted by offline personnel, so that the universality is not high.

(16) On-line (On-line) learning: namely online learning, and when a game or an application runs on an electronic device, training and adjusting of the model are carried out in real time. In this process, developers cannot constantly stare at and optimize learning results. The model can only be learned from steps and procedures originally planned by the developer. However, if the online learning can obtain good effect, the model can be widely used, so that the model does not need to be trained again off line due to game or application revision.

In the prior art, the selection of the frequencies of the operation resources (such as CPU, DDR, GPU and NPU) are independent. In the frequency selection of the CPU, whether the conventional DVFS or the EAS proposed later, the load is determined by using a window method, and the load is used to determine whether to increase the current frequency. Under empirical rules, the current CPU frequency solution chooses to prevent the fluctuation of the task load with 1.25 times efficiency.

In an electronic device, such as a smart phone, multiple devices or modules (CPU, GPU, Modem) all use a Double Data Rate (DDR) at the same time, so the DDR frequency is selected by voting from each module and selecting the highest frequency. The native CPU and DDR frequency modulation strategies of smartphones are also usually left with some performance margin for general use. But with performance margins it also represents a waste of power consumption. When multiple devices in the system vote for the DDR frequency and the frequency is the highest, some devices may exaggerate the DDR frequency required by themselves, which causes the voted DDR frequency to be higher, resulting in waste of power consumption.

In addition, the frequency selection of the native DDR and the frequency selection of the CPU are independent of each other. However, there is some coupling relationship between the DDR frequency and the CPU frequency: at some time, the low-frequency DDR and high-frequency CPU and the high-frequency DDR and low-frequency CPU may both prevent the game from dropping frames; however, in comparison, the former may save power consumption (determined by actual measurement), so that the CPU frequency or DDR frequency is adjusted only, which is likely to cause waste of power consumption.

Based on the above problem, an embodiment of the present invention provides a frequency modulation method, which implements an optimal frequency point combination of at least one module with the lowest power consumption on the premise of not dropping frames.

Specifically, the following describes embodiments of the present invention in detail by taking an electronic device as an example of a smart phone.

Referring to fig. 1, fig. 1 is a schematic structural diagram of a frequency modulation system according to an embodiment of the present invention;

as shown in fig. 1, the fm system 100 is applied to an electronic device, such as a smart phone, and the fm system is a hardware system architecture of the electronic device, and the fm system 100 includes: central processing unit 110, graphics processing unit 120, memory 130, neural network processing unit 140, bus 150, and hardware accelerator 160.

It will be appreciated that the illustrated frequency modulation system 100 may be included in one or more chips to form an electronic system. The system may be located in an electronic device which may be a wireless terminal, a wired terminal, a user device or a connectionless device. For example, the electronic device may be a smartphone, a personal computer, a tablet computer, a game console, or the like.

The central processing unit 110, i.e., a CPU, is shown coupled to a graphics processing unit 120, a memory 130, a neural network processing unit 140, and a hardware accelerator 160 via a bus 150.

The graphics processing unit 120, i.e., a GPU, is shown connected to the central processing unit 110, the memory 130, the neural network processing unit 140, and the hardware accelerator 160 via a bus 150.

The memory 130 shown may be, for example, a DDR memory, and the memory 130 may be used to store program codes and data of the frequency modulation system 100. Memory 130 is connected to central processing unit 110, graphics processing unit 120, and neural network processing unit 140 by bus 150.

The neural network processing unit 140, i.e., (network process units, NPUs), is shown coupled to the central processing unit 110, the graphics processing unit 120, the memory 130, and the hardware accelerator 160 via a bus 150.

It is understood that the bus 150 may be any type of bus such as a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 150 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one line is shown in FIG. 1, but it is not intended that there be only one bus or one type of bus.

The central processing unit 110, the graphics processing unit 120, and the neural network processing unit 140 may be configured to process an image, and the central processing unit 110, the graphics processing unit 120, and the neural network processing unit 140 may be on a chip, and read software codes stored in the memory 130 through an interface coupled to the outside of the chip to implement corresponding functions. The software code is read from memory 130 into the chip interior via interface and bus 150 and used by at least one of central processing unit 110, graphics processing unit 120, or neural network processing unit 140.

The hardware accelerator 160 is shown coupled to the cpu 110, the gpu 120, the memory 130, and the neural network processing unit 140 via the bus 150. The hardware accelerator 160 may include an application-specific integrated circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic device, transistor logic device, hardware component, or any combination thereof.

In the following, the frequency modulation system 100 shown in fig. 1 is described in detail by taking an electronic device as a smart phone as an example.

Referring to fig. 2, fig. 2 is a schematic diagram of a reinforcement learning model according to an embodiment of the present invention;

as shown in fig. 2, the reinforcement learning model 20 includes: agent21 and Environment 22.

The illustrated Environment22 is an internal Environment of the smartphone, the illustrated Agent21 is an Agent, the Agent is a model that must be learned, the Agent makes an Action, i.e., an Action, by receiving State, i.e., a load State, the Action is a frequency point combination of at least one module, the at least one module includes a CPU and a DDR in fig. 2, and the frequency point combination includes a CPU frequency and a DDR frequency, i.e., a CPU frequency and a DDR frequency.

The State, that is, the State in the embodiment of the present invention, refers to a load State, and the State in fig. 2 includes a number of instructions (instructions) required for processing a current frame, a number of times of occurrence of a data cache miss (cache miss), and a utilization rate (util), where the cache miss includes a number of times of occurrence of a first-level data cache miss, a number of times of occurrence of a second-level data cache miss, and a number of times of occurrence of a third-level data cache miss, and the util includes a CPU utilization rate, a GPU utilization rate, and an NPU utilization rate.

In this embodiment of the present invention, the load state may further include internal information of the electronic device, such as a CPU frequency, a GPU frequency, a DDR frequency, a network speed, and a temperature, that is, the load state includes at least one of the number of instructions required to process the current frame, the number of times of occurrence of a first-level data cache miss, the number of times of occurrence of a second-level data cache miss, the number of times of occurrence of a third-level data cache miss, the CPU frequency, the CPU usage rate, the GPU frequency, the GPU usage rate, the DDR frequency, the NPU usage rate, the network speed, and the temperature.

As shown in fig. 2, the Environment sends a state, the Environment sends the state to an Agent, and the Agent receives the state to generate an Action, that is, recommend a group of frequency point combinations of the Environment. After the system of the electronic device, for example, a mobile phone system, runs on the frequency point combination, the running result (such as whether the game is dropped and thus saves more power than before) is transmitted back to the Agent in the form of Reward.

Among them, the Reward possible design modes include: when the game drops frames, -1 is given. When the game has no dropped frames and the frequency is lower than the frequency recommended by the handset system, the difference in frequency is given as Reward (positive bonus). When the game does not drop frames, but is equal to the frequency of system selection, 0 is given.

The reinforcement learning Model is called a Model Free Model, wherein an Agent of the Model Free Model generally needs to use a Talular Solution in reinforcement learning, such as Monte Carlo:

various frequency point combinations are tried in a random trial and error mode for a period of time at the beginning, and the result is recorded in a first table, wherein the first table is a high-dimensional table, and the first table comprises each state (Si) and the last Reward obtained by adopting different actions (aj).

During the period of supply, that is, during the period of configuring the frequency point combination, when the State is generated as Si, the Action (Ai) which can receive the highest reward when Si is generated is searched in the table. Finally, the CPU and DDR frequencies (i, j) considered best by the model are obtained, and this is called the explore.

During the period of the supply, the potentially best Action may be changed due to the change of the game scene, so there must be a slight chance to try whether other actions will get higher rewarded, which is called random trial and error (explorer), and e-greedy algorithm (epsilon greedy).

It will be appreciated that since the tables are discrete and of limited number, but the states of many systems are continuous, the states of these systems must be sliced. For example: the value range of the instructions is [0,1e10], and different load steps can be determined according to the number of instructions (instructions) required for processing the current frame, so as to conveniently establish the first table:

the Model Free Model can be learned online due to low calculation amount, and is not influenced by an arithmetic chip and a game due to online learning. However, during the learning process, due to the random trial and error (explore), frame dropping may be obvious and the user experience may be reduced.

In summary, to overcome the above disadvantages, offline learning must be replaced. However, offline learning must have its own model (i.e., high dimensional tables) for different computing chips, games, applications, etc. And when the game is updated, the model must be changed accordingly. For cell phone vendors, if updated information for a game or application is not determined in real time, the user experience may be affected. And because different models are trained aiming at different games and different operation chips, huge manpower resources are needed to be spent.

However, learning either online or offline has its limitations. Therefore, the present invention further proposes the following improved Model Based on reinforcement learning, i.e. Model Based Model, to avoid the above-mentioned disadvantages.

Referring to fig. 3, fig. 3 is a schematic diagram of an improved model based on reinforcement learning according to an embodiment of the present invention;

as shown in fig. 3, the improved model 30 based on reinforcement learning includes:

agent31, i.e. Agent;

environment32, i.e., the internal Environment of the smartphone;

and a load prediction model 33, configured to obtain a load state of at least one frame before the current frame, and determine a predicted load of the current frame according to the load state of the at least one frame before the current frame.

And the performance model 34 is used for receiving the predicted load sent by the load prediction unit, and predicting the running time Tij of the current thread required by running based on each preset frequency point combination of at least one module according to the predicted load and based on the preset frequency point combination set of at least one module.

And the power consumption model 35 is used for receiving the running time sent by the performance model and determining the power consumption Pij generated by the current thread based on the combined running of each preset frequency point of the at least one module according to the running time and at least one power consumption characteristic.

An Optimizer36, which is configured to determine an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time, and the power consumption, and adjust the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, where the optimal frequency point combination satisfies the minimum power consumption under the premise that no frame dropping occurs.

When the demand, that is, when configuring the frequency point combination, the whole flow is as follows:

the Environment32 transmits the system state, i.e., the load state, of at least one frame previous to the current frame to the Agent31 and the load prediction model 33.

The load prediction model 33 generates a predicted load based on the load state, and outputs the predicted load to the performance model. The predicted load comprises at least one of the number of instructions needed for processing the current frame, the frequency of occurrence of first-level data cache misses, the frequency of occurrence of second-level data cache misses, the frequency of occurrence of third-level data cache misses, CPU frequency, CPU utilization rate, GPU frequency, GPU utilization rate, DDR frequency, NPU utilization rate, network speed and temperature;

the performance model 34 obtains the predicted load predicted by the load prediction model 33, and estimates the running time of the current thread, such as the running time runtime (tij) of the game thread. Where Tij represents a running time after the current thread is executed through a frequency point combination of at least one module, for example: and executing the Runtime after the game thread by using the CPU frequency i and the DDR frequency j.

The input of the power consumption model 35 is the running time Tij of the current thread sent by the performance model 34 and at least one power consumption characteristic, where the power consumption characteristic includes at least one of a frame length, a CPU frequency, a GPU frequency, a DDR frequency, an NPU frequency, a temperature, and an electric quantity, and the output of the power consumption model is the power consumption Pij generated by the current thread running based on each preset frequency point combination of the at least one module, for example, the power consumption required for executing a game thread using the CPU frequency i and the DDR frequency j.

The inputs to the Optimizer36 are Tij, Pij for all bin combinations of the preset set of bin combinations for at least one module, such as: tij and Pij of all permutation combinations of CPU and DDR.

The Optimizer obtains a Time Constraint Tc (Tc) from the Agent, and calculates an optimal frequency point combination corresponding to at least one module with the minimum power consumption, such as a CPU and a DDR frequency point combination (i, j), under the condition that the game is not dropped.

The system of the electronic device, for example, the mobile phone system, executes a game of one frame by using the above-mentioned optimal frequency point combination (i x, j), and feeds back the operation result (the frame length, whether to drop the frame, the temperature, the current …, and other system states) to the Agent, and allows the Agent to adjust the system state, i.e., the load state, so that the next frame can more accurately obtain the reasonable time constraint Tc.

In the embodiment of the present invention, the Agent includes a Time Constraint model (Time Constraint model), wherein the Time Constraint model, an Optimizer (Optimizer) and a mobile phone system form a reinforcement learning mechanism:

the Agent observes the state of the mobile phone (such as instructions, cache miss, CPU frequency, DDR frequency, GPU frequency, temperature and current of the previous frames) and takes Action (outputs Tc or outputs after correcting Tc of the previous moment). The Optimizer builds the Tc, selects a proper frequency point combination, such as a CPU frequency and a DDR frequency, and recommends the frequency point combination to the mobile phone system, so that the mobile phone system operates by using the frequency point combination in the current frame.

When the current frame is finished (namely when the next frame is started), whether the frame is dropped or not can be known by the frequency point combination, and whether the frame is dropped or not can be fed back to the Agent. And the Agent receives feedback, and repeats the steps after adjusting the system state, and the process is circulated continuously.

The time constraint model includes a model constructed by a statistical Method such as a Monte Carlo Method, and may also be an Artificial Neural network model (Artificial Neural Networks).

Wherein the Optimizer, after obtaining the appropriate Tc, will solve the following Constraint Optimization (Constraint Optimization) by equation (1):

argminP_ij s.t.T_ijtc ≦ Tc formula (1)

Wherein:

i, j represents running the game at the CPU frequency i and the DDR frequency j;

pij represents the power consumption generated when the game is run by using the CPU frequency i and the DDR frequency j;

tij represents the running time Runtime required for running the game at the CPU frequency i and the DDR frequency j.

It can be understood that the above formula (1) is equivalent to solving the combination of the CPU and the DDR frequency point with smaller power consumption under the condition of satisfying no frame drop;

as can be seen from the above equation, increasing or decreasing Tc changes the selectable CPU and DDR frequency bins. When Tc is larger, Tij satisfying the time constraint is larger, and a lower frequency CPU and DDR frequency point are more likely to be selected, so that more power consumption can be saved. It can be understood that increasing Tc may suggest that the mobile phone system uses lower frequency CPU, DDR, which may be considered as suggesting that the mobile phone system decelerates; otherwise, decreasing Tc may be considered as speeding up the mobile phone system.

Referring to fig. 4, fig. 4 is a schematic flow chart of a frequency modulation method according to an embodiment of the present invention;

as shown in fig. 4, the frequency modulation method includes:

step S10: determining the predicted load of the current frame according to the load state of at least one frame before the current frame;

specifically, the load state includes at least one of instruction numbers (instructions) required for processing the current frame, the number of times of occurrence of primary data cache misses (L1 data caches), the number of times of occurrence of secondary data cache misses (L2 data caches), the number of times of occurrence of tertiary data cache misses (L3 data caches), a CPU frequency, a CPU utilization rate, a GPU frequency, a GPU utilization rate, a DDR frequency, an NPU utilization rate, a network speed, and a temperature;

specifically, the determining the predicted load of the current frame according to the load state of at least one frame before the current frame includes:

and averaging the load state of at least one frame before the current frame, and determining the average as the predicted load of the current frame, wherein the average is determined as the first predicted load of the current frame.

Specifically, the load state of each frame in at least one frame before the current frame is determined, the load state of each frame in at least one frame before the current frame is averaged, and the average is determined as the predicted load of the current frame. It is understood that at least one frame before the current frame should be a continuous multi-frame to improve the accuracy of the predicted load.

Specifically, referring back to fig. 5, fig. 5 is a detailed flowchart of step S10 in fig. 4;

as shown in fig. 5, the step S10: determining the predicted load of the current frame according to the load state of at least one frame before the current frame, wherein the method comprises the following steps:

step S11: dividing the load state into at least two load gears, and establishing a load gear model;

specifically, the load status includes a number of instructions (instructions) required for processing the current frame, where a value range of the number of instructions (instructions) is [0,1e10], so that the number of instructions (instructions) required for processing the current frame may be divided into a plurality of load stages, for example: according to the number of the instructions, dividing the instruction number into 5 load gears, which are respectively as follows:

first load gear: [0, 2e9]

The second load gear: [2e9,4e9]

A third load gear: [4e9,6e9]

A fourth load gear: [6e9,8e9]

A fifth load gear: [8e9,1e10]

And establishing a load gear model according to the load gear, where the load gear model includes a load gear of at least one frame before the current frame and a load gear of a next frame after the current frame, and the load gear model further includes the number of occurrences of the load gear corresponding to the next frame under the condition that the load gear corresponding to each frame in the at least one frame before the current frame is the same, for example: when the load steps of the first three frames before the current frame are [3,5,4], the current frame is 1. It is understood that the load step model includes [ k, 3,5,4, i ═ 1-5], where k is the number of occurrences and i is the load step, which may have a value of 1 to 5, and represents the load step of the current frame.

Step S12: determining a load gear corresponding to each frame in at least one frame before the current frame;

specifically, the load gear corresponding to each frame in at least one frame before the current frame is determined according to the instruction number of each frame in at least one frame before the current frame.

Step S13: predicting a load gear corresponding to the predicted load of the current frame according to the load gear corresponding to each frame in at least one frame before the current frame;

specifically, referring back to fig. 6, fig. 6 is a detailed flowchart of step S13 in fig. 5;

as shown in fig. 6, step S13: predicting the load gear corresponding to the predicted load of the current frame according to the load gear corresponding to each frame in at least one frame before the current frame, wherein the method comprises the following steps:

step S131: acquiring the occurrence frequency of the load gear corresponding to the next frame under the condition that the load gear corresponding to each frame in at least one frame before the current frame in the load gear model is the same;

specifically, the load shift model includes the number of occurrences of a load shift corresponding to each frame in at least one frame before a current frame, for example: when the load steps of the first three frames before the current frame are [3,5,4], the current frame is 1.

Step S132: and determining the load gear with the largest occurrence number as the load gear corresponding to the predicted load of the current frame.

Specifically, if the load steps of the first three frames before the current frame are [3,5,4] respectively, and the occurrence frequency of the current frame being 1 is the largest, it is determined that the load step corresponding to the predicted load of the current frame is 1.

It will be appreciated that after the current frame has been processed, the load gear model will be updated, for example: the load gear is updated and the number of occurrences is redetermined.

Step S14: and determining the predicted load of the current frame according to the load gear corresponding to the predicted load of the current frame.

Specifically, according to the load gear corresponding to the predicted load of the current frame, the predicted load of the current frame is determined as the median of the load gear, for example: and if the load gear corresponding to the predicted load of the current frame is the first load gear and the value range of the first load gear is [0, 2e9], determining that the load gear corresponding to the predicted load of the current frame is the median of the value range of the first load gear, namely 1e9, wherein the predicted load of the current frame is determined according to the load gear corresponding to the predicted load of the current frame, and the predicted load is determined as the second predicted load.

In this embodiment of the present invention, the determining the predicted load of the current frame according to the load status of at least one frame before the current frame includes:

training a load prediction model;

Specifically, a load prediction model is trained by acquiring a load state in a preset time period and taking the load state in the preset time period as a training set, the load prediction model may be a neural network model, such as a Long Short-Term Memory (LSTM), the load state of at least one frame before the current frame is taken as a test set, the load state of at least one frame before the current frame is input into the load prediction model, and the predicted load of the current frame is output. And inputting the load state of at least one frame before the current frame into the load prediction model, and outputting the predicted load of the current frame, wherein the predicted load is a third predicted load.

In an embodiment of the present invention, the method further comprises:

and averaging the first predicted load, the second predicted load and the third predicted load according to the first predicted load, the second predicted load and the third predicted load, and taking the average as a final predicted load.

The predicted load is determined through various modes respectively, and then the predicted load determined through various modes respectively serves as the final predicted load.

Step S20: according to the predicted load, based on a preset frequency point combination set of at least one module, predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module;

specifically, referring back to fig. 7, fig. 7 is a detailed flowchart of step S20 in fig. 4;

as shown in fig. 7, the step S20: according to the predicted load, based on a preset frequency point combination set of at least one module, the method for predicting the running time required by the current thread based on each preset frequency point combination of at least one module comprises the following steps:

step S21: off-line training a performance model, wherein the performance model stores the corresponding relation between the preset frequency point combination of at least one module and the running time;

specifically, the performance model may be obtained by supervised Learning (supervisory Learning). It will be appreciated that the performance model is only relevant to the computing chip of the electronic device and not to the game or application being run, and therefore the performance model can be implemented by off-line training or off-line training. That is, the performance model needs to be retrained only when the computing chip of the electronic device is replaced.

In an embodiment of the present invention, the performance Model includes one or more of a machine Learning Model, an Artificial Neural network Model (Artificial Neural Networks), a Deep Learning Model (Deep Learning Model), a statistical Learning Model, such as a Linear Regression Model (Linear Regression), and a Curve Fitting Model (Curve Fitting).

Step S22: and determining the running time required by the current thread to run based on the preset frequency point combination of at least one module based on the performance model according to the preset frequency point combination of at least one module.

Specifically, the input of the performance model is a load state predicted by the load prediction model, and the method includes: the method comprises the steps of processing at least one of the number of instructions needed by a current frame, the number of times of first-level data cache misses, the number of times of second-level data cache misses, the number of times of third-level data cache misses, CPU frequency, CPU utilization rate, GPU frequency, GPU utilization rate, DDR frequency, NPU utilization rate, network speed and temperature, wherein the output of a performance model is the running time (Runtime) needed by the current frame of a current thread to run by the aid of a preset frequency point combination of at least one module, for example, the output of the performance model is the running time Tij needed by the current frame to run by the aid of CPU frequency i and the DDR frequency j.

Step S30: determining the power consumption generated by the current thread based on the combined operation of each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic;

specifically, referring back to fig. 8, fig. 8 is a detailed flowchart of step S30 in fig. 4;

as shown in fig. 8, the step S30: determining the power consumption generated by the current thread based on the combined operation of each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic, wherein the determining comprises the following steps:

step S31: training a power consumption model offline, wherein the power consumption model stores the corresponding relation between the running time and at least one power consumption characteristic and the power consumption;

the running time is the running time predicted by the performance model, that is, the running time required by the at least one module running at a preset frequency point combination, the power consumption feature includes at least one of a frame length, a CPU frequency, a GPU frequency, a DDR frequency, an NPU frequency, a temperature, and an electric quantity, and the power consumption feature is a feature related to power consumption.

Specifically, the power consumption model may be obtained by supervised Learning (supervisory Learning). It will be appreciated that the power consumption model is only relevant to the computing chip of the electronic device, and not to the running game or application, and therefore can be done by off-line training or off-line training. That is, the power consumption model needs to be retrained only when the arithmetic chip of the electronic device is replaced.

In an embodiment of the present invention, the power consumption Model includes one or more of a machine Learning Model, an Artificial Neural network Model (Artificial Neural Networks), a Deep Learning Model (Deep Learning Model), a statistical Learning Model, such as a Linear Regression Model (Linear Regression), and a Curve Fitting Model (Curve Fitting).

Step S32: and determining the power consumption generated by the current thread based on the combined operation of each preset frequency point of the at least one module based on the power consumption model according to the operation time and the at least one power consumption characteristic.

Specifically, the input of the power consumption model is the running time and at least one power consumption characteristic, and the output of the power consumption model is the power consumption generated by the combined running of the current thread based on each preset frequency point of the at least one module.

Step S40: determining initial time constraint, determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, running time and power consumption, and adjusting the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping frames.

Specifically, the determining the initial time constraint includes:

calculating the average frame rate in the preset time period;

Specifically, the preset time period may be set manually, for example, set to be half a second, one second, or the like, or the preset time period is determined by a frame number, for example: the frame number is 40 frames, the preset time period before the current frame comprises a plurality of frames, and the average frame rate of the plurality of frames is used as the average frame rate of the preset time period by counting the frame rate of each frame of the plurality of frames and calculating the average frame rate of the plurality of frames.

Assuming that the Average frame rate is AFPS (Average Frames per Second, AFPS), and the Expected frame rate is EFPS (Expected Frames per Second, EFPS), when AFPS is less than EFPS, the time constraint Tc of the previous frame of the current frame is reduced, and when AFPS is greater than or equal to EFPS, the time constraint Tc of the previous frame of the current frame is increased.

In an embodiment of the present invention, the method further comprises:

when the average frame rate AFPS ≈ expected frame rate EFPS, that is, the average frame rate AFPS is approximately equal to the expected frame rate EFPS, at this time, a time constraint Tc is tentatively increased, a thread is executed through different parameter configurations, the parameter configurations are frequency point combinations, and the parameter configurations with the minimum frame drop situation and the lowest power consumption are determined, that is, the frequency point combination with the minimum frame drop situation and the lowest power consumption is determined, so that a time constraint corresponding to the frequency point combination with the minimum frame drop situation and the lowest power consumption is determined, and the initial time constraint is adjusted to the time constraint corresponding to the frequency point combination with the minimum frame drop situation and the lowest power consumption.

In an embodiment of the present invention, the method further comprises:

dividing a preset time period into a plurality of time segments, and counting the average frame rate of each time segment in the preset time period;

Specifically, the preset time period is divided into a plurality of time segments, which is equivalent to dividing the preset time period into a plurality of cells, each cell is a time segment, and the length of each time segment may be equal or unequal.

Determining a time constraint adjustment trend of each time slice by comparing the average frame rate of each time slice with the expected frame rate, wherein the time constraint adjustment trend comprises increasing time constraints or decreasing time constraints, and is equivalent to voting to decide whether to increase or decrease the time constraints;

determining the time constraint of the previous frame of the current frame by counting the number of each time constraint adjustment tendency in a preset time period and determining the time constraint of the current frame by increasing or decreasing the time constraint of the previous frame of the current frame according to the number of each time constraint adjustment tendency in the preset time period;

specifically, if the number of the increased time constraints is greater than the number of the decreased time constraints, determining to increase the time constraint of the previous frame of the current frame, and determining the increased time constraint as the time constraint of the current frame; and if the number of the increased time constraints is less than the number of the decreased time constraints, determining to decrease the time constraint of the previous frame of the current frame, and determining the decreased time constraint as the time constraint of the current frame. The increased or decreased time constraint value is determined by a preset rule, for example, the preset rule includes: and determining the increased or decreased time constraint value according to the difference value of the number of the increased time constraints and the number of the decreased time constraints, wherein the time constraint of the current frame is the time constraint of the previous frame of the current frame +/-the increased or decreased time constraint value.

In the embodiment of the invention, by counting the frame rate change in the preset time period and tentatively increasing or reducing the time constraint, the invention can determine better time constraint under the condition of not causing frame dropping greatly, and is favorable for determining the frequency point combination with lower power consumption on the premise of not causing frame dropping.

In this embodiment of the present invention, the determining the initial time constraint further includes:

and determining the initial time constraint according to the running time of the drawing thread of at least one frame before the current frame, or determining the initial time constraint according to the experience value of an application program corresponding to the current frame.

Referring back to fig. 9, fig. 9 is a detailed flowchart of step S40 in fig. 4;

as shown in fig. 9, step S40: determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, wherein the method comprises the following steps:

step S41: dynamically adjusting the initial time constraint, and determining a first frequency point combination set meeting the initial time constraint based on a preset frequency point combination set of at least one module;

specifically, the meeting of the initial time constraint means that the running time of running the current frame based on the preset frequency point combination is less than the initial time constraint, and the initial time constraint is dynamically adjusted, for example: and increasing or decreasing the initial time constraint, determining an updated initial time constraint, and acquiring a first frequency point combination set meeting the updated initial time constraint from a preset frequency point combination set of the at least one module based on the updated initial time constraint.

Step S42: searching a frequency point combination meeting a preset condition according to the first frequency point combination set, wherein the preset condition comprises that the running time is less than or equal to an initial time constraint and the power consumption is minimum;

specifically, the first frequency point combination set is a plurality of frequency point combinations satisfying the initial time constraint, and the first frequency point combination set is screened according to a preset condition to determine an optimal frequency point combination. The preset conditions include that the running time is less than or equal to the initial time constraint and the power consumption is minimum, which is equivalent to screening the frequency point combination with the running time less than or equal to the initial time constraint and the minimum power consumption from the first frequency point combination set, and determining the frequency point combination as the optimal frequency point combination.

Step S43: and determining the frequency point combination meeting the preset condition as the optimal frequency point combination corresponding to the at least one module.

Specifically, the frequency point combination with the running time less than or equal to the initial time constraint and the minimum power consumption is screened from the first frequency point combination set, the frequency point combination is determined to be the optimal frequency point combination corresponding to the at least one module, and the optimal frequency point combination is recommended to a system of electronic equipment, such as a mobile phone system, so that the mobile phone system runs the current frame based on the optimal frequency point combination.

In an embodiment of the present invention, the method further comprises:

specifically, the operation result includes at least one of a frame length, a CPU frequency, a GPU frequency, a DDR frequency, a frame dropping situation, a temperature, and a current.

Specifically, the load state of the current frame is determined according to the operation result, so that the predicted load of the next frame of the current frame is determined, and the operation time required by the current thread to operate based on each preset frequency point combination of at least one module is estimated according to the predicted load of the next frame of the previous frame and based on the preset frequency point combination set of at least one module; determining the power consumption generated by the current thread based on the combined operation of each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic; and determining the updated initial time constraint so that the next frame of the current frame determines the corresponding optimal frequency point combination based on the updated initial time constraint.

In an embodiment of the present invention, a frequency modulation method is provided and applied to an electronic device, where the method includes: determining the predicted load of the current frame according to the load state of at least one frame before the current frame; according to the predicted load, based on a preset frequency point combination set of at least one module, predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module; determining the power consumption generated by the current thread based on the combined operation of each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic; determining initial time constraint, determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, running time and power consumption, and adjusting the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping frames. The method and the device for controlling the frequency point combination of the at least one module determine the optimal frequency point combination corresponding to the at least one module based on the preset frequency point combination set of the at least one module through initial time constraint, running time and power consumption.

Referring to fig. 10 again, fig. 10 is a schematic structural diagram of a frequency modulation device according to an embodiment of the present invention;

as shown in fig. 10, the frequency modulation apparatus 10 is applied to an electronic device, and the frequency modulation apparatus 10 includes:

a load prediction unit 101, configured to determine a predicted load of a current frame according to a load state of at least one frame before the current frame;

the running time unit 102 is connected with the load prediction unit and used for predicting the running time required by the current thread to run based on each preset frequency point combination of at least one module according to the predicted load and based on the preset frequency point combination set of at least one module;

the power consumption unit 103 is connected with the operation time unit and is used for determining power consumption generated by the combined operation of the current thread based on each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic;

and an optimal frequency point combination unit 104, connected to the load prediction unit, the operation time unit, and the power consumption unit, and configured to determine an initial time constraint, determine an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the operation time, and the power consumption, and adjust a working frequency point of a current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, where the optimal frequency point combination meets minimum power consumption on the premise of no frame drop.

In an embodiment of the present invention, the load prediction unit includes:

In an embodiment of the present invention, the runtime unit includes:

In an embodiment of the present invention, the power consumption unit includes:

In this embodiment of the present invention, the optimal frequency point combining unit includes:

a time constraint model for determining an initial time constraint;

In an embodiment of the present invention, the load prediction unit is configured to:

In an embodiment of the present invention, the load prediction unit is further configured to:

In this embodiment of the present invention, the predicting a load tap corresponding to a predicted load of a current frame according to a load tap corresponding to each frame in at least one frame before the current frame includes:

training a load prediction model;

In an embodiment of the present invention, the runtime unit is configured to:

In an embodiment of the present invention, the power consumption unit is configured to:

In this embodiment of the present invention, the optimal frequency point combining unit is configured to:

calculating the average frame rate in the preset time period;

dividing a preset time period into a plurality of time segments;

counting the average frame rate of each time segment in the preset time period;

comparing the average frame rate of each time slice with the expected frame rate, and determining a time constraint adjusting tendency of each time slice, wherein the time constraint adjusting tendency comprises increasing time constraints or decreasing time constraints;

In this embodiment of the present invention, the optimal frequency point combining unit is further configured to:

In the embodiment of the present invention, the operation result includes at least one of a frame length, a CPU frequency, a GPU frequency, a DDR frequency, a frame dropping condition, a temperature, and a current.

In an embodiment of the present invention, the load status includes: processing at least one of the number of instructions required by the current frame, the number of times of occurrence of first-level data cache misses, the number of times of occurrence of second-level data cache misses, the number of times of occurrence of third-level data cache misses, a CPU frequency, a CPU utilization rate, a GPU frequency, a GPU utilization rate, a DDR frequency, an NPU utilization rate, a network speed and a temperature;

In an embodiment of the present invention, there is provided a frequency modulation apparatus applied to an electronic device, the apparatus including: the load prediction unit is used for determining the predicted load of the current frame according to the load state of at least one frame before the current frame; the running time unit is connected with the load prediction unit and used for predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module based on the preset frequency point combination set of at least one module according to the predicted load; the power consumption unit is connected with the operation time unit and used for determining the power consumption generated by the combined operation of the current thread based on each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic; and the optimal frequency point combination unit is connected with the load prediction unit, the running time unit and the power consumption unit and used for determining initial time constraint, determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time and the power consumption, and adjusting the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping the frame. The method and the device for controlling the frequency point combination of the at least one module determine the optimal frequency point combination corresponding to the at least one module based on the preset frequency point combination set of the at least one module through initial time constraint, running time and power consumption.

Referring to fig. 11, fig. 11 is a schematic diagram of a hardware structure of an electronic device according to various embodiments of the present invention;

as shown in fig. 11, the electronic device 11 includes but is not limited to: radio frequency unit 111, network module 112, audio output unit 113, input unit 114, sensor 115, display unit 116, user input unit 117, interface unit 118, memory 119, processor 1110, and power supply 1111, and the like. Those skilled in the art will appreciate that the configuration of the electronic device shown in fig. 11 does not constitute a limitation of the electronic device, which may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. In the embodiment of the present invention, the electronic device includes, but is not limited to, a television, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer, and the like.

A processor 1110, configured to determine a predicted load of a current frame according to a load status of at least one frame before the current frame; according to the predicted load, based on a preset frequency point combination set of at least one module, predicting the running time required by the running of the current thread based on each preset frequency point combination of at least one module; determining the power consumption generated by the current thread based on the combined operation of each preset frequency point of the at least one module according to the operation time and the at least one power consumption characteristic; determining initial time constraint, determining an optimal frequency point combination corresponding to the at least one module according to the initial time constraint, running time and power consumption, and adjusting the working frequency point of the current frame processed by the at least one module to the optimal frequency point combination corresponding to the at least one module, wherein the optimal frequency point combination meets the minimum power consumption under the premise of not dropping frames.

In the embodiment of the invention, the optimal frequency point combination corresponding to at least one module is determined based on the preset frequency point combination set of the at least one module through the initial time constraint, the running time and the power consumption.

It should be understood that, in the embodiment of the present invention, the radio frequency unit 111 may be used for receiving and sending signals during a message transmission and reception process or a call process, and specifically, receives downlink data from a base station and then processes the received downlink data to the processor 1110; in addition, uplink data is transmitted to the base station. In general, radio frequency unit 111 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 111 may also communicate with a network and other devices through a wireless communication system.

The electronic device 11 provides the user with wireless broadband internet access via the network module 112, such as helping the user send and receive e-mails, browse web pages, access streaming media, and the like.

The audio output unit 113 may convert audio data received by the radio frequency unit 111 or the network module 112 or stored in the memory 119 into an audio signal and output as sound. Also, the audio output unit 113 may also provide audio output related to a specific function performed by the electronic apparatus 11 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 113 includes a speaker, a buzzer, a receiver, and the like.

The input unit 114 is used to receive an audio or video signal. The input Unit 114 may include a Graphics Processing Unit (GPU) 1141 and a microphone 1142, and the Graphics Processing Unit 1141 processes a target image of a still picture or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 116. The image frames processed by the graphic processor 1141 may be stored in the memory 119 (or other storage medium) or transmitted via the radio frequency unit 111 or the network module 112. The microphone 1142 may receive sound and may be capable of processing such sound into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 111 in case of the phone call mode.

The electronic device 11 also includes at least one sensor 115, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor that adjusts the brightness of the display panel 1161 according to the brightness of ambient light, and a proximity sensor that turns off the display panel 1161 and/or the backlight when the electronic device 11 is moved to the ear. As one type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of an electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), and vibration identification related functions (such as pedometer, tapping); the sensors 115 may also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which are not described in detail herein.

The display unit 116 is used to display information input by the user or information provided to the user. The Display unit 116 may include a Display panel 1161, and the Display panel 1161 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 117 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device. Specifically, the user input unit 117 includes a touch panel 1171 and other input devices 1172. Touch panel 1171, also referred to as a touch screen, may collect touch operations by a user on or near it (such as operations by a user on touch panel 1171 or near touch panel 1171 using a finger, stylus, or any suitable object or attachment). Touch panel 1171 can include two portions, a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, and sends the touch point coordinates to the processor 1110, and receives and executes commands sent from the processor 1110. In addition, the touch panel 1171 can be implemented by various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1171, the user input unit 117 may also include other input devices 1172. Specifically, the other input devices 1172 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a track ball, a mouse, and a joystick, which are not described herein.

Further, touch panel 1171 can be overlaid on display panel 1161, and when touch panel 1171 detects a touch operation thereon or nearby, the touch operation can be transmitted to processor 1110 to determine the type of touch event, and then processor 1110 can provide a corresponding visual output on display panel 1161 according to the type of touch event. Although in fig. 11, the touch panel 1171 and the display panel 1161 are two independent components to implement the input and output functions of the electronic device, in some embodiments, the touch panel 1171 and the display panel 1161 may be integrated to implement the input and output functions of the electronic device, and is not limited herein.

The interface unit 118 is an interface for connecting an external device to the electronic apparatus 11. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 118 may be used to receive input from external devices (e.g., data information, power, etc.) and transmit the received input to one or more elements within the electronic device 11 or may be used to transmit data between the electronic device 11 and the external devices.

The memory 119 may be used to store software programs as well as various data. The memory 119 may mainly include a program storage area and a data storage area, wherein the program storage area may store an application program 1191 (such as a sound playing function, an image playing function, etc.) and an operating system 1192, etc. required by at least one function; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 119 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 1110 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, performs various functions of the electronic device and processes data by operating or executing software programs and/or modules stored in the memory 119 and calling data stored in the memory 119, thereby performing overall monitoring of the electronic device. Processor 1110 may include one or more processing units; preferably, the processor 1110 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 1110.

The electronic device 11 may further include a power supply 1111 (e.g., a battery) for supplying power to various components, and preferably, the power supply 1111 may be logically connected to the processor 1110 via a power management system, so as to manage charging, discharging, and power consumption management functions via the power management system.

In addition, the electronic device 11 includes some functional modules that are not shown, and are not described in detail herein.

Preferably, an embodiment of the present invention further provides an electronic device, which includes a processor 1110, a memory 119, and a computer program stored in the memory 119 and capable of running on the processor 1110, where the computer program, when executed by the processor 1110, implements each process of the above-mentioned battery detection method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not described here again.

The electronic device 11 of the present embodiment exists in a variety of forms, including but not limited to:

(1) a mobile communication device: such devices are characterized by mobile communications capabilities and are primarily targeted at providing voice, data communications. Such electronic devices include smart phones (e.g., iphones), multimedia phones, functional phones, and low-end phones, among others.

(2) The mobile personal computer equipment belongs to the category of personal computers, has calculation and processing functions and generally has the characteristic of mobile internet access. Such electronic devices include: PDA, MID, and UMPC devices, etc., such as ipads.

(3) A portable entertainment device: such devices can display and play video content, and generally also have mobile internet access features. This type of device comprises: video players, handheld game consoles, and intelligent toys and portable car navigation devices.

(4) And other electronic equipment with a video playing function and an internet surfing function.

The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by one or more processors, the computer program implements each process of the above battery detection method embodiment, and can achieve the same technical effect, and is not described herein again to avoid repetition. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above-described embodiments of the apparatus or device are merely illustrative, wherein the unit modules described as separate parts may or may not be physically separate, and the parts displayed as module units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (which may be a mobile terminal, a personal computer, a server, or a network device) to execute the method according to the embodiments or some parts of the embodiments of the present invention.

Finally, it should be noted that: the embodiments described above with reference to the drawings are only for illustrating the technical solutions of the present invention, and the present invention is not limited to the above-mentioned specific embodiments, which are only illustrative and not restrictive; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A frequency modulation method is applied to electronic equipment, and is characterized by comprising the following steps:

2. The method of claim 1, wherein determining the predicted load of the current frame according to the load status of at least one frame before the current frame comprises:

3. The method of claim 1, wherein determining the predicted load of the current frame according to the load status of at least one frame before the current frame comprises:

4. The method according to claim 3, wherein predicting the load step corresponding to the predicted load of the current frame according to the load step corresponding to each frame in at least one frame before the current frame comprises:

5. The method of claim 1, wherein determining the predicted load of the current frame according to the load status of at least one frame before the current frame comprises:

training a load prediction model;

6. The method of claim 1, wherein the predicting, according to the predicted load, a running time required for the current thread to run based on a preset frequency point combination of at least one module comprises:

7. The method of claim 1, wherein determining, according to the running time and at least one power consumption characteristic, power consumption generated by the current thread running based on each preset frequency point combination of the at least one module comprises:

8. The method of claim 1, wherein determining an initial time constraint comprises:

calculating the average frame rate in the preset time period;

9. The method of claim 8, further comprising:

dividing a preset time period into a plurality of time segments;

counting the average frame rate of each time segment in the preset time period;

10. The method of claim 1, wherein the determining the optimal frequency point combination corresponding to the at least one module according to the initial time constraint, the running time, and the power consumption comprises:

11. The method of claim 1, further comprising:

12. The method of claim 11, wherein the operation result comprises at least one of frame length, CPU frequency, GPU frequency, DDR frequency, frame drop condition, temperature, and current.

13. The method according to any of claims 1-12, wherein the load state comprises: processing at least one of the number of instructions required by the current frame, the number of times of occurrence of first-level data cache misses, the number of times of occurrence of second-level data cache misses, the number of times of occurrence of third-level data cache misses, a CPU frequency, a CPU utilization rate, a GPU frequency, a GPU utilization rate, a DDR frequency, an NPU utilization rate, a network speed and a temperature;

14. A frequency modulation device applied to electronic equipment is characterized by comprising:

15. The apparatus of claim 14, wherein the load prediction unit comprises:

16. The apparatus of claim 15, wherein the runtime unit comprises:

17. The apparatus of claim 16, wherein the power consuming unit comprises:

18. The apparatus of claim 17, wherein the optimal frequency point combining unit comprises:

a time constraint model for determining an initial time constraint;

19. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of frequency modulation according to any one of claims 1-13.

20. A computer-readable storage medium, comprising a computer program which, when run on a computer or processor, causes the computer or processor to carry out the frequency modulation method according to any one of claims 1-13.