CN116450363B

CN116450363B - Resource scheduling method and electronic equipment

Info

Publication number: CN116450363B
Application number: CN202310695606.0A
Authority: CN
Inventors: 董达
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2023-06-13
Filing date: 2023-06-13
Publication date: 2023-11-14
Anticipated expiration: 2043-06-13
Also published as: CN116450363A

Abstract

The application provides a resource scheduling method and electronic equipment, and relates to the technical field of terminals. After the game application is started, when the electronic equipment renders the t frame image of the game application, whether the rendering interval time between the t frame image and the t-1 frame image is larger than the first preset interval time or not can be judged, so that whether the rendering delay of the electronic equipment occurs or not is judged. Under the condition that the rendering delay is larger than the first preset interval time, the electronic equipment is indicated to have rendering delay, and the electronic equipment can determine the latest running requirement of the game application based on the number of vertex shaders, fragment shaders and drawing instructions used for rendering the t-th frame image, namely, determine a first load value corresponding to the t-th frame image, so that the electronic equipment utilizes the first load value to improve the frequency of a processor, improve the performance of the electronic equipment, realize timely and accurate adjustment of the performance of the electronic equipment and ensure the running fluency of the game application.

Description

Resource scheduling method and electronic equipment

Technical Field

The present application relates to the field of terminal technologies, and in particular, to a resource scheduling method and an electronic device.

Background

With the development of electronic devices, the types and the number of game applications that can be installed on the electronic devices are increasing. However, the performance requirement of the game application on the electronic device is high, and when the performance of the electronic device cannot meet the requirement, the game application may have the conditions of frame dropping, blocking and the like, so that the use experience of the user is affected.

Therefore, there is a need for a method of optimizing resources (e.g., processors) of an electronic device to improve the performance of the electronic device, thereby ensuring the smoothness of the game application.

Disclosure of Invention

The embodiment of the application provides a resource scheduling method and electronic equipment, which are used for improving the performance of the electronic equipment.

In a first aspect, a resource scheduling method is provided, after a first game application of an electronic device is started, whether a rendering interval time between a t-th frame image and a t-1 th frame image, which are newly rendered by the electronic device, is greater than a first preset interval time can be determined, where the t-1 th frame image represents a previous frame image of the t-th frame image.

Under the condition that the time is longer than the first preset interval time, the electronic equipment can determine how much resources are occupied by the electronic equipment for rendering the t frame image based on the image rendering information corresponding to the t frame image so as to obtain a first load value corresponding to the t frame image, that is, the first load value represents the latest running requirement of the first game application.

And then, the electronic equipment can perform resource scheduling operation based on the first load value corresponding to the t-th frame image so as to adjust the performance of the electronic equipment. Wherein the resource scheduling operation may include increasing a frequency of a processor in the electronic device.

In the application, after the electronic device finishes rendering a frame of game frame (such as the t frame image), the electronic device can judge whether the rendering interval time between the t frame image and the t-1 image is longer than the first preset interval time so as to judge whether the rendering is overtime, namely, judge whether the problem of frame dropping and blocking possibly occurs. Under the condition that the rendering interval time is longer than the first preset interval time, the occurrence of rendering timeout is indicated, namely that the performance of the electronic equipment possibly cannot meet the running requirement of the first game application, namely that the electronic equipment possibly has the problem of frame dropping and blocking, the electronic equipment can determine a first load value corresponding to a t frame image based on image rendering information corresponding to the t frame image, namely determining the latest running requirement of the first game application, so that the electronic equipment can utilize the latest running requirement of the first game application, the frequency of a processor is improved, the performance of the electronic equipment can meet the running requirement of the first game application, and the smoothness of the first game application is ensured. And after one frame of image is rendered, the application judges whether the electronic equipment is likely to have the problem of frame dropping and blocking, thereby carrying out frequency boosting in time when the problem of frame dropping and blocking is likely to occur and realizing the timely adjustment of the performance. The frequency boosting method and the device are used for carrying out frequency boosting based on the running requirement of the first game application, ensuring the accuracy of performance adjustment, and avoiding the situation that the electronic equipment has larger power consumption caused by overhigh performance adjustment and cannot meet the running requirement of the first game application caused by overlow performance adjustment.

The image rendering information corresponding to the t-th frame image may include at least one of the number of vertex shaders, the number of fragment shaders, and the number of drawing instructions.

In a possible implementation manner of the first aspect, when the rendering interval time is smaller than or equal to the first preset interval time, it indicates that no rendering timeout occurs, and there is less possibility that the electronic device has a problem of frame dropping and blocking, so that the electronic device does not need to perform a resource scheduling operation, that is, does not need to perform performance adjustment, and avoids wasting resources due to unnecessary performance adjustment.

In one possible implementation manner of the first aspect, in order to avoid resource waste caused by frequent performance adjustment, the electronic device may indicate that the rendering timeout degree is serious, that is, indicate that the possibility of frame dropping and blocking of the electronic device is high, where the rendering interval time is greater than the first preset interval time, and a difference between the rendering interval time and the first preset interval time is greater than the first preset time difference, so that the electronic device may increase the frequency of the processor to improve performance of the electronic device. And under the condition that the rendering interval time is larger than the first preset interval time, but the difference between the rendering interval time and the first preset interval time is smaller than the first preset time difference, the fact that the rendering timeout occurs is indicated, the timeout degree is light, the current performance of the electronic equipment can still meet the running requirement of the first game application, the electronic equipment does not need to improve the frequency of a processor, frequent adjustment of the performance is avoided, and high power consumption of the electronic equipment due to high performance is avoided.

In a possible implementation manner of the first aspect, performing the resource scheduling operation may further include reducing a frequency of the processor. Correspondingly, the process of performing the resource scheduling operation by the electronic device based on the first load value corresponding to the t-th frame image may include:

the electronic device may use a load level corresponding to a preset load range to which the first load value corresponding to the t-th frame image belongs as the first load level corresponding to the t-th frame image.

The electronic device determines whether the first load level is higher than the second load level to determine whether the running requirement of the first game application becomes high. The second load level represents a load level corresponding to a game frame rendered before the t-th frame image, such as a load level corresponding to the t-1 st frame image.

Under the condition that the load level is higher than the second load level, the running requirement of the first game application is indicated to be high, the current performance of the electronic equipment cannot meet the running requirement of the first game application, the frequency of the processor can be increased by the electronic equipment, so that the performance of the electronic equipment is improved, and the running fluency of the first game application is ensured.

Under the condition that the load level is lower than the second load level, the running requirement of the first game application is lower, the current performance of the electronic equipment can meet the running requirement of the first game application, and the frequency of the processor can be reduced by the electronic equipment so as to reduce the power consumption.

In a possible implementation manner of the first aspect, below the second load level, it indicates that the running requirement of the first game application is low, and the electronic device may not perform the resource scheduling operation, so as to avoid a resource loss caused by performing the resource scheduling operation.

And under the condition of being equal to the second load level, indicating that the running requirement of the first game application is unchanged, the electronic equipment can not perform the resource scheduling operation so as to avoid the resource loss caused by performing the resource scheduling operation.

In a possible implementation manner of the first aspect, after the game frame is rendered, the electronic device stores the game frame in a frame buffer, and when the game frame needs to be displayed, the electronic device may read from the frame buffer. In general, the rendered game frames of the electronic device are not immediately displayed, but are delayed. Therefore, when the rendering timeout occurs in the electronic device, the electronic device may not have the display timeout, that is, the jam problem may not occur. In order to avoid frequent adjustment of the performance, when the rendering timeout occurs, the electronic device may continue to determine whether the display timeout occurs, that is, whether the display interval time between the kth frame image and the kth-1 frame image is greater than the second preset interval time. The kth frame image represents a game frame currently displayed by the electronic device, and the kth-1 frame image represents a previous frame image of the kth frame image.

Under the condition that the display time is longer than the second preset interval time, the electronic equipment is indicated to have overtime display, namely the problem of jamming occurs, the electronic equipment can improve the frequency of a processor based on the first load value so as to improve the performance of the electronic equipment, solve the problem of jamming, ensure the operation smoothness of the first game application, and improve the use experience of users.

Under the condition that the rendering time-out is smaller than or equal to the second preset interval time, the electronic equipment is indicated to be overtime, but the display time-out does not occur, and the electronic equipment does not have the problem of blocking, so that the frequency of a processor is not required to be improved, and the power consumption loss caused by the improvement of the performance is avoided.

In a possible implementation manner of the first aspect, in order to avoid resource waste caused by frequent performance adjustment, the electronic device may indicate that the display timeout degree is serious and the problem of blocking of the electronic device is serious when the display interval time is greater than the second preset interval time and the difference between the display interval time and the second preset interval time is greater than the second preset time difference, so that the electronic device may increase the frequency of the processor to improve the performance of the electronic device. And under the condition that the display interval time is larger than the second preset interval time, but the difference between the display interval time and the second preset interval time is smaller than the second preset time difference, the display timeout appears, but the timeout degree is lighter, the performance of the electronic equipment can still meet the running requirement of the first game application, the electronic equipment does not need to improve the frequency of the processor, and therefore frequent adjustment of the performance is avoided, and high power consumption of the electronic equipment due to high performance is avoided.

In a possible implementation manner of the first aspect, the process of increasing the frequency of the processor by the electronic device may include:

firstly, the electronic equipment acquires a target gear corresponding to a first load level corresponding to the t-th frame image. The electronic device may then adjust the frequency of the processor based on the frequency (or referred to as the first frequency) corresponding to the target gear. In one case, the electronic device may directly use the frequency corresponding to the target gear as the target frequency, so as to adjust the frequency of the processor to the target frequency. In another case, the electronic device may comprehensively determine the target frequency based on the first frequency and the operation information of the electronic device, so as to adjust the frequency of the processor to the target frequency. In the other case, the electronic device uses the first frequency as a reference factor, and performs resource scheduling in combination with the overall situation of the electronic device, so as to realize accurate scheduling of resources.

The operation information may include a current temperature of the electronic device and/or other application conditions of the electronic device. For example, the first frequency is 2.3Ghz, but due to the higher current temperature, the electronic device may target 2Ghz to avoid excessive temperature of the electronic device due to the high frequency.

In a possible implementation manner of the first aspect, the first load value corresponding to the t-th frame image is a sum of all amounts in the image rendering information corresponding to the t-th frame image. For example, the first load value may be a sum of a number of vertex shaders, a number of fragment shaders, and a number of drawing instructions.

In a second aspect, the present application provides an electronic device comprising a memory and one or more processors; the memory is coupled to the processor; the memory is used for storing computer program codes, and the computer program codes comprise computer instructions; the computer instructions, when executed by the processor, cause the electronic device to perform the method as described above.

In a third aspect, the application provides a computer readable storage medium comprising computer instructions which, when run on an electronic device, cause the electronic device to perform the method as described above.

In a fourth aspect, the application provides a computer program product for, when run on an electronic device, causing the electronic device to perform the method as described above.

It will be appreciated that the advantages achieved by the electronic device according to the second aspect, the computer readable storage medium according to the third aspect, and the computer program product according to the fourth aspect provided above may refer to the advantages in the first aspect and any possible design manner thereof, and are not described herein.

Drawings

Fig. 1 is a schematic hardware structure of an electronic device according to an embodiment of the present application;

fig. 2 is a block diagram of an electronic device according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an interface for starting a game application according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an interface for running a game application according to an embodiment of the present application;

FIG. 5 is a second interface diagram illustrating a game application according to an embodiment of the present application;

FIG. 6 is a third interface diagram illustrating the operation of a game application according to an embodiment of the present application;

FIG. 7 is a diagram illustrating an interface for running a game application according to an embodiment of the present application;

FIG. 8 is a schematic diagram of an interface for running a game application according to an embodiment of the present application;

fig. 9 is a schematic flow chart of a resource scheduling method according to an embodiment of the present application;

FIG. 10 is a diagram illustrating an interface for running a game application according to an embodiment of the present application;

FIG. 11 is a diagram of an interface seventh for running a game application according to an embodiment of the present application;

FIG. 12 is a schematic diagram of an interface eight for running a game application according to an embodiment of the present application;

fig. 13 is a second flowchart of a resource scheduling method according to an embodiment of the present application;

fig. 14 is a schematic diagram of a resource scheduling process according to an embodiment of the present application;

fig. 15 is a schematic diagram of an information acquisition process according to an embodiment of the present application.

Detailed Description

The terms "first" and "second" are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

The terms related to the present application will be described first.

Vertex: a three-dimensional (3D) object is formed by vertices, e.g., a triangle has three vertices. One vertex is a point in 3D space formed by X, Y, Z coordinates.

Vertex shader: the method is generally used for realizing coordinate transformation of vertexes and vertex space transformation, and performing illumination calculation on a vertex by vertex basis to provide a data source for a fragment shader stage.

The primitives: is composed of vertices, and the primitive types may include points, line segments, or polygons.

Fragments: the pixel is converted into pixel data in an image frame.

Drawing (draw call) instruction: pointing to a list of primitives to be rendered, which is initiated from a central processor (central processing unit, CPU), received by a graphics processor (graphics processing unit, GPU).

Having described the terms involved in the present application, the following description of the technical solution of the present application will begin.

By way of example, fig. 1 shows a schematic diagram of an electronic device 200. As shown in fig. 1, the electronic device 200 may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (universal serial bus, USB) interface 230, a charge management module 211, a power management module 212, a battery 213, an antenna 1, an antenna 2, a mobile communication module 240, a wireless communication module 250, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, keys 290, a motor 291, an indicator 292, a camera 293, a display 294, a user identification module (subscriber identification module, SIM) card interface 295, and the like.

It should be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 200. In other embodiments of the application, electronic device 200 may include more or fewer components than shown, or certain components may be combined, or certain components may be separated, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 210 may include one or more processing units such as, for example: the processor 210 may include an application processor (application processor, AP), a modem processor, a GPU, an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The controller may be a neural hub and a command center of the electronic device 200, among others. The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 210 for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may hold instructions or data that the processor 210 has just used or recycled. If the processor 210 needs to reuse the instruction or data, it may be called directly from the memory. Repeated accesses are avoided and the latency of the processor 210 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 210 may include a CPU.

In some embodiments, processor 210 may include one or more interfaces. The interfaces may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serial bus, USB) interface, among others.

It should be understood that the connection relationship between the modules illustrated in the embodiment of the present application is only illustrative, and does not limit the structure of the electronic device 200. In other embodiments of the present application, the electronic device 200 may also employ different interfacing manners, or a combination of interfacing manners, as in the above embodiments.

The charge management module 211 is configured to receive a charge input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 211 may receive a charging input of a wired charger through the USB interface 230. In some wireless charging embodiments, the charging management module 211 may receive wireless charging input through a wireless charging coil of the electronic device 200. The charging management module 211 may also supply power to the electronic device through the power management module 212 while charging the battery 213.

The wireless communication function of the electronic device 200 can be implemented by the antenna 1, the antenna 2, the mobile communication module 240, the wireless communication module 250, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 200 may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 240 may provide a solution for wireless communication including 2G/3G/4G/5G, etc., applied on the electronic device 200. The mobile communication module 240 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module 240 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 240 may amplify the signal modulated by the modem processor, and convert the signal into electromagnetic waves through the antenna 1 to radiate the electromagnetic waves. In some embodiments, at least some of the functional modules of the mobile communication module 240 may be disposed in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 240 may be disposed in the same device as at least some of the modules of the processor 210.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 270A, receiver 270B, etc.), or displays images or video through display screen 294. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 240 or other functional module, independent of the processor 210.

The wireless communication module 250 may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN) (e.g., wireless fidelity (wireless fidelity, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field wireless communication technology (near field communication, NFC), infrared technology (IR), etc., as applied on the electronic device 200. The wireless communication module 250 may be one or more devices that integrate at least one communication processing module. The wireless communication module 250 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 210. The wireless communication module 250 may also receive a signal to be transmitted from the processor 210, frequency modulate it, amplify it, and convert it to electromagnetic waves for radiation via the antenna 2.

The electronic device 200 implements display functions through a GPU, a display screen 294, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or change display information.

The display 294 is used to display images, videos, and the like. The display 294 includes a display panel. The display panel may employ a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED) or an active-matrix organic light-emitting diode (matrix organic light emitting diode), a flexible light-emitting diode (flex), a mini, a Micro led, a Micro-OLED, a quantum dot light-emitting diode (quantum dot light emitting diodes, QLED), or the like. In some embodiments, the electronic device 200 may include 1 or N display screens 294, N being a positive integer greater than 1.

The electronic device 200 may implement a photographing function through an ISP, a camera 293, a video codec, a GPU, a display 294, an application processor, and the like.

The ISP is used to process the data fed back by the camera 293. For example, when photographing, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing and is converted into an image visible to naked eyes. ISP can also optimize the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in the camera 293.

The camera 293 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a Complementary Metal Oxide Semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transferred to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the electronic device 200 may include 1 or N cameras 293, N being a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process other digital signals besides digital image signals. For example, when the electronic device 200 is selecting a frequency bin, the digital signal processor is used to fourier transform the frequency bin energy, or the like.

Video codecs are used to compress or decompress digital video. The electronic device 200 may support one or more video codecs. In this way, the electronic device 200 may play or record video in a variety of encoding formats, such as: dynamic picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

The NPU is a neural-network (NN) computing processor, and can rapidly process input information by referencing a biological neural network structure, for example, referencing a transmission mode between human brain neurons, and can also continuously perform self-learning. Applications such as intelligent cognition of the electronic device 200 may be implemented by the NPU, for example: image recognition, face recognition, speech recognition, text understanding, etc.

The external memory interface 220 may be used to connect an external memory card, such as a Micro SD card, to enable expansion of the memory capabilities of the electronic device 200.

The internal memory 221 may be used to store computer executable program code that includes instructions. The processor 210 executes various functional applications of the electronic device 200 and data processing by executing instructions stored in the internal memory 221. The internal memory 221 may include a storage program area and a storage data area. The storage program area may store an application program (such as a sound playing function, an image playing function, etc.) required for at least one function of the operating system, etc. The storage data area may store data (e.g., voice, phonebook, etc.) created during use of the electronic device 200, and so on. In addition, the internal memory 221 may include a high-speed random access memory, and may further include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and the like.

The electronic device 200 may implement audio functions through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an ear-headphone interface 270D, an application processor, and the like. Such as music playing, recording, etc.

The audio module 270 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal.

Speaker 270A, also referred to as a "horn," is used to convert audio electrical signals into sound signals.

A receiver 270B, also referred to as a "earpiece", is used to convert the audio electrical signal into a sound signal.

Microphone 270C, also referred to as a "microphone" or "microphone," is used to convert sound signals into electrical signals.

The earphone interface 270D is for connecting a wired earphone.

Keys 290 include a power on key, a volume key, etc. The keys 290 may be mechanical keys. Or may be a touch key.

The indicator 292 may be an indicator light, which may be used to indicate a state of charge, a change in power, a message indicating a missed call, a notification, etc.

The sensor module 280 may include a pressure sensor, a gyroscope sensor, a barometric sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

Fig. 2 is a block diagram of an electronic device according to an embodiment of the present application. The software system of the electronic device 200 may employ a layered architecture, an event driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The layered architecture divides the software into several layers, each with distinct roles and branches. The layers communicate with each other through a software interface. In some embodiments, the system is divided into four layers, from top to bottom, an application layer, an application framework layer, runtime (run) and system libraries, and a kernel layer.

The application layer may include a series of application packages.

As shown in fig. 2, the application package may include applications for cameras, gallery, games, talk, map, navigation, WLAN, bluetooth, music, video, short messages, etc.

The application framework layer provides an application programming interface (application programming interface, API) and programming framework for application programs of the application layer. The application framework layer includes a number of predefined functions.

As shown in FIG. 2, the application framework layer may include a game service, a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

The game service is used to determine the frequency of the processor required to meet the fluent operational requirements of the game application based on the rendering of the game frames in the game application.

The window manager is used for managing window programs. The window manager can acquire the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make such data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc.

The view system includes visual controls, such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, a display interface including a text message notification icon may include a view displaying text and a view displaying a picture.

The telephony manager is used to provide the communication functions of the electronic device 200. Such as the management of call status (including on, hung-up, etc.).

The resource manager provides various resources for the application program, such as localization strings, icons, pictures, layout files, video files, and the like.

The notification manager allows the application to display notification information in a status bar, can be used to communicate notification type messages, can automatically disappear after a short dwell, and does not require user interaction. Such as notification manager is used to inform that the download is complete, message alerts, etc. The notification manager may also be a notification in the form of a chart or scroll bar text that appears on the system top status bar, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in a status bar, a prompt tone is emitted, the electronic device vibrates, and an indicator light blinks, etc.

runtimes include core libraries and virtual machines. runtime is responsible for scheduling and management of the android system.

The core library consists of two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. The virtual machine executes java files of the application program layer and the application program framework layer as binary files. The virtual machine is used for executing the functions of object life cycle management, stack management, thread management, security and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. Such as integrated frequency modulation modules, surface manager (surface manager), media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., openGL ES), two-dimensional graphics engines (e.g., SGL), etc.

The integrated frequency modulation module is used to determine the frequency that the processor needs to reach based on the frequency of the processor that is needed to meet the fluent operational requirements of the gaming application and the condition of other applications running on the electronic device 200.

The surface manager is used to manage the display subsystem and provides a fusion of two-dimensional and 3D layers for multiple applications.

Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like. The media library may support a variety of audio and video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc.

The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

In some embodiments, the runtime and system library layers may also be referred to as native (native) layers. The native layer belongs to the local framework.

A two-dimensional graphics engine is a drawing engine that draws two-dimensional drawings.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a frequency modulation interface, a display driver, a camera driver, an audio driver and a sensor driver.

The frequency modulation interface is used for realizing frequency adjustment of a processor on the electronic device 200 and realizing communication between a software layer and a hardware layer. Illustratively, the integrated frequency modulation module may adjust the frequency of the processor to the target frequency by invoking a frequency modulation interface.

The electronic equipment is limited in size, heat dissipation, power consumption and performance are always balanced, for example, when the performance of the electronic equipment is in a high state for a long time, the power consumption of the mobile phone is high, and the mobile phone needs to dissipate heat so as to avoid the heating problem. Many users have installed gaming applications on electronic devices. During the running of a game application on an electronic device, the game application may experience a frame drop problem, i.e., a stuck problem, because the performance of the electronic device may not meet the running requirements of the game application. For example, as shown in FIG. 3, the user clicks on the icon 10 of the first gaming application. The electronic device starts the first game application in response to a click operation of the icon 10 of the first game application. During the process of launching the first gaming application, the handset may first display a launch animation 20 as shown in FIG. 4, and then the electronic device may display a load interface 21 as shown in FIG. 5. The starting animation and the loading interface do not have any control for the user to operate. After the loading of the startup content is completed, the electronic device may display a login interface 30 as shown in fig. 6, where the login interface 30 may be used to input information such as a user account number, a password, and the like. After the login is successful, the mobile phone may display a game hall interface 40 as shown in fig. 7, where the game hall interface 40 may include user information such as a user's avatar 41, name 42, and level 43, and may further include a game mode such as a networking mode, a stand-alone mode, and the user may select the game mode by clicking any one of the game modes. And then, the electronic equipment can render and display an interface corresponding to the game mode so as to enable the user to experience the game. For example, the user clicks the networking mode, and the electronic device displays an interface, i.e., an image, corresponding to the networking mode as shown in fig. 8 in response to the clicking operation of the networking mode. The interface includes a virtual character (i.e., a small bright) corresponding to the user, other virtual characters (e.g., a small red) corresponding to players using the first game application, a game scene in which the virtual character is located, and the like.

During the running period of the first game application, for example, after the electronic equipment enters a certain game mode, the interface rendering task of the electronic equipment is heavier, the performance requirement on the electronic equipment is higher, if the performance of the electronic equipment cannot meet the requirement of the game application, frame dropping and game picture blocking stagnation can occur, the fluency of the game application is influenced, and the user experience is lower.

In some embodiments, to ensure game smoothness, the electronic device may schedule resources of the electronic device based on a current system load condition, e.g., the current system load is large, and the electronic device may increase a frequency of a processor (e.g., a CPU) in the electronic device to ensure performance of the electronic device. However, a scheduling policy based on a system load may not realize timely scheduling of resources of the electronic device, for example, when a game application is blocked in time, the frequency of the processor is increased to improve the performance of the electronic device, and the smoothness of the game cannot be effectively ensured.

Therefore, in view of the above-mentioned problems, the present application provides a resource scheduling method, after a first game application in an electronic device is started, after rendering a frame of image (such as a t frame of image), the electronic device may determine whether a rendering interval time between the t frame of image and the t-1 frame of image is greater than a first preset interval time. Wherein t is a positive integer.

Under the condition that the rendering interval time between the t frame image and the t-1 frame image is longer than the first preset interval time, the electronic equipment is indicated to have higher possibility of rendering overtime and is blocked, so that the performance of the electronic equipment needs to be improved, the electronic equipment can determine a first load value corresponding to the t frame image, namely, the load size corresponding to the first game application, based on the image rendering information corresponding to the t frame image, so that resources of the electronic equipment can be scheduled by utilizing the load size corresponding to the first game application, such as the frequency of a processor in the electronic equipment is improved, and the accurate improvement of the performance of the electronic equipment is realized. And under the condition that the rendering interval time between the t frame image and the t-1 frame image is longer than the first preset interval time, the rendering time of the electronic equipment for rendering the t frame image exceeds the expected time, and the rendering of the t frame image is completed after the rendering of the t-1 frame image is completed for a long time, so that the performance of the mobile phone at the moment cannot meet the running requirement of the first game application, and rendering timeout occurs, so that the electronic equipment possibly has the problem of frame dropping and blocking at the moment, the performance of the electronic equipment needs to be improved, the accurate judgment of the frequency increasing time is realized, the timely adjustment of the performance of the electronic equipment is realized, the performance of the electronic equipment can meet the running requirement of the first game application, the smoothness of the first game application is further ensured, and the use experience of a user is improved.

Under the condition that the rendering interval time between the t frame image and the t-1 frame image is smaller than or equal to the first preset interval time, the electronic equipment is indicated to have no rendering timeout, namely the possibility that the electronic equipment has display blocking is lower, in other words, the performance of the electronic equipment can meet the running requirement of the first game application, the electronic equipment does not need to improve the performance of the electronic equipment, so that unnecessary performance improvement is avoided, and the power consumption of the electronic equipment can be ensured.

The electronic device in the embodiment of the present application may be a device capable of installing an application, such as a mobile phone, a tablet computer, a wearable device, a personal digital assistant (personal digital assistant, PDA), a vehicle-mounted terminal, or an internet of things device, and the specific form of the electronic device is not particularly limited.

The resource scheduling method provided by the application will be described below by taking the electronic device as an example of a mobile phone. As shown in fig. 9, the resource scheduling method may include S301 to S309.

S301, responding to a first operation by the mobile phone, and starting a first game application.

The first operation is used for triggering the mobile phone to start a first game application, and the first game application is one game application on the mobile phone. Such as when the user wants to launch the first game application, the icon of the first game application may be clicked, where the operation of clicking the icon of the first game application by the user is the first operation.

S302, after the first game application is started, under the condition that the t frame image of the first game application is rendered, the mobile phone counts image rendering information corresponding to the t frame image in the first game application. Wherein the image rendering information includes at least one of a number of vertex shaders, a number of fragment shaders, and a number of drawing instructions.

Illustratively, after the first game application is started, that is, during the running of the first game application by the mobile phone, a game picture, that is, an image (or referred to as a game frame) needs to be rendered. The mobile phone may count at least one of the number of vertex shaders, the number of fragment shaders, and the number of drawing instructions required to render the t frame image of the first game application, so as to obtain image rendering information corresponding to the t frame image. Wherein t is a positive integer, and the image of the t frame represents a game frame which is newly rendered by the mobile phone.

The number of vertex shaders, the number of fragment shaders and the number of drawing instructions corresponding to the t frame image represent the number of vertex shaders, the number of fragment shaders and the number of drawing instructions utilized for rendering the t frame image. The more the number of vertex shaders, the number of fragment shaders and the number of drawing instructions corresponding to the image of the t frame are, the more the content to be rendered of the image of the t frame is, the heavier the rendering task is, the larger the rendering load corresponding to the image is, namely, the higher the performance requirement on the mobile phone is when the image is rendered.

It should be understood that the number of vertex shaders and fragment shaders on the mobile phone is fixed, and that when the mobile phone renders a game frame (i.e., an image of a game application), a certain number of vertex shaders and fragment shaders may be selected according to the requirements. For example, there are 100 vertex shaders, 100 fragment shaders on the handset. When rendering the t-th frame image, the mobile phone uses 1 vertex shader and 1 fragment shader, and correspondingly, the number of vertex shaders corresponding to the t-th frame image is 1, and the number of fragment shaders is 1.

It should be noted that, the rendering process of the image of the game application represents rendering the three-dimensional model (such as the patch model corresponding to the 3D object) and the texture in the game application onto the two-dimensional screen, and displaying the three-dimensional model and the texture in the form of pixels. The rendering process of the image may include an application stage, a geometry stage, and a rasterization stage.

Wherein, the application stage comprises the following steps: the method comprises the steps of obtaining data to be drawn of textures, materials, rays and colors to be rendered, setting corresponding rendering states, and sending a drawing instruction to the GPU by the CPU to inform the GPU to execute rendering.

The geometric phase described above: data submitted by the application stage is acquired, vertex transformation calculation is performed, model vertices (such as vertices of triangle primitives) are transformed from model space into screen space, and main tasks of the geometric stage include transforming three-dimensional vertex coordinates and performing illumination calculation of Zhu Hang vertices, which can be realized through vertex shaders.

The rasterization stage described above: setting the triangle segment after the geometric stage transformation, performing fragment-by-fragment operation, such as color modification, to obtain a corresponding rendering result, namely a displayable image, and storing the image into a frame buffer. The image in the frame buffer is for display on a display screen. Wherein the fragment-by-fragment operation may be implemented by a vertex shader.

Briefly, the above-mentioned rendering process of the image of the game application, that is, the rendering process of the game frame, may be generally divided into three parts, and first, the mobile phone may determine a patch model corresponding to the 3D object on the game frame. Then, the mobile phone can determine the visual element data such as texture, material, light, color and the like adopted by the patch model corresponding to the 3D object. And then, the mobile phone can call related APIs (such as Graphics APIs) based on the visual element data such as textures, materials, rays, colors and the like adopted by the surface patch model, so that the surface patch model is drawn, and the rendering of game frames is realized. Based on the finding, the rendering complexity corresponding to the game frame, that is, the rendering load size corresponding to the game frame, that is, the level of the mobile phone performance requirement has strong correlation with the vertex (vertex), texture (texture), drawCall, shader and other elements of the game frame. Therefore, the rendering load size corresponding to the game frame can be determined using these elements.

S303, the mobile phone calculates the difference between the rendering time of the t frame image and the rendering time of the t-1 frame image to obtain the rendering interval time.

S304, the mobile phone judges whether the difference between the rendering interval time and the first preset interval time is larger than the first preset time difference.

In one case of the embodiment of the present application, the rendering time of the t-th frame image represents the time when the rendering of the t-th frame image is completed. The rendering time of the t-1 st frame image represents the time when the rendering of the t-1 st frame image is completed. In another case, the rendering time of the t-th frame image represents the time when the t-th frame image starts to be rendered. The rendering time of the t-1 st frame image represents the time when the t-1 st frame image starts to be rendered. The t-1 frame image is the last frame image of the t frame image. Of course, the application can be used for judging whether the rendering time of one frame of image is the time of starting the rendering or the time of finishing the rendering according to the requirements.

Firstly, the mobile phone can subtract the rendering time of the t-1 frame image from the rendering time of the t frame image to obtain the rendering interval time between the t frame image and the t-1 frame image, namely the rendering interval time corresponding to the t frame image. The rendering interval time corresponding to the t frame image represents the actual frame rate of the mobile phone corresponding to the t frame image, namely the actual frame rate of the mobile phone when the rendering of the t frame image is completed. The larger the rendering interval time corresponding to the t frame image is, the longer the time for rendering the t frame image by the mobile phone is, that is, if the mobile phone currently displays the t-1 frame image, the mobile phone can display the t frame image after a longer time, and the rendering time-out occurs. In other words, the greater the rendering interval time, the greater the likelihood that the handset will experience a display drop frame stuck problem.

Then, the mobile phone can determine whether the rendering interval time corresponding to the t-th frame image has a larger difference from the first preset interval time, i.e. determine whether the difference between the rendering interval time and the first preset interval time is larger than the first preset time difference. When the difference is smaller than or equal to the first preset time difference, it indicates that the rendering interval time is smaller than the first preset interval time, that is, the mobile phone does not have rendering timeout, or the rendering timeout occurs, but the timeout time is shorter, in other words, even if the mobile phone has a problem of game screen jamming due to the rendering timeout, the game screen jamming degree is also lighter, in order to avoid the increase of power consumption caused by frequent adjustment of the mobile phone performance, the mobile phone may not need to perform adjustment of resources, and the mobile phone may execute S305.

Under the condition that the difference between the rendering interval time and the first preset interval time is larger than the first preset time difference, the fact that the difference between the rendering interval time and the first preset interval time is larger indicates that the mobile phone finishes rendering of the t-1 frame image after the rendering of the t-1 frame image is finished for a longer time, namely the mobile phone is indicated to have rendering timeout, the timeout time is longer, the probability of the mobile phone that the display is blocked is higher, in other words, the current performance of the mobile phone possibly cannot meet the game frame rendering requirement, the mobile phone needs to improve the performance of the mobile phone, and the mobile phone can execute S306, so that accurate judgment of the performance opportunity of the mobile phone is realized, and timely adjustment of the performance of the mobile phone is further realized.

The first preset interval time represents an expected rendering interval time, which may be a preset rendering interval time between images, or may be a standard frame rate corresponding to the mobile phone, that is, an interval time corresponding to the frame rate of the mobile phone in an ideal state. For example, the frame rate of the mobile phone is 30fps (frames per second), and the interval time corresponding to the frame rate of the mobile phone may be 1/33 second.

In addition, the first preset time difference may be set according to actual requirements, which is not limited by the present application.

It should be appreciated that in general, the first predetermined time difference is a non-negative number. Assuming that the first preset time difference is a positive number, the difference between the rendering interval time corresponding to the t-th frame image and the first preset interval time is less than or equal to the first preset time difference, which includes two cases. In one case, when the difference between the rendering interval time corresponding to the t-th frame image and the first preset interval time is smaller than or equal to 0, it indicates that the rendering interval time is smaller than or equal to the first preset interval time, the rendering time of the t-th frame image is smaller than or equal to the ideal rendering time, and the mobile phone does not have rendering timeout. In another case, the difference between the rendering interval time corresponding to the t-th frame image and the first preset interval time is greater than 0, but less than the first preset interval time, which indicates that the rendering interval time is slightly greater than the first preset interval time, the rendering time of the t-th frame image is slightly greater than the ideal rendering time, and the mobile phone has a rendering timeout but has a lighter timeout degree.

For example, assume that the first preset interval time is 20ms, and the first preset time difference is 10% of the first preset interval time, i.e., 2ms. The rendering interval time corresponding to the t-th frame image is 23ms. The difference between the rendering interval time and the first preset interval time is 3ms, and since 3ms is greater than 2ms, the mobile phone may perform S306.

In some embodiments, in order to ensure that the performance of the mobile phone can timely meet the running requirement of the first game application, so as to ensure the smoothness of the first game application, the mobile phone can directly determine whether the rendering interval time corresponding to the t-th frame image is greater than the first preset interval time. Under the condition that the rendering interval time is smaller than or equal to the first preset interval time, the fact that the rendering interval time does not exceed the expected rendering interval time is indicated, the fact that the mobile phone does not have rendering timeout is indicated, namely the fact that the actual frame rate of the mobile phone meets the expected requirement is indicated, namely the fact that the performance of the mobile phone can meet the running requirement of the first game application is indicated, the probability that the mobile phone has the display card problem is low, therefore, the mobile phone does not need to adjust the performance of the mobile phone, and the mobile phone can execute S305. Under the condition that the rendering interval time is larger than the first preset interval time, the fact that the rendering interval time exceeds the expected rendering interval time is indicated that the mobile phone has a rendering timeout, namely that the actual frame rate of the mobile phone does not meet the expected frame rate, namely that the performance of the mobile phone cannot meet the running requirement of the first game application, and the mobile phone can possibly have a display clamping problem is indicated, so that the mobile phone can execute S306 to improve the performance of the mobile phone.

S305, the mobile phone does not perform resource scheduling operation.

In some embodiments, when the difference between the first rendering interval time and the first preset interval time is less than or equal to the first preset time difference, it indicates that the performance of the mobile phone can meet the rendering requirement of the first game application, and the mobile phone may not need to improve the performance of the mobile phone. Of course, other applications, such as applications corresponding to rendering windows displayed by the mobile phone and background applications of the mobile phone, may be also run by the mobile phone in the process of running the first game application, so that the mobile phone can also schedule resources according to the running conditions of other applications, that is, based on the comprehensive load condition of the mobile phone, so as to ensure that the performance of the mobile phone can meet the running requirements of other applications.

S306, the mobile phone determines a first load level corresponding to the t frame image based on the image rendering information corresponding to the t frame image.

The method includes that when a difference between the first rendering interval time and the first preset interval time is larger than a first preset time difference, it is indicated that time for rendering the t frame image of the mobile phone is longer, current performance of the mobile phone may not need to meet a rendering requirement of the first game application, the mobile phone can obtain a first load value corresponding to the t frame image based on image rendering information corresponding to the t frame image to determine a latest rendering requirement of the first game application, that is, determine a current load condition corresponding to the first game application, so that resources of the mobile phone can be adjusted based on the current load condition corresponding to the first game application, and the resources of the mobile phone can meet the rendering requirement of the first game application. It should be appreciated that the load condition corresponding to the first game application represents the resources that the first game application needs to occupy for smooth operation.

The first load value corresponding to the t frame image represents the amount of resources occupied by the mobile phone needed (or supposed) to render the t frame image. The larger the first load value is, the more resources are occupied by the mobile phone for rendering the t-th frame image, and the higher the performance requirement of the mobile phone is.

Then, the mobile phone can acquire a preset load value range (or referred to as a first load value range) to which a first load value corresponding to the t frame image belongs, and takes a level corresponding to the first load value range as a first load level corresponding to the t frame image, so as to determine the condition of resource adjustment by using the first load level.

Wherein each preset load value range corresponds to a load level. The higher the load value included in the preset load value range, the higher the load level corresponding to the preset load value range, and the higher the mobile phone performance requirement.

In some embodiments, the load LEVEL (Render-weight-LEVEL) may include one or more of an initialization LEVEL (level_init), a light rendering LEVEL (level_low), and a heavy rendering LEVEL (level_high). Wherein the initialization level generally corresponds to a game application initialization scenario. When the mobile phone renders an image during the initialization of the game application, that is, the initialization interface (such as the interface shown in fig. 4), since the drawing function is not called, it may not use instructions such as a vertex shader, a fragment shader, a drawing instruction, etc., so that the load when the mobile phone renders the image is small, in other words, the performance requirement of the game application on the mobile phone is low. For example, the range of preset load values corresponding to the initialization level may be [0,3 ]. When the first load value corresponding to the t frame image is determined to be [0, 3), the first load level corresponding to the t frame image can be determined to be an initialization level, and the first game application is in an initialization scene and has low requirements on the performance of the mobile phone.

The above light rendering level generally corresponds to a light rendering scene of the game application, and the rendering content corresponding to an image of the light rendering scene (such as a login interface shown in fig. 6 and a game hall interface shown in fig. 7) is relatively small, so that when the mobile phone renders an image (i.e., an interface) of the light rendering scene of the game application, the number of instructions such as a vertex shader, a fragment shader, a drawing instruction and the like that need to be utilized is relatively small, and therefore, the load when the mobile phone renders the image is relatively small, in other words, the performance requirement of the game application on the mobile phone is relatively low. Illustratively, the range of pre-set load values for the mild rendering level may be [3, 120). When the first load value corresponding to the t frame image is determined to be [3, 120), the first load level corresponding to the t frame image can be determined to be a light rendering level, the first game application is in a light rendering scene, and the requirement on the performance of the mobile phone is low.

The heavy rendering level generally corresponds to a heavy rendering scene of the game application, and the image of the heavy rendering scene corresponds to a large amount of rendering content, such as an interface in a fight scene. When the mobile phone is used for heavily rendering images of scenes of game applications, the number of instructions such as vertex shaders, fragment shaders, drawing instructions and the like to be utilized is large, so that the load of the mobile phone when rendering the images is large, in other words, the performance requirement of the game applications on the mobile phone is high. By way of example only, and not by way of limitation, the range of preset load values for the heavy rendering level may be 120, ++ infinity A kind of electronic device. When the first load value corresponding to the t frame image is determined to be [120, ++ infinity), the first load level corresponding to the t frame image can be determined to be the heavy rendering level, the first game application is in the heavy rendering scene, and the requirement on the performance of the mobile phone is low.

In some embodiments, a rendering scene generally corresponds to rendering of a plurality of images, rendering contents corresponding to different images in the plurality of images may be different, and rendering complexity may be different, and thus, performance requirements of a mobile phone may be different, and a load level corresponding to the rendering scene may be further divided into a plurality of sub-levels, where each sub-level still corresponds to a preset load value range, so as to achieve accurate determination of a load level corresponding to the image, and thus, accurate adjustment of resources is achieved by using the load level corresponding to the image. Wherein, the preset load value range corresponding to each sub-level of one load level belongs to the preset load value range corresponding to the load level.

Illustratively, the heavy rendering level is divided into four sub-levels (Render-weight-sub-levels). As shown in table 1, the four Sub-levels may include a severe Sub-Level 1 (sub_level_1), a severe Sub-Level 2 (sub_level_2), a severe Sub-Level 3 (sub_level_3), and a severe Sub-Level 4 (sub_level_4), each of which corresponds to a preset load value range. The larger the preset load value range corresponding to the sub-level, that is, the larger the numerical value in the preset load value range corresponding to the sub-level, the more complex the rendering is, and the higher the requirement on the performance of the mobile phone is. For example, a severe sub-level 1 may correspond to a situation where a game scene is transformed, such as an image in which a game environment in which a Ming player is located is changed as shown in fig. 10, which corresponds to a lower rendering complexity than other sub-levels. For another example, severe sub-level 2 may correspond to a double fight situation in a fight scene (as shown in fig. 8 above), which corresponds to a lower rendering complexity than severe sub-levels 3 and 4. For another example, severe sub-level 3 may correspond to a three-person fight situation in a fight scene (as shown in fig. 11), which may correspond to a lower rendering complexity than severe sub-level 4. As another example, a severe sub-level 3 may correspond to a more aggressive condition in the aggressive scene (as shown in fig. 12), which corresponds to the highest degree of rendering complexity.

TABLE 1

In some embodiments, the load value corresponding to an image (e.g., the first load value corresponding to the t-th frame image) may be the sum of all the amounts in the image rendering information corresponding to the image, that is, the sum of the amounts of various information included in the image rendering information corresponding to the image.

Taking the example that the image rendering information corresponding to the image includes the number of vertex shaders, the load value corresponding to the image may be the number of vertex shaders.

Taking the example that the image rendering information corresponding to the image includes the number of fragment shaders, the load value corresponding to the image may be the number of fragment shaders.

Taking the number of drawing instructions included in the image rendering information corresponding to the image as an example, the load value corresponding to the image may be the number of drawing instructions.

Taking the example that the image rendering information corresponding to the image includes the number of vertex shaders and the number of fragment shaders, the load value corresponding to the image may be F (Material Complexity, geometry Complexity), F () represents the summing function, material Complexity represents the number of vertex shaders, and Geometry Complexity represents the number of fragment shaders. That is, the load value to which the image corresponds may be the sum of the number of vertex shaders and the number of fragment shaders.

Taking the example that the image rendering information corresponding to the image includes the number of fragment shaders and the number of drawing instructions, the load value corresponding to the image may be F (Geometry Complexity, drawcall Complexity), and Drawcall Complexity represents the number of drawing instructions. That is, the load value to which the image corresponds may be the sum of the number of fragment shaders and the number of drawing instructions.

Taking the example that the image rendering information corresponding to the image includes the number of vertex shaders and the number of drawing instructions, the load value corresponding to the image may be F (Material Complexity, drawcall Complexity). That is, the load value to which the image corresponds may be the sum of the number of vertex shaders and the number of drawing instructions.

Taking the example that the image rendering information corresponding to the image includes the number of vertex shaders and the number of fragment shaders, the load value corresponding to the image may be F (Material Complexity, geometry Complexity, drawcall Complexity). That is, the load value to which the image corresponds may be the sum of the number of vertex shaders, the number of fragment shaders, and the number of drawing instructions.

Of course, the above determination of the load value corresponding to the image based on the sum of the amounts of the various information included in the image rendering information corresponding to the image is merely an example, and the mobile phone may determine the load value corresponding to the image based on the various information included in the image rendering information corresponding to the image in other manners, for example, may calculate an average value based on a weighted summation manner, etc., which is not limited by the present application.

In some embodiments, the mobile phone may determine the type of information that needs to be utilized by the load value corresponding to the image according to the requirement, that is, the type of information included in the image rendering information. For example, to increase the accuracy of the load value determination, the handset may determine the load value using the number of vertex shaders, the number of fragment shaders, and the number of drawing instructions.

It should be noted that, because the performance requirements of the mobile phone, that is, the load size of the mobile phone has strong correlation with the elements such as vertices (vertexes), textures (textures), drawCall, and shaader of the game frame, the mobile phone can determine the load situation corresponding to the game application, that is, the resource situation required to be occupied by using the number of vertex shaders, the number of fragment shaders, and the number of drawing instructions, so that the accurate adjustment of the performance of the mobile phone can be realized by using the load situation.

S307, the mobile phone judges whether the first load level is higher than the second load level.

The second load level represents a load level corresponding to an image before the t frame image in the first game application. In some embodiments, the second load level may represent the load level corresponding to the t-1 st frame image, i.e., represent how much resources are required to be occupied to render the t-1 st frame image.

The higher the second load level is, the heavier the rendering task corresponding to the t-1 frame image is, the more complex the rendering content is, the more resources are needed to be occupied, and the higher the performance requirement on the mobile phone is. Of course, the second load level may also represent a load level corresponding to other images rendered before the t-th frame image is rendered, which is only required to be smaller than the preset number of images by the number of images between the other images and the t-th frame image.

In the embodiment of the application, the mobile phone can compare the first load level with the second load level to determine whether the first load level is higher than the second load level, namely, whether the load becomes larger when the mobile phone renders the t frame image is judged, namely, whether the performance requirement of the first game application on the mobile phone becomes higher. Under the condition that the first load level is lower than or equal to the second load level, the load of the mobile phone is indicated not to be increased, namely that the requirement of rendering the t-th frame image on the performance of the mobile phone is indicated not to be increased, the performance of the mobile phone can meet the running requirement of the first game application, the mobile phone does not need to improve the performance of the mobile phone, and the mobile phone can execute S308. Under the condition that the first load level is higher than the second load level, the load of the mobile phone is indicated to be increased, that is, the performance requirement of the mobile phone for rendering the t frame image by the mobile phone is indicated to be increased, the performance of the mobile phone may not meet the running requirement of the first game application, the mobile phone may need to perform resource adjustment, and then the mobile phone may execute S309.

S308, the mobile phone does not perform resource scheduling operation.

In some embodiments, when the first load level is equal to the second load level, it indicates that the rendering load corresponding to the t-th frame image, that is, the frame load, is not changed, that is, the performance requirement of the rendered t-th frame image and the image before the rendering (for example, the t-1-th frame image) on the mobile phone is not changed, and the performance of the mobile phone can meet the running requirement of the first game application, so that the mobile phone does not need to perform a resource scheduling operation at this time, for example, the performance of the mobile phone is improved.

Under the condition that the first load level is lower than the second load level, the frame load is indicated to be small, that is, the performance requirements of rendering the t-th frame image and the image before rendering on the mobile phone are reduced, so that in order to avoid that the power consumption of the mobile phone is faster due to high performance, the mobile phone can perform resource scheduling operation, such as reducing the frequency of a processor. The processor may include a CPU and/or a GPU.

For example, the process of reducing the frequency of the processor by the handset may include: the mobile phone obtains a target gear corresponding to the first load level, so that the gear of the processor is adjusted to the target gear, namely the frequency of the processor is adjusted to the frequency corresponding to the target gear, the frequency of the processor is reduced, the power consumption of the mobile phone is reduced, the heating problem of the mobile phone can be avoided, and the use experience of a user is ensured.

And S309, the mobile phone performs resource scheduling operation based on the first load level, wherein the adjustment operation comprises increasing the frequency of a processor in the mobile phone.

The mobile phone can acquire the first frequency corresponding to the first load level under the condition that the first load level is higher than the second load level, and the mobile phone can adjust the frequency of the processor to the first frequency so as to improve the performance of the mobile phone, so that the performance of the mobile phone can meet the running requirement of the first game application, and the smoothness of game running is ensured.

In some embodiments, after obtaining the first load level, the mobile phone may perform the resource scheduling operation directly based on the first load level, that is, adjust the performance of the mobile phone to a performance matching the first load level, without performing the step S307, that is, perform the resource scheduling operation only when the first load level is higher than the second load level or when the first load level is lower than the second load level. After the first load level is obtained, the mobile phone can determine a first frequency corresponding to the first load level, so that the frequency of the processor is adjusted to the first frequency, the performance of the mobile phone can ensure the rendering requirement of the first game application, the frame dropping is avoided, and the fluency of the first game application is ensured.

It should be noted that, the above resource scheduling operation by using the load level is only an example, and the mobile phone may also directly use the first load value corresponding to the t frame image to perform the resource scheduling operation. For example, the handset may determine a preset load value range (or referred to as a first load value range) to which the first load value belongs. Then, the mobile phone can adjust the frequency of the processor to a first frequency corresponding to the first load value range. Or, the mobile phone may determine whether the first load value range is greater than the second load value range, where the second load value range may represent a preset load value range to which a load value (or referred to as a second load value) corresponding to an image before the t-th frame image belongs. And under the condition that the first load value range is larger than the second load value range, the performance of the first game application on the mobile phone is higher, and the mobile phone can adjust the frequency of the processor to a first frequency corresponding to the first load value range. For another example, the mobile phone may determine a first frequency corresponding to the first load value, so as to adjust the frequency of the processor to the first frequency corresponding to the first load value. That is, the present application is not limited to the manner in which the first frequency is determined based on the first load value.

Wherein one value range is larger than another value range, and as the first load value range is larger than the second load value range, the values in the one value range are larger than any value in the other value range.

In some embodiments, to avoid frequent adjustment of resources, the mobile phone may increase the frequency of the processor in the mobile phone when the first load level is higher than the second load level and the number of levels of the interval between the first load level and the second load level is greater than or equal to the preset number of levels. Otherwise, the handset may not increase the frequency of the processor. Wherein the number of preset grades is greater than or equal to 1.

For example, the load level includes sub-levels, and the preset number of levels is 2. The first load level is a severe sub-level 4, the second load level is a severe sub-level 1, the first load level is higher than the second load level, and the intermediate interval severe sub-level 2 and severe sub-level 3, i.e. the number of levels of intervals is 2. The number of the levels is equal to the number of the preset levels, which indicates that the rendering requirement of the first game application is greatly changed, the performance of the mobile phone cannot meet the rendering requirement of the first game application at present, and the mobile phone may have a problem of serious jamming degree, so that the mobile phone can improve the frequency of the processor to improve the performance of the mobile phone. Wherein the number of preset grades is greater than or equal to 1.

For another example, the load level includes sub-levels, and the preset number of levels is 2. The first load level is a heavy sub-level 3, the second load level is a light rendering level, the first load level is higher than the second load level, and the intermediate interval heavy sub-level 1 and the heavy sub-level 2, i.e. the number of levels of intervals is 2. The number of the levels is equal to the number of the preset levels, which indicates that the rendering requirement of the first game application is greatly changed, the performance of the mobile phone cannot meet the rendering requirement of the first game application at present, and the mobile phone may have a problem of serious jamming degree, so that the mobile phone can improve the frequency of the processor to improve the performance of the mobile phone.

In the embodiment of the application, when the mobile phone renders the image in the first game application of one frame, the mobile phone can determine whether the load is changed greatly when the mobile phone renders the image by utilizing the rendering interval time between the image and the previous frame image, if the load is changed greatly, the rendering timeout appears, namely the mobile phone possibly appears display timeout, namely the frame dropping problem is indicated, so that the mobile phone can carry out frequency lifting, strongly correlates the frequency lifting with the frame dropping, realizes accurate judgment of the frequency lifting time, and ensures the rationality and timeliness of the frequency lifting time. And the mobile phone can determine the frame load corresponding to the image by utilizing the image rendering information corresponding to the image, namely, determine the latest occupied resource condition of the first game application, and schedule the resources of the mobile phone by utilizing the resource condition so as to improve the performance of the mobile phone and meet the running requirement of the first game application, thereby ensuring the running fluency of the first game application.

In some embodiments, the first frequency represents a frequency of a processor corresponding to meeting an operating requirement of the first game application. The mobile phone can directly take the first frequency as the target frequency. Or when the frequency of the processor is adjusted, the frequency of the processor can be determined by combining the whole operation condition of the mobile phone, namely, after the mobile phone determines the first frequency, the target frequency can be determined based on the first frequency and the operation information of the mobile phone, so that the frequency of the processor is adjusted to the target frequency, and the target frequency and the first frequency can be the same value or different values. The running information of the mobile phone can include information of other applications running on the mobile phone except the first game application and/or running state parameters of the mobile phone. The information of other applications may include identification of other applications (e.g., package name, name), occupancy of resources (e.g., memory), etc. The operation state parameters may include parameters such as temperature, electric quantity, etc. of the mobile phone.

In some embodiments, since the image of the first game application displayed by the mobile phone is an image obtained from the frame buffer, each time the mobile phone finishes rendering one frame of image, the frame image can be saved to the frame buffer. And then, the mobile phone reads the image from the frame buffer memory so as to display the read image on a display screen of the mobile phone. The image of the first game application currently displayed by the handset may not be the image of the first game application currently rendered, but rather the image previously rendered. Therefore, even if the interval time between the rendering images of the mobile phone is long, that is, even if the rendering time-out occurs to the mobile phone, the mobile phone does not have the frame dropping condition because enough images can be displayed by the mobile phone in the frame buffer memory, and the mobile phone does not need to improve the performance of the mobile phone. The process by which the handset determines whether improved handset performance is required based on the displayed image will be described in detail below in conjunction with figure 13. As shown in fig. 13, the process may include S401-S404.

S401, the mobile phone calculates the difference between the display interval time between the currently displayed image and the previous frame of display image and the second preset interval time.

Wherein the display interval time between the currently displayed image (or referred to as a kth frame image) and the previous frame display image (or referred to as a kth-1 frame image) may be a difference between the display time of the kth frame image and the display time of the kth-1 frame image. In this case, the display time of the k-1 frame image may be the time at which the k-1 frame image starts to be displayed. The display time of the kth frame image may be the time at which the display of the kth frame image starts, that is, the time at which the display of the previous frame image ends. In another case, the display time of the kth frame image indicates the time when the display of the kth frame image ends. The display time of the k-1 frame image indicates the time when the display of the k-1 frame image ends. Of course, the present application can be applied to whether the display time of one frame of image is the time of starting display or the time of ending display according to the requirements.

Correspondingly, the longer the display interval time between the kth frame image and the kth-1 frame image, the longer the display interval time indicates that the mobile phone continuously displays the kth-1 frame image.

S402, the mobile phone judges whether the difference between the display interval time and the second preset interval time is larger than a second preset time difference.

In the embodiment of the application, under the condition that the first load level is higher than the second load level, the rendering load of the mobile phone is increased, and the mobile phone possibly has the display blocking condition, so that the mobile phone can determine whether the mobile phone has the display blocking problem or not based on the display duration of the game frame, namely, the mobile phone can calculate the difference value between the display interval time between the kth frame image and the kth-1 frame image and the second preset interval time. Then, the mobile phone can judge whether the difference is larger than a second preset time difference. That is, it is determined whether a new image is displayed after a longer time has elapsed after the display of the k-1 th frame image, in other words, whether the display period of the k-1 th frame image is excessively long.

Under the condition that the difference between the display interval time and the second preset interval time is larger than the second preset time difference, the display of the new image is indicated to be performed after the display of the k-1 frame image is performed for a long time, namely, the display time of the k-1 frame image is too long, and the problem of frame dropping and blocking occurs when the display time of the mobile phone is overtime, so that the mobile phone can execute S403 to improve the performance of the mobile phone, enable the performance of the mobile phone to meet the running requirement of the first game application, solve the problem of display blocking of the mobile phone, ensure the fluency of the first game application and improve the user experience.

Under the condition that the difference between the display interval time and the second preset interval time is smaller than or equal to the second preset time difference, the display duration of the k-1 frame image is shorter and accords with the expected display duration, namely the actual frame rate currently displayed by the mobile phone accords with the expected display duration, and although the mobile phone has the rendering timeout, no frame dropping is blocked or the blocking degree is lighter, so that the performance of the mobile phone is not required to be improved, S404 can be executed, frequent resource scheduling is avoided, and higher power consumption of the mobile phone due to high performance can be avoided.

The second preset interval time represents an expected display interval time, which may be a preset display interval time between images, or may be a standard frame rate corresponding to the mobile phone, that is, an interval time corresponding to the frame rate of the mobile phone in an ideal state. The second preset interval time and the first preset interval time may be the same value or different values.

In addition, the second preset time difference may be set according to actual requirements, for example, the second preset time difference is 10% of the second preset interval time, which is not limited by the present application.

It should be appreciated that in general, the second predetermined time difference is a non-negative number. Assuming that the second preset time difference is a positive number, the difference between the display interval time between the kth frame image and the kth-1 frame image and the second preset interval time is less than or equal to the second preset time difference includes two cases. In one case, when the difference between the display interval time and the second preset interval time is smaller than or equal to 0, the display interval time is smaller than or equal to the second preset interval time, the duration of the mobile phone continuously displaying the k-1 frame image is shorter, and the display delay of the mobile phone does not occur, namely the problem of frame dropping and blocking does not occur. In another case, the difference between the display interval time and the second preset interval time is greater than 0, but less than the second preset interval time, which indicates that the display interval time is slightly greater than the second preset interval time, the display duration of the t-1 frame image is slightly greater than the ideal display duration, and the mobile phone has display timeout, but the timeout degree is light, so the mobile phone can not improve the performance first.

S403, the mobile phone increases the frequency of a processor in the mobile phone.

S404, the mobile phone does not increase the frequency of the processor.

The process of increasing the frequency of the processor by the mobile phone may refer to the process of increasing the frequency of the processor by the mobile phone, for example, increasing based on the target gear corresponding to the first load level.

For example, the above-mentioned t-th frame image is a 3 rd frame image. After the 3 rd frame image is rendered, the mobile phone stores the 3 rd frame image into a frame buffer. A rendering interval time between the 3 rd frame image and the 2 nd frame image is calculated. When the difference between the rendering interval time and the first preset interval time is greater than the first preset time difference, the mobile phone may determine a load level (i.e., a first load level) corresponding to the 3 rd frame image based on the image rendering information corresponding to the 3 rd frame image. Under the condition that the first load level is higher than the load level (namely the second load level) corresponding to the 2 nd frame image, the rendering requirement of the first game application is increased, the load of the mobile phone is increased, and the mobile phone possibly has a clamping problem, so that the mobile phone can continuously judge whether the clamping problem occurs.

In one example, assuming that the handset currently displays the 2 nd frame image, after reading the 3 rd frame image from the frame buffer, the 3 rd frame image is displayed. The 2 nd frame image becomes the previous frame display image, and the 3 rd frame image becomes the current display image.

The handset may calculate the display interval time between the 3 rd frame image and the 2 nd frame image. Then, the mobile phone can calculate the difference between the display interval time and the second preset time interval. Under the condition that the difference value is larger than the second preset time difference, the mobile phone reads the 3 rd frame image from the frame buffer after the 2 nd frame image is displayed for a long time, and the mobile phone has a problem of blocking due to the fact that the rendering time-out of the 3 rd frame image is larger, so that the mobile phone needs to improve the frequency of a processor, the performance of the mobile phone is improved, the running requirement of the first game application is met, and the fluency of the first game application is guaranteed.

In another example, assume that the handset is currently displaying the 1 st frame image, and the 2 nd frame image already exists in the frame buffer. After the mobile phone normally displays the 1 st frame image according to the standard frame rate, the mobile phone can continuously read the 2 nd frame image from the frame buffer memory and display the 2 nd frame image. At this time, although the rendering of the 3 rd frame image is overtime, the display of the mobile phone is not affected, that is, the game frame of the first game application can be smoothly displayed, the problem of blocking does not occur, the mobile phone does not need to improve the performance of the mobile phone, frequent adjustment of the performance of the mobile phone is avoided on the basis of meeting the running requirement of the application, the increase of power consumption caused by the improvement of the performance of the mobile phone is avoided, and the use experience of a user is improved.

After that, because the 3 rd frame image has a rendering timeout, it cannot be stored in the frame buffer in time, so that the 2 nd frame image has a display timeout, and a display blocking problem occurs, and the mobile phone needs to improve the performance.

In some embodiments, in order to implement timely adjustment of resources, that is, to implement timely adjustment of performance of the mobile phone, the mobile phone directly determines whether a display interval time between a currently displayed image and a last frame of display image is greater than a second preset interval time. And under the condition that the display interval time is larger than the second preset interval time, indicating that the rendering interval time exceeds the expected display interval time, the time for displaying the k-1 frame of image by the mobile phone is longer, the mobile phone is blocked, and the mobile phone can execute S403. If the display interval is less than or equal to the second preset interval, it indicates that the display interval does not exceed the expected display interval, and the mobile phone displays the kth frame image after displaying the kth-1 frame image for a short time, so that the first game application is still smooth, and no frame dropping and blocking problem occurs, and the mobile phone can execute S404.

In some embodiments, the mobile phone may also directly calculate the difference between the display interval time between the kth frame image and the kth-1 frame image and the second preset interval time in real time or periodically. And then, the mobile phone can judge whether the difference between the display interval time and the second preset interval time is larger than the second preset time difference. And under the condition that the time difference is larger than the second preset time difference, the mobile phone display is indicated to be blocked, namely, the mobile phone display is overtime, and the mobile phone can directly improve the mobile phone performance. And under the condition that the time difference is smaller than or equal to the second preset time difference, the mobile phone is indicated that the display jamming problem does not exist at present, and the mobile phone can not increase the frequency of the processor. And whether the rendering timeout occurs or not is judged without using the rendering interval time. And under the condition that the rendering time-out occurs, the mobile phone continuously judges whether the display time-out occurs or not.

In some embodiments, the step numbers do not represent the sequence of writing steps, the sequence of steps may be other sequences, for example, the mobile phone may determine the first load level first, and in the case that the first load level is higher than the second load level, the mobile phone may continuously determine whether the difference between the first rendering interval time corresponding to the t-th frame image and the first preset interval time is greater than the preset time difference.

It should be noted that, the above adjustment of the frequency of the processor is only one example of resource scheduling, that is, the processor is only one example of resources, and the mobile phone may also implement the adjustment of the performance of the mobile phone by adjusting other resources of the mobile phone.

In some embodiments, since the game application (such as the first game application) generally needs to use a game engine or an EGL library during running, the mobile phone can bind a slice model code (abbreviated as a slice model) with the game application (or a process described as the game application) through a instrumentation method when the game application starts to call the game engine or the EGL library, so that cross-process communication can be realized by using the slice model, so that other processes can determine whether to perform frequency boosting by using the rendering condition of the game application, and timely adjustment of the performance of the mobile phone is realized. The process of implementing cross-process communication using a slice model and determining whether to perform frequency boosting will be described below with reference to fig. 2. As shown in fig. 14, the process may include S1-S13.

S1, when a first game application process in the mobile phone is initialized, the first game application process is bound with the slice model, and first game application information is sent to the slice model.

The first game application information may include a package name corresponding to the first game application process, that is, a package name of the first game application.

In the embodiment of the application, the mobile phone starts the first game application in response to the first operation, that is, starts to create the first game application process corresponding to the first game application. Thereafter, when the first game application process is initialized, the game engine is invoked, and the slice model is already nested in the game engine, so that the first game application process is bound with the slice model.

It should be appreciated that the game engine described above may alternatively be described as an EGL library. Typically, the gaming application will call the EGL library.

S2, the slice model sends first game application information to a game service in the mobile phone based on a cross-process communication mode.

In the embodiment of the application, the first game application process and the game service belong to different processes, and cannot be directly communicated, so that the first game application process can communicate with the game service in the mobile phone by utilizing a slice model and combining a cross-process communication mode, thereby realizing cross-process communication.

The cross-process communication modes can include modes such as Activity, content Provider, broadcast and Service.

And S3, the application type identification module in the game service judges whether the mobile phone is in a game scene or not based on the first game application information, and if so, the application type identification module sends a handle of the game service to the slicing model.

In the embodiment of the application, since the game engine may be called by the non-game application, the application type identification module in the game service can firstly judge whether the type of the foreground application started by the mobile phone is the game type or not by using the first game application information, namely, judge whether the mobile phone is in a game scene or not. And under the condition that the type of the foreground application started by the mobile phone is a game type, indicating that the mobile phone is in a game scene, and the application type identification module can send a handle of the game service to the slice model. After the slice model obtains the handle of the game service, the initialization is completed, that is, when the mobile phone is in a game scene, the slice model completes the initialization of the slice model along with the creation of the first game progress.

And under the condition that the type of the foreground application started by the mobile phone is not the game type, indicating that the mobile phone is not in a game scene, and the game service does not need to send a handle to the slice model. The cell phone may perform resource scheduling based on other scheduling policies, such as temperature, load size of the cell phone, without using a slice model for resource scheduling.

S4, the first game application process acquires image rendering information corresponding to the t frame image in the process of rendering the t frame image.

S5, when the difference between the display interval time between the t frame image and the t-1 frame image and the first preset interval time is larger than the first preset time difference, the first game application process determines a first load level corresponding to the t frame image based on the image rendering information corresponding to the t frame image.

Wherein the first gaming application process may include a plurality of threads, wherein the plurality of threads have one rendering thread that loads the rendering work of the game frame of the first gaming application.

Illustratively, since the first gaming application process has been bound to the slice model, and the slice model has been initialized. Thus, the first gaming application process may gather image rendering information corresponding to each game frame, such as triples (number of vertex shaders, number of fragment shaders, number of drawing instructions) as the rendering thread renders each game frame. Wherein the newly rendered game frame can be used as the t-th frame image. And then, the first game application process can judge whether the difference value between the display interval time between the t frame image and the t-1 frame image and the first preset interval time is larger than the first preset time difference or not so as to judge whether the mobile phone has rendering delay or not, namely judging the probability of frame dropping of the mobile phone.

Under the condition that the rendering delay is larger than the first preset time difference, the probability of occurrence of frame dropping is higher, so that the first game application process can calculate the running requirement of the first game application based on the image rendering information corresponding to the t frame image so as to obtain a first load level corresponding to the t frame image. The first load level characterizes a resource condition required by the first gaming application process.

Under the condition that the first load level is smaller than or equal to the first preset time difference, the probability of occurrence of frame dropping is low when no rendering delay occurs, so that the first game application process does not need to calculate the first load level, unnecessary calculation is avoided, and resource waste is avoided.

S6, the first game application process judges whether the first load level is higher than the second load level. And sending the first load level to the slice model when the first load level is higher than the second load level.

S7, the slice model sends a first load level to the game service based on the cross-process communication mode.

In the embodiment of the application, when the first load level is lower than or equal to the second load level, the performance of the mobile phone can meet the running requirement of the first game application without improving the performance, so that the first load level is not required to be sent to the game service based on the cross-process communication mode, frequent reporting of data is avoided, and resource waste and power consumption loss caused by unnecessary cross-process communication are reduced.

In some embodiments, as shown in FIG. 15, the first game application process, after invoking the first function, for the application side, the rendering of the t-1 th frame image ends. Then, the first game application process calls a second function to start rendering the t-th frame image. The first game application process may perform instruction collection to record the number of instructions (e.g., the number of vertex shaders, the number of fragment shaders, and the number of drawing instructions) used to render the t-th frame image. After the first function is called, indicating that the rendering of the t frame image is finished, and obtaining image rendering information corresponding to the t frame image. Then, the mobile phone can send the image rendering information corresponding to the t frame image to the game service directly through the slice model under the condition that the difference between the display interval time between the t frame image and the t-1 frame image and the first preset interval time is larger than the first preset time difference, so that the game service calculates the first load level corresponding to the t frame image. And when the difference between the display interval time between the t frame image and the t-1 frame image and the first preset interval time is smaller than or equal to the first preset time difference, the first game application process does not need to send image rendering information corresponding to the t frame image to the game service through a slice model, so that cross-process communication is reduced, and the loss of power consumption caused by the cross-process communication is reduced. The first function may be an eglswappuffer function. The second function may be a deque buffer function.

Of course, the first game application process may also directly send the image rendering information of each frame of image to the game service through the slice model after collecting the image rendering information of each frame of image, and the game service process determines whether the rendering timeout occurs.

It will be appreciated that the rendering time of the above-mentioned t-th frame image represents the time when the rendering of the t-th frame image is completed, such as the time when the first function is invoked. Alternatively, the rendering time of the t-th frame image represents a time when the t-th frame image starts to be rendered, such as a time when the second function is called.

And S8, the data receiving module in the game service receives the first load level and sends the first load level to the game scene sensing module.

S9, the game scene perception module determines a first frequency gear of the processor corresponding to the first load level.

S10, the game scene perception module sends a first frequency gear to the scheduling module.

S11, the scheduling module calculates a difference value between the display interval time between the kth frame image and the kth-1 frame image and a second preset interval time. And under the condition that the difference value is larger than the second preset time difference, the scheduling module sends the first frequency gear to the comprehensive scheduling module.

In the embodiment of the application, under the condition that the time difference is smaller than or equal to the second preset time difference, the mobile phone is indicated to have no clamping condition, the performance can not be improved, the scheduling module does not need to send the first frequency gear to the comprehensive scheduling module, and the cross-process communication is reduced, so that the loss of power consumption caused by the cross-process communication is reduced.

And S12, the comprehensive scheduling module sends the target frequency to the kernel layer based on the first frequency gear.

And S13, the frequency modulation interface in the kernel layer sends the target frequency to the processor so as to adjust the frequency of the processor to the target frequency.

The integrated scheduling module may determine the scheduling policy, for example, determine the target frequency, based on the first frequency corresponding to the first frequency gear and the operation information of the mobile phone. And then, the comprehensive scheduling module can adjust the frequency of the processor to the target frequency by calling the frequency modulation interface.

In some embodiments, the game scene awareness module may also send the first frequency directly to the integrated scheduling module.

In some embodiments, since the slice model is bound to the game application process, in other words, since the slice model is integrated into the game application process, the slice model is synchronized with the lifecycle of the game application process, e.g., when the game application process is created, one slice model is bound to the game application process when the game application process is created, enabling the creation of the slice model. When the game application process is finished, the slicing model is also finished, so that the automatic management of the life cycle of the slicing model is realized, and the automatic recovery of the memory and the system resources can be realized.

It should be appreciated that the above-described gaming services may be in a one-to-many relationship with gaming application processes. The game service may receive load levels sent by the plurality of slice models. In general, however, a mobile phone foreground typically runs a game application, that is, a game service typically has a one-to-one relationship with a game application process.

In the embodiment of the application, the slice model is integrated in the first game application process, so that the image rendering information corresponding to each frame of image can be acquired in the process of rendering the image by the rendering thread in the first game application process, and the automatic acquisition of data is realized. And related data (such as load level) can be reported to the game service through the slice model, so that automatic cross-process reporting of the data is realized. In addition, the game service can determine the frequency of the processor based on the load level corresponding to the image, namely, determine the resource scheduling condition based on the actual frame rate, so that the comprehensive scheduling module can realize timely and accurate resource scheduling, and smooth game experience is ensured.

In some embodiments, the application provides a computer readable storage medium comprising computer instructions which, when run on an electronic device, cause the electronic device to perform the method as described above.

In some embodiments, the application provides a computer program product which, when run on an electronic device, causes the electronic device to perform the method as described above.

It will be apparent to those skilled in the art from this description that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to perform all or part of the functions described above.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and the parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions for causing a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read Only Memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely illustrative of specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for scheduling resources, comprising:

after a first game application of the electronic equipment is started, under the condition that the rendering interval time between a t-th frame image and a t-1 th frame image of the first game application is longer than a first preset interval time, the electronic equipment determines a first load value corresponding to the t-th frame image based on image rendering information corresponding to the t-th frame image; wherein the image rendering information includes at least one of a number of vertex shaders, a number of fragment shaders, and a number of drawing instructions; the t frame image represents a game frame which is newly rendered by the electronic equipment; the first load value corresponding to the t frame image represents the amount of resources which the electronic equipment needs to occupy when rendering the t frame image; the first load value is determined according to the quantity of image rendering information corresponding to the t-th frame image; the first load value is used for determining a first load level corresponding to the t-th frame image, and the first load level is a load level corresponding to a preset load value range to which the first load value belongs;

If the first load level corresponding to the t frame image is higher than the second load level, and the display interval time between the k frame image and the k-1 frame image of the first game application is longer than the second preset interval time, the electronic equipment increases the frequency of a processor in the electronic equipment based on the frequency corresponding to the first load level corresponding to the t frame image; wherein the kth frame image represents a game frame currently displayed by the electronic device; the kth frame image is the same as or different from the t frame image; the second load level represents a load level corresponding to an image before a t-th frame image in the first game application.

2. The method according to claim 1, wherein the method further comprises:

the electronic device reduces the frequency of the processor if the first load level is lower than a second load level; the second load level represents a load level corresponding to a game frame of the first game application rendered before the t-th frame image.

3. The method according to claim 1, wherein the determining, by the electronic device, the first load value corresponding to the t-th frame image based on the image rendering information corresponding to the t-th frame image, in a case where a rendering interval time between the t-th frame image and the t-1-th frame image of the first game application is greater than a first preset interval time, includes:

And under the condition that the rendering interval time is larger than the first preset interval time and the difference value between the rendering interval time and the first preset interval time is larger than a first preset time difference, the electronic equipment determines a first load value corresponding to the t-th frame image based on the image rendering information corresponding to the t-th frame image.

4. The method of claim 1, wherein, in the case where the display interval time between the kth frame image and the kth-1 frame image of the first game application is greater than a second preset interval time, the electronic device increases the frequency of the processor in the electronic device based on the frequency corresponding to the first load level corresponding to the kth frame image, including:

and under the condition that the display interval time is larger than a second preset interval time and the difference value between the display interval time and the second preset interval time is larger than a second preset time difference, the electronic equipment increases the frequency of the processor based on the frequency corresponding to the first load value.

5. The method of any one of claims 1, 3, or 4, wherein the electronic device increasing the frequency of a processor in the electronic device based on a frequency corresponding to a first load level corresponding to the t-th frame image comprises:

The electronic equipment acquires a target gear corresponding to a first load level corresponding to the t-th frame image; the first load level is a load level corresponding to a preset load value range to which the first load value belongs;

and the electronic equipment adjusts the frequency of the processor based on the frequency corresponding to the target gear.

6. The method according to claim 1, wherein the method further comprises:

in the case where the display interval time between the kth frame image and the kth-1 frame image of the first game application is less than or equal to the second preset interval time, the electronic device does not increase the frequency of the processor in the electronic device.

7. The method of any one of claims 1, 3, or 4, wherein the first load value for the t-th frame image is a sum of all amounts in the image rendering information for the t-th frame image.

8. An electronic device comprising a display screen, a memory, and one or more processors; the display screen, the memory and the processor are coupled; the display screen is used for displaying images generated by the processor, and the memory is used for storing computer program codes, and the computer program codes comprise computer instructions; the computer instructions, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 7.

9. A computer readable storage medium comprising computer instructions which, when run on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 7.