WO2023065873A1

WO2023065873A1 - Frame rate adjustment method, terminal device, and frame rate adjustment system

Info

Publication number: WO2023065873A1
Application number: PCT/CN2022/117608
Authority: WO
Inventors: 肖利军; 赵京
Original assignee: 荣耀终端有限公司
Priority date: 2021-10-22
Filing date: 2022-09-07
Publication date: 2023-04-27
Also published as: CN115016869A; CN115016869B

Abstract

Embodiments of the present application provide a frame rate adjustment method, a terminal device, and a frame rate adjustment system, applicable to the technical field of terminals. In the method, the image type of a target image is determined according to attribute information of a view control in the target image, when the image types of N consecutive target image frames are all non-visually sensitive frames, it is determined that a current image to be drawn is an image frame with low user attention, and image drawing is performed at a low second frame rate. Thus, by reducing the drawing frame rate, the power consumption of the terminal device is reduced, and the battery life of the terminal device is improved.

Description

Frame rate adjustment method, terminal device and frame rate adjustment system

This application claims the priority of the Chinese patent application with the application number 202111236271.3 and the application name "frame rate adjustment method, terminal equipment and frame rate adjustment system" submitted to the State Intellectual Property Office of China on October 22, 2021, the entire content of which Incorporated in this application by reference.

technical field

The present application relates to the technical field of terminals, and in particular to a frame rate adjustment method, a terminal device, and a frame rate adjustment system.

Background technique

With the continuous development of terminal technology, terminal devices such as smart phones and tablet computers have been widely used in people's life and work. Terminal devices continuously draw and render images to present various display images at different times.

At present, in order to improve the user's visual sensory experience, most terminal devices support a higher rendering frame rate. It also brings a greater power consumption overhead, resulting in a decrease in the battery life of the terminal device.

Contents of the invention

Embodiments of the present application provide a frame rate adjustment method, a terminal device, and a frame rate adjustment system, so as to reduce the power consumption of the terminal device and improve the battery life of the terminal device by reducing the drawing frame rate of image frames with low user attention .

In the first aspect, the embodiment of the present application proposes a frame rate adjustment method, which is applied to a terminal device. The terminal device acquires a target image from the image data to be drawn; the terminal device acquires the attribute information of the view control in the target image; the terminal device Determine the image type of the target image according to the attribute information; when it is detected that the image types of the consecutive N frames of the target image in the image data are all non-visually sensitive frames, the terminal device switches from the first frame rate to the second frame rate, at the first frame rate Image rendering is performed at a second frame rate; N is a positive integer, and the second frame rate is lower than the first frame rate.

Wherein, the view control may also be referred to as a view control. In this way, when the present application detects that the image types of the consecutive N frames of target images are all non-visually sensitive frames, it is determined that the current image to be drawn is an image frame with low user attention, and at this time, the second lower frame rate Image drawing is performed, thereby reducing the drawing frame rate of image frames with low user attention, thereby reducing the power consumption of the terminal device and improving the battery life of the terminal device.

In an optional implementation manner, the terminal device determines the image type of the target image according to the attribute information, including: the terminal device determines the first score value of each view control according to the attribute information of each view control in the target image ; The terminal device calculates the second score value of the target image according to the first score value of each view control; the terminal device determines the image type of the target image according to the second score value. Since the first score value of the view control in different images is different, the application can calculate the second score value of the target image based on the first score value of each view control in the target image, and determine the target according to the second score value Whether the image type of the image is a visually sensitive frame or a non-visually sensitive frame, so as to improve the accuracy of the type judgment result of the target image.

In an optional implementation manner, the terminal device determines the first score value of each view control according to the attribute information of each view control in the target image, including: the terminal device searches the target score value lookup table for the target The third score value corresponding to the attribute information of each view control in the image; the terminal device calculates the first score value of each view control according to the third score value. In this way, the third score value corresponding to the attribute information of the view control is searched by means of table lookup, thereby further determining the first score value of the view control, which simplifies the calculation process of the first score value of the view control.

In an optional implementation, the attribute information is any one of type information, location information, and area information; the terminal device calculates the first score value of each view control according to the third score value, including: the terminal The device determines the third score value corresponding to the attribute information of each view control as the first score value of each view control. In this way, the third score value of an attribute information is directly determined as the first score value of the corresponding view control, which reduces the computational complexity of the first score value of the view control.

In an optional implementation manner, the attribute information includes at least two of type information, location information, and area information; the terminal device calculates the first score value of each view control according to the third score value, including: the terminal The device looks up the weight value corresponding to the attribute information of each view control from the target weight lookup table; the terminal device performs weighted summation according to the corresponding weight value for the third scoring value corresponding to the attribute information of each view control to obtain each The first score value of a view control. Since in different applications, the weight value of each attribute information of view control may be different, therefore, the third score value corresponding to the attribute information of each view control is weighted and summed according to the corresponding weight value, which can improve the calculation. The accuracy of the first scoring value of , thereby further improving the accuracy of the type judgment result of the target image.

In an optional implementation manner, before the terminal device looks up the third score value corresponding to the attribute information of each view control in the target image from the target score lookup table, the terminal device further includes: the terminal device determines that the target image belongs to The target application program; the terminal device searches the target score lookup table corresponding to the target application program from the preset score lookup tables corresponding to each application program. According to the actual needs of different applications, set its corresponding score lookup table for different applications, so that the score in each score lookup table can more accurately reflect the actual score of the attribute information of the view control in the target image, Therefore, the accuracy of the type judgment result of the target image is improved.

In an optional implementation manner, before the terminal device searches the weight value corresponding to the attribute information of each view control from the target weight lookup table, the terminal device further includes: the terminal device determines the target application program to which the target image belongs; the terminal device Find the target weight lookup table corresponding to the target application program from the preset weight lookup table corresponding to each application program. According to the actual needs of different applications, set their corresponding weight lookup tables for different applications, so that the weight value in each weight lookup table can more accurately reflect the proportion of the attribute information of the view control in the target image, so that Improve the accuracy of the type judgment result of the target image.

In an optional implementation manner, the terminal device calculates the second score value of the target image according to the first score value of each view control, including: the terminal device calculates an average value of the first score values of each view control , to get the second score value of the target image. In this way, by comprehensively considering the first scoring values of multiple view controls in the target image, the second scoring value of the target image is finally calculated, so that the accuracy of the type judgment result of the target image determined according to the second scoring value is higher.

In an optional implementation manner, the terminal device determines the image type of the target image according to the second score value, including: the terminal device compares the second score value with a preset threshold; when the second score value is greater than the preset threshold When , it is determined that the image type of the target image is a visually sensitive frame; when the second score value is less than or equal to a preset threshold, it is determined that the image type of the target image is a non-visually sensitive frame. In this way, the image type of the target image is judged directly through the size relationship between the second scoring value and the preset threshold, and the judgment process is relatively simple.

In an optional implementation, after the terminal device switches from the first frame rate to the second frame rate and performs image drawing at the second frame rate, it also includes: when detecting consecutive M frames of target images in the image data When the image types of are all visually sensitive frames, the terminal device switches from the second frame rate to the first frame rate, and draws images at the first frame rate; M is a positive integer. In this way, when it is detected that the image types of the consecutive M frames of target images in the image data are all visually sensitive frames, it is determined that the image in the image data is a scene picture with a higher degree of user attention, and therefore, a higher first The frame rate is used for image drawing, which makes the image picture after drawing, rendering and display smoother, and improves the user's visual experience.

In an optional implementation, when it is detected that the image types of the consecutive N frames of target images in the image data are all non-visually sensitive frames, the terminal device switches from the first frame rate to the second frame rate, and uses the second The frame rate is used for image drawing, including: when it is detected that the image types of the consecutive N frames of target images in the image data are all non-visually sensitive frames, the terminal device performs the Nth non-visually sensitive frame and each subsequent frame of image, each Draw a frame of images at intervals of R frames of images until it is detected that there are consecutive M frames of target images in the image data, all of which are visually sensitive frames; R is a positive integer.

In an optional implementation, when it is detected that the image types of the M consecutive target images in the image data are all visually sensitive frames, the terminal device switches from the second frame rate to the first frame rate, and uses the first frame rate Image drawing at a high rate, including: when it is detected that the image types of the M consecutive target images in the image data are all visually sensitive frames, the terminal device draws the Mth visually sensitive frame and every frame thereafter until It is detected that there are consecutive N frames of target images in the image data, all of which are non-visually sensitive frames.

In an optional implementation manner, the view control is a view control whose display content changes in the image data. In this way, when the view control targeted by the embodiment of the present application is a view control whose display content in the image data changes, the number of view controls involved in the calculation can be reduced, and the complexity of subsequent calculation of the image type of each target image can be reduced, thereby Reduce the workload of terminal devices when performing computing operations.

In an optional implementation manner, the terminal device obtains the target image from the image data to be drawn, including: the terminal device extracts a frame of image every X frames from the multi-frame images included in the image data as the target image; X is a positive integer. In this way, the number of extracted target images can be reduced, and the complexity of subsequently calculating the image type of each target image is reduced, thereby reducing the workload of the terminal device when performing calculation operations.

In the second aspect, the embodiment of the present application proposes a terminal device, the terminal device includes a processing unit, and the processing unit is configured to acquire a target image from image data to be drawn; acquire attribute information of a view control in the target image; Attribute information to determine the image type of the target image; when it is detected that the image types of the continuous N frames of the target image in the image data are all non-visually sensitive frames, switch from the first frame rate to the second frame rate, and use the second frame rate Perform image drawing; N is a positive integer, and the second frame rate is lower than the first frame rate.

In the third aspect, the embodiment of the present application proposes a terminal device, including a memory and a processor, the memory is used to store a computer program, and the processor is used to call the computer program to perform the following steps: acquire a target image from the image data to be drawn; Obtain the attribute information of the view control in the target image; determine the image type of the target image according to the attribute information; Switch to the second frame rate, and draw images at the second frame rate; N is a positive integer, and the second frame rate is lower than the first frame rate.

In an optional implementation manner, the processor is specifically configured to perform the following steps: determine the first score value of each view control according to the attribute information of each view control in the target image; A score value, calculating a second score value of the target image; according to the second score value, determining the image type of the target image.

In an optional implementation manner, the processor is specifically configured to perform the following steps: from the target score lookup table, look up the third score value corresponding to the attribute information of each view control in the target image; Value, calculate the first score value of each view control.

In an optional implementation manner, the attribute information is any one of type information, position information, and area information; the processor is specifically configured to perform the following steps: the third score value corresponding to the attribute information of each view control , determine the first score value for each view control.

In an optional implementation manner, the attribute information includes at least two of type information, position information, and area information; the processor is specifically configured to perform the following steps: Find the attribute of each view control from the target weight lookup table The weight value corresponding to the information; the third score value corresponding to the attribute information of each view control is weighted and summed according to the corresponding weight value to obtain the first score value of each view control.

In an optional implementation manner, the processor is further configured to perform the following steps: determine the target application program to which the target image belongs; and search for the target application program corresponding to the target application program from the preset score lookup table corresponding to each application program. Score lookup table.

In an optional implementation manner, the processor is further configured to perform the following steps: determine the target application program to which the target image belongs; and search for the target weight corresponding to the target application program from the preset weight lookup table corresponding to each application program lookup table.

In an optional implementation manner, the processor is specifically configured to perform the following step: calculating an average value of the first score values of each view control to obtain the second score value of the target image.

In an optional implementation manner, the processor is specifically configured to perform the following steps: comparing the second score value with a preset threshold; when the second score value is greater than the preset threshold, determine that the image type of the target image is visual A sensitive frame; when the second scoring value is less than or equal to a preset threshold, determine that the image type of the target image is a non-visually sensitive frame.

In an optional implementation manner, the processor is further configured to perform the following steps: when it is detected that the image types of the M consecutive frames of target images in the image data are all visually sensitive frames, switch from the second frame rate to the first Frame rate, image rendering is performed at the first frame rate; M is a positive integer.

In an optional implementation, the processor is specifically configured to perform the following steps: when it is detected that the image types of the N consecutive frames of the target image in the image data are all non-visually sensitive frames, the non-visually sensitive frame of the Nth frame And for each frame of image thereafter, draw a frame of image every R frame of image until it is detected that there are consecutive M frames of target images in the image data, and the image types of the target image are all visually sensitive frames; R is a positive integer.

In an optional implementation manner, the processor is specifically configured to perform the following steps: when it is detected that the image types of the consecutive M frames of target images in the image data are all visually sensitive frames, for the Mth visually sensitive frame and subsequent Each frame of image is drawn until it is detected that there are consecutive N frames of target images in the image data, all of which are non-visually sensitive frames.

In an optional implementation manner, the view control is a view control whose display content changes in the image data.

In an optional implementation manner, the processor is specifically configured to perform the following steps: from the multiple frames of images included in the image data, extract a frame of image every X frames as the target image; X is a positive integer.

In a fourth aspect, the embodiment of the present application provides a computer-readable storage medium, in which a computer program or instruction is stored, and when the computer program or instruction is executed, the above frame rate adjustment method is realized.

In a fifth aspect, the embodiment of the present application provides a computer program product, including a computer program, which, when the computer program is run, causes the computer to execute the above frame rate adjustment method.

In the sixth aspect, the embodiment of the present application proposes a frame rate adjustment system, including a server and the above-mentioned terminal device; the server is used to convert the changed score to The query table is sent to the terminal device; and when the data in the weight query table stored in the server is changed, the changed weight query table is sent to the terminal device.

The effects of the possible implementations of the second aspect to the sixth aspect are similar to those of the first aspect and possible designs of the first aspect, and will not be repeated here.

Description of drawings

FIG. 1 is a schematic structural diagram of a terminal device hardware system provided by an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a terminal device software system provided by an embodiment of the present application;

FIG. 3 is a flow chart of a frame rate adjustment method provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of changing the image rendering frame rate in the embodiment of the present application;

Fig. 5 is the schematic diagram of two kinds of target images in the embodiment of the present application;

FIG. 6 is a flowchart of determining the image type of a target image according to attribute information in an embodiment of the present application;

FIG. 7 is a schematic diagram of a hardware structure of a terminal device provided in an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a frame rate adjustment system provided by an embodiment of the present application.

Detailed ways

In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same function and effect. For example, the first chip and the second chip are only used to distinguish different chips, and their sequence is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the number and execution order, and words such as "first" and "second" do not necessarily limit the difference.

It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used as examples, illustrations or descriptions. Any embodiment or design described herein as "exemplary" or "for example" is not to be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete manner.

In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A, B can be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one item (piece) of a, b, or c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, c can be single or multiple .

In order to improve the problem of reduced battery life of the terminal device caused by a high rendering frame rate, in related technologies, an application list can be preset, and the application list includes a set rendering frame rate corresponding to multiple applications. Exemplary , set a higher drawing frame rate for video applications or game applications, for example, the drawing frame rate is 120 frames per second (frames per second, fps), and set a lower drawing frame rate for news browsing applications , such as drawing at a frame rate of 30fps. When the image to be drawn is acquired, the application program to which the image to be drawn belongs is determined first, then the set drawing frame rate corresponding to the application program is obtained from the application program list, and finally the image is drawn based on the acquired set drawing frame rate.

Therefore, in the related art, some application programs use a lower set rendering frame rate for image rendering to reduce the power consumption of the terminal device, thereby improving the battery life of the terminal device. However, this method of adjusting the drawing frame rate cannot dynamically adjust the drawing frame rate in real time according to the display content actually required by the screen, resulting in a poor effect on improving the display effect and power consumption of the terminal device.

Exemplarily, for a certain video application, image drawing is always performed at a higher drawing frame rate, but in some cases it is not necessary to maintain such a high drawing frame rate, for example, when the image to be drawn is the When setting the interface of a video application, it does not contain video images, but only contains some text content and/or pictures. The setting interface does not require very high drawing frame rate. When drawing the setting interface at a higher drawing frame rate , which will lead to an increase in the power consumption of the terminal device; or, for a news browsing application, image rendering will always be performed at a lower rendering frame rate, but in some cases, image rendering will be performed at a lower rendering frame rate , it will cause the final display screen to appear stuttering and other phenomena. For example, when the image to be drawn is a video screen that needs to be played in the news browsing application, it needs to draw the image at a higher rendering frame rate to ensure that the image If the image is drawn at a lower frame rate, it will cause the final display screen to appear stuttering and other phenomena.

Based on this, the embodiment of the present application provides a frame rate adjustment method, by determining whether the image type of the target image is a visually sensitive frame or a non-visually sensitive frame according to the attribute information of the view control in the target image, when consecutive M frames of the target When the image types of the images are all visually sensitive frames, it is determined that the current image to be drawn is an image frame with high user attention, and at this time, the image is drawn at a higher first frame rate until there are consecutive N frames of the target image The image types are all non-visually sensitive frames, thereby improving the fluency of the picture and improving the user's visual sensory experience; and when the image types of the continuous N frames of target images are all non-visually sensitive frames, it is determined that the current image to be drawn is For image frames with low user attention, image rendering is performed at a lower second frame rate until the image type of the target image in consecutive M frames is a visually sensitive frame, thereby reducing the number of terminal devices by reducing the rendering frame rate. Power consumption overhead, improve the battery life of the terminal equipment.

Therefore, the embodiment of the present application can dynamically adjust the drawing frame rate in real time according to the display content actually required by the screen, and increase the drawing frame rate of image frames with high user attention, so as to improve the display effect of the terminal device, and reduce user attention. The drawing frame rate of the low-resolution image frame is reduced, thereby reducing the power consumption of the terminal device, so as to improve the display and power consumption of the terminal device.

The frame rate adjustment method provided in the embodiment of the present application can be applied to a terminal device with a display function, and the terminal device can also be called a terminal (terminal), user equipment (user equipment, UE), mobile station (mobile station, MS), mobile terminal (mobile terminal, MT), etc. The terminal device can be mobile phone, smart TV, wearable device, tablet computer (Pad), computer with wireless transceiver function, virtual reality (virtual reality, VR) terminal device, augmented reality (augmented reality, AR) terminal Equipment, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grid, transportation Wireless terminals in transportation safety, wireless terminals in smart city, wireless terminals in smart home, etc. The embodiment of the present application does not limit the specific technology and specific device form adopted by the terminal device.

In order to better understand the embodiment of the present application, the structure of the terminal device in the embodiment of the present application is introduced below. Exemplarily, FIG. 1 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.

The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

It can be understood that, the structure shown in the embodiment of the present application does not constitute a specific limitation on the terminal device 100 . In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be recalled from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuitsound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver) /transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and/or Universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 may be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 communicates with the touch sensor 180K through an I2C bus interface to realize the touch function of the terminal device 100 .

The I2S interface can be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both I2S interface and PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160 . For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through a CSI interface to realize the shooting function of the terminal device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the terminal device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the terminal device 100, and can also be used to transmit data between the terminal device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules shown in the embodiment of the present application is a schematic description, and does not constitute a structural limitation of the terminal device 100 . In other embodiments of the present application, the terminal device 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through the wireless charging coil of the terminal device 100 . While the charging management module 140 is charging the battery 142 , it can also supply power to the terminal device through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input of the battery 142 and/or the charging management module 140, and supplies power for the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110 . In some other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the terminal device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. The antennas in the terminal device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the terminal device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area (wireless local area networks, WLAN) (such as wireless fidelity (wireless fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation, etc. applied on the terminal device 100. Satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. Wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband code division Multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and/or IR technology etc. GNSS can include global positioning system (global positioning system, GPS), global navigation satellite system (global navigation satellite system, GLONASS), Beidou satellite navigation system (beidounavigation satellite system, BDS), quasi-zenith satellite system (quasi-zenith satellite system) , QZSS) and/or satellite based augmentation systems (SBAS).

The terminal device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used for displaying images, displaying videos, receiving sliding operations, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrixorganic light-emitting diode). , AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the terminal device 100 may include one or more display screens 194 .

The terminal device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.

The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the optical signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the terminal device 100 may include one or more cameras 193 .

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The terminal device 100 may support one or more video codecs. In this way, the terminal device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the terminal device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data created during the use of the terminal device 100 (such as audio data, phonebook, etc.) and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like. The processor 110 executes various functional applications and data processing of the terminal device 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

The terminal device 100 may implement an audio function through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, and an application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .

Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. The terminal device 100 can listen to music through the speaker 170A, or listen to hands-free calls.

Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the terminal device 100 receives a phone call or voice information, the receiver 170B can be placed close to the human ear to receive the voice.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C. The terminal device 100 may be provided with at least one microphone 170C. In some other embodiments, the terminal device 100 may be provided with two microphones 170C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the terminal device 100 can also be provided with three, four or more microphones 170C to realize sound signal collection, noise reduction, identify sound sources, and realize directional recording functions, etc.

The earphone interface 170D is used for connecting wired earphones. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the terminal device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The terminal device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions.

The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100 . In some embodiments, the angular velocity of the terminal device 100 around three axes (ie, x, y and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the terminal device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the terminal device 100 through reverse motion to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The terminal device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip holster. In some embodiments, when the terminal device 100 is a clamshell machine, the terminal device 100 may detect opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the acceleration of the terminal device 100 in various directions (generally three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to recognize the posture of terminal equipment, and it can be applied to applications such as horizontal and vertical screen switching, pedometer, etc.

The distance sensor 180F is used to measure the distance. The terminal device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The terminal device 100 emits infrared light through the light emitting diode. The terminal device 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100 . When insufficient reflected light is detected, the terminal device 100 may determine that there is no object near the terminal device 100 . The terminal device 100 can use the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear to make a call, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.

The ambient light sensor 180L is used for sensing ambient light brightness. The terminal device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in the pocket to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The terminal device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access to the application lock, take pictures with fingerprints, answer incoming calls with fingerprints, and so on.

The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the terminal device 100 may reduce the performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In some other embodiments, when the temperature is lower than another threshold, the terminal device 100 heats the battery 142 to avoid abnormal shutdown of the terminal device 100 caused by the low temperature. In some other embodiments, when the temperature is lower than another threshold, the terminal device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the terminal device 100 , which is different from the position of the display screen 194 .

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The terminal device 100 may receive key input and generate key signal input related to user settings and function control of the terminal device 100 .

The motor 191 can generate a vibrating reminder. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.

The SIM card interface 195 is used for connecting a SIM card. The SIM card can be connected and separated from the terminal device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The terminal device 100 may support one or more SIM card interfaces. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of multiple cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the terminal device 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100 .

The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In this embodiment of the present application, an Android system with a layered architecture is taken as an example to illustrate the software structure of the terminal device 100 .

FIG. 2 is a block diagram of a software structure of a terminal device 100 according to an embodiment of the present application.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces. In some embodiments, the Android system is divided into four layers, which are application layer, application framework layer, Android runtime and system library, and kernel layer from top to bottom.

The application layer can include a series of application packages.

As shown in FIG. 2, the application program package may include application programs such as telephone, email, calendar, and camera.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 2, the application framework layer may include an input system, an activity manager, a location manager, a package manager, a notification manager, a resource manager, a phone manager, and a view system, etc.

The input system is a program used to manage input devices. For example, the input system can determine input actions such as mouse clicks, keyboard input actions, and touch swipes.

The Activity Manager is used to manage the lifecycle of individual applications and navigation fallback functionality. Responsible for the creation of the main thread of Android and the maintenance of the life cycle of each application.

The location manager is used to provide location services for applications, including querying the last known location, registering and unregistering from a periodic location update, etc.

The package manager is used for program management within the system, such as application installation, uninstallation, and upgrade.

The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the download completion, message reminder, etc. The notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is issued, the terminal device vibrates, and the indicator light flashes, etc.

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.

The phone manager is used to manage mobile device functions, including: phone call status, get phone information (device, sim card, network information), monitor phone status and call the phone dialer to make a call

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

In this embodiment of the application, the view system can sequentially extract the target image from the image data to be drawn, and obtain the attribute information of the view control in the target image, and then the view system can determine according to the attribute information of the view control in the target image The image type of the target image is a visually sensitive frame or a non-visually sensitive frame. And, when the view system detects that the image types of the M consecutive target images in the image data are all visually sensitive frames, the view system calls the image drawing module in the system library to draw the images in the image data at the first frame rate , until the view system detects that the image types of the consecutive N frames of target images are all non-vision-sensitive frames; and when the view system detects that the image types of the consecutive N frames of target images in the image data are all non-visual-sensitive frames, the view system Call the image drawing module in the system library to draw the images in the image data at the second frame rate until the view system detects that the image types of the continuous M frames of target images are all visually sensitive frames, and the second frame rate is less than the second frame rate One frame rate.

Android runtime includes core library and virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

The core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

A system library can include multiple function modules. For example: image drawing module, image rendering module, image synthesis module, function library, and input processing library, etc.

The image drawing module is used for drawing two-dimensional or three-dimensional images. The image rendering module is used for rendering of two-dimensional or three-dimensional images. The image synthesis module is used for the synthesis of two-dimensional or three-dimensional images.

In a possible implementation, the application draws the image through the image drawing module, and then the application renders the drawn image through the image rendering module, and then the application sends the rendered image to the cache queue for display synthesis process. Whenever the vsync signal arrives, the display compositing process (for example, surface flinger) sequentially acquires a frame of image to be composited from the buffer queue, and then performs image compositing through the image compositing module.

The function library provides macros, type definitions, string manipulation functions, mathematical calculation functions, and input and output functions used in C language

The input processing library is a library for processing input devices, which can implement mouse, keyboard and touch input processing, etc.

The kernel layer is the layer between hardware and software. The kernel layer includes at least a touch panel driver, an LCD/LED screen driver, a display driver, a Bluetooth driver, a WIFI driver, a keyboard driver, a shared memory driver, and a camera driver.

Hardware can be audio devices, bluetooth devices, camera devices, sensor devices, etc.

The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments may be implemented independently, or may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

Exemplarily, FIG. 3 is a flowchart of a method for adjusting a frame rate provided by an embodiment of the present application. Referring to Figure 3, the frame rate adjustment method may include the following steps:

In step 301, the terminal device acquires a target image from image data to be drawn.

There are multiple applications installed on the terminal device, such as game applications, video applications, or news browsing applications. After the application is started, the application prepares the image data to be drawn. Push to the view system of the terminal device, or the view system obtains pre-cached image data from the cache.

Among them, the image data includes multiple frames of continuous images, and a frame refers to a single picture in the smallest unit of interface display. One frame can be understood as a pair of still pictures, and displaying multiple consecutive frames in rapid succession can form an object movement. illusion.

After the view system of the terminal device receives the image data to be drawn, the view system can sequentially extract target images from the image data. In an optional implementation manner, the view system of the terminal device may use each frame of image in the image data as a target image; in another optional implementation manner, the view system of the terminal device includes the Among the multiple frames of images, one frame of image is extracted every X frames as the target image, and X is a positive integer, for example, X can be a value such as 1, 2, 4 or 5.

As shown in Figure 4, the image data to be drawn includes 23 consecutive frames of images, which are the first frame image, the second frame image to the 23rd frame image, and one frame can be extracted every 4 frames (that is, X=4) The image is used as the target image. At this time, the target images extracted from the 23 frames of images are the 1st frame image, the 6th frame image, the 11th frame image, the 16th frame image and the 21st frame image.

When the value of X is smaller, the number of target images extracted from the image data is more, which makes the subsequent frame rate switching more timely, which makes the display effect of the terminal device better or the power consumption of the terminal device is lower; when When the value of X is larger, the number of target images extracted from the image data is smaller, which reduces the complexity of subsequent calculation of the image type of each target image, thereby reducing the workload of the terminal device when performing calculation operations.

Step 302, the terminal device acquires attribute information of the view control in the target image.

In the embodiment of the present application, each frame of image in the image data is composed of one or more view (view) controls, and these view controls constitute the drawing elements of a frame of image, therefore, each frame of image extracted from the image data The frame target image also includes one or more view controls.

In the embodiment of the present application, the view control includes: TextureView control, SurfaceView control, ImageView control, TextView control, ListView control, RecycleView control, Button (button) control, EditView (input box) control, custom view control, etc. Among them, TextureView control and SurfaceView control can be used to display video content, ImageView control can be used to display picture content, TextView control can be used to display text content, ListView control and RecycleView control can be used to display list content.

Each view control contains its corresponding feature attributes, which include: type information, width, height, coordinate position, background color, transparency, whether it supports click events, whether it has focus, whether it is visible, etc., the view of the terminal device The system can obtain the characteristic properties of each view control through the interface function. For example, the view system obtains the width of the view control through the interface function View.getWidth(), and the view system obtains the height of the view control through the interface function View.getHeight().

In the embodiment of this application, only three important attribute information of each view control (they are type information, position information and area information) can be used to determine whether the image type of the target image is a visually sensitive frame or a non-visually sensitive frame. frame, therefore, the view system of the terminal device can only obtain the type information, width, height and coordinate position of each view control.

The type information refers to the type of content displayed by the view control. For example, the type information of the SurfaceView control refers to the video type, the type information of the ImageView control refers to the image type, and the type information of the TextView control refers to the text type.

The view system of the terminal device can calculate the area information of the view control based on the width of the view control and the height of the view control. The area information refers to the proportion of the view control in the target image, that is, the ratio between the area of the view control and the area of the target image, and the area of the view control is the product of the width and height of the view control.

The view system of the terminal device may calculate the position information of the view control based on the coordinate position of the view control. The location information refers to the distance between the center coordinates of the view control and the center coordinates of the target image.

Taking the upper left corner of the target image as the coordinate origin, the view system can obtain the coordinate position (x ₁ , y ₁ ) of the upper left corner of the view control, the coordinate position (x ₁ , y ₂ ) of the lower left corner of the view control, and the coordinate position of the view control The coordinate position of the upper right corner (x ₂ , y ₁ ) and the coordinate position of the lower right corner of the view control (x ₂ , y ₂ ), based on the coordinate position of the upper left corner (x ₁ , y ₁ ), the coordinate position of the lower left corner (x ₁ , y ₂ ), the coordinate position of the upper right corner (x ₂ , y ₁ ) and the coordinate position of the lower right corner (x ₂ , y ₂ ), calculate the center coordinate (x ₃ , y ₃ ) of the view control, namely x ₃ =(x ₁ +x ₂ )/2, y ₃ =(y ₁ +y ₂ )/2; or, the view system can also obtain the coordinate position (x ₁ , y ₁ ) of the upper left corner of the view control, the coordinate position of the view control Any one of the coordinate position (x ₁ , y ₂ ) of the lower left corner, the coordinate position (x ₂ , y ₁ ) of the upper right corner of the view control, and the coordinate position (x ₂ , y ₂ ) of the lower right corner of the view control, For example, the view system only obtains the coordinate position (x ₁ , y ₁ ) of the upper left corner of the view control, and then determines the center coordinates (x ₃ , y ₃ ) of the view control according to the width d and height h of the view control, that is, x ₃ =x ₁ +d/2, y ₃ =y ₁ +h/2.

After calculating the center coordinates (x ₃ , y ₃ ) of the view control, calculate the distance between the center coordinates (x ₃ , y ₃ ) of the view control and the center coordinates (x ₀ , y ₀ ) of the target image, which can be Euclidean distance D, therefore, the distance D between the center coordinate (x ₃ , y ₃ ) of the view control and the center coordinate (x ₀ , y ₀ ) of the target image is:

Therefore, in this embodiment of the present application, the attribute information of the view control acquired by the view system of the terminal device from the target image may include at least one of the above-mentioned type information, position information, and area information.

In addition, in the multi-frame images included in the image data, the content displayed by the same view control in different images may be different, and the view control with different content displayed in different images is called a view control whose display content changes; In the multiple frames of images included in the image data, the same view control may display the same content in different images, and the view control that displays the same content in different images is called a view control that does not change the display content.

Therefore, the target image may include a view control whose display content changes and a view control whose display content does not change. For the view control whose display content does not change, whether the frame rate of the image is drawn at a high frame rate or a low frame rate depends on this The display content of some view controls will not be affected. For view controls whose display content changes, if the image drawing frame rate is low, the display content of these view controls will be affected. Therefore, the embodiment of the present application may only focus on the view control whose display content changes, that is to say, in step 302, the terminal device may acquire the attribute information of the view control whose display content changes.

Moreover, there is also a content identifier in the feature attribute of each view control, and the content identifier indicates whether the display content of the view control has changed. Therefore, the view system of the terminal device can determine the content of each view control through the content identifier of each view control. The view control is a view control whose display content changes, or a view control whose display content does not change.

When the view control targeted by the embodiment of the present application is a view control whose display content in the image data changes, the number of view controls involved in the calculation can be reduced, and the complexity of subsequent calculation of the image type of each target image is reduced, thereby reducing The workload of an end device performing a computing operation.

As shown in (a) in Figure 5, for the display screen of a certain video application program, the display content in the display screen is "subscribe", "selection", "discovery", "tv series" and "anime". and other view controls belong to the view controls whose display content does not change. Therefore, the view system does not need to obtain the attribute information of these view controls, and the view control 51 corresponding to the recommended video 1 and the view control 52 corresponding to the recommended video 2 belong to the display The view control whose content changes, therefore, the view system can obtain the attribute information of the view control 51 corresponding to the recommended video 1 and the attribute information of the view control 52 corresponding to the recommended video 2 .

Correspondingly, as shown in (b) in FIG. 5 , for a display screen of a certain news browsing application program, the display content in the display screen is "follow", "recommended", "hot list", "video" The view controls such as "novel" and "novel" belong to the view controls whose display content does not change, while the view control 53 corresponding to the advertisement animation picture shown in the upper left corner and the view control 54 corresponding to the video content of the news feed 4 belong to the view control that changes the display content. The view control. In addition, the view control corresponding to news feed 1, the view control corresponding to news feed 2, the view control corresponding to news feed 3, the view control corresponding to the picture content of news feed 3, and the view control corresponding to news feed 4, all belong to the display content A view control that does not change.

It should be noted that the display screens shown in (a) and (b) in Figure 5 are display screens that have been drawn, rendered and displayed, which is mainly to illustrate that there may be display content that does not occur in one frame of image at the same time. A view control that changes, and a view control that displays content that changes. In the actual application process, the attribute information of the view control in the target image is extracted before the target image is drawn.

Step 303, the terminal device determines the image type of the target image according to the attribute information; the image type of the target image is a visually sensitive frame or a non-visually sensitive frame.

In this embodiment of the application, after the view system of the terminal device acquires the attribute information of the view control in the target image, the view system of the terminal device judges whether the image type of the target image is based on the attribute information of the view control in the target image. Visually sensitive frames or non-visually sensitive frames.

Among them, the visually sensitive frame refers to the scene picture that the user is more sensitive to the smoothness of the picture. At this time, if the image is drawn at a lower rendering frame rate, the user's visual experience will be poor; the non-visually sensitive frame refers to the It is a scene picture in which users are less sensitive to picture fluency.

As shown in Figure 6, a score lookup table corresponding to each application is set in advance in the database corresponding to the server, and the server can send the score lookup tables corresponding to multiple application programs to the terminal device, and the terminal device sends the score lookup table corresponding to each application program to the terminal device. The value lookup table is stored locally, and the score lookup table is shown in Table 1 below:

Table I

For the application program A, the specific content of the score lookup table corresponding to the application program A is introduced below. Wherein, the score lookup table corresponding to application A includes the third score value corresponding to the type information, position information and area information of multiple view controls, such as SurfaceView control, ImageView control and TextView control, and their respective type information, position information and area information. The third score value corresponding to the area information.

Assume that application A is a video application. For video applications, the user's attention to the display content is as follows from high to low: video playback content, picture display content, and text display content. Therefore, the type information of the SurfaceView control The corresponding third score value A ₁₁ , the third score value A ₁₂ corresponding to the type information of the ImageView control, and the third score value A ₁₃ corresponding to the type information of the TextView control decrease successively. For example, the third scoring value _A11 corresponding to the type information of the SurfaceView control is set to 10 points, the third scoring value _A12 corresponding to the type information of the ImageView control is set to 5 points, and the third scoring value corresponding to the type information of the TextView control A ₁₃ is set to 2 points.

Moreover, for the same type of view control, when the position information is different, its corresponding third score value is also different. Usually, the position that the user focuses on is located at the center of the display screen. Therefore, when the position information of the view control (actual above refers to the distance between the center coordinates of the view control and the center coordinates of the target image), the smaller the value, the larger the third scoring value corresponding to the position information of the view control is set, and when the position information of the view control is greater , the smaller the third score value corresponding to the position information of the view control is set.

The position information corresponding to each type of view control is divided into different distance intervals in advance, as shown in Table 1, the position information corresponding to each type of view control is divided into 4 distance intervals, that is, the first distance interval [0, D ₁ ), the second distance interval [D ₁ , D ₂ ), the third distance interval [D ₂ , D ₃ ), and the fourth distance interval [D ₃ , D ₄ ]. D ₄ may refer to the maximum distance from the center coordinates of the target image in the target image, for example, D ₁ =D ₄ /4, D ₂ =D ₄ /2, D ₃ =3D ₄ /4.

When the position information corresponding to the SurfaceView control is within the first distance interval [0, D ₁ ), that is, 0≤D<D ₁ , the third score value corresponding to the position information of the SurfaceView control at this time is A ₂₁ ; when the SurfaceView control When the corresponding position information is within the second distance interval [D ₁ , D ₂ ), that is, D ₁ ≤ D<D ₂ , the third score value corresponding to the position information of the SurfaceView control at this time is A ₃₁ ; when the SurfaceView control corresponds to When the position information of the SurfaceView control is within the third distance interval [D ₂ , D ₃ ), that is, D ₂ ≤ D<D ₃ , the third score value corresponding to the position information of the SurfaceView control at this time is A ₄₁ ; when the SurfaceView control corresponds to When the position information is within the fourth distance interval [D ₃ , D ₄ ], that is, D ₃ ≤ D ≤ _{D 4} , the third score value corresponding to the position information of the SurfaceView control at this time is A ₅₁ . Therefore, for the SurfaceView control, the third scoring value A ₂₁ corresponding to the first distance interval [0, D ₁ ), the third scoring value A 31 corresponding to the second distance interval [D ₁ , D ₂ ), the third scoring value A ₃₁ corresponding to the third distance interval [ The third scoring value A ₄₁ corresponding to D ₂ , D ₃ ) and the third scoring value A ₅₁ corresponding to the fourth distance interval [D ₃ , D ₄ ] decrease successively.

For example, for the SurfaceView control, when its corresponding position information is within the first distance interval [0, D ₁ ), the third scoring value _A21 can be set to 10 points; when its corresponding position information is within the second distance interval When it is within [D ₁ , D ₂ ), the third score value A ₃₁ can be set to 7 points, and when its corresponding position information is within the third distance interval [D ₂ , D ₃ ), the third score value A 31 can be set to A ₄₁ is set to 4 points, and when its corresponding position information is within the fourth distance interval [D ₃ , D ₄ ], the third score value A ₅₁ may be set to 1 point.

Correspondingly, for the ImageView control, the third score value A ₂₂ corresponding to the first distance interval [0, D ₁ ), the third score value A ₃₂ corresponding to the second distance interval [D ₁ , D ₂ ), and the third distance interval The third scoring value A ₄₂ corresponding to [D ₂ , D ₃ ] and the third scoring value A ₅₂ corresponding to the fourth distance interval [D ₃ , D ₄ ] also decrease sequentially. For the TextView control, the third scoring value A ₂₃ corresponding to the first distance interval [0, D ₁ ), the third scoring value A 33 corresponding to the second distance interval [D ₁ , D ₂ ), the third scoring value A ₃₃ corresponding to the third distance interval [D ₂ , D ₃ ) corresponding to the third scoring value A ₄₃ and the third scoring value A ₅₃ corresponding to the fourth distance interval [D ₃ , D ₄ ] also decrease sequentially.

In addition, for the same type of view control, when the area information is different, the corresponding third score value is also different. Usually, what the user focuses on is the content displayed in a window with a larger area. Therefore, when the area information of the view control is smaller, the third score value corresponding to the area information of the view control is set to be smaller. When the area information of the view control is larger, the third score value corresponding to the area information of the view control is set to be larger.

The area information corresponding to each type of view control is divided into different area intervals in advance, as shown in Table 1, the area information corresponding to each type of view control is divided into 4 area intervals, that is, the first area interval (0, S ₁ ), the second area interval [S ₁ , S ₂ ), the third area interval [S ₂ , S ₃ ] and the fourth area interval [S ₃ , S ₄ ]. Exemplarily, S ₁ may be 25%, S ₂ may be 50%, S ₃ may be 75%, and S ₄ may be 100%.

When the area information corresponding to the SurfaceView control is within the first area interval (0, S ₁ ), that is, 0<S<S ₁ , the third scoring value corresponding to the area information of the SurfaceView control at this time is A ₆₁ ; when the SurfaceView control When the corresponding area information is within the second area interval [S ₁ , S ₂ ), that is, S ₁ ≤ S<S ₂ , the third scoring value corresponding to the area information of the SurfaceView control at this time is A ₇₁ ; when the SurfaceView control corresponds to When the area information of the SurfaceView control is within the third area interval [S ₂ , S ₃ ), that is, S ₂ ≤ S<S ₃ , the third scoring value corresponding to the area information of the SurfaceView control at this time is A ₈₁ ; when the SurfaceView control corresponds to When the area information is within the fourth area interval [S ₃ , S ₄ ], that is, S ₃ ≤ S ≤ _{S 4} , the third score value corresponding to the area information of the SurfaceView control at this time is A ₉₁ . Therefore, for the SurfaceView control, the third scoring value A ₆₁ corresponding to the first area interval (0, S ₁ ), the third scoring value A ₇₁ corresponding to the second area interval [S ₁ , S ₂ ], the third area interval [ The third scoring value A ₈₁ corresponding to S ₂ , S ₃ ) and the third scoring value A ₉₁ corresponding to the fourth area interval [S ₃ , S ₄ ] increase sequentially.

For example, for the SurfaceView control, when its corresponding area information is within the first area interval (0, S ₁ ), the third scoring value A ₆₁ can be set as 1 point; when its corresponding area information is within the second area interval When it is within [S ₁ , S ₂ ), the third scoring value A ₇₁ can be set to 4 points, and when its corresponding area information is within the third area interval [S ₂ , S ₃ ), the third scoring value A 71 can be set to A ₈₁ is set to 7 points, and when its corresponding area information is within the fourth area interval [S ₃ , S ₄ ], the third scoring value A ₉₁ can be set to 10 points.

Correspondingly, for the ImageView control, the third score value A ₆₂ corresponding to the first area interval (0, S ₁ ), the third score value A ₇₂ corresponding to the second area interval [S ₁ , S ₂ ], the third area interval The third scoring value A ₈₂ corresponding to [S ₂ , S ₃ ) and the third scoring value A ₉₂ corresponding to the fourth area interval [S ₃ , S ₄ ] also increase sequentially. For the TextView control, the third score value A ₆₃ corresponding to the first area interval (0, S ₁ ), the third score value A ₇₃ corresponding to the second area interval [S ₁ , S ₂ ], the third area interval [S ₂ , S ₃ ) corresponding to the third scoring value A ₈₃ and the third scoring value A ₉₃ corresponding to the fourth area interval [S ₃ , S ₄ ] also increase sequentially.

It should be noted that the score lookup table stored in the server includes score lookup tables corresponding to multiple applications. The specific content of the lookup table is similar to that of the score lookup table corresponding to the application program A, and will not be repeated here. Moreover, the view controls corresponding to each application are not limited to the SurfaceView controls, ImageView controls, and TextView controls shown in Table 1, and may also include other view controls; in addition, the division of distance intervals and area intervals is not limited to the above-mentioned way of division.

As shown in Figure 6, in the database corresponding to the server, a weight query table corresponding to each application is also set in advance, the server can send the weight query tables corresponding to multiple applications to the terminal device, and the terminal device queries the weight The table is stored locally, and the weight query table is shown in Table 2 below:

应用名称Application Name	类型信息type information	位置信息location information	面积信息area information
应用程序AApplication A	W ₁₁ W ₁₁	W ₁₂ W ₁₂	W ₁₃ W ₁₃

application B

W ₂₁

W ₂₂

W ₂₃

Table II

In the actual application process, the weight values corresponding to the attribute information of different application programs can be set for different application programs in combination with the functions of different application programs and the usage scenarios of users, so as to generate weight lookup tables corresponding to different application programs. It can be understood that, due to different functions of different applications, the weight values of the same attribute information in weight lookup tables of different applications may be different.

Assume that application A is a video application. For video applications, the display content that users focus on is generally the video playback content with a large area. Therefore, the type information and area information have a greater impact on the score value of the target image. The weight value corresponding to the type information and the weight value corresponding to the area information may be set larger, and the weight value corresponding to the position information may be set smaller. For example, the weight value W ₁₁ corresponding to the type information of application A can be set to 0.4, the weight value W ₁₂ corresponding to the location information of application A can be set to 0.2, and the weight value W ₁₃ corresponding to the area information of application A can be set to 0.4.

Assume that application B is a news browsing application. For news browsing applications, the display content that users focus on is generally located in the center of the display screen. Therefore, the location information has a greater impact on the score value of the target image. The location information can be set to The weight value corresponding to the information is set larger, and the weight value corresponding to the type information and the weight value corresponding to the area information are set smaller. For example, the weight value W ₂₁ corresponding to the type information of application B can be set to 0.2, the weight value W ₂₂ corresponding to the location information of application B can be set to 0.6, and the weight value W ₂₃ corresponding to the area information of application B can be set to 0.2.

It should be noted that, for the same application, the weight values corresponding to the same attribute information of different view controls can be set to be equal or unequal; in addition, each third score value set in Table 1, and Each weight value set can be set according to experience value.

In this embodiment of the application, after the view system of the terminal device obtains the attribute information of the view control in the target image, it first determines the target application program to which the target image belongs, that is, determines which application program the target image belongs to when it is displayed. The interface is displayed, and then, the view system of the terminal device searches the target score lookup table corresponding to the target application program from the score lookup tables corresponding to each application program stored locally.

As shown in Figure 6, after the view system of the terminal device finds the target score lookup table, the view system of the terminal device executes step 601 to look up the attribute information of each view control in the target image from the target score lookup table Corresponding to the third score value, the attribute information includes at least two of type information, location information and area information.

For example, if the target application to which the target image belongs is application A, then look up the score lookup table corresponding to application A, if the target image includes 1 SurfaceView control and 1 TextView control, and the position information of the SurfaceView control is located at the first distance In the interval [0, D ₁ ), the area information of the SurfaceView control is located in the third area interval [S2, S3), the position information of the TextView control is located in the fourth distance interval [D ₃ , D ₄ ], the area information of the TextView control Located in the first area interval (0, S1), then obtain the third score value A ₁₁ corresponding to the type information of the SurfaceView control, the third score value A _{21 corresponding to the position information of the SurfaceView control, and the third score value A 21} corresponding to the area information of the SurfaceView control The third scoring value A ₈₁ , the third scoring value A ₁₃ corresponding to the type information of the TextView control, the third scoring value A ₅₃ corresponding to the position information of the TextView control, and the third scoring value A ₆₃ corresponding to the area information of the TextView control.

When the attribute information includes at least two of type information, location information, and area information, it is necessary to obtain the weight value corresponding to each attribute information of the view control. Specifically, the view system of the terminal device may first determine the target application program to which the target image belongs, and then, the view system of the terminal device searches the target weight lookup table corresponding to the target application program from the weight lookup table corresponding to each application program stored locally. .

As shown in FIG. 6 , after the view system of the terminal device finds the target weight lookup table, the view system of the terminal device performs step 602 to look up the weight value corresponding to the attribute information of each view control from the target weight lookup table.

For example, the target application to which the target image belongs is application A, and the target image includes a SurfaceView control and a TextView control, then the weight value corresponding to the type information of the SurfaceView control and the TextView control is W ₁₁ , and the SurfaceView control and the TextView control The weight value corresponding to the position information of the SurfaceView control and the TextView control _{is W 13} _.

Next, the view system of the terminal device executes step 603, performing weighted summation according to the corresponding weight value for the third score value corresponding to the attribute information of each view control, to obtain the first score value of each view control.

For example, for the target image corresponding to application A, the third score value corresponding to the type information of the SurfaceView control included in it is A ₁₁ , the corresponding weight value is W ₁₁ , and the third score value corresponding to the position information of the SurfaceView control is A ₂₁ , its corresponding weight value is W ₁₂ , the third scoring value corresponding to the area information of the SurfaceView control is A ₈₁ , and its corresponding weighting value is W ₁₃ , therefore, the first scoring value F ₁ =A corresponding to the SurfaceView control ₁₁ ×W ₁₁ +A ₂₁ ×W ₁₂ +A ₈₁ ×W ₁₃ . Correspondingly, for the target image corresponding to application A, the third score value corresponding to the type information of the TextView control included in it is A ₁₃ , the corresponding weight value is W ₁₁ , and the third score value corresponding to the position information of the TextView control is is A ₅₃ , its corresponding weight value is W ₁₂ , the third scoring value corresponding to the area information of the TextView control is A ₆₃ , and its corresponding weighting value is W ₁₃ , therefore, the first scoring value F ₂ corresponding to the TextView control = A ₁₃ ×W ₁₁ +A ₅₃ ×W ₁₂ +A ₆₃ ×W ₁₃ .

After calculating the first score value of each view control in the target image, when the target image includes multiple view controls, the view system of the terminal device may execute step 604 to calculate the average of the first score values of each view control value to get the second score value of the target image.

For example, the target image includes a SurfaceView control and a TextView control, and the first score value of the SurfaceView control is F ₁ , and the corresponding first score value of the TextView control is F ₂ , therefore, the second score value of the target image is F ₀ =(F ₁ +F ₂ )/2.

Then, the view system of the terminal device executes step 605, comparing the calculated second score value with a preset threshold, and judging whether the second score value is greater than the preset threshold. When the second score value is greater than the preset threshold, the view system of the terminal device performs step 606 to determine that the image type of the target image is a visually sensitive frame; when the second score value is less than or equal to the preset threshold, the view system of the terminal device Step 607 is executed to determine that the image type of the target image is a non-visually sensitive frame. The preset threshold can be set according to empirical values, for example, the preset threshold can be set to a value such as 5 or 6.

Therefore, based on the method shown in FIG. 6, the first score value of each view control can be determined according to the attribute information of each view control in the target image, and then the view system of the terminal device can determine the first score value of each view control according to the first score value of each view control. A score value, calculating a second score value of the target image, and according to the second score value, determining the image type of the target image as a visually sensitive frame or a non-visually sensitive frame.

Of course, in some embodiments, the attribute information of each view control is any one of type information, position information and area information. A score value, after step 601 is executed, the view system of the terminal device may determine the third score value corresponding to the attribute information of each view control in the target image as the first score value of each view control.

Moreover, in some other embodiments, the target image may also include a view control. Therefore, after step 603 is executed, the view system of the terminal device may determine the first score value of the view control as the second score of the target image value.

Taking the target image shown in (a) in FIG. 5 as an example, it includes two view controls whose display content changes, which are respectively the view control 51 corresponding to the recommended video 1 and the view control 52 corresponding to the recommended video 2 . The view control 51 is a SurfaceView control, the third score value corresponding to its type information is A ₁₁ , the distance between the center coordinates of the view control 51 and the center coordinates of the target image is relatively low, and its corresponding position information is located in the first distance interval [0, D ₁ ), the third score value corresponding to its position information is A ₂₁ , the area information of the view control 51 is 40%, which is located in the second area interval [S ₁ , S ₂ ), and its area information The corresponding third score value is A ₇₁ , and the weight values corresponding to the type information, location information and area information of the view control 51 in sequence are W ₁₁ , W ₁₂ and W ₁₃ , therefore, the first score value F of the view control 51 is ₁ = A ₁₁ ×W ₁₁ +A ₂₁ ×W ₁₂ +A ₇₁ ×W ₁₃ . Correspondingly, the view control 52 is a SurfaceView control, the third score value corresponding to its type information is A ₁₁ , the position information of the view control 52 is located in the second distance interval [D ₁ , D ₂ ), and the third score value corresponding to its position information is The score value is A ₃₁ , the area information of the view control 52 is 40%, which is located in the second area interval [S ₁ , S ₂ ), and the third score value corresponding to the area information is A ₇₁ , and the view control 52’s The weight values corresponding to the type information, position information and area information in turn are W ₁₁ , W ₁₂ and W ₁₃ , therefore, the first score value F ₂ of the view control 52 = A ₁₁ ×W ₁₁ +A ₃₁ ×W ₁₂ +A ₇₁ ×W ₁₃ . Therefore, the second score value F ₀ =(F ₁ +F ₂ )/2 of the target image.

For example, if A ₁₁ is 10 points, A ₂₁ is 10 points, A ₇₁ is 4 points, A ₃₁ is 7 points, W ₁₁ is 0.4, W ₁₂ is 0.2, W ₁₃ is 0.4, therefore, the first view control 51 Score value F ₁ =10×0.4+10×0.2+4×0.4=7.6 points, the first score value F ₂ of view control 52 =10×0.4+7×0.2+4×0.4=7 points, therefore, Fig. 5 The second scoring value F ₀ of the target image shown in (a) in (a) = (7+7.6)/2=7.3 points. Assuming that the preset threshold value is 5 points, then the second scoring value of the target image shown in (a) in Fig. 5 is greater than the preset threshold value, then it is determined that the image type of the target image shown in (a) in Fig. 5 is visual sensitive frame.

Correspondingly, taking the target image shown in (b) in Figure 5 as an example, it includes two view controls whose display content changes, which are respectively the view control 53 corresponding to the advertisement animation picture shown in the upper left corner and the news summary The view control 54 corresponding to the video content of 4. The view control 53 is an ImageView control, the third score value corresponding to its type information is B ₁₂ , the distance between the center coordinates of the view control 53 and the center coordinates of the target image is far, and its corresponding position information is located in the fourth distance interval In [D ₃ , D ₄ ], the third scoring value corresponding to its position information is B ₅₂ , and the area information of the view control 53 is 1%, which is located in the first area interval (0, S ₁ ), and its area information The corresponding third score value is B ₆₂ , and the weight values corresponding to the type information, location information and area information of the view control 53 in turn are W ₂₁ , W ₂₂ and W ₂₃ , therefore, the first score value F of the view control 52 is ₁ = B ₁₂ ×W ₂₁ +B ₅₂ ×W ₂₂ +B ₆₂ ×W ₂₃ . Correspondingly, the view control 54 is a SurfaceView control, the third score value corresponding to its type information is B ₁₁ , the position information of the view control 54 is located in the second distance interval [D ₁ , D ₂ ), and the third score value corresponding to its position information is The score value is B ₃₁ , the area information of the view control 54 is 30%, which is located in the second area interval [S ₁ , S ₂ ), and the third score value corresponding to the area information is B ₇₁ , and the view control 54’s The weight values corresponding to type information, position information and area information in turn are W ₂₁ , W ₂₂ and W ₂₃ , therefore, the first score value F ₂ of view control 54 = B ₁₁ ×W ₂₁ +B ₃₁ ×W ₂₂ +B ₇₁ ×W ₂₃ . Therefore, the second score value F ₀ of the target image =(F ₁ +F ₂ )/2.

For example, if B ₁₂ is 5 points, B ₅₂ is 1 point, B ₆₂ is 1 point, B ₁₁ is 2 points, B ₃₁ is 7 points, B ₇₁ is 4 points, W ₂₁ is 0.2, W ₂₂ is 0.6, W ₂₃ is 0.2, therefore, the first scoring value F ₁ of the view control 53 = 5×0.2+1×0.6+1×0.2=1.8 points, and the first scoring value F ₂ of the view control 54 =2×0.2+7×0.6 +4×0.2=5.4 points, therefore, the second score value F ₀ of the target image shown in (b) in FIG. 5 =(1.8+5.4)/2=3.6 points. Assuming that the preset threshold is 5 points, then the second scoring value of the target image shown in (b) in Figure 5 is less than the preset threshold, then determine that the image type of the target image shown in (b) in Figure 5 is non- Visually sensitive frames.

In the embodiment of the present application, according to the above-mentioned calculation method, it is determined whether the image type of each frame of the target image in FIG. 4 is a visually sensitive frame or a non-visually sensitive frame. The target images in Figure 4 are the first frame image, the sixth frame image, the eleventh frame image, the sixteenth frame image and the twenty-first frame image, then according to the above method, by obtaining the first frame image, the sixth frame image , the attribute information of the 11th frame image, the 16th frame image and the 21st frame image, it can be determined that the image types of the 1st frame image and the 6th frame image are visually sensitive frames, while the 11th frame image, the 16th frame image and the 21st frame image The image types of the 21 frames of images are all non-visually sensitive frames.

Step 304, when it is detected that the image types of the consecutive N frames of target images in the image data are all non-visually sensitive frames, the terminal device switches from the first frame rate to the second frame rate, and performs image drawing at the second frame rate; The second frame rate is less than the first frame rate.

In the embodiment of this application, after the terminal device performs image rendering at the first frame rate, when the view system of the terminal device detects that the image types of the target images in the image data are all non-visually sensitive frames, it determines this The image in the real-time image data is a scene picture with low user attention, therefore, the terminal device may perform image drawing at a lower second frame rate, so as to reduce power consumption of the terminal device. Wherein, N is a positive integer, for example, N can be 1, 2, 3 and other numerical values.

As shown in Figure 4, assuming that N is 2, when it is detected that the image types of the 11th frame image and the 16th frame image are both non-visual sensitive frames, it is determined that the image types of the detected 2 consecutive frames of target images are all non-visual Sensitive frames, at this time, image drawing is performed at the second frame rate until the image types of the consecutive M frames of target images are detected to be visually sensitive frames, so the terminal device draws the 16th to the 16th frame images at the second frame rate 23 frames of images.

In fact, the frame rate adjustment of the terminal device can be realized through the view system and the image drawing module of the terminal device. When the view system of the terminal device detects that the image types of the target images in consecutive N frames in the image data are all non-visually sensitive frames, The view system of the terminal device extracts one frame of image at intervals of R frame images for the Nth non-visually sensitive frame and each frame of image thereafter, and passes the extracted image to the image drawing module through the interface, while the image not extracted by the view system Discarded, not passed to the image drawing module. Therefore, the image drawing module draws a frame of image at intervals of R frame images for the Nth non-visual sensitive frame and each subsequent frame image, until it detects that there are consecutive M frames of target images in the image data. sensitive frame. Wherein, R is a positive integer, and R can be 1, 2, 3 and other values.

For example, R is equal to 1, that is, one frame of image is drawn at intervals of one frame, assuming that the maximum drawing frame rate supported by the terminal device is 120fps, the second frame rate of the terminal device is 60fps. At this time, as shown in Figure 4, the view system transfers the 16th frame image, the 18th frame image, the 20th frame image, and the 22nd frame image to the image drawing module, and the image drawing module performs the 16th frame image, the 18th frame image image, the 20th frame image and the 22nd frame image are drawn, and the view system discards the 17th frame image, the 19th frame image, the 21st frame image and the 23rd frame image, and does not pass them to the image drawing module. Alternatively, R is equal to 3, that is, drawing a frame of image every 3 frames, then the second frame rate of the terminal device is 30fps. The embodiment of the present application does not specifically limit the value of R.

Step 305, when it is detected that the image types of the M consecutive target images in the image data are all visually sensitive frames, switch from the second frame rate to the first frame rate, and perform image rendering at the first frame rate.

In this embodiment of the application, after the terminal device performs image rendering at the second frame rate, when the view system of the terminal device detects that the image types of the M consecutive target images in the image data are all visually sensitive frames, it is determined that The image in the image data is a scene with high attention of the user. Therefore, the terminal device needs to draw the image at a higher first frame rate, so that the image picture after drawing, rendering and display is smoother, and the user's vision is improved. experience. Wherein, M is a positive integer, for example, M can be 1, 2, 3 and other numerical values.

As shown in Figure 4, assuming that M is 2, when the image types of the first frame image and the sixth frame image are successively detected as visually sensitive frames, it is determined that the image types of the detected two consecutive frames of target images are all visually sensitive frames, At this time, the image is drawn at the first frame rate until it is detected that the image types of the consecutive N frames of the target image are all non-visually sensitive frames, because the image types of the 11th frame image and the 16th frame image are all non-visually sensitive frames, therefore, the terminal device draws images from the sixth frame to the fifteenth frame at the first frame rate.

In fact, when the view system of the terminal device detects that the image types of the consecutive M frames of target images in the image data are all visually sensitive frames, the view system of the terminal device uses the interface to convert the Mth frame of visually sensitive frames and each The frame images are all passed to the image drawing module, and then the image drawing module draws the M-th visually sensitive frame and each frame image thereafter until it is detected that the image types of continuous N frames of target images in the image data are all non- Visually sensitive frames so far.

The image drawing performed by the terminal device at the first frame rate actually refers to: the drawing frame rate of the image drawing module is the first frame rate, and the drawing frame rate is the number of images drawn per unit time (eg, 1 second). When the drawing frame rate is higher, a smoother and more realistic picture can be obtained.

If the maximum rendering frame rate supported by the terminal device is 120fps, when the image rendering module renders the M-th visually sensitive frame and every subsequent image frame, the first frame rate is 120fps. At this time, as shown in Figure 4, the view system transfers each frame of images from the sixth frame to the fifteenth frame to the image drawing module, and the image drawing module performs the processing of the sixth to fifteenth frame images Each frame of the image is drawn.

Of course, it can be understood that when the image types of the target images in consecutive M frames are detected to be visually sensitive frames, the view system of the terminal device can also discard 1 image frame every K frames, and pass the remaining images to the image The drawing module performs image drawing. K is a positive integer. For example, K is equal to 4, that is, one frame of image is discarded every 4 frames. Assuming that the maximum drawing frame rate supported by the terminal device is 120fps, the first frame rate of the terminal device is 96fps.

In addition, the embodiment of the present application is mainly aimed at the image data that needs to be drawn, rendered and displayed in the process of using the application program. After the user triggers the touch operation on the application program icon, until the detection of consecutive M frames of visually sensitive frames or The image to be drawn before N frames of non-visually sensitive frames may be drawn at the first frame rate, or at the second frame rate, or at other frame rates, which are not discussed in the embodiments of this application. limited.

For example, as shown in Figure 4, M is 2, and when the image types of the image to be drawn in the first frame and the image to be drawn in the sixth frame are both visually sensitive frames, the terminal device can be used for the images in the first to fifth frames The image is drawn at the first frame rate, that is, the view system transfers the first frame image value and the fifth frame image to the image drawing module for image drawing. The first frame image can be understood as what needs to be displayed after the user clicks the application icon Frame 1 image.

It should be noted that when the view system of the terminal device acquires the target image from the image data and calculates the image type of the target image, the image drawing module of the terminal device also draws the frame of the target image according to the corresponding frame rate The previous image frame to draw on. That is to say, when the view system of the terminal device calculates the image type of the target image, the image drawing module will still perform the image drawing operation instead of waiting for the view system to calculate the image type of the target image before the image drawing module starts to perform image drawing operate.

For example, assuming that the target image is the 6th frame image in the image data, when the view system calculates the image type for the 6th frame image, the image drawing module will use the image frame before the 6th frame image (such as the 3rd frame image or the 4th frame image) frame image, etc.) to perform image drawing operations.

It is worth noting that after the terminal device draws images at the first frame rate, if it detects that the number of consecutive non-visually sensitive frames is less than N, it does not need to switch the first frame rate to the second frame rate. Perform image drawing at the first frame rate, and switch the first frame rate to the second frame rate until consecutive N frames of non-visually sensitive frames are detected, and use the second frame rate for the Nth frame of non-visually sensitive frames and subsequent frames The image is drawn until it is detected that the image types of the continuous M frames of the target image are visually sensitive frames; correspondingly, after the terminal device draws the image at the second frame rate, if the number of consecutive visually sensitive frames detected is When it is less than M, it does not need to switch the second frame rate to the first frame rate, and still draws images at the second frame rate, and does not switch the second frame rate to the first frame rate until consecutive M frames of visually sensitive frames are detected. One frame rate, at the first frame rate, image rendering is performed on the Mth visually sensitive frame and subsequent images until it is detected that the image types of the consecutive N frames of target images are all non-visually sensitive frames.

And, when a non-visually sensitive frame is detected, the count value of the visually sensitive frame is cleared to 0 until the visually sensitive value is detected again; when a visually sensitive frame is detected, the count value of the non-visually sensitive frame is cleared to 0 until The counting is restarted when a non-visually sensitive frame is detected again.

Wherein, M and N may be equal, for example, M and N are both equal to 2; or M and N may not be equal, for example, M is 2, and N is 3.

It should be noted that, in the embodiment of the present application, the process of determining whether the image type of the target image is a visually sensitive frame or a non-visually sensitive frame is performed before drawing the target image. After the image drawing module draws the image in the image data, it needs to pass the drawn image to the image rendering module. After the image rendering module renders, it passes the rendered image to the LCD/LED screen driver at the kernel layer, so that The LED screen driver drives the display to display the rendered image.

When the drawing frame rate of the image drawing module is the first frame rate, the rendering frame rate of the image rendering module and the display frame rate of the display screen can also be the first frame rate; when the drawing frame rate of the image drawing module is the second frame rate When the frame rate is set, the rendering frame rate of the image rendering module and the display frame rate of the display screen may both be the second frame rate.

FIG. 7 is a schematic diagram of a hardware structure of a terminal device provided by an embodiment of the present application. The terminal device 700 shown in FIG. 7 includes: a memory 701, a processor 702, and a communication interface 703, wherein the memory 701, the processor 702, and the communication interface 703 can communicate; for example, the memory 701, the processor 702, and the communication interface 703 Communication is possible via a communication bus.

The memory 701 may be a read only memory (read only memory, ROM), a static storage device, a dynamic storage device or a random access memory (random access memory, RAM). The memory 701 may store a computer program, which is controlled and executed by the processor 702 and communicated by the communication interface 703, so as to realize the frame rate adjustment method provided by the above-mentioned embodiments of the present application.

Processor 702 may be a general-purpose central processing unit (central processing unit, CPU), microprocessor, application specific integrated circuit (application specific integrated circuit, ASIC), GPU, or one or more integrated circuits.

The processor 702 may also be an integrated circuit chip, which has a signal processing capability. During implementation, the functions of the frame rate adjustment method of the present application may be implemented by an integrated logic circuit of hardware in the processor 702 or instructions in the form of software. The above-mentioned processor 702 can also be a general-purpose processor, DSP, an application-specific integrated circuit (ASIC), a ready-made programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware The components can implement or execute the methods, steps and logic block diagrams disclosed in the following embodiments of the present application. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the methods disclosed in the following embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory 701, and the processor 702 reads the information in the memory 701, and combines its hardware to complete the function of the frame rate adjustment method in the embodiment of the present application.

The communication interface 703 in the chip may be an input/output interface, a pin or a circuit, and the like.

The terminal device 700 in this embodiment can be correspondingly used to execute the steps executed in the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.

The embodiment of the present application also provides a computer-readable storage medium. The methods described in the foregoing embodiments may be fully or partially implemented by software, hardware, firmware or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media may include computer storage media and communication media, and may include any medium that can transfer a computer program from one place to another. A storage media may be any target media that can be accessed by a computer.

In a possible implementation, the computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage or other magnetic storage devices, or target or any other medium that stores the required program code in the form of instructions or data structures and can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial Cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes compact disc, laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. A computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line) or wireless (eg, infrared, wireless, microwave, etc.) to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server, a data center, etc. integrated with one or more available media. The usable medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), etc.

FIG. 8 is a schematic structural diagram of a frame rate adjustment system provided by an embodiment of the present application. Referring to FIG. 8 , an embodiment of the present application also provides a frame rate adjustment system, which includes the above-mentioned terminal device 700 And the server 800 that is communicatively connected to the terminal device 700 stores a score lookup table and a weight lookup table corresponding to each application program in the server 800 .

When the data in the score lookup table stored in the server 800 changes, such as the third score value corresponding to a certain attribute information of the view control changes, and when the third score value corresponding to some attribute information is added or deleted, the server 800 The changed score lookup table is sent to the terminal device 700 , and the terminal device 700 updates the locally previously stored score lookup table according to the score lookup table sent by the server 800 .

When there is a change in the data in the weight lookup table stored in the server 800, such as the weight value corresponding to the attribute information changes, the server 800 sends the changed weight lookup table to the terminal device 700, and the terminal device 700 according to the weight value sent by the server 800 The lookup table updates the previously stored weight lookup table locally.

For the specific execution process of the terminal device 700, reference may be made to the foregoing description, and details are not repeated here to avoid repetition.

Embodiments of the present application are described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processing unit of other programmable data processing equipment to produce a machine such that the instructions executed by the processing unit of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

The specific implementation above has further described the purpose, technical solutions and beneficial effects of the application in detail. It should be understood that the above is only a specific implementation of the application and is not used to limit the scope of protection of the application. On the basis of the technical solution of this application, any modification, equivalent replacement, improvement, etc. should be included in the protection scope of this application.

Claims

A frame rate adjustment method, characterized in that it is applied to a terminal device, the method comprising:

The terminal device obtains the target image from the image data to be drawn;

The terminal device acquires attribute information of the view control in the target image;

The terminal device determines the image type of the target image according to the attribute information;

When it is detected that the image types of the target images in the continuous N frames of the image data are all non-visually sensitive frames, the terminal device switches from the first frame rate to the second frame rate, and the second frame rate Perform image rendering; the N is a positive integer, and the second frame rate is lower than the first frame rate.
The method according to claim 1, wherein the terminal device determines the image type of the target image according to the attribute information, comprising:

The terminal device determines the first score value of each view control according to the attribute information of each view control in the target image;

The terminal device calculates a second score value of the target image according to the first score value of each view control;

The terminal device determines the image type of the target image according to the second scoring value.
The method according to claim 2, wherein the terminal device determines the first score value of each view control according to the attribute information of each view control in the target image, including:

The terminal device searches a target score lookup table for a third score value corresponding to the attribute information of each of the view controls in the target image;

The terminal device calculates the first score value of each view control according to the third score value.
The method according to claim 3, wherein the attribute information is any one of type information, location information, and area information; and the terminal device calculates each of the The first score value of the view control, including:

The terminal device determines the third score value corresponding to the attribute information of each view control as the first score value of each view control.
The method according to claim 3, wherein the attribute information includes at least two of type information, location information, and area information; and the terminal device calculates each of the The first score value of the view control, including:

The terminal device searches the weight value corresponding to the attribute information of each view control from the target weight lookup table;

The terminal device performs weighted summation on the third scoring value corresponding to the attribute information of each of the view controls according to the corresponding weight value, to obtain the first scoring value of each of the viewing controls.
The method according to claim 3, wherein, before the terminal device looks up the third score value corresponding to the attribute information of each of the view controls in the target image from the target score lookup table, Also includes:

The terminal device determines the target application program to which the target image belongs;

The terminal device searches the target score lookup table corresponding to the target application program from preset score lookup tables corresponding to each application program.
The method according to claim 5, wherein, before the terminal device searches the weight value corresponding to the attribute information of each of the view controls from the target weight lookup table, it also includes:

The terminal device determines the target application program to which the target image belongs;

The terminal device searches the target weight lookup table corresponding to the target application program from preset weight lookup tables corresponding to each application program.
The method according to claim 2, wherein the terminal device calculates the second score value of the target image according to the first score value of each of the view controls, including:

The terminal device calculates an average value of the first score values of each of the view controls to obtain the second score value of the target image.
The method according to claim 2, wherein the terminal device determines the image type of the target image according to the second scoring value, comprising:

The terminal device compares the second scoring value with a preset threshold;

When the second scoring value is greater than the preset threshold, determine that the image type of the target image is a visually sensitive frame;

When the second scoring value is less than or equal to the preset threshold, it is determined that the image type of the target image is a non-visually sensitive frame.
The method according to any one of claims 1 to 9, characterized in that, after the terminal device switches from the first frame rate to the second frame rate and performs image drawing at the second frame rate, further comprising :

When it is detected that the image types of the target images in the M consecutive frames in the image data are all visually sensitive frames, the terminal device switches from the second frame rate to the first frame rate, and uses the second frame rate Image rendering is performed at a frame rate; the M is a positive integer.
The method according to claim 1, wherein when it is detected that the image types of the target images in the N consecutive frames in the image data are all non-visually sensitive frames, the terminal device starts from the first frame The frame rate is switched to the second frame rate, and the image drawing is performed at the second frame rate, including:

When it is detected that the image types of the target images in the N consecutive frames in the image data are all non-visually sensitive frames, the terminal device performs an operation on the Nth frame of the non-visually sensitive frame and each subsequent frame of image Draw a frame of images at intervals of R frames of images until it is detected that there are consecutive M frames in the image data. The image types of the target images are all visually sensitive frames; the R is a positive integer.
The method according to claim 10, wherein when it is detected that the image types of the target images in the M consecutive frames in the image data are all visually sensitive frames, the terminal device starts from the second The frame rate is switched to the first frame rate, and image drawing is performed at the first frame rate, including:

When it is detected that the image types of the target image in consecutive M frames in the image data are all visually sensitive frames, the terminal device draws the visually sensitive frame of the Mth frame and each frame of image thereafter, Until it is detected that there are N consecutive frames in the image data, the image types of the target image are all non-visually sensitive frames.
The method according to any one of claims 1 to 9, wherein the view control is a view control whose display content changes in the image data.
The method according to any one of claims 1 to 9, wherein the terminal device obtains the target image from the image data to be drawn, comprising:

The terminal device extracts one frame of image every X frames from the multiple frames of images included in the image data as the target image; the X is a positive integer.
A terminal device, characterized in that it includes a memory and a processor, the memory is used to store a computer program, and the processor is used to call the computer program to execute the method described in any one of claims 1 to 14 Frame rate adjustment method.
A computer-readable storage medium, characterized in that, computer programs or instructions are stored in the computer-readable storage medium, and when the computer programs or instructions are executed, the computer program or instructions described in any one of claims 1 to 14 can be realized. The frame rate adjustment method described above.
A computer program product, characterized by comprising a computer program, which causes a computer to execute the frame rate adjustment method according to any one of claims 1 to 14 when the computer program is executed.
A frame rate adjustment system, characterized by comprising a server and a terminal device according to claim 15;

The server is configured to send the changed score lookup table to the terminal device when there is a change in the data in the score lookup table stored in the server; and when the weight stored in the server When the data in the lookup table is changed, the changed weight lookup table is sent to the terminal device.