WO2020207251A1

WO2020207251A1 - Image update method and apparatus, and terminal and storage medium

Info

Publication number: WO2020207251A1
Application number: PCT/CN2020/081134
Authority: WO
Inventors: 彭德良
Original assignee: Oppo广东移动通信有限公司
Priority date: 2019-04-09
Filing date: 2020-03-25
Publication date: 2020-10-15
Also published as: CN109992232B; CN109992232A

Abstract

An image update method and apparatus, and a terminal and a storage medium, relating to the technical field of terminals. The method comprises: sending image frame data to a display screen by means of a synthesis thread, the image frame data being obtained after surface synthesis by the synthesis thread (401); obtaining the number of times an image frame is currently displayed on the display screen (402); and if the number of times is greater than a number-of-times threshold, resetting a vertical synchronization signal starting point of the display screen, and updating, according to the image frame data, the image frame displayed on the display screen (403). With respect to the method in the related art that the display screen needs to receive a hardware vertical synchronization signal before it can perform image update, in the method, by resetting the vertical synchronization starting point of the display screen, the display screen can perform image update immediately according to the image frame data upon receipt of a new image frame when the same image frame is repeatedly displayed, so as to shorten the time during which the display screen waits for the hardware vertical synchronization signal, thereby facilitating improvement of the speed of displaying an image on the display screen.

Description

Image update method, device, terminal and storage medium

This application claims the priority of a Chinese patent application filed on April 9, 2019 with the application number 201910279696.9 and the invention title "Image update method, device, terminal and storage medium", the entire content of which is incorporated into this application by reference .

Technical field

The embodiments of the present application relate to the field of terminal technology, and in particular, to an image update method, device, terminal, and storage medium.

Background technique

Vertical synchronization (Vsync) is a technology that improves the smoothness of terminal screens.

After the terminal applies the vertical synchronization technology, when the application receives the vertical synchronization signal, it will draw the picture; when the synthesis thread receives the vertical synchronization signal, it will synthesize multiple visible layers and send the synthesized image frame data to the display Screen: When the display screen receives the vertical synchronization signal, it updates the image frame.

Summary of the invention

The embodiments of the present application provide an image update method, device, terminal, and storage medium. The technical solutions are as follows:

On the one hand, an embodiment of the present application provides an image update method, the method includes:

Sending image frame data to the display screen through a synthesis thread, where the image frame data is obtained after layer synthesis by the synthesis thread;

Acquiring the display times of the image frame currently displayed on the display screen;

In response to the display times being greater than the times threshold, resetting the starting point of the vertical synchronization signal of the display screen, and updating the image frames displayed on the display screen according to the image frame data.

On the other hand, an embodiment of the present application provides an image update device, which includes:

The first sending module is configured to send image frame data to the display screen through a synthesis thread, where the image frame data is obtained after layer synthesis by the synthesis thread;

The first acquiring module is configured to acquire the display times of the image frame currently displayed on the display screen;

The first update module is configured to reset the starting point of the vertical synchronization signal of the display screen in response to the display times being greater than the times threshold, and update the image frames displayed on the display screen according to the image frame data.

On the other hand, the present application provides a terminal including a processor, a memory and a display screen connected to the processor, and program instructions stored on the memory, and the processor executes the program The image update method described in the above aspect is implemented when instructed.

In another aspect, a computer-readable storage medium is provided, and program instructions are stored thereon, and when the program instructions are executed by a processor, the image update method as described in the above aspect is implemented.

The beneficial effects brought by the technical solutions provided by the embodiments of the present application include at least:

In the embodiment of the application, when the image frame data is sent to the display screen through the synthesis thread, the display times of the image frames currently displayed on the display screen are obtained, and when the display times are greater than the times threshold, the starting point of the vertical synchronization signal of the display screen is reset for display The screen updates the image frame displayed on the display screen according to the image frame data; compared with the related art, the display screen needs to receive the hardware vertical synchronization signal before it can update the image. In the embodiment of the present application, the vertical synchronization starting point of the display screen is reset , So that when the display screen receives new image frame data while repeatedly displaying the same image frame, it can immediately update the image according to the image frame data, thus shortening the time for the display screen to wait for the hardware vertical synchronization signal and helping to improve the display The speed of the screen display.

Description of the drawings

FIG. 1 is a schematic structural diagram of a terminal provided by an exemplary embodiment of the present application;

Figure 2 is a schematic diagram of the graphic display process in the Android system;

Figure 3 is a schematic diagram of the principle of the image update process in the related art;

Fig. 4 shows a method flowchart of an image update method provided by an exemplary embodiment of the present application;

FIG. 5 is a schematic diagram of the implementation process of the image update method shown in FIG. 4;

Fig. 6 is a schematic diagram of the vertical synchronization process in the related art;

Fig. 7 shows a method flowchart of an image update method provided by another exemplary embodiment of the present application;

Fig. 8 shows a method flowchart of an image update method provided by another exemplary embodiment of the present application;

FIG. 9 is a schematic diagram of the implementation process of the image update method shown in FIG. 8;

FIG. 10 is a schematic structural diagram of an image updating device provided by an embodiment of the present application.

detailed description

In order to make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the application as detailed in the appended claims.

In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance. In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral Ground connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above-mentioned terms in this application can be understood under specific circumstances. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more. "And/or" describes the association relationship of the associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean: A alone exists, both A and B exist, and B alone exists. The character "/" generally indicates that the associated objects are in an "or" relationship.

Before explaining the embodiments of the present application, the application scenarios of the embodiments of the present application are first described. Fig. 1 shows a schematic structural diagram of a terminal provided by an exemplary embodiment of the present application.

The terminal 100 is an electronic device installed with a target application. The target application can be a system program or a third-party application. Among them, third-party applications are applications made by third parties other than the user and the operating system. For example, the target application can be a game application or a video playback application.

Optionally, the terminal 100 includes a processor 120 and a memory 140.

The processor 120 may include one or more processing cores. The processor 120 uses various interfaces and lines to connect various parts of the entire terminal 100, and executes the terminal by running or executing instructions, programs, code sets, or instruction sets stored in the memory 140, and calling data stored in the memory 140. 100 various functions and processing data. Optionally, the processor 120 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). A kind of hardware form to realize. The processor 120 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is used to render and draw the content that the display needs to display; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 120, but may be implemented by a chip alone.

The memory 140 may include random access memory (RAM) or read-only memory (Read-Only Memory). Optionally, the memory 140 includes a non-transitory computer-readable storage medium. The memory 140 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 140 may include a program storage area and a data storage area. The program storage area may store instructions for implementing the operating system and instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), Instructions used to implement the following method embodiments; the storage data area can store the data involved in the following method embodiments.

The terminal 120 in the embodiment of the present application further includes a display screen 160. Optionally, the display screen 160 is a touch display screen, which is used to receive a user's touch operation on or near any suitable object such as a finger, a touch pen, etc., and to display a user interface of each application program. The display screen 160 is usually arranged on the front panel of the terminal 100 or, at the same time, arranged on the front panel and the rear panel of the terminal 100. The display screen 160 can be designed as a full screen, a curved screen or a special-shaped screen. The display screen 160 can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in this embodiment.

Optionally, the display screen 160 involved in the embodiment of the present application may be a video screen or a command screen. Among them, the control integrated circuit (Integrated Circuit, IC) of the video screen does not include Random-Access Memory (RAM), while the control IC of the command screen includes RAM. When the video screen is working, the control signal and image frame data are transmitted to the video screen through the bus in the form of messages, and the video screen displays images according to the received image frame data; when the command screen is working, the image frame data passes through the bus After being transmitted to the command screen, it is written into the frame buffer of RAM, and the command screen reads the image frame data from the frame buffer under the control of the timer, thereby displaying the image according to the image frame data. Of course, the display screen 160 in the embodiment of the present application may also support the video mode and the command mode at the same time, which is not limited in this embodiment.

In addition, those skilled in the art can understand that the structure of the terminal 100 shown in the above drawings does not constitute a limitation on the terminal 100, and the terminal may include more or less components than those shown in the figure, or combine certain components. Components, or different component arrangements. For example, the terminal 100 also includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (Wireless Fidelity, WiFi) module, a power supply, a Bluetooth module, etc., which are not repeated here.

For ease of understanding, the graphic display system in the terminal is first described below, and the following embodiments take an Android graphic display system as an example for schematic description.

The graphics display process of the terminal is divided into three stages: layer drawing, layer composition and image frame update. Among them, the layer drawing is executed by the application program, the layer composition is executed by the synthesis thread, and the image frame update is executed by the display screen.

As shown in Figure 2, the display screen 21 reads the image frame data from the frame buffer, and the reading process is: starting from the start address of the frame buffer, proceeding from top to bottom, from left to right. Scan to map the scanned content on the display.

Since the content displayed in the display 21 needs to be continuously updated, if reading and writing operations are performed in the same frame buffer, multiple frames of content will be displayed on the display 21 at the same time. Therefore, the terminal adopts a double buffering mechanism, among which, One of the double buffers is used for content reading and display, while the other buffer is used for background graphics synthesis and writing.

Schematically, as shown in FIG. 2, the front buffer 22 is a frame buffer for the content to be displayed on the display screen, and the rear buffer 23 is a frame buffer for synthesizing the next frame of graphics. When the current frame is displayed and the next frame is written, the display screen 21 reads the content in the back buffer 23. Correspondingly, the next frame of graphics is synthesized in the front buffer 22 (the front and rear buffer roles interact with each other). change).

SurfaceFlinger, as a layer compositor (including a compositing thread), is used to synthesize multiple layers (surface) passed by the upper layer (application), and submit it to the buffer of the display 21 for the display 21 to read display. As shown in Figure 2, the content in the back buffer 23 is synthesized by SurfaceFlinger from multiple surfaces 24. Among them, each surface corresponds to a window (window) of the upper layer, such as a dialog box, a status bar, and an activity (Activity).

In order to enable the application program and the synthesis thread to draw and synthesize the screen according to the refresh frequency of the display screen, so as to avoid screen freezes, the display screen is based on the hardware vertical synchronization signal (the generation frequency of the hardware vertical synchronization signal is consistent with the display refresh frequency) The image frame is updated, and the application program and the compositing thread perform picture drawing and synthesis by requesting the software vertical synchronization signal (to keep in sync with the hardware synchronization signal).

As shown in FIG. 3, it shows a schematic diagram of the principle of the process of updating the image of the display screen in the related art. The display screen displays A frame image frame in the vertical synchronization signal period of 0-T1, and receives the image frame data of B frame in the period of display 0-T1, so that in the vertical synchronization signal period of T1-T2 B frame image frame is displayed inside. However, since the display screen does not receive new image frame data during the period of the vertical synchronization signal of T1-T2, the display screen needs to wait for the new image frame data and display B frame image frames again. When the C frame image frame data is received at T3 (the interval between T3 and T2 is less than the period of the vertical synchronization signal), since the vertical synchronization signal has not been received, the display screen needs to continue to display the B frame image frame, and at time T3' When the vertical synchronization signal is received, the C frame image frame is displayed according to the new image frame data.

Obviously, in the related art, since the display screen only updates the image frame when the vertical synchronization signal is received, the display of the new image frame is delayed.

In order to solve the problem of delay in the screen display due to the long waiting time of the vertical synchronization signal in the related art, in the image update method provided by the embodiment of the present application, when the image frame data is sent to the display screen through the synthesis thread, the current display screen is obtained. Displays the display times of the image frame, and resets the starting point of the vertical synchronization signal of the display screen when the display times are greater than the threshold of times, so that the display screen can update the image frame displayed on the display screen according to the image frame data, thus shortening the display screen waiting for the vertical synchronization signal The time can help increase the speed of the display screen to update the picture and reduce the display delay of the picture. Illustrative embodiments are used for description below.

Please refer to FIG. 4, which shows a method flowchart of an image update method provided by an exemplary embodiment of the present application. The method may include the following steps.

Step 401: Send the image frame data to the display screen through the synthesis thread, and the image frame data is obtained after layer synthesis by the synthesis thread.

In a possible implementation manner, after the application program draws the layer, the drawn layer is sent to the composition thread, and the composition thread synthesizes multiple layers to obtain image frame data, thereby sending the image frame data To the display.

Optionally, after the display screen receives the image frame data, it writes the image frame data into the buffer.

Step 402: Obtain the display times of the image frame currently displayed on the display screen.

In the related art, the image frame data sent by the synthesis thread is written into the buffer of the display screen, and when the hardware vertical synchronization signal is received, the display screen updates the image frame according to the image frame data in the buffer.

In a possible scenario, if the new image frame data sent by the synthesis thread is not received, the display screen needs to repeatedly display the previous image frame until the synthesis thread sends the new image frame data, in the next vertical synchronization signal cycle Update the image frame, that is, the same image frame is repeatedly displayed multiple times.

In order to improve the update speed of image frames in this scenario, in this embodiment of the application, when receiving the image frame data sent by the synthesis thread, the terminal does not always update the image frame when it receives the hardware vertical synchronization signal, but Determine whether the image frame needs to be updated immediately according to whether the image frame currently displayed on the display screen is repeatedly displayed.

In a possible implementation manner, when the display screen is displaying, it records the display times of the current image frame. When receiving the image frame data sent by the synthesis thread, the terminal obtains the display times of the current image frame. Wherein, when the number of display times is 1, it indicates that the current image frame is displayed for the first time, and when the number of display times is greater than 1, it indicates that the current image frame is being repeatedly displayed.

Step 403, in response to the display times being greater than the times threshold, reset the starting point of the vertical synchronization signal of the display screen, and update the image frames displayed on the display screen according to the image frame data.

When the number of display times is greater than the number threshold, it indicates that the current image frame is a repeated display image frame. In order to update the image frame as soon as possible, the terminal does not need to update the image frame according to the hardware vertical synchronization signal, but resets the starting point of the vertical synchronization signal of the display screen. , So that the display screen can immediately update the image frames displayed on the display screen according to the received image frame data.

Among them, the terminal resets the starting point of the vertical synchronization signal by modifying the register of the display screen, and after resetting the starting point of the vertical synchronization signal, the display screen starts from the starting point of the vertical synchronization signal and regenerates the hardware vertical synchronization signal according to the period of the vertical synchronization signal.

Optionally, the threshold of the number of times is 1 time, that is, when the currently displayed image frame is not displayed for the first time, the terminal will re-start the vertical synchronization signal of the display screen when the new image frame data is synthesized.

In an illustrative example, as shown in Figure 5, the display screen displays A frame image frame during the period of 0-T1 vertical synchronization signal, and it receives B frame image during display period 0-T1 Frame data, so that B-frame image frames are displayed in the period of the vertical synchronization signal of T1-T2. However, since the display screen does not receive new image frame data during the period of the vertical synchronization signal of T1-T2, the display screen needs to wait for the new image frame data and display B frame image frames again. When the C frame image frame data is received at T3 (the interval between T3 and T2 is less than the period of the vertical synchronization signal), since the number of times the B frame image frame currently displayed on the display screen is displayed is greater than 1, the terminal T3 will reset at this time Set the starting point of the vertical synchronization signal, and display C frames in the period of the vertical synchronization signal T3-T4.

Different from the related art that needs to wait for the hardware vertical synchronization signal to update the image frame, the dynamic vertical synchronization signal period mechanism is introduced in the embodiment of the present application, so that the display screen can immediately update the image frame according to the received image frame data. Improve the display speed of images.

To sum up, in the embodiment of the present application, when the image frame data is sent to the display screen through the synthesis thread, the display times of the image frames currently displayed on the display screen are obtained, and when the display times are greater than the times threshold, the vertical synchronization of the display screen is reset Signal starting point, so that the display screen can update the image frame displayed on the display screen according to the image frame data; compared with the related art, the display screen needs to receive the hardware vertical synchronization signal before it can update the image. In this embodiment of the application, the display is reset The vertical synchronization starting point of the screen enables the screen to update the image immediately according to the image frame data when it receives new image frame data while repeatedly displaying the same image frame, thus shortening the time for the display screen to wait for the hardware vertical synchronization signal. Helps increase the speed of the display screen.

Optionally, after the image frame data is sent to the display screen through the synthesis thread, the method further includes:

Acquiring the current display state of the display screen;

In response to the display screen being in the sleep state in the command mode, the starting point of the vertical synchronization signal of the display screen is reset, and the image frame displayed on the display screen is updated according to the image frame data.

Optionally, the resetting the start point of the vertical synchronization signal of the display screen and updating the image frame displayed on the display screen according to the image frame data includes:

Determining the reset cost for resetting the starting point of the vertical synchronization signal according to the current hardware state, the reset cost including the reset time;

In response to the resetting cost being less than the cost threshold, resetting the starting point of the vertical synchronization signal, and updating the image frame displayed on the display screen according to the image frame data.

Optionally, before the sending the image frame data to the display screen through the synthesis thread, the method further includes:

Receiving a vertical synchronization signal request sent by a requesting party, where the vertical synchronization signal request is used to request to obtain a vertical synchronization signal;

Acquiring the first request time of the vertical synchronization signal request and the second request time of the last time the vertical synchronization signal request was received;

In response to the request time interval being greater than the duration threshold, send an analog vertical synchronization signal to the requester, where the request time interval is the time interval between the first request moment and the second request moment, and the requester uses Performing a predetermined operation according to the analog vertical synchronization signal.

Optionally, the duration threshold is determined according to a vertical synchronization signal period, the vertical synchronization signal period is used to indicate the time interval between two adjacent vertical synchronization signals, and the duration threshold is greater than or equal to the vertical synchronization signal period .

Optionally, the requesting party is an application program;

The receiving the vertical synchronization signal request sent by the requesting party includes:

Receiving a first vertical synchronization signal request sent by the application program;

The sending an analog vertical synchronization signal to the requesting party includes:

Sending a first analog vertical synchronization signal to the application program, where the application program is used to draw a picture according to the first analog vertical synchronization signal.

Optionally, after the acquiring the first request time of the vertical synchronization signal request and the second request time of the vertical synchronization signal request last time, the method further includes:

In response to the request time interval being less than the duration threshold, generating a first vertical synchronization signal according to a standard vertical synchronization signal, and the standard vertical synchronization signal is generated by a signal generation thread;

Sending the first vertical synchronization signal to the application program, where the application program is used to draw a picture according to the first vertical synchronization signal.

Optionally, the requester is the composition thread;

Receiving a second vertical synchronization signal request sent by the composition thread;

Sending a second analog vertical synchronization signal to the composition thread, where the composition thread is used to perform layer composition according to the second analog vertical synchronization signal.

In response to the request time interval being less than the duration threshold, generating a second vertical synchronization signal according to a standard vertical synchronization signal, the standard vertical synchronization signal being generated by a signal generation thread;

The second vertical synchronization signal is sent to the composition thread, and the composition thread is used for layer composition according to the second vertical synchronization signal.

Since a certain amount of overhead is required to reset the starting point of the vertical synchronization signal, in order to avoid the situation that the resetting cost is too large, in a possible implementation manner, the above step 403 may include the following steps:

1. Determine the reset cost for resetting the starting point of the vertical synchronization signal according to the current hardware state, and the reset cost includes the reset time.

Optionally, if the display times of the current image frame are greater than 1, the terminal first determines the reset cost for resetting the starting point of the vertical synchronization signal according to the current hardware state, and the reset cost includes at least the reset time. The reset time may be the time for the terminal to reset the register of the display screen.

Optionally, the current hardware state includes at least one of processor load, display refresh frequency, and remaining memory, which is not limited in this embodiment.

2. In response to the reset cost being less than the cost threshold, reset the starting point of the vertical synchronization signal, and update the image frame displayed on the display screen according to the image frame data.

Optionally, the cost threshold is determined according to the waiting cost, and the waiting cost can be the time period from the next vertical synchronization signal. For example, when the cost threshold is the length of time from the next vertical synchronization signal, if the reset time is greater than the cost threshold, it indicates that when the reset of the start point of the vertical synchronization signal is completed, the next vertical synchronization signal has been received, and the reset start point is invalid at this time. Therefore, there is no need to reset the starting point.

When the reset cost is less than the cost threshold, the terminal resets the starting point of the vertical synchronization signal; if the reset cost is greater than the cost threshold, the terminal continues to display the current image frame until the hardware vertical synchronization signal is received to update the image frame.

In this embodiment, by determining the reset cost of resetting the starting point of the vertical synchronization signal, and comparing the reset cost with the cost threshold, the starting point of the vertical synchronization signal is reset when the reset cost is small, which avoids resetting If the cost is too high, reset the starting point of the vertical synchronization signal.

In another possible application scenario, for the display screen that supports the Command mode, when the display screen processes the sleep state (in the sleep state, the display screen displays images according to the image frame data in its own memory) and receives new In the related art, the display screen needs its hardware vertical synchronization signal to update the image frame, resulting in a delay in the image update. In order to reduce the image update delay, optionally, the terminal obtains the current display state of the display screen; if the display screen is in the sleep state in command mode, reset the start point of the vertical synchronization signal of the display screen and update according to the image frame data The image frame displayed on the display. For the process of resetting the starting point of the vertical synchronization signal of the display screen, reference may be made to the above-mentioned embodiment, which will not be repeated here.

As shown in FIG. 6, it shows a schematic diagram of the vertical synchronization process in the related art. When the application needs to draw a layer, it sends a vertical synchronization signal request to SurfaceFlinger, and waits for SurfaceFlinger to feed back the vertical synchronization signal to the application at the next vertical synchronization time point. After the application receives the vertical synchronization signal, it will draw the layer and send the layer data obtained by drawing to SurfaceFlinger. After SurfaceFlinger receives the layer data, when it reaches the next vertical synchronization time point, it performs layer synthesis according to the layer data, and sends the generated image frame data to the display screen. If the image frame is currently being displayed, the display screen will update the image frame according to the received image frame data at the next vertical synchronization point in time.

It can be seen that in related technologies, the application, SurfaceFlinger, and the display screen all need to receive the vertical synchronization signal before they can work, and the application, SurfaceFlinger, and the display screen waiting for the vertical synchronization signal will cause a delay in the screen display. For example, for a display with a refresh rate of 60Hz, the vertical synchronization signal interval is 16.7ms. In extreme cases, if the application, SurfaceFlinger, and the display all wait 16.7ms, it will take 50ms for a frame of image to be drawn to display.

The method provided in the above embodiment can shorten the waiting time of the display screen from the hardware level. In order to further improve the screen display speed, the waiting time of the application and SurfaceFlinger can be optimized from the software level. Illustrative embodiments are used for description below.

In a possible implementation manner, based on FIG. 4, as shown in FIG. 7, the following steps may be further included before step 401.

Step 701: Receive a vertical synchronization signal request sent by the requester, where the vertical synchronization signal request is used to request to obtain a vertical synchronization signal.

Optionally, the requester includes at least one of an application program and a compositing thread. Correspondingly, the vertical synchronization signal request is a vertical synchronization signal request sent by the application program before the layer is drawn, or the compositing thread is in progress. The vertical synchronization signal request sent before the layer composition.

Step 702: Obtain the first request time of the vertical synchronization signal request and the second request time of the last received vertical synchronization signal request.

In the embodiment of the present application, the terminal records the request time of the vertical synchronization signal request every time after receiving the vertical synchronization signal request sent by the requesting party. When receiving the vertical synchronization signal request sent by the requester again, the terminal obtains the first request moment of the current vertical synchronization signal request and the second request moment of the last request sent by the requester of the vertical synchronization signal. The request time and the second request time calculate the request time interval between two adjacent vertical synchronization signal requests, where the request time interval = the first request time-the second request time.

Step 703: In response to the request time interval being greater than the duration threshold, send an analog vertical synchronization signal to the requester. The request time interval is the time interval between the first request moment and the second request moment, and the requester is used to perform execution according to the analog vertical synchronization signal. Scheduled operation.

In a possible implementation manner, the SurfaceFlinger process includes a signal generation thread and a signal transmission thread, where the signal generation thread is used to generate standard vertical synchronization signals, and the signal transmission thread is used to process standard vertical synchronization signals (such as After increasing the time offset), the vertical synchronization signal is sent to the requester.

Different from the related technology, the signal sending thread needs to receive the standard vertical synchronization signal generated by the signal generation thread before it feeds back the vertical synchronization signal to the requester. Optionally, in the embodiment of the present application, when the requester makes two requests When the request time interval between is greater than the duration threshold (indicating that the vertical synchronization signal is received after a long waiting time after the previous vertical synchronization signal request is sent), the signal sending thread generates an analog vertical synchronization signal and sends the analog vertical synchronization signal Sent to the requester.

Optionally, the signal sending thread in the embodiment of the present application has the function of generating an analog vertical synchronization signal, and the generated analog vertical synchronization signal is the same as the real vertical synchronization signal (the vertical synchronization signal sent to the requesting party according to the standard vertical synchronization signal) .

Correspondingly, after receiving the analog vertical synchronization signal, the requesting party performs a predetermined operation according to the analog vertical synchronization signal. Since there is no need to wait for the real vertical synchronization signal, the time for the requester to request the vertical synchronization signal is shortened, thereby reducing the display delay of the picture.

In a possible implementation manner, the duration threshold is determined according to the period of the vertical synchronization signal, the period of the vertical synchronization signal is used to indicate the time interval between two adjacent vertical synchronization signals, and the duration threshold is greater than or equal to the period of the vertical synchronization signal. For example, the duration threshold is one vertical synchronization signal period, or 1.2 vertical synchronization signal periods, or 1.5 vertical synchronization signal periods.

In an illustrative example, when the refresh frequency of the display screen is 60 Hz, the period of the vertical synchronization signal is 16.7 ms, and accordingly, the duration threshold is determined to be 16.7 ms.

Optionally, when the request time interval between two adjacent requests of the requester is less than the duration threshold, the signal sending thread sends the (real) vertical synchronization signal to the requester according to the standard vertical synchronization signal generated by the signal generation thread, so that the requester can follow The vertical synchronization signal performs a predetermined operation.

In the embodiment of the present application, after receiving the vertical synchronization signal request sent by the requesting party, the request time interval is calculated according to the first request time of the vertical synchronization signal request and the second request time of the last time the vertical synchronization signal request is received , And when the request time interval is greater than the duration threshold, send an analog vertical synchronization signal to the requester so that the requester can perform a predetermined operation according to the analog vertical synchronization signal; compared with the related technology, it needs to wait for the real vertical synchronization request to perform the operation. The introduction of a vertical synchronization signal simulation mechanism allows the requester to perform a predetermined operation without waiting for the real vertical synchronization signal, thereby shortening the time for the requesting party to wait for the vertical synchronization signal and helping to increase the speed of the display screen.

In a possible implementation, the SurfaceFlinger process includes a signal generation thread (VsyncThread) that generates a standard vertical synchronization signal (ie, a software vertical synchronization signal), and a first active thread (EventThread (APP) that focuses on the vertical synchronization signal of the application program). ), the second active thread (EventThread (Surface)) that pays attention to the vertical synchronization signal of the composition thread, the signal sending thread (DispSyncThread) that sends the vertical synchronization signal according to the standard vertical synchronization signal, and the composition thread that is responsible for the composition layer. The following embodiments are used to describe the process of the application program requesting to obtain the vertical synchronization signal and the synthesis thread requesting to obtain the vertical synchronization signal.

Please refer to FIG. 8, which shows a method flowchart of an image updating method provided by another exemplary embodiment of the present application. The method may include the following steps.

Step 801: Receive a first vertical synchronization signal request sent by an application program.

In a possible implementation, when the application needs to draw a layer, it sends a first vertical synchronization signal request to the first active thread in the SurfaceFlinger process, and the first active thread sends the first vertical synchronization signal to the signal sending thread. Synchronization signal request. Correspondingly, the signal sending thread receives the first vertical synchronization signal request.

Step 802: Obtain the first request moment of the first vertical synchronization signal request and the second request moment of the last vertical synchronization signal request.

Optionally, the signal sending thread of the SurfaceFlinger process obtains the first request moment of the first vertical synchronization signal request and the second request moment of the first vertical synchronization signal request sent by the application program last time, and calculates the first request The request time interval between time and the second request time.

In an illustrative example, as shown in Figure 9, the SurfaceFlinger process (the signal sending thread in) acquires the first request time as t1 and the second request time as t2, and the calculated request time interval is t1-t2= 20ms.

Step 803: In response to the request time interval being greater than the duration threshold, send a first analog vertical synchronization signal to the application program. The request time interval is the time interval between the first request moment and the second request moment. The vertical sync signal draws the picture.

Optionally, the duration threshold is a vertical synchronization signal period of the display screen. For example, when the display screen of the terminal is 60 Hz, the period of the vertical synchronization signal is 16.7 ms.

Correspondingly, when the request time interval between the first request moment and the second request moment is greater than one vertical synchronization signal period, the signal sending thread in the SurfaceFlinger thread generates the first analog vertical synchronization signal through the vertical synchronization signal simulation function, And send the first analog vertical synchronization signal to the application program (first send the first analog vertical synchronization signal to the first active thread, and then the first active thread to the application program).

With reference to the example in step 802, since the request time interval (20 ms) between two adjacent requests is greater than one vertical synchronization signal period (16.7 ms), SurfaceFlinger immediately sends the first analog vertical synchronization signal to the application.

In other possible implementation manners, when n-1 request time intervals between consecutive n (n≥3) first vertical synchronization signal requests are greater than the duration threshold, the terminal sends the first simulation to the application through the SurfaceFlinger process The vertical synchronization signal is not limited in this embodiment.

Optionally, after receiving the first analog vertical synchronization signal, the application program performs layer drawing, and sends the layer data obtained by drawing to SurfaceFlinger. Schematically, as shown in FIG. 9, the application program performs layer drawing after receiving the first analog vertical synchronization signal, and sends the layer data to the SurfaceFlinger process (composite thread in).

Step 804, in response to the request time interval being less than the duration threshold, generate a first vertical synchronization signal according to the standard vertical synchronization signal, and the standard vertical synchronization signal is generated by the signal generation thread.

When the request time interval between two adjacent requests is less than the duration threshold, it indicates that the application program waits for the vertical synchronization signal for a short time, so the terminal can generate the first vertical synchronization signal according to the standard vertical synchronization signal.

In a possible implementation manner, when the request time interval is less than the duration threshold, the signal sending thread generates the first vertical synchronization signal after receiving the standard vertical synchronization signal generated by the signal generation thread.

Step 805: Send a first vertical synchronization signal to the application program, where the application program is used to draw a picture according to the first vertical synchronization signal.

Further, the terminal sends the first vertical synchronization signal to the application program, so that the application program performs picture drawing according to the first vertical synchronization signal.

In a possible implementation manner, the signal sending thread sends the first vertical synchronization signal to the first active thread, and the first active thread sends the first vertical synchronization signal to the application program.

Step 806: Receive a second vertical synchronization signal request sent by the composition thread.

In a possible implementation, when the composition thread receives the layer data sent by the application program and needs to perform layer composition, it sends a second vertical synchronization signal request to the second active thread in the SurfaceFlinger process. The two active threads send the second vertical synchronization signal request to the signal sending thread. Correspondingly, the signal sending thread receives the second vertical synchronization signal request.

Step 807: Obtain the first request time of the second vertical synchronization signal request and the second request time of the last received second vertical synchronization signal request.

Optionally, the signal sending thread of the SurfaceFlinger process obtains the first request moment of the second vertical synchronization signal request and the second request moment of the second vertical synchronization signal request sent by the synthesis thread last time, and calculates the first request The request time interval between time and the second request time.

In an illustrative example, the SurfaceFlinger process (signal sending thread) acquires the first request time as t1 and the second request time as t2, and calculates that the request time interval is t1-t2=25ms.

Step 808: In response to the request time interval being greater than the duration threshold, send a second analog vertical synchronization signal to the composition thread, and the composition thread is used to perform layer composition according to the second analog vertical synchronization signal.

Correspondingly, when the request time interval between the first request moment and the second request moment is greater than one vertical synchronization signal period, the signal sending thread in the SurfaceFlinger thread generates the second analog vertical synchronization signal through the vertical synchronization signal simulation function, And send the second analog vertical synchronization signal to the composition thread (first send the second analog vertical synchronization signal to the second active thread, and the second active thread to the composition thread).

With reference to the example in step 807, since the request time interval (25ms) between two adjacent requests is greater than one vertical synchronization signal period (16.7ms), SurfaceFlinger immediately sends the second analog vertical synchronization signal to the synthesis thread.

In other possible implementation manners, when n-1 request time intervals between consecutive n (n≥3) second vertical synchronization signal requests are greater than the duration threshold, the terminal sends the second simulation to the synthesis thread through the SurfaceFlinger process The vertical synchronization signal is not limited in this embodiment.

Optionally, after receiving the second analog vertical synchronization signal, the synthesis thread performs layer synthesis, and sends the synthesized image frame data to the display screen.

Combining the example in the above steps, as shown in Figure 9, because the request time interval (25ms) between two adjacent requests is greater than one vertical synchronization signal period (16.7ms), therefore, the signal sending thread in the SurfaceFlinger process (not shown in the figure) (Shown) Send a second analog vertical synchronization signal (not shown in the figure) to the composition thread. After the compositing thread receives the second analog vertical synchronization signal, it can perform layer synthesis without waiting for the real vertical synchronization signal, thereby sending the synthesized image frame data to the display screen.

Step 809: In response to the request time interval being less than the duration threshold, generate a second vertical synchronization signal according to the standard vertical synchronization signal, and the standard vertical synchronization signal is generated by the signal generation thread.

When the request time interval between two adjacent requests is less than the duration threshold, it indicates that the synthesis thread waits for the vertical synchronization signal for a short time, so the terminal can generate the second vertical synchronization signal according to the standard vertical synchronization signal.

In a possible implementation manner, when the request time interval is less than the duration threshold, the signal sending thread generates the second vertical synchronization signal after receiving the standard vertical synchronization signal generated by the signal generation thread.

Step 810: Send a second vertical synchronization signal to the composition thread, and the composition thread is used to perform layer composition according to the second vertical synchronization signal.

Further, the terminal sends a second vertical synchronization signal to the composition thread, so that the composition thread performs layer composition according to the second vertical synchronization signal.

In a possible implementation manner, the signal sending thread sends the second vertical synchronization signal to the second active thread, and the second active thread sends the second vertical synchronization signal to the composition thread.

In step 811, the image frame data is sent to the display screen through the synthesis thread, and the image frame data is obtained after layer synthesis by the synthesis thread.

Step 812: Obtain the display times of the image frame currently displayed on the display screen.

Step 813, in response to the display times being greater than the times threshold, reset the starting point of the vertical synchronization signal of the display screen, and update the image frames displayed on the display screen according to the image frame data.

For the implementation process of the foregoing steps 811 to 813, reference may be made to steps 401 to 403, which will not be repeated in this embodiment.

In this embodiment, when the request time interval between two adjacent requests of the application program is greater than the duration threshold, a vertical synchronization signal is simulated to the application program, so that the application program can immediately draw the layer, avoiding the application program from getting real data for a long time. Vertical synchronization signal, thereby improving the screen drawing efficiency of the application.

In this embodiment, when the request time interval between two adjacent requests of the compositing thread is greater than the duration threshold, a vertical synchronization signal is simulated to the compositing thread, so that the compositing thread can immediately perform the layer compositing operation, and avoid the compositing thread from getting the truth for a long time. The vertical synchronization signal, thereby improving the efficiency of layer synthesis.

In order to verify the optimization effect of the above method, the testers performed statistics on the average time drawn to the screen display under the desktop click scene and the in-app click scene respectively. The experimental data obtained are shown in Table 1.

Table I

From the data in Table 1, it can be found that after applying the above method, the average time drawn to the screen display is optimized by 8ms (desktop click scene) and 9ms (in-app click scene), and the standard deviation is smaller, that is, the frame rate of the picture is higher. stable.

The following are device embodiments of this application, which can be used to implement the method embodiments of this application. For details not disclosed in the device embodiment of this application, please refer to the method embodiment of this application.

Please refer to FIG. 10, which shows a schematic structural diagram of an image updating apparatus provided by an embodiment of the present application. The device can be implemented as all or part of the terminal in Figure 1 through a dedicated hardware circuit, or a combination of software and hardware. The device includes:

The first sending module 1001 is configured to send image frame data to the display screen through a synthesis thread, where the image frame data is obtained after layer synthesis by the synthesis thread;

The first obtaining module 1002 is configured to obtain the display times of the image frame currently displayed on the display screen;

The first update module 1003 is configured to reset the starting point of the vertical synchronization signal of the display screen in response to the display times being greater than the times threshold, and update the image frames displayed on the display screen according to the image frame data.

Optionally, the device further includes:

The second acquiring module is used to acquire the current display state of the display screen;

The second update module is configured to reset the starting point of the vertical synchronization signal of the display screen if the display screen is in the sleep state in the command mode, and update the image frame displayed on the display screen according to the image frame data .

Optionally, the first update module 1003 or the second update module includes:

A cost determining unit, configured to determine a reset cost for resetting the starting point of the vertical synchronization signal according to the current hardware state, the reset cost including the reset time;

The update unit is configured to reset the starting point of the vertical synchronization signal if the reset cost is less than the cost threshold, and update the image frame displayed on the display screen according to the image frame data.

Optionally, the device further includes:

The first receiving module is configured to receive a vertical synchronization signal request sent by the requesting party, where the vertical synchronization signal request is used to request to obtain a vertical synchronization signal;

The third acquiring module is configured to acquire the first request moment of the vertical synchronization signal request and the second request moment of the last received vertical synchronization signal request;

The second sending module is configured to send an analog vertical synchronization signal to the requesting party in response to the request time interval being greater than the duration threshold, where the request time interval is the time between the first request moment and the second request moment Interval, the requester is used to perform a predetermined operation according to the analog vertical synchronization signal.

Optionally, the requesting party is an application program;

The first receiving module is used to:

The second sending module is used to:

Optionally, the device further includes:

A first generation module, configured to generate a first vertical synchronization signal according to a standard vertical synchronization signal in response to the request time interval being less than the duration threshold, and the standard vertical synchronization signal is generated by a signal generation thread;

The third sending module is configured to send the first vertical synchronization signal to the application program, and the application program is configured to perform picture drawing according to the first vertical synchronization signal.

Optionally, the requester is the composition thread;

The first receiving module is used to:

The second sending module is used to:

Optionally, the device further includes:

A second generation module, configured to generate a second vertical synchronization signal according to a standard vertical synchronization signal in response to the request time interval being less than the duration threshold, and the standard vertical synchronization signal is generated by a signal generation thread;

The fourth sending module is configured to send the second vertical synchronization signal to the composition thread, and the composition thread is used to perform layer composition according to the second vertical synchronization signal.

It should be noted that the device provided in the above embodiment, when implementing its functions, only uses the division of the above functional modules for illustration. In practical applications, the above functions can be allocated by different functional modules as needed, namely The internal structure of the terminal is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above-mentioned embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

The present application also provides a computer-readable medium on which program instructions are stored, and the program instructions implement the image update method provided by the foregoing method embodiments when the program instructions are executed by the processor.

The present application also provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the image update method described in each of the foregoing embodiments.

The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

Those of ordinary skill in the art can understand that all or part of the steps in the frame rate control method of the foregoing embodiments can be implemented by hardware, or by a program to instruct relevant hardware to complete, and the program can be stored in a computer readable Among the storage media, the aforementioned storage media may be read-only memory, magnetic disks, or optical disks. The above descriptions are only preferred embodiments of this application and are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection of this application. Within range.

Claims

An image update method, the method includes:

Sending image frame data to the display screen through a synthesis thread, where the image frame data is obtained after layer synthesis by the synthesis thread;

Acquiring the display times of the image frame currently displayed on the display screen;

In response to the display times being greater than the times threshold, resetting the starting point of the vertical synchronization signal of the display screen, and updating the image frames displayed on the display screen according to the image frame data.
The method according to claim 1, wherein, after the image frame data is sent to the display screen through the synthesis thread, the method further comprises:

Acquiring the current display state of the display screen;

In response to the display screen being in the sleep state in the command mode, the starting point of the vertical synchronization signal of the display screen is reset, and the image frame displayed on the display screen is updated according to the image frame data.
The method according to claim 1 or 2, wherein the resetting the start point of the vertical synchronization signal of the display screen and updating the image frame displayed on the display screen according to the image frame data comprises:

Determining the reset cost for resetting the starting point of the vertical synchronization signal according to the current hardware state, the reset cost including the reset time;

In response to the resetting cost being less than the cost threshold, resetting the starting point of the vertical synchronization signal, and updating the image frame displayed on the display screen according to the image frame data.
The method according to claim 1 or 2, wherein before the sending the image frame data to the display screen through the synthesis thread, the method further comprises:

Receiving a vertical synchronization signal request sent by a requesting party, where the vertical synchronization signal request is used to request to obtain a vertical synchronization signal;

Acquiring the first request time of the vertical synchronization signal request and the second request time of the last time the vertical synchronization signal request was received;

In response to the request time interval being greater than the duration threshold, send an analog vertical synchronization signal to the requester, where the request time interval is the time interval between the first request moment and the second request moment, and the requester uses Performing a predetermined operation according to the analog vertical synchronization signal.
The method according to claim 4, wherein the duration threshold is determined according to a vertical synchronization signal period, the vertical synchronization signal period is used to indicate the time interval between two adjacent vertical synchronization signals, and the duration threshold is greater than Equal to the period of the vertical synchronization signal.
The method according to claim 4, wherein the requesting party is an application program;

The receiving the vertical synchronization signal request sent by the requesting party includes:

Receiving a first vertical synchronization signal request sent by the application program;

The sending an analog vertical synchronization signal to the requesting party includes:

Sending a first analog vertical synchronization signal to the application program, where the application program is used to draw a picture according to the first analog vertical synchronization signal.
7. The method according to claim 6, wherein said acquiring the first request time of the vertical synchronization signal request and after the second request time of the last receiving the vertical synchronization signal request, the method further comprises:

In response to the request time interval being less than the duration threshold, generating a first vertical synchronization signal according to a standard vertical synchronization signal, and the standard vertical synchronization signal is generated by a signal generation thread;

Sending the first vertical synchronization signal to the application program, where the application program is used to draw a picture according to the first vertical synchronization signal.
The method according to claim 4, wherein the requestor is the composition thread;

The receiving the vertical synchronization signal request sent by the requesting party includes:

Receiving a second vertical synchronization signal request sent by the composition thread;

The sending an analog vertical synchronization signal to the requesting party includes:

Sending a second analog vertical synchronization signal to the composition thread, where the composition thread is used to perform layer composition according to the second analog vertical synchronization signal.
The method according to claim 8, wherein, after said acquiring the first request time of the vertical synchronization signal request and after the second request time of the last receiving the vertical synchronization signal request, the method further comprises:

In response to the request time interval being less than the duration threshold, generating a second vertical synchronization signal according to a standard vertical synchronization signal, the standard vertical synchronization signal being generated by a signal generation thread;

The second vertical synchronization signal is sent to the composition thread, and the composition thread is used for layer composition according to the second vertical synchronization signal.
An image update device, characterized in that the device includes:

The first sending module is configured to send image frame data to the display screen through a synthesis thread, where the image frame data is obtained after layer synthesis by the synthesis thread;

The first acquiring module is configured to acquire the display times of the image frame currently displayed on the display screen;

The first update module is configured to reset the starting point of the vertical synchronization signal of the display screen in response to the display times being greater than the times threshold, and update the image frames displayed on the display screen according to the image frame data.
The device according to claim 10, wherein the device further comprises:

The second acquiring module is used to acquire the current display state of the display screen;

The second update module is configured to reset the starting point of the vertical synchronization signal of the display screen in response to the display screen being in the sleep state in the command mode, and update the image frame displayed on the display screen according to the image frame data .
The device according to claim 10 or 11, wherein the first update module or the second update module comprises:

A cost determining unit, configured to determine a reset cost for resetting the starting point of the vertical synchronization signal according to the current hardware state, the reset cost including the reset time;

The updating unit is configured to reset the starting point of the vertical synchronization signal in response to the reset cost being less than the cost threshold, and update the image frame displayed on the display screen according to the image frame data.
The device according to claim 10 or 11, wherein the device further comprises:

The first receiving module is configured to receive a vertical synchronization signal request sent by the requesting party, where the vertical synchronization signal request is used to request to obtain a vertical synchronization signal;

The third acquiring module is configured to acquire the first request moment of the vertical synchronization signal request and the second request moment of the last received vertical synchronization signal request;

The second sending module is configured to send an analog vertical synchronization signal to the requester in response to the request time interval being greater than the duration threshold, where the request time interval is the time between the first request moment and the second request moment Interval, the requester is used to perform a predetermined operation according to the analog vertical synchronization signal.
The apparatus according to claim 13, wherein the duration threshold is determined according to a vertical synchronization signal period, the vertical synchronization signal period is used to indicate the time interval between two adjacent vertical synchronization signals, and the duration threshold is greater than Equal to the period of the vertical synchronization signal.
The device according to claim 13, wherein the requesting party is an application;

The first receiving module is used to:

Receiving a first vertical synchronization signal request sent by the application program;

The second sending module is used to:

Sending a first analog vertical synchronization signal to the application program, where the application program is used to draw a picture according to the first analog vertical synchronization signal.
The device according to claim 15, wherein the device further comprises:

A first generation module, configured to generate a first vertical synchronization signal according to a standard vertical synchronization signal in response to the request time interval being less than the duration threshold, and the standard vertical synchronization signal is generated by a signal generation thread;

The third sending module is configured to send the first vertical synchronization signal to the application program, and the application program is configured to perform picture drawing according to the first vertical synchronization signal.
The apparatus according to claim 13, wherein the requestor is the composition thread;

The first receiving module is used to:

Receiving a second vertical synchronization signal request sent by the composition thread;

The second sending module is used to:

Sending a second analog vertical synchronization signal to the composition thread, where the composition thread is used to perform layer composition according to the second analog vertical synchronization signal.
The device according to claim 17, wherein the device further comprises:

A second generation module, configured to generate a second vertical synchronization signal according to a standard vertical synchronization signal in response to the request time interval being less than the duration threshold, and the standard vertical synchronization signal is generated by a signal generation thread;

The fourth sending module is configured to send the second vertical synchronization signal to the composition thread, and the composition thread is used to perform layer composition according to the second vertical synchronization signal.
A terminal, characterized in that the terminal includes a processor, a memory and a display screen connected to the processor, and program instructions stored on the memory. When the processor executes the program instructions, The image update method according to any one of claims 1 to 9.
A computer-readable storage medium, characterized in that program instructions are stored thereon, and when the program instructions are executed by a processor, the image update method according to any one of claims 1 to 9 is realized.