WO2023040593A1 - Image data transmission method and apparatus, terminal and medium - Google Patents

Abstract

An image data transmission method and apparatus, a terminal and a medium. The method comprises: transmitting an mth frame of image data to a DDIC (301); on the basis of a historical TE signal outputted by the DDIC during the display process of n frames of images closest to the mth frame image, determining n historical refresh rates of the DDIC for each frame image from an (m-n)th to an (m-1)th frame image (302); when the n historical refresh rates meet display delay conditions, performing a display delay operation on an (m+1)th frame of image data, wherein the display delay operation is used to delay the transmission of the (m+1)th frame of image data to the DDIC (303); and transmitting the (m+1)th frame of image data to the DDIC when the display delay operation is complete (304). By means of introducing a display delay mechanism, the problem of screen flickering and jitter caused by the frequency hopping of the DDIC refresh rate due to fluctuations of the frame rate outputted by an AP is avoided, and the stability of the DDIC refresh rate during the image display process is improved.

Description

Image data transmission method, device, terminal and medium

This application claims the priority of the Chinese patent application with the application number 202111078920.1 and the title of the invention "image data transmission method, device, terminal and medium" filed on September 15, 2021, the entire contents of which are incorporated by reference in this application .

technical field

The embodiments of the present application relate to the field of display technology, and in particular to an image data transmission method, device, terminal and medium.

Background technique

With the continuous development of display technology, more and more displays that can support high refresh rate display have emerged as the times require. When running high frame rate applications or during sliding operations, by setting the display to high refresh rate mode It can improve the fluency of the picture.

For a display screen that adopts the application processor (Application Processor, AP)-display driver integrated circuit (DDIC)-display panel (Panel) drive architecture, during the image display process, the DDIC outputs the frame rate according to the AP (that is, the output The rate of image data) adaptively adjusts the refresh frequency to realize adaptive frequency conversion.

Contents of the invention

Embodiments of the present application provide an image data transmission method, device, terminal, and medium. Described technical scheme is as follows:

On the one hand, an embodiment of the present application provides an image data transmission method for an application processor AP, the method comprising:

Transmit the image data of the mth frame to the display driver chip DDIC, where m is a positive integer;

Based on the historical TE signal output by the DDIC during the display process of the n-th frame image closest to the m-th frame image, determine the n historical refresh frequencies of the DDIC for each frame image in the m-nth to m-1th frame images, where n is An integer greater than or equal to 2;

In the case that the n historical refresh frequencies meet the display delay condition, the display delay operation is performed on the m+1th frame of image data, and the display delay operation is used to delay the transmission of the m+th frame to the DDIC 1 frame of image data;

When the display delay operation is completed, the m+1th frame of image data is transmitted to the DDIC.

On the other hand, an embodiment of the present application provides an image data transmission device, the device includes:

The transmission module is used to transmit the image data of the mth frame to the display driver chip DDIC, where m is a positive integer;

The first determination module is configured to determine n of each frame image in the m-nth to m-1th frame images of the DDIC based on the historical TE signal output by the DDIC during the display process of the n-th frame image closest to the m-th frame image historical refresh frequency, n is an integer greater than or equal to 2;

A delay module, configured to perform a display delay operation on the m+1th frame of image data when the n historical refresh frequencies meet the display delay condition, and the display delay operation is used to delay transmission to the DDIC The m+1th frame of image data;

The transmission module is further configured to transmit the m+1th frame of image data to the DDIC when the display delay operation is completed.

On the other hand, an embodiment of the present application provides a terminal, the terminal includes an AP, a display screen, and a DDIC, and the AP and the DDIC are connected through a Mobile Industry Processor Interface (MIPI), The AP is used to execute at least one program in the memory to realize the image data transmission method described above.

On the other hand, an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores at least one program, and the at least one program is used to be executed by a processor to implement the above-mentioned image data transmission method .

On the other hand, an embodiment of the present application provides a computer program product, the computer program product includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; the processor of the terminal reads the computer program from the computer-readable storage medium. Instructions, the processor executes the computer instructions, so that the terminal executes the image data transmission method provided in the above aspect.

Description of drawings

Figure 1 is a schematic diagram of the image display process under the AP-DDCI-Panel architecture;

Fig. 2 is a schematic diagram of the principle of the image data transmission method provided by the embodiment of the present application;

FIG. 3 shows a flowchart of an image data transmission method shown in an exemplary embodiment of the present application;

Figure 4 is a comparison chart of the refresh frequency when the display delay mechanism is introduced and the display delay mechanism is not introduced;

Fig. 5 is a flow chart of a process of determining the historical refresh frequency shown in an exemplary embodiment of the present application;

Fig. 6 is an implementation schematic diagram of a historical refresh frequency determination process shown in an exemplary embodiment of the present application;

FIG. 7 shows a flowchart of an image data transmission method shown in another exemplary embodiment of the present application;

Fig. 8 is an implementation schematic diagram of the implementation process of the image data transmission method shown in Fig. 7;

FIG. 9 shows a structural block diagram of an image data transmission device provided by an embodiment of the present application;

Fig. 10 shows a structural block diagram of a terminal provided by an exemplary embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings.

The "plurality" mentioned herein means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently. The character "/" generally indicates that the contextual objects are an "or" relationship.

As shown in Figure 1, under the AP-DDIC-Panel architecture, the AP side first performs layer drawing and rendering through the application program (Application, App), and then performs layering on the drawn layer through SurfaceFlinger (layer compositor) Synthesize the image data, and then send the image data to display (write) DDIC through MIPI. The DDIC stores the image data sent by the AP in the buffer (Buffer), and controls the Panel to refresh and display the image (Display) by scanning (reading) the image data in the Buffer. When implementing adaptive frequency conversion, the DDIC will adaptively adjust the refresh frequency according to the output frame rate of the AP (that is, the amount of image data transmitted by the AP to the DDIC per unit time, or the speed at which the AP transmits image data to the DDIC). For example, when the output frame rate of the AP decreases, the DDIC lowers the refresh rate, and when the output frame rate of the AP increases, the DDIC increases the refresh rate.

In the process of adaptive frequency conversion, the refresh frequency changes in a small range in a short time without affecting the image display quality, but when the refresh frequency changes in a large range in a short time, problems such as flickering and jitter will occur, which will affect the image display quality.

For example, in some scenarios, due to fluctuations in the speed of preparing image data on the AP side, the output frame rate of the AP changes from 60 Hz to 45 Hz in a short period of time, and then changes from 45 Hz to 72 Hz, and the refresh rate of the DDIC changes from 60 Hz. When changing to 45Hz, it will not cause flickering and jittering of the screen, but when the refresh rate of DDIC changes from 45Hz to 72Hz, the flickering and jittering of the screen will appear because the refresh rate changes too much.

In order to solve the above technical problem, in the embodiment of the present application, the AP side introduces a display delay mechanism. Under this mechanism, as shown in Figure 2, the AP determines the historical refresh frequency of the DDIC during the display process of the latest n frames of images based on the historical TE signal output by the DDIC (that is, the refresh frequency of the DDIC when displaying each frame of the latest n frames of images) , and based on the DDIC refresh frequency stabilization algorithm, the display delay condition detection is performed on the historical refresh frequency, so that when the delay condition is met, the display delay operation is performed on the next frame of image data to avoid the problem of a large jump in the refresh frequency , to achieve the effect of stabilizing the DDIC refresh rate, thereby reducing the flickering problem of the display screen caused by it.

For example, the AP obtains the historical refresh rate of the DDIC during the display of the last two frames of images. When the process of displaying the last two frames of images is detected, the historical refresh rates of the DDIC are 60 Hz (refresh frame image refresh rate), if the AP directly transmits the next frame of image data to the DDIC after completing the data preparation, the refresh rate of the DDIC will become 72Hz; and after the display delay mechanism is introduced, the AP detects the historical refresh of the DDIC The frequency satisfies the delay condition of sending display, so that the next frame of image data is transmitted to DDIC after a certain period of time delay, so that the refresh frequency of DDIC becomes 60Hz, thus avoiding the sharp jump of DDIC refresh frequency directly from 45Hz to 72Hz.

The method provided in the embodiment of the present application is applied to a terminal, and the above image data transmission method is executed by an AP in the terminal. The terminal may include a smart phone, a tablet computer, a wearable device (such as a smart watch), a portable personal computer, a smart TV, etc. The embodiment of the present application does not limit the specific type of the terminal.

Please refer to FIG. 3 , which shows a flowchart of an image data transmission method shown in an exemplary embodiment of the present application. The method includes:

Step 301, transmit the image data of the mth frame to the DDIC, where m is a positive integer.

In a possible implementation, the AP and the DDIC are connected through MIPI. After the image data preparation is completed, the AP transmits the image data to the DDIC through the MIPI, and then the DDIC controls the display screen (Panel) to display images based on the image data.

Step 302, based on the historical TE signal output by the DDIC during the display process of the n-th frame image closest to the m-th frame image, determine the n historical refresh frequencies of the DDIC for each frame image in the m-nth to m-1th frame image, where n is greater than An integer equal to 2.

In order to avoid jumping the refresh frequency of DDIC, before transmitting the next frame of image data (that is, the m+1th frame of image data) to DDIC, the AP needs to determine the latest n frames of images (that is, the m-nth to m-1th frame of image) During the display process, n historical refresh frequencies of DDIC can be used to detect whether the display delay condition is satisfied based on n historical refresh frequencies.

The process of displaying a frame of image includes the process of the DDIC performing frame scanning, and the process of waiting for the next frame of image data after the frame scanning is completed. While waiting for the next frame of image data, DDIC will output the TE signal (instructing the AP to transmit the ready image data), and stop outputting the TE signal when receiving the next frame of image data, so as to frame based on the next frame of image data scanning. Therefore, in a possible implementation manner, the AP may determine the historical refresh frequency of the DDIC based on the historical TE signal output by the DDIC. Regarding the specific implementation manner of determining the historical refresh frequency on the DDIC side, the following embodiments will describe in detail.

In a possible implementation manner, during the image display process, the AP monitors the refresh rate of the DDIC in real time during the image display process of each frame, and performs a corresponding refresh rate on the n-frame images closest to the m-th frame (currently displayed frame). Storage, that is, the AP stores the historical refresh frequency of the last n frames. When transmitting the m+1th frame of image data, the AP obtains the stored n historical refresh frequencies.

In a schematic example, when m=10 and n=2, during the process of displaying the 10th frame of image and before transmitting the 11th frame of image data, the AP determines the first DDIC during the display of the 8th frame of image The historical refresh rate, and the second historical refresh rate of DDIC during the image display of the ninth frame.

Step 303: When the n historical refresh frequencies meet the display delay condition, perform a display delay operation on the m+1th frame of image data, and the display delay operation is used to delay the transmission of the m+1th frame of image data to the DDIC.

In some embodiments, the display delay condition is used to filter sporadic acceleration requests on the AP side, preventing the DDIC from directly increasing from a low refresh rate to a high refresh rate. Among them, when the image data preparation speed on the AP side changes suddenly in a short period of time, the AP generates sporadic acceleration requests, and after the sporadic acceleration requests, the speed of preparing image data on the AP side drops and cannot be maintained for a long time. Occurs after the delay in preparing image frame data on the AP side.

In a possible implementation, the AP judges whether there is an image preparation delay in the latest n frames of images based on the historical refresh frequency of the latest n frames of images. +1 frame of image data is sent to the display with a delay operation to avoid that when the m+1th frame of image data is prepared in advance, the refresh rate jumps due to the AP immediately transmitting the m+1th frame of image data to the DDIC (because the previous frame is sent Display delay will shorten the sending interval between two adjacent frames of image data, which will lead to an increase in refresh rate).

In the embodiment of this application, the purpose of the display delay operation is to reduce the refresh frequency of the DDIC, so as to avoid the sudden increase in the image preparation speed of the AP side that causes the DDIC refresh frequency to jump and rise when there is an image preparation delay in the latest n frames of images. .

Optionally, when the historical refresh frequency of the latest n frames does not meet the display delay condition, the AP does not need to perform the display delay operation on the m+1th frame of image data, and transmits the m+1th frame to the DDIC according to the conventional display logic image data.

Step 304 , when the delay operation of sending to display is completed, transmit the m+1th frame of image data to the DDIC.

In a possible implementation manner, after completing the display delay operation for the m+1th frame of image data, the AP transmits the m+1th frame of image data to the DDIC according to the TE signal output by the DDIC. After completing the transmission of the image data of the m+1 frame, and before transmitting the image data of the m+2 frame, the AP re-executes the above steps 302 to 303, which will not be repeated here in this embodiment.

Optionally, when the refresh frequency of the DDIC changes, in order to avoid the impact of the frequency change on the screen display, the DDIC displays according to the refresh frequency corresponding to the frame register (the register used to store the correspondence between the refresh frequency and the display screen parameters in the DDIC). The parameters of the display screen are adjusted. The adjusted display screen parameters may include Gamma parameters and Demura parameters, which are not limited in this embodiment.

In a schematic example, as shown in Figure 4, in the process of displaying the 5th frame and the 6th frame, the refresh frequency of the DDIC jumps from 45Hz to 72Hz without introducing a display delay mechanism. In the process of displaying the 13th frame and the 14th frame, the refresh frequency of the DDIC jumps from 51Hz to 72Hz.

After introducing the display delay mechanism, before sending the seventh frame of image data to the DDIC, the AP determines that the historical refresh frequencies of the DDIC during the display of the fourth and fifth frames are 60 Hz and 45 Hz, respectively, so it is determined that the display delay condition is met. In this way, the display delay operation is implemented, and the transmission of the seventh frame of image data to the DDIC is delayed, so that the refresh frequency of the DDIC is reduced to 60Hz during the display of the sixth frame of the image, that is, in the process of displaying the fifth and sixth frames of images, the DDIC’s Refresh frequency increased from 45Hz to 60Hz, but did not jump directly to 72Hz; another example: after introducing the display delay mechanism, before sending the 15th frame of image data to DDIC, the AP displays the images based on the 12th and 13th frames The historical refresh frequencies of the DDIC are 60Hz and 51Hz respectively, so it is determined that the display delay condition is satisfied, so that the transmission of the 15th frame of image data to the DDIC is delayed, so that the refresh rate of the DDIC is reduced to 60Hz during the display of the 14th frame of image, that is, when the DDIC is displayed During the 13th and 14th frames, the refresh rate of DDIC increased from 51Hz to 60Hz, but did not directly jump to 72Hz.

To sum up, in the embodiment of the present application, by introducing a display delay mechanism, after the AP transmits the image data of the m-th frame to the DDIC, it determines the The n historical refresh frequencies of DDIC, and further determine whether the display delay condition is satisfied based on the n historical refresh frequencies, so that when the display delay condition is satisfied, after the display delay operation is performed on the m+1th frame image data, and then sent to The DDIC transmits the image data of the m+1th frame, avoiding the DDIC refresh frequency jump caused by the fluctuation of the AP output frame rate, which in turn causes the problem of flickering and jittering of the screen, which helps to improve the stability of the DDIC refresh frequency during the image display process, and achieves an improvement. The effect of image display quality.

Optionally, the AP is used to transmit image data when the high level of the single-TE signal is detected, and the single-TE signal is a continuous high-level TE signal output by the DDIC;

Based on the historical TE signal output by the DDIC during the display process of the n-th frame image closest to the m-th frame image, determine the n historical refresh frequencies of the DDIC for each frame image in the m-nth to m-1th frame images, including:

Determine the falling edge of the historical single-TE signal output by the DDIC during the display of the n-frame image closest to the m-th frame image;

Based on the time interval between adjacent falling edges, determine n historical refresh frequencies.

Optionally, determine n historical refresh frequencies based on the time interval between adjacent falling edges, including:

Determine the i-th falling edge interval between the nearest i-th falling edge and the nearest i+1-th falling edge, where i is a positive integer smaller than n;

Based on the i-th falling edge interval, determine the historical refresh frequency corresponding to the m-i-th frame image.

Optionally, in the case that the n historical refresh frequencies meet the delay condition for display transmission, the display transmission delay operation is performed on the image data of the m+1th frame, including:

In the case that there is at least one frame of image whose historical refresh rate is less than the target refresh rate, it is determined that the display delay condition is satisfied, and the display delay operation is performed on the m+1th frame of image data. The target refresh rate is the same as that of the foreground application during running to match the base frame rate.

Optionally, data transmission is performed between the AP and the DDIC through the mobile industry processor interface MIPI;

After transmitting the mth frame of image data to the DDIC, the method also includes:

Start the first timer, wherein the MIPI is in the access state within the timer duration of the first timer;

Perform display delay operation on the image data of the m+1th frame, including:

When the timer duration of the first timer is reached, a second timer is started, and the MIPI is set to a blocking state within the timer duration of the second timer;

Wherein, the timer duration of the first timer and the timer duration of the second timer are set based on the target refresh frequency.

Optionally, the target refresh rate is i, and the highest refresh rate required by the DDIC during the running of the foreground application is j, where j is greater than i;

So the timer duration of the first timer is less than 1/j;

The sum of the timer duration of the first timer and the timer duration of the second timer is greater than 1/j and less than 1/i.

Optionally, in the case of completing the display delay operation, transmit the m+1th frame of image data to the DDIC, including:

When the timer duration of the second timer is reached, the MIPI is set to a pass state, and the m+1th frame of image data is transmitted to the DDIC.

Optionally, the method also includes:

Determine the base frame rate of the foreground application;

Set the display delay condition based on the base frame rate.

In a possible implementation manner, when the DDIC completes the image display based on the image data transmitted by the AP and is ready to refresh the next frame of image, it will output a continuous high-level single-tear effect (single-TE) signal, correspondingly, When the AP detects the high level of the single-TE signal, it transmits the prepared image data to the DDIC.

Schematically, as shown in Figure 5, after receiving the image data corresponding to the image frame A sent by the AP, the DDIC performs frame scanning on the image frame A, and outputs a continuous high-level single-TE signal when the frame scanning is completed . When the AP completes the data preparation of the image frame B and detects the high level of the single-TE signal, it transmits the image data corresponding to the image frame B to the DDIC. Correspondingly, when the DDIC scans the image frame B, the single-TE signal set low. After the frame scanning of image frame B is completed, the DDIC outputs a single-TE signal with a continuous high level again, and waits for the AP to transmit the image data of image frame C.

From the above frame scanning process, it can be seen that the interval between the falling edges (Falling Trigger) of adjacent single-TE signals output by the DDIC is the display duration of one frame of image, so the AP can detect the falling edge of the single-TE signal To determine the historical refresh frequency of DDIC. In a possible implementation manner, as shown in FIG. 6 , the process of determining n historical refresh frequencies may include the following steps.

Step 302A, determine the falling edge of the historical single-TE signal output by the DDIC during the image display process of the n-frame closest to the m-th frame.

In a possible implementation, when it is necessary to determine the n historical refresh frequencies corresponding to the latest n frames of images, the AP obtains the falling edges of the last n+1 historical single-TE signals output by the DDIC, where n+1 Falling edges of two adjacent historical single-TE signals in the historical single-TE signal are used to determine a historical refresh frequency.

Schematically, as shown in Figure 5, when n=2, if the image data of the image frame C transmitted by the AP is received, the AP obtains the falling edges of the last three historical single-TE signals, which are respectively DDIC to image frame C The falling edge of the single-TE signal output before frame scanning (that is, the first falling edge), and the falling edge of the single-TE signal output by DDIC before frame scanning of image frame B (that is, the second falling edge most recently) , and the falling edge (that is, the latest third falling edge) of the single-TE signal output by the DDIC before frame scanning of the image frame A.

Step 302B, based on the time interval between adjacent falling edges, determine n historical refresh frequencies.

Further, for the determined n falling edges, AP calculates the time interval Δt between two adjacent falling edges, determines the display duration of historical image frames between adjacent falling edges, and then determines the corresponding History refresh frequency, wherein, history refresh frequency (unit is Hz)=1000/time interval Δt (unit is ms).

In a possible implementation, when determining the historical refresh frequency corresponding to the m-i frame image, the AP determines the i-th falling edge interval between the latest i-th falling edge and the latest i+1-th falling edge (i is less than A positive integer of n), so that based on the i-th falling edge interval, the historical refresh frequency corresponding to the m-i-th frame image is determined.

When m=3, n=2, i=1, when determining the historical refresh frequency corresponding to the 3-1th frame (2nd frame) image, the AP determines the first one closest to the mth frame (3rd frame) The first falling edge interval between the falling edge and the second falling edge closest to the mth frame (the third frame), so that the historical refresh frequency corresponding to the second frame image can be determined; similarly, when m=3, When n=2, i=2, when determining the historical refresh frequency corresponding to the 3-2th frame (1st frame), the AP determines the second falling edge nearest to the mth frame (3rd frame) The second falling edge interval between the nearest third falling edge of the m frame (the third frame), so that the historical refresh frequency corresponding to the image of the first frame can be determined.

Schematically, as shown in Figure 5, the AP determines the history corresponding to image frame B (equivalent to the second frame image above) based on the first falling edge interval between the most recent first falling edge and the most recent second falling edge The refresh frequency is 45Hz (that is, the refresh frequency of DDIC is 45Hz in the process of displaying image frame B); based on the second falling edge interval between the nearest second falling edge and the nearest third falling edge, determine the image frame A ( The historical refresh frequency corresponding to the above-mentioned first frame image) is 60 Hz (that is, the refresh frequency of the DDIC during the process of displaying image frame A is 60 Hz).

It should be noted that, in other possible implementation manners, the AP may also use other methods to determine the historical refresh frequency of the DDIC, which is not limited in this embodiment.

In a possible scenario, when the base frame rate of the front-end application is 60FPS (Frame Per Second, Frame Per Second), the refresh rate of DDIC should be designed to stabilize the frame with 60Hz as the target refresh rate, that is, the target refresh rate The frequency matches the base frame rate while the foreground app is running. Wherein, the match between the target refresh rate and the reference frame rate means that the difference between the target refresh rate and the reference frame rate is less than a threshold (for example, 5FPS). Optionally, the target refresh rate is equal to the reference frame rate, or the target refresh rate is slightly lower than greater than the base frame rate, or, the target refresh rate is slightly less than the base frame rate.

Among them, when the AP prepares the image data on time (that is, according to the frequency of 60Hz) or when the delay is ready (that is, less than 60Hz), the DDIC can wait appropriately to ensure that the refresh rate of the DDIC is not greater than the target refresh rate in most scenarios. range (such as 45Hz to 60Hz).

And when the AP prepares the image data in advance (that is, when there is a demand for acceleration), in order to avoid the DDIC’s refresh rate from directly jumping from a low refresh rate to a high refresh rate (that is, from a refresh rate lower than 60Hz to a refresh rate higher than 60Hz) Refresh frequency), in this application, the AP performs display delay operation.

Since the refresh frequency hopping occurs when the historical refresh frequency is low and the current refresh frequency is high, in a possible implementation, after the AP obtains n historical refresh frequencies, it detects whether each historical refresh frequency is lower than the target Refresh frequency. If the historical refresh frequency corresponding to at least one frame of image is less than the target refresh frequency, it is determined that the display delay condition is satisfied, and then the display delay operation is performed on the (m+1)th frame of image data.

If the historical refresh frequency corresponding to each frame image is greater than or equal to the target refresh frequency, it is determined that the display delay condition is not met, and the m+1th frame of image data is transmitted to the DDIC according to the conventional display logic (that is, the single-TE signal is detected to be high level, the AP transmits the m+1th frame of image data to the DDIC).

Since different applications correspond to different base frame rates, different target refresh rates need to be set for different applications. In a possible implementation manner, before the image data is transmitted, the AP determines the base frame rate of the foreground application, so as to set the display delay condition based on the base frame rate.

In a schematic example, when the current foreground application is a game application, and the base frame rate of the game application is 60FPS, the AP sets the display delay condition as follows: among the historical refresh rates corresponding to the last two frames of images, there is a historical refresh rate less than 60Hz. That is, as long as the historical refresh frequency corresponding to an image is less than 60Hz, the AP will perform the display delay operation; if the historical refresh frequency corresponding to two frames of images is not less than 60Hz, the AP does not need to perform the display delay operation.

As for the specific way of delaying image data transmission, in a possible implementation, since data transmission is performed between the AP and DDIC through MIPI, when the historical refresh frequency satisfies the delay condition for display transmission, and the m+1th When the frame image data is ready, the AP can delay sending the display by blocking MIPI. The following uses an exemplary embodiment for description.

Please refer to FIG. 7 , which shows a flowchart of an image data transmission method shown in another exemplary embodiment of the present application. The method includes:

Step 701, transmit the image data of the mth frame to the DDIC, where m is a positive integer.

Step 702, start the first timer, wherein, the MIPI is in the pass state within the timer duration of the first timer.

In a possible implementation manner, after transmitting the image data to the DDIC, the AP starts the first timer and ensures that the MIPI is in the pass state within the timer duration of the first timer, so that the AP can pass the MIPI during the frame scanning process. Instructions other than image data are transmitted to the DDIC, wherein the first timer can be started by the AP by calling a timing thread.

Schematically, as shown in FIG. 8 , the AP starts the first timer after transmitting the image data of the image frame C to the DDIC.

Step 703, based on the historical TE signal output by the DDIC during the display process of the n-th frame image closest to the m-th frame image, determine the n historical refresh frequencies of the DDIC for each frame image in the m-nth to m-1th frame image, where n is greater than An integer equal to 2.

Schematically, as shown in FIG. 8, after the AP transmits the image data of image frame C to the DDIC, it determines that the historical refresh frequency of the DDIC when displaying image frame A is 60 Hz, and that the historical refresh frequency of DDIC when displaying image frame B is 45 Hz.

Step 704: In the case that there is at least one frame of image whose historical refresh frequency is less than the target refresh frequency, when the timer duration of the first timer is reached, start the second timer, and when the timer duration of the second timer Set MIPI to blocking state.

In this embodiment, when the first timer reaches the timer duration, the AP sets the MIPI from the channel state to the blocking state, and starts the second timer to ensure that the MIPI remains in the blocking state within the timer duration of the second timer. Since MIPI is in the blocking state, the TE signal cannot be detected during the second timer period, so the AP cannot transmit the m+1th frame of image data to the DDIC during the second timer period, thus achieving the effect of delayed display.

Wherein, the timer duration of the first timer and the timer duration of the second timer are set based on the target refresh frequency. In a possible implementation manner, when the target refresh frequency is i, and the highest refresh frequency required by DDIC during the running of the foreground application is j, the timer duration of the first timer is less than 1/j, and the first timer The sum of the timer duration of the second timer and the second timer is greater than 1/j and less than 1/i, so that MIPI enters the blocking state before the rising edge of the next TE signal, and makes the refresh of DDIC during the image display process of the mth frame Frequency is the target refresh rate.

Schematically, as shown in FIG. 8, when the target refresh frequency of the foreground application running is 60 Hz, and the highest refresh frequency required by the DDIC during the running of the foreground application is 72 Hz, the AP sets the timer duration of the first timer to 13 ms ( less than 1000÷72=13.9ms), and the timer duration of the second timer is set as 2ms (the sum of the timer duration of the first timer and the timer duration of the second timer is 15ms, and 15ms is less than 1000÷60= 16.7ms). During the duration of the second timer, because the MIPI is in a blocking state, the AP cannot transmit the image data of the image frame D to the DDIC within this period.

Step 705, when the timer duration of the second timer is reached, set the MIPI to the channel state, and transmit the m+1th frame of image data to the DDIC.

Optionally, when the timer duration of the second timer is reached, the AP resets the MIPI to the channel state, and when detecting the high level of the single-TE signal, transmits the m+1th frame of image data to the DDIC, so that the first During the m-frame image display process, the refresh rate of the DDIC is the target refresh rate.

Schematically, as shown in Figure 8, when the timer duration of the second timer is reached, the AP restores the MIPI to the path state, and when detecting the high level of the single-TE signal output by the DDIC, transmits the image frame to the DDIC D image data. Without introducing the display delay mechanism, the refresh frequency of DDIC is 72Hz in the process of displaying image frame C, but after introducing the display delay mechanism, the refresh frequency of DDIC is reduced to 60Hz in the process of displaying image frame C, thus avoiding refresh The frequency jumps from 45Hz to 72Hz.

It should be noted that the above-mentioned embodiment only uses the delayed display sending method of blocking MIPI as an example for illustration. In other possible implementation manners, the AP may delay the display sending timing in other ways. The explicit way constitutes a limitation.

Please refer to FIG. 9 , which shows a structural block diagram of an image data transmission device provided by an embodiment of the present application. The unit includes:

The transmission module 901 is configured to transmit the mth frame of image data to the display driver chip DDIC, where m is a positive integer;

The first determination module 902 is configured to determine, based on the historical TE signal output by the DDIC during the display process of the n-th frame image closest to the m-th frame image, the DDIC for each frame image in the m-nth to m-1th frame image n historical refresh frequencies, n is an integer greater than or equal to 2;

The delay module 903 is configured to perform a display delay operation on the (m+1)th frame of image data when the n historical refresh frequencies meet the display delay condition, and the display delay operation is used to delay sending to the DDIC transmitting the m+1th frame of image data;

The transmission module 901 is further configured to transmit the m+1th frame of image data to the DDIC when the display delay operation is completed.

Optionally, the AP is used to perform image data transmission when a high level of a single-TE signal with a single tearing effect is detected, and the single-TE signal is a continuous high-level TE signal output by the DDIC;

The first determining module 902 includes:

The first determination unit is configured to determine the falling edge of the historical single-TE signal output by the DDIC during the display process of the n-frame image closest to the m-th frame image;

The second determination unit is configured to determine n historical refresh frequencies based on the time interval between adjacent falling edges.

Optionally, the second determination unit is configured to:

Determine the i-th falling edge interval between the nearest i-th falling edge and the nearest i+1-th falling edge, where i is a positive integer smaller than n;

Based on the i-th falling edge interval, the historical refresh frequency corresponding to the m-i-th frame image is determined.

Optionally, the delay module 903 includes:

A delay unit, configured to determine that the display delay condition is satisfied when the historical refresh frequency corresponding to at least one frame of image is less than the target refresh frequency, and perform the sending of the m+1th frame of image data operation with significant delay, the target refresh rate matches the base frame rate during the running of the foreground application.

Optionally, data transmission is performed between the AP and the DDIC through a mobile industry processor interface MIPI;

The device also includes:

A starting module, configured to start a first timer, wherein the MIPI is in a pass state within the timer duration of the first timer;

The delay unit is used for:

When the timer duration of the first timer is reached, start a second timer, and set the MIPI to a blocking state within the timer duration of the second timer;

Wherein, the timer duration of the first timer and the timer duration of the second timer are set based on the target refresh frequency.

Optionally, the target refresh frequency is i, and the highest refresh frequency required by the DDIC during the running of the foreground application is j, where j is greater than i;

So the timer duration of the first timer is less than 1/j;

The sum of the timer duration of the first timer and the timer duration of the second timer is greater than 1/j and less than 1/i.

Optionally, the transmission module 901 is configured to:

When the timer duration of the second timer is reached, the MIPI is set to a pass state, and the m+1th frame of image data is transmitted to the DDIC.

Optionally, the device also includes:

A second determining module, configured to determine the reference frame rate of the foreground application;

A setting module, configured to set the display delay condition based on the reference frame rate.

To sum up, in the embodiment of the present application, by introducing a display delay mechanism, after the AP transmits the image data of the mth frame to the DDIC, based on the historical TE signal output by the DDIC during the display process of the n-frame image closest to the m-th image, Determine the n historical refresh frequencies of DDIC, and further determine whether the display delay condition is satisfied based on the n historical refresh frequencies, so that when the display delay condition is satisfied, after the display delay operation is performed on the m+1th frame of image data, and then The image data of the m+1th frame is transmitted to the DDIC to avoid the jump of the DDIC refresh rate caused by the fluctuation of the AP output frame rate, which in turn causes the problem of flickering and jittering of the screen, which helps to improve the stability of the DDIC refresh rate during the image display process, and achieves Improve the image display quality effect.

Please refer to FIG. 10 , which shows a structural block diagram of a terminal 1000 provided by an exemplary embodiment of the present application. The terminal 1000 may be a smart phone, a tablet computer, a notebook computer, and the like. The terminal 1000 in this application may include one or more of the following components: a processor 1010 , a memory 1020 , and a display screen module 1030 .

The processor 1010 may include one or more processing cores, and the processor 1010 may be the AP described in the foregoing embodiments. The processor 1010 uses various interfaces and lines to connect various parts of the entire terminal 1000, and executes the terminal by running or executing instructions, programs, code sets or instruction sets stored in the memory 1020, and calling data stored in the memory 1020. 1000's of various functions and processing data. Optionally, the processor 1010 may adopt at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in the form of hardware. The processor 1010 may integrate one or more of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a neural network processor (Neural-network Processing Unit, NPU) and a modem, etc. The combination. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used to render and draw the content that the touch display module 1030 needs to display; the NPU is used to realize the artificial intelligence (Artificial Intelligence, AI) function; the modem Used to handle wireless communications. It can be understood that, the above-mentioned modem may not be integrated into the processor 1010, but may be realized by a single chip.

The memory 1020 may include random access memory (Random Access Memory, RAM), and may also include read-only memory (Read-Only Memory, ROM). Optionally, the memory 1020 includes a non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, codes, sets of codes or sets of instructions. The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), Instructions and the like for implementing various method embodiments of the present application; the storage data area may store data created according to the use of the terminal 1000 (such as audio data, phone book) and the like.

The display screen module 1030 is a display component for displaying images, and is usually arranged on the front panel of the terminal 1000 . The display module 1030 can be designed as a full screen, a curved screen, a special-shaped screen, a double-sided screen or a folding screen. The display screen module 1030 can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in this embodiment.

In the embodiment of the present application, the display screen module 1030 includes a DDIC 1031 and a display screen 1032 (panel). Wherein, the display screen 1032 may be an OLED display screen, which may be a low temperature polysilicon (Low Temperature Poly-Silicon, LTPS) AMOLED display screen or a low temperature polycrystalline oxide (Low Temperature Polycrystalline Oxide, LTPO) AMOLED display screen.

The DDIC1031 is used to drive the display screen 1032 to display images. In addition, the DDIC 1031 is connected to the processor 1010 through a MIPI interface, and is used for receiving image data and instructions issued by the processor 1010 .

In a possible implementation manner, the display screen module 1030 also has a touch function. Through the touch function, the user can use any suitable object such as a finger or a touch pen to perform touch operations on the display screen module 1030 .

In addition, those skilled in the art can understand that the structure of the terminal 1000 shown in the above drawings does not constitute a limitation on the terminal 1000, and the terminal may include more or less components than those shown in the figure, or combine some components, or different component arrangements. For example, the terminal 1000 also includes components such as a microphone, a speaker, a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (Wireless Fidelity, WiFi) module, a power supply, and a bluetooth module, which will not be repeated here.

The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores at least one program, and the at least one program is used to be executed by a processor to implement the image data transmission method as described in the above-mentioned embodiments.

The embodiment of the present application also provides a computer program product, the computer program product includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; the processor of the terminal reads the computer instructions from the computer-readable storage medium, and processes The computer executes the computer instruction, so that the terminal executes the image data transmission method provided by the above embodiment.

Those skilled in the art should be aware that, in the foregoing one or more examples, the functions described in the embodiments of the present application may be implemented by hardware, software, firmware or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

The above are only optional embodiments of the application, and are not intended to limit the application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the application shall be included in the protection of the application. within range.

Claims (19)

1. An image data transmission method for an application processor AP, the method comprising:

   Transmit the image data of the mth frame to the display driver chip DDIC, where m is a positive integer;

   Based on the historical TE signal output by the DDIC during the display process of the n-th frame image closest to the m-th frame image, determine the n historical refresh frequencies of the DDIC for each frame image in the m-nth to m-1th frame images, where n is An integer greater than or equal to 2;

   In the case that the n historical refresh frequencies meet the display delay condition, the display delay operation is performed on the m+1th frame of image data, and the display delay operation is used to delay the transmission of the m+th frame to the DDIC 1 frame of image data;

   When the display delay operation is completed, the m+1th frame of image data is transmitted to the DDIC.
2. The method according to claim 1, wherein the AP is used to perform image data transmission when a high level of a single-TE signal of a single tearing effect is detected, and the single-TE signal is a continuous signal output by the DDIC High level TE signal;

   Based on the historical TE signal output by the DDIC during the display process of the n-th frame image closest to the m-th frame image, determine the n historical refresh frequencies of the DDIC for each frame image in the m-nth to m-1th frame images, include:

   determining the falling edge of the historical single-TE signal output by the DDIC during the display process of the n-frame image closest to the m-th frame image;

   Based on the time interval between adjacent falling edges, determine n historical refresh frequencies.
3. The method according to claim 2, wherein said determining n historical refresh frequencies based on the time interval between adjacent falling edges comprises:

   Determine the i-th falling edge interval between the nearest i-th falling edge and the nearest i+1-th falling edge, where i is a positive integer smaller than n;

   Based on the i-th falling edge interval, the historical refresh frequency corresponding to the m-i-th frame image is determined.
4. The method according to any one of claims 1 to 3, wherein, in the case where the n historical refresh frequencies meet the display delay condition, performing a display delay operation on the m+1th frame of image data includes:

   In the case that the historical refresh frequency corresponding to at least one frame of image is less than the target refresh frequency, it is determined that the display delay condition is met, and the display delay operation is performed on the m+1th frame of image data, so The above target refresh rate matches the base frame rate when the foreground application is running.
5. The method according to claim 4, wherein data transmission is performed between the AP and the DDIC through a mobile industry processor interface MIPI;

   After the m-th frame of image data is transmitted to the DDIC, the method further includes:

   Start a first timer, wherein the MIPI is in a pass state within the timer duration of the first timer;

   The performing the display delay operation on the m+1th frame of image data includes:

   When the timer duration of the first timer is reached, start a second timer, and set the MIPI to a blocking state within the timer duration of the second timer;

   Wherein, the timer duration of the first timer and the timer duration of the second timer are set based on the target refresh frequency.
6. The method according to claim 5, wherein the target refresh frequency is i, the highest refresh frequency required by the DDIC during the running of the foreground application is j, and j is greater than i;

   So the timer duration of the first timer is less than 1/j;

   The sum of the timer duration of the first timer and the timer duration of the second timer is greater than 1/j and less than 1/i.
7. The method according to claim 5, wherein said transmitting the m+1th frame of image data to the DDIC after completing the delay operation of sending display comprises:

   When the timer duration of the second timer is reached, the MIPI is set to a pass state, and the m+1th frame of image data is transmitted to the DDIC.
8. The method according to claim 4, wherein the method further comprises:

   determining the reference frame rate of the foreground application;

   The display delay condition is set based on the reference frame rate.
9. An image data transmission device, the device comprising:

   The transmission module is used to transmit the image data of the mth frame to the display driver chip DDIC, where m is a positive integer;

   The first determination module is configured to determine n of each frame image in the m-nth to m-1th frame images of the DDIC based on the historical TE signal output by the DDIC during the display process of the n-th frame image closest to the m-th frame image historical refresh frequency, n is an integer greater than or equal to 2;

   A delay module, configured to perform a display delay operation on the m+1th frame of image data when the n historical refresh frequencies meet the display delay condition, and the display delay operation is used to delay transmission to the DDIC The m+1th frame of image data;

   The transmission module is further configured to transmit the m+1th frame of image data to the DDIC when the display delay operation is completed.
10. The device according to claim 9, wherein the AP is configured to perform image data transmission when a high level of a single-TE signal of a single tearing effect is detected, and the single-TE signal is a continuous signal output by the DDIC High level TE signal;

    The first determination module includes:

    The first determination unit is configured to determine the falling edge of the historical single-TE signal output by the DDIC during the display process of the n-frame image closest to the m-th frame image;

    The second determination unit is configured to determine n historical refresh frequencies based on the time interval between adjacent falling edges.
11. The device according to claim 10, wherein the second determining unit is configured to:

    Determine the i-th falling edge interval between the nearest i-th falling edge and the nearest i+1-th falling edge, where i is a positive integer smaller than n;

    Based on the i-th falling edge interval, the historical refresh frequency corresponding to the m-i-th frame image is determined.
12. The device according to any one of claims 9 to 11, wherein the delay module comprises:

    A delay unit, configured to determine that the display delay condition is satisfied when the historical refresh frequency corresponding to at least one frame of image is less than the target refresh frequency, and perform the sending of the m+1th frame of image data operation with significant delay, the target refresh rate matches the base frame rate during the running of the foreground application.
13. The device according to claim 12, wherein data transmission is performed between the AP and the DDIC through a mobile industry processor interface (MIPI);

    The device also includes:

    A starting module, configured to start a first timer, wherein the MIPI is in a pass state within the timer duration of the first timer;

    The delay unit is used for:

    When the timer duration of the first timer is reached, start a second timer, and set the MIPI to a blocking state within the timer duration of the second timer;

    Wherein, the timer duration of the first timer and the timer duration of the second timer are set based on the target refresh frequency.
14. The device according to claim 13, wherein the target refresh frequency is i, the highest refresh frequency required by the DDIC during the running of the foreground application is j, and j is greater than i;

    So the timer duration of the first timer is less than 1/j;

    The sum of the timer duration of the first timer and the timer duration of the second timer is greater than 1/j and less than 1/i.
15. The device according to claim 13, wherein the transmission module is configured to:

    When the timer duration of the second timer is reached, the MIPI is set to a pass state, and the m+1th frame of image data is transmitted to the DDIC.
16. The apparatus according to claim 12, wherein said apparatus further comprises:

    A second determining module, configured to determine the reference frame rate of the foreground application;

    A setting module, configured to set the display delay condition based on the reference frame rate.
17. A terminal, the terminal includes an application processor AP, a display screen and a display driver circuit chip DDIC, the AP and the DDIC are connected through a mobile industry processor interface MIPI, and the AP is used to execute at least A program to realize the image data transmission method as claimed in any one of claims 1 to 8.
18. A computer-readable storage medium, the computer-readable storage medium stores at least one program, and the at least one program is used to be executed by a processor to implement the image data transmission method according to any one of claims 1 to 8.
19. A computer program product, the computer program product includes computer instructions, the computer instructions are stored in a computer-readable storage medium; the processor AP of the terminal reads the computer instructions from the computer-readable storage medium, the The processor AP executes the computer instructions, so that the processor AP of the terminal executes the image data transmission method according to any one of claims 1-8.

Images